Cannon of the times required round iron cannonballs. A ship's master wanted to store the cannonballs such that they could be available for instant use when needed, but in a manner that would not let them roll around the gun deck.
The solution devised was to stack them up in a square-based pyramid next to the cannon.
Four levels would provide a stack of 30 cannonballs.
When temperature falls, brass contracts faster than iron. As it got cold on the gun decks, the indentations in the brass monkey would get smaller than the iron cannonballs they were holding. If the temperature got cold enough, the bottom layer of cannonballs would pop out of the indentations, spilling the entire pyramid over the deck. Thus it was, quite literally, "cold enough to freeze the balls off a brass monkey." And so, another familiar phrase became part of the language. Now, aren't you glad you took the time to read this historical piece?