This is a common phrase that is spoken to ask someone who is in a hurry to slow down or stop for a moment. It is a reference to a person driving a vehicle that is pulled by horses. “Holding” the horses means to stop the horses and wait.

For Example:

“Hold your horses! Grab an umbrella before you walk out into the rain!”