This phrase is often used to indicate that something is very unlikely to happen.

For Example:

If your friend said, “hey, maybe you’ll win the lottery tomorrow!” You could respond, “well, don’t hold your breath.” What you are saying is “don’t hold your breath until this happens because if you did it would be the last breath you took.” The implication is that it, whatever “it” is, is “not going to happen.”