I just don't carry change. I don't often use cash if I don't have to. When I get change, it's usually at a convenience store and I look for the "take a penny/leave a penny" tray or a donation bin. If it makes it to the car, it goes straight into the cupholder. If it makes it as far as the house, it goes in a change jar.
How big is your wallet? I carry a "minimalist" wallet. Just big enough to carry a few cards, DL, and some cash. I used to carry a slightly bigger tri-fold wallet, but when I started carrying a phone in the same pocket... and then that phone started getting BIGGER... I went minimal with the wallet.