Rule out some things
If the atmosphere is more dense, then it will refract more. However, light refraction is highly wavelength dependent, so if it refracts more light, the sky will be more blue. The equation can be seen here. Different compounds have different scattering cross-sections, so they will scatter different amounts, but all of them will have the same wavelength dependent curve.
Every yellow star will have a blue sky if there are no colored compounds in the air. If the star was redder, then the sky would be shifted to red. If the red, yellow, and blue components to the sky's scattering were equal (a possible result of a red star), then the sky would appear white. If the star were bluer, then the sky would be even bluer or violet.
So basically, we are limited to a tinted sky to get the orange color you want.
Tinted sky
If your sky was tinted with something orange, then it would be orange. However, there would still be a blue-ish tint to the atmosphere due to reflection from the much more common nitrogen and oxygen. Your best bet is to have planetary cloud cover that is tinted orange; then there is always an orange backdrop at a couple of km altitude.
The list of things that could be in the clouds and are orange is not long. For gasses there are basically none, but Nitrogen Dioxide and Bromine might be close. Nitrogen Dioxide clouds are plausible in some sort of biologically dominated system. Bromine sounds pretty dangerous. While it is very reactive and won't normally be found as a bromine gas, sunlight converts organo-bromide compounds to free bromine at the top of the atmosphere, so that gives us a plausible reason for bromine haze. Unfortunately, free bromine destroys the ozone layer, so that is a bummer.
As far as aqueous compounds suspended in water droplets as a method for coloring clouds, we have Dichromate (Cr$_2$O$_7^{2-}$) and Cobalt ammine {Co(NH$_3$)$_6^{3+}$}. I don't know much about either, but neither one sounds very healthy. Depending on your goals for this planet, you could either have the geological and life cycles on this planet be adapted to having high levels of chromium or cobalt around, or just make your human explorers wear masks.