As the first poster said,
Do what makes you happy, or you won't last.
Also, I have no doubt that you will use programming languages in some of your classes. I'm sure your required to take some sort of "programming for engineers." If you no for sure that you wont learn any programming, the first year sequence of CS classes should be enough (intro to program, then intro to data structures).
To not confuse you:
Follow through with Engineering Physics and H math minor
Take first 2 cs classes as electives
Take a intro to probability class (most likely will count towards math minor)
You math minor should cover Calc 1-3, Lin Alg, Diff Eqs, Analysis, Probability, PDE's and I am sure some of those are also required for your engineering physics degree. Take a numerical analysis class also if you can