The issue is whether you are accepting and then reneging on an offer from a program where you would be paying all the tuition yourself, vs. a program where the institution is offering you a fellowship, assistantship, and/or tuition waiver.
In the former case, where you are paying the tuition out of your own pocket, you can (technically) do whatever you want with no consequences.
In the latter case, if you have accepted a "funded" offer before April 15th, you can rescind your acceptance up until April 15th and accept admission to a different program (likely, one which has offered you admission later, after their "more desirable" candidates have turned them down in favor of more prestigious institutions.)
If as of close of business on April 15th you have accepted (and not rescinded) an offer which includes financial aid from one of the institutions whose name appears on this list:
then you are not allowed to renege on your commitment to that institution to accept a late admission offer at another institution unless the first institution agrees, in writing, to release you from your commitment.
Of course, since most MFE programs are structured in such a way that you pay everything yourself, this situation doesn't apply (it's mainly oriented toward Ph.D. programs which offer a multi-year fully-funded arrangement to the "top" candidates on their list.)
Dropping a course is very different than dropping a program. No one cares if you say "I want to take DiffEqs next semester rather than now." However, if you tell a school that you are attending their program, then reneg on that, it makes you look unstable and untrustworthy, not to mention that the world of finance is a small one where everyone knows everyone else. This could very easily come back to bite you. Of course, I also think it is unethical. So whether you're looking at it from a personal perspective or from an ethical perspective, I think it is just something you should not do.