I'm not going to say I'm "sure"/confident like some of the other posts here, but I will comment on some observations garnered over the few years I've been working. I've noticed that some jobs value technical skill more and others value drive, social skills, etc. People that get the entry-level IB jobs seem to fit a certain mold. They are generally pretty confident (perhaps overly so), know what they want (and have known since earlier in their college career), and have a lot of energy.
Unfortunately going to a math finance program, and then applying for IB pretty much shows you have NOT known what you wanted. That's already one strike against you. Are you out? No, not yet. I'd really take a step back and take an honest self-assessment. If not for the "glamour" and anticipated money, would you be thinking about it?