Not at all.
I joined MFE hoping to go into Sales (strange, I know). That quickly changed into me wanting to moving into Risk, and now I'm working in S&T as a Strat and very happy.
Moving around is exciting. Learning new roles and areas of finance is exciting. Just make sure when you are on a job interview you sound very confident that you are 100% committed and eager to be a part of that role.