Maybe, it rather depends upon the technology you use.
(say) 10 years ago, there was no big difference between the tech used by quants and housekeeping IT.
Then like now quants used C/
C++ one of the VBA family, and some sort of graphic/calc tool like Matlab. Often they used the same SQL database as everyone else.
Housekeeping IT now has a lot more Java and web technologies in general, typically they use Oracle and/or MS SQL, and their programmers go to a
lot more mettings. The head of front office IT at one truly huge bank has had a guy in his team whose sole role is to go to meetings for him, yes really.
As I say to everyone considering the CQF, "you will do more programming in this line of work than if your job title is currently programmer".
Quants now use
C++, VBA, C#, KDB, Matlab/R and very little Java.
They don't do "business analysis", and testing is alien to them. User interfaces are a form of minimal gothic, and obey no UI guidelines except to copy Excel and some cool thing they used once. Housekeeping IT at large banks have people who do nothing all day except dream up these standards. At one large bank I utterly defeated a political enemy because he didn't realise just how attached senior management were to the colour scheme they had adopted. I did. I won. He lost.
Yes, and there's politics too.
So if you get a job using the rights skills, it can be an OK place to start, but of course there is the issue of not only maintiaing your math skills, but moving them forward.