I hate it too. I said this the first week of it. I have spent anywhere from 400-600 gems on class switching so far.
They don’t want to remove it to make sure people stick with one class, yet their own data shows that most people stick with one class (1/3 have no class, another 1/3 have Warlord). The easiest way to fix the “not problem” is to add some kind of incentive to sticking with one class, so people would rarely switch even if the option was available for free.