I assume you mean ‘level’ 54, in which case it seems odd that you would only be getting level 100+. The matchmaking system is a bit of a mystery, but at level 250, I get matches from as low as 125 or so up to level 1000. One thing to be aware of is that as you move up the PVP ranks, you will face increasingly difficult opposition. At ranks 15 to 10, it is usually pretty easy. At ranks 5 to 1, it is usually pretty tough. No matter what, there is a lot of random variation.
I always scout to see which troops I am facing, how many traits they have and what level they are. If something looks like it is going to be too much, I pay the 10 gold and try a different match. Better to spend 60 gold to scout and re-roll than enter a fight I don’t have a good chance of winning.