How to prevent magical supremacy?

Question

My world is one where there are billions of humans (about 3.5). These humans are just like us i.e. normal. However, there is a race which mutated from humans early on in time. This race has magical powers which range in severity (think of Skulduggery Pleasant if it helps). Apart from their abilities, they're like humans in every other way (they're literally just humans with magic).

My question:

Obviously, this magical race could crush and rule over human civilisation without too much difficulty if they started a war. So how can I prevent them from trying to strive for world dominance?

Note:

1. Most normal humans are mildly racist towards this magical race
2. This magical race has no known home as such (normal humans think these magical people have no cities/countries of their own). But, this magical race do have their own country and have concealed it using their magic but the normal humans don't know about it.
3. There was a war between races a few hundred years back where the humans thought they'd won. They didn't. They sustained heavy losses and the magicals realised that the humans would die out and never give up so they cast a huge illusion (ignore the realism for now). The mild racist attitude towards these magical people stems from the intense hatred from the war.

Answer 1

HORRIBLE SIDE EFFECTS The magic users CAN do incredible feats, but have to pay a price of some sort for anything they do. Small stuff like lighting a candle with a hand gesture or making somebody have a head ache isn't really costly. Maybe they get a migraine once or twice a month or stomach cramps. Wanna knock down a castle wall or obliterate an opponent in a blast of ravening death? You run the risk of a grand-mal seizure or a heart attack. large magical tasks (like concealing a city) require many MANY individuals all contributing a safe to use tiny amount of power. A magic user's ability isn't measured by their actual power level so much as how efficiently they manage to utilize this power (instead of incinerating a charging enemy why not simply light his hat on fire then stab him when hes freaking out. Instead of knocking down the castle wall why not identify and loosen a single key stone so that a battering ram is more effective?) Using magic effectively and safely requires years and years of study and training and you run the risk of having severe health effects the longer and more often you use it. Heavy magic practitioners often die younger and sickly from myriad cumulative physical effects.

Answer 2

1. "anti-magic" exists. think DnD's anti-magic field or dispel magic, this would obviously need to be usable by non-magic humans so perhaps there are certain "magic-less" areas that occur naturally and normal's have built all their settlements around them, perhaps there is some material which is resistant to magic perhaps there are machines with anti-magical abilities, perhaps magic goes away every eclipse the list goes on.
2. there are gods/god-like beings/spirits with a vested interest in either normal humans or in preventing magicians from gaining to much power. these could also be a source of anti-magic. perhaps only the faith of normal humans feeds them, perhaps they just fear magicians. normal humans would not need to know of their existance

3. Magical creatures exist that normal humans have bred and trained. (think dragons, sphinxes, giant anythings's or perhaps even vampires or werewolves.), while magical people could also train these it would certainly level the playing field

4. Long term exposure to magic destroys complex machinery. in this way your non-magical people could have anything from muskets to tanks to balance out the magic and magical people would not. (if you have few enough magical people its possible they just don't have the economies of scale needed to build high-tech stuff).

5. Non-magical people simply vastly outnumber magical people, and are reasonably well organised and united. this is not unprecedented many historians believe the reason Homo sapiens out competed Homo neanderthals is simply that we had better cooperation even though neanderthals had most other advantages. You would of course have to come up with a lore reason why Mages don't just reproduce like mad with fertility spells or something but hey. this works quite well with a variant of number 4

6. Magic is so wild and unpredictable that Magicians pose as much of a threat to each other on the battle-field as they do their enemies. This would make fielding them on the battle-field nigh impossible

7. Killing another human being using magic has side-effects of increasing severity, making using magic to wage war or worse genocide undesirable for all parties involved

8. Magic is dependant upon some reasonably scarce resource, perhaps most spells require gold or a special spice which is a hassle to grow.

9. they where in charge of the world for a great deal of time until event X, the recent state of affairs is an outlier.

10. some pick and mix of the above reasons.