Re: Looking for a fantasy game system
An older game, but Reve: the Dragon Oneiros should match the request as well.
You have a plethora of skills, which normally improve only when you manage a critical success with one, under difficult circumstances. So an apprentice swordfighter will be able to improve by trying out a few simple feints, wile an expert mage needs to cast the most difficult spells in order to get any better.
Player characters have a further advantage, in that they are striving to meet an archetype (which you can define yourself - it's a matter of picking skills) and can develop their talent for archetype skills much faster. However, even the archetype has limits, so once you hit the best any of your past incarnations mananged, you're back to looking for danger.
In fact, any of the Chaosium systems (Call of Chthulu, Runequest, Elric, etc.) fall into this category, with skills expressed as percentiles and a chance to improve them if you use the skill during an adventure, and then roll higher than the skill while training. So as you get better, it becomes harder and harder to improve.
A lot of early rule-heavy RPGs had this type of system, if you want to dig through the archives and pull up a few.
On the pure xp for skills front, there's Exalted, I suppose.