Personally i dont think that would solve the problem at all. I think that would just make things worse.
Well, let me clarify then.
You enforce a 5 (or 8 to start the process) foreigners rule throughout all UEFA leagues.
Then you take the example of a few teams. Let's say: Real Madrid, Man City and Juventus. I'll make a list of their foreigners (dual nationalities accepted).
Real: Cristiano Ronaldo, Modric, Casemiro, Kroos, Varane, Benzema, Kovacic, Marcelo, Bale. (iirc Navas holds a Spanish passport and so do the first team youngsters that aren't considered Spanish nationals)
That is 9 foreigners.
Man City: Aguero, Gabriel Jesus, Sane, de Bruyne, Toure, Ederson, Fernandinho(?), Otamendi, David Silva, Bernardo Silva, Bravo, Mendy, Danilo, Gundogan, Kompany, Mangala.
That is 16 foreigners.
Juventus: Dybala, Higuain, Benatia, Matuidi, Khedira, Bentancur, Mandzukic, Asamoah, Liechtsteiner, Howedes, Sczeszny, Alex Sandro, Douglas Costa, Cuadrado, Pjanic.
That is 15 foreigners.
Now imagine that you have to cut players off these teams to make it to 5 foreigners. They won't keep more than a couple for European competitions only and not many would accept to sit out the whole seasons just to be eligible to play for 6+2+2+2+1 games at most in the Champions League. Only a few fringe players would accept something like that. Or someone like Winston Bogarde.
Where do the rest of the players go? Well, if they want to play at a more significant league, probably to these teams' rivals. But most will be too expensive for most teams so they will have to settle for a more economic solution. Or they would go home if there are clubs there that can afford them.
Now, I randomly 'remove' the redundant foreign players and will come up with the list of players that need to find a new club.
That would be:
Bale, Benzema, Varane and Kovacic from Real Madrid. Bale could move to the EPL with no issues. Benzema and Varane could move to France with no issues. Kovacic would have less options.
Fernandinho (if not a holder of a UK passport), Otamendi, David Silva, Bernardo Silva, Bravo, Mendy, Danilo, Gundogan, Kompany, Mangala, Toure (there's the question of Otamendi staying over one of the two forwards but whatever, this doesn't need to be practical) from Man City.
Benatia, Matuidi, Asamoah, Liechtsteiner, Howedes, Szesczny, Douglas Costa, Mandzukic, Cuadrado and Bentancur would have no room at Juventus.
That means that these players will need to find other clubs to play, either on permanent deals or on loan. That means that if they want to stay in Europe, they will have to settle for less money at lesser clubs.
Might sound unfair to the players, but it's more fair to the competition and more fair to every other club who will not see money as the only way to lose. Sure, they still will not be able to keep Bernardeschi from going to Juventus, but they will not have a trident of Dybala-Pjanic-Higuain backed by another trident of Cuadrado-Douglas Costa-Mandzukic.
And when you're already capped with foreigners, you will not be spending 50m on a rotation player like City did with both Danilo and Mendy. Getting rid of players will also come at a discount so transfer fees will eventually drop a bit but the big clubs will still be able to offer 100 and 200 million. The only problem is, with this system, they won't be able to do so as often as it will be a waste and a 100m player does not move when he knows that there's a chance he'll be limited to 15 games a year instead of 50+.
That also means that Bernardo Silva does not move to Man City but he might have moved elsewhere or stayed at Monaco.
The market is too fluid as things are and such a return to the past will limit it. That will eventually make the market more stable and less crazy.