I completely agree with this. At the time I'm writing this, there are 1500 players on Skywars Chaos. I very rarely play chaos mode, but when I do, I've noticed that it usually sits around 60 players, and the games take quite some time to fill. I've had it where I was waiting in my cage in the pregame for nearly 10 minutes and nobody joined. Although it was at a low hour for the server, this seems ridiculous as each game only needs 4 players.
Making it free permanently would make it more enjoyable for players who DO enjoy playing the game, as the games would fill much faster and make it easier to grind for leaderboard. And as you said, it would likely draw more players into the server, as it is a fun mode, it just needs more people who enjoy playing it.
I know you want to reward VIP's, but I think in the future you could do that in better ways.
An idea I have for this would be VIP exclusive cosmetics. It would likely be a huge update to the entire server, but if they were to add some sort of server currency that isn't minecoins, like Hypixel has, it would be cool to see each VIP level (and non VIPs) get their own rewards.
As an example, non VIPs would get some default cosmetics, while VIP level 1s would get the same cosmetics, but with a few of their own, and have that pattern continue to VIP level 5. Again, this is just an example, but I personally think this would be cool.