You bet.
And so is every other theory that has any grounding in reality. There is just no escaping the fact that some humans are better than others in certain areas, and there are some areas where being better means being a more capable leader.
Now, the problem with most elitist views is the elite. If you went to high school in North America, you were probably either good-looking, rich, athletic or basically screwed. There was an elite, and you weren't about it forget it. But the system sucked because the qualities that these people had didn't make them good social role models.
That's problem number one. Problem number two is this: how do we decide who should be the elite? Anyone who says its an easy problem to solve isn't paying attention. In democratic societies, we nominally decide on the basis of popularity, and it isn't too bad of system. Still, it's come to mean Government by Marketing, and while it could be doing a lot worse, it cold be doing a lot better, too.
The Confucian criteria was two-fold: virtue and learning. You have to be informed enough to know what the problems are and what possible solutions exist, and you have to be virtuous enough to seek to implement the solutions without thought of personal gain.
Sure, it's simple on paper. It's a lot harder to decide who actually has these traits. Confucius has some guidelines, but he knew, too, that it was a tough path.
And it also seems unlikely that in the Rule by Advertising society that we live in, a learned, virtuous person will even come to our attention. Still, it's worth thinking about. Elections aren't the only place where personnel decisions are made. You make people choices all the time: who will be your lawyer, your employee, your car salesmen, etc. It might be possible to start picking your own elite right away.