Why is marriage restricted to four wives?

Allah clearly permits marriage to multiple wives — insofar as they can be treated justly — in Qur'an 4:3, where He says "...*then marry those that please you of [other] women, two or three or four.*..." The conventional opinion of this verse, however, seems to be that it sets a hard limit of four wives; a man may not under any conditions take a fifth wife without first divorcing one of the four. Given the general principle of Islamic jurisprudence wherein anything that is not explicitly forbidden is permissible (even if not recommendable) by default, why then is marriage limited to four wives? The above verse clearly permits up to four, but does not expressly forbid more. It is also well-established through the hadith literature that the prophet himself had significantly more than four wives during his life, most of them simultaneously. Admittedly, just because the prophet was granted certain privileges does not mean they are applicable to Muslims on the whole, but this fact combined with the distinct *lack* of an explicit prohibition otherwise, suggests to me that that marrying beyond the four-wife cap would be permissible behavior. Has this opinion been argued by any reputable scholars in the past, and if so (or even if not) why did the apparent majority opinion of no-more-than-four win out?