John Battelle recently posted a report which states that Wikipedia pages are taking over the organic search results of the major search engines: "Today 27% of Google’s results on the first link alone come from Wikipedia, as do 31% of Yahoo’s."
What magic power causes a free site like Wikipedia to own between a quarter and a third of results for the single most powerful page position at the world's two largest search engines?
It's simple. Money.
Google and Yahoo make more money by putting Wikipedia and sites like it first in their results.
Let's look at a scenario that Google hates. Christmas is coming, and a searcher puts a broad but valuable query term like HDTV into Google. The results that come back are clearly delineated between organic content and paid ads:
But here's what drives Google (and Yahoo) absolutely crazy. There in the third link position is CNET, with a feature called HDTV World. CNET is a media company which creates great, rich content about tech topics like HTDV. Google can hardly ignore them. But boy, do they wish they could! Because in two short clicks, a user goes from the organic results on Google to a highly monetized buying guide and store for Samsung HDTVs on CNET -- and Google makes exactly $0.
It's enough to cause heartburn at the 'plex.
Now let's walk through that scenario with a similar search for hearing aids. In passing, it's interesting to note that Google has no top-line ads for 'HDTV' (which I presume young, savvy people search for) and it displays three top-line ads for 'hearing aid' - which I presume older, less savvy searchers tend to type in. Nothing to see here, they're still not evil, just as long as they keep it below 49%.
Ahem. Back to our topic at hand, Wikipedia love.
Here's the paid/unpaid split for hearing aids:
We can see that like the HDTV search, Wikipedia is the first organic result, which means that it will collect the bulk of the clicks for unpaid traffic on the page. This delights Google, because there are no ads on Wikipedia. By displaying Wikipedia links in the most prominent free position, not only can they deliver a useful result to a searcher, they can't get into the CNET/HDTV situation which they hate - letting a searcher with strong commercial intent escape without the lucky advertiser paying the Google tax.
There's a further benefit to Google in promoting Wikipedia in organic results. Even within the context of a buying decision, searchers strongly type for commercial and non-commercial intent of a particular query. When a searcher is in the exploration and research phase of a hearing-aid buying decision, they want information, overview, context. When they have found out what they want, they switch to a transactional mode, and are ready to be marketed to. Over time, Google can use free, non-commercial content like Wikipedia to actively train searchers that they won't find commercial offers in the organic results:
...and over time, encourage them to look first in the paid ad positions when they want to buy something. Commercial nirvana.
That commercial nirvana is why Google and Yahoo love Wikipedia, and you'll continue to see more Wikipedia links in high organic positions over time.
Unless Jimmy Wales allows ads.
Update: I've just noticed Matt Cutts' new post on Google's efforts to nuke people who sell links with PageRank. Matt asks, rather piously, "Now, think about how you would feel if your medical search was influenced by pages like this," and points to a paid content page about the Gamma Knife, which is apparently a cancer treatment.
Hmm, Matt, I'd feel about the same as I do when Google influences my search by prominently showing a Gamma Knife ad on the first SERP:
Let's be clear - piety aside, Google isn't worried about your finding out about the Gamma Knife - they just want someone to pay them. That's why they are happy to prominently feature it in their ads at the same time that Matt is finger-wagging about it in his blog.
This battle is all about searcher differentiation, and paying the Google Tax on the way to commercial nirvana.
Note: For more on query-type differentiation, see my previous post, "Why Google owns your lunch."