Google doesn't like Wikipedia-watch.org, I'm getting my tin-foil hat out of the closet on this one Options

[Daniel Brandt]

Google doesn't like www.wikipedia-watch.org and I'm beginning to suspect a conspiracy. This site is almost nine months old, and should be out of the so-called Google "sandbox," which has been much-discussed on many Google forums for a couple of years.

Yes, I'm missing a few links to my site from Wikipedia itself, because they took them down when I redirected them to this site. But those redirects were in place for just two months, and that doesn't explain why Google has buried Wikipedia-Watch for almost nine months.

I'm not the only one who has noticed this. Wikipedia critic Matthew White also mentioned it in the last paragraph of his June 25 entry. That was entirely his own observation -- I've never communicated with Matthew.

In a two-word search for "wikipedia watch" without the quotes and no hyphen, my home page comes up with the following rank in these engines:

MSN -- number 1
Yahoo -- number 1
Ask.com -- number 1
Dogpile -- number 3
Clusty -- number 1

How does Google fare? The average rank across 25 of the most-used Google IP addresses, as reported on my special Scroogle tool, is about 45. That means page five if you are set to 10 results per page. And look at all the junk on those first four pages!

That's for the home page, which reports a PageRank of 5 out of 10. The deeper pages all show a PageRank of zero! They are indexed, but they almost never show up in searches unless your search terms are very specific.

What about backlinks? The www.wikipedia-watch.org home page has 499 backlinks, according to this tool, which counts external backlinks reported by Yahoo. (Google only shows a sampling of backlinks, and for years has been worthless for backlink analysis). The hivemind.html page, with a PageRank of zero, has 137 backlinks all by itself.

Yes, I'm aware that Google-lovers and Wikipedia-lovers will jump on me through various blogs, talk pages, and IRC channels, and start snickering about my tin-foil hat. All I'm saying is that Google has hand-tweaked my wikipedia-watch site so that it performs poorly in the rankings.

It happened to me with my "out-of-touch executives" Googlebomb two years ago. All of a sudden, a month after it was mentioned in the New York Times, it vanished from the number one spot in Google to somewhere between 400 and 800. It happened overnight, and there wasn't even a Google update in progress. (By the way, that was such a successful Googlebomb that it is still number one in MSN and Yahoo, even though my links were taken down two years ago. All those bloggers talking about it has kept it at number one even without my links.)

Why would Google take action against www.wikipedia-watch.org when they haven't taken action against my two anti-Google sites, www.google-watch.org and www.scroogle.org ?

Here's my theory: On my two anti-Google sites, there is almost no mention of Google AdWords and AdSense, except the occasional cartoon. I'm not an expert on ads, because I've never had an ad on any of my sites. I'm interested in Google's ad programs as part of the "big picture" of where the web is headed, but I'm not interested in them enough to experiment with ads myself, the way I experimented with my bio on Wikipedia.

Apparently Google doesn't feel threatened by those two sites. But they do feel threatened by the notion that Wikipedia could tighten up their operation and restrict bots from scraping their content.

Google makes tons of money from scraped spam sites that are generated automatically as "made for AdSense" pages. They love this spam, and couldn't survive without it. Over 95 percent of Google's total revenue comes from ads. If you don't think Google loves spam, check out their Domainpark ad program. This is custom-built for typosquatting spammers.

The scraping of Wikipedia has gone through the roof. Articles, talk pages, user pages, and user talk pages all get scraped. Try doing searches in any of the major engines on a unique username, and you will see a number of sites that specialize in scraping this user information. Some of the scrapers even scramble the words on the page, just so that they have some content to trigger some ad placement. This is all worthless spam, and almost all of it all carries AdSense.

Remember, my bio exists today because as soon as SlimVirgin and I agreed to delete it in October, a pro-Google blogger named Philipp Lenssen (Google loves this guy -- he even gets his blog in Google News!) complained to Jimmy. Shortly after that, Canderson7 reverted SlimVirgin's deletion of my bio. Then the fights started. The original bio concentrated on two or three substandard links from Google-lovers who hated me, which was originally the major problem I had with it. A couple of these Google-lovers were experts at gaming Google, and their hate pages ranked well, but the content on those pages was not encyclopedic, to say the least. (That's one of the problems with Wikipedians -- they think it must be true if it shows up on page one of a Google search.)

Is it possible that www.wikipedia-watch.org is a bigger threat to Google than my two anti-Google sites? It sure looks that way.

[Jonathan]

I think that when it comes to actual direct criticism, such as your Google Watch pages, Google do not appear to have the same policy as Wikipedia. Unlike Wikipedia, Google actually ARE a huge organisation that everyone has heard of. And everyone knows that there is criticism of Google, so if Google were to censor any sites critical of them, it would look exceptionally bad for them (in other words, Wikipedia can get away with it, Google can't). Thus, Google Watch has a high PageRank. It would pretty much be the same for MSN Criticism sites on MSN search, and likewise for Yahoo.

With Wikipedia Watch, well that's more dangerous for Google. We all know how heavily Google relies on Wikipedia with regards to searches, as the Wikipedia pages tend to be very much high up in terms of PageRank. Wikipedia Watch is dangerous because quite clearly some WikiPersonell feel threatened by it, and what is contained within WW could destroy the credibility of certain people within Wikipedia, and thus severely damage the credibility of Wikipedia itself. And Google quite clearly can't have that, so Wikipedia Watch is ranked low in PageRank.

There's also the fact that "Wikipedia Watch" was deleted and merged into the Brandt article as well, despite my protests that this was actually a notable site.

[Somey]

Before we just assume the Conspiracy Theory version is true, is it possible they're dinging you because of all the obscentities on the findchat.html page? Some of those are pretty over-the-top!

You might try hiding that page from their spiders, or turning the table into a bitmap and resubmitting, but it will probably take another two months for it to forget that there was blue language there. Then again, I'm still seeing the site on page 4 even after I change my filter setting to "None"...

But if that's what it is, then we'll all have to deal with the whole "supreme irony" thing for quite a while!

[Daniel Brandt]

Before we just assume the Conspiracy Theory version is true, is it possible they're dinging you because of all the obscentities on the findchat.html page? Some of those are pretty over-the-top!

No, the findchat.html page has only existed for one month, and the obscenities at the bottom of that page for only two weeks. The two hivemind pages aren't even indexed by Google right now, according to this search. But the first hivemind page, hivemind.html, has been there with the same essay on it (plenty of meat for a crawler), since December. I've been dinged since October.

Maybe Google cut a deal with Jimmy, and Jimmy hasn't told anyone that all of Wikipedia will be covered with AdSense any day now. Remember, Mozilla Foundation -- also a nonprofit like Wikimedia Foundation -- is getting over $50 million per year from their ad income from Google. It was only confirmed within the last six months, but was rumored for six months before that. They like to keep things like this quiet as long as they can.

[Daniel Brandt]

UPDATE:
I duplicated the first post in this thread over at the searchenginewatch.com forum three days ago. This is where GoogleGuy hangs out (he is Matt Cutts, a Google engineer and unofficial apologist for Google's massive, frequent, and often horrific screw-ups). Yesterday my rank for a search for "wikipedia watch" without the quotes on Google jumped dramatically from around number 45 to number 8. This happened across more than 50 Google IP addresses. They all instantly changed to exactly 8. (Of course, Google will never admit that anything at all happened.)

There was a malicious filter on my domain, and someone at the Googleplex decided that it was time to lift it because I was getting irritated.

Right now the only subpage at wikipedia-watch.org that shows up in Google is the article by Seigenthaler. Hivemind is nowhere. But I am now optimistic that in another 30 days or so, the other subpages will begin to rank in Google. I'm presuming that the penalty was lifted on the domain, which means the subpages will no longer inherit the penalty once another indexing cycle has time to work its way into Google's results. Hivemind does reasonably well on Yahoo and MSN, and it will on Google also once everything is normal.

I think someone should unplug the entire Internet and let us start all over again. This time, make sure that Sergey Brin, Larry Page, Jimmy Wales, and any admins from Wikipedia are not allowed to play, by order of the U.N. Security Council or whatever.

[Somey]

Matt Cutts! I know of him. Uncyclopedia has been practically banned from Google for months, and until very recently I was convinced it was because there are a few pages on the site that use black-on-black text to humorously indicate that something has been "censored." It's ridiculous that they couldn't make an exception for a site like that, but the fact is, the articles on Wikipedia and Google itself aren't exactly fawning.

In recent weeks there's been an effort to move the pages with color-on-same-color (we've taken to calling it "homotextual") content to a separate namespace that's listed in robots.txt as not to be indexed, but so far it's had precious little effect. There are still several pages that haven't been moved, but I'm beginning to think now that I was wrong in believing that this effect wasn't the result of direct action on Google's part.

This post has been edited by Somey: Sat 8th July 2006, 7:36pm

[Daniel Brandt]

It does indeed look like uncyclopedia.org is practically banned from Google. I cannot imagine that the homotextual links are the reason. The main page itself, as well as every subpage linked from that main page (dozens of links), all show a PageRank of zero. The site:uncyclopedia.org search shows lots of indexed pages, but with a PageRank of zero, almost nothing will show up in searches.

My guess would be that Jimmy and/or Matt, or someone else at the Googleplex, were concerned that uncyclopedia as a parody site would dilute Jimmy's "mission from God" of scraping all of the world's information making all of the world's information available to everybody. Indexing the content but ranking it at the bottom of the deepest ocean, is a convenient way to solve a pesky problem without raising too many eyebrows.

Anyone interested in Google should read this court case. Let's hope it makes it to the discovery stage so that Google is ordered by the court to cough up some information about top-secret ranking shenanigans at the Googleplex.

[Somey]

Sounds familiar, all righty!

Has there been any speculation in the KinderStart.com case as to why Google would have sandboxed them? There are lots of specialty search engines out there... Why go after them in particular? I'm just curious.

The problem for Uncyclopedia may actually be worse, because as along as the text-coloring "issues" exist on any one of its 18,500 pages, Google can claim that as the reason for the zero PageRanks - putting a huge burden on the site to find and "fix" each and every instance, after which there's no assurance whatsoever that the situation would improve, especially since anyone can edit Uncyclopedia, which means anyone can come along and add more pages that fit that description.

And like I say, I'm becoming more and more convinced that the situation wouldn't improve, with each passing day...

Presumably they wouldn't do this to Wikipedia, though it might be interesting to see what might happen if somebody tried it anyway!

[Somey]

Anyone interested in Google should read this court case. Let's hope it makes it to the discovery stage so that Google is ordered by the court to cough up some information about top-secret ranking shenanigans at the Googleplex.

Alas, it didn't make it to the discovery stage.

Oh well, at least they haven't banned this site yet... Probably just a matter of time, of course...

[Placeholder]

/

This post has been edited by Joey: Sun 15th October 2006, 10:21pm

[omobomo]

I think someone should unplug the entire Internet and let us start all over again. This time, make sure that Sergey Brin, Larry Page, Jimmy Wales, and any admins from Wikipedia are not allowed to play, by order of the U.N. Security Council or whatever.

Sig! I claim it!