Hi all.

I just got an e-mail from Andrew Orlowski asking me if I have any material to add for a report he wishes to publish in relation to a Brittanica response to the Nature article about Wikipedia. Rather than just think something up by myself, I thought that I would give this to the forum to see what people think.

Anyway, here is the e-mail that I received:



Obviously, it'd be easier if Andrew would register an account and post here himself, but I am happy to be a go-between in this instance.

Does anyone have anything to say on this matter?

wikipedia fan boys have something to say

http://en.wikipedia.org/wiki/User_talk:Jimbo_Wales#Nature

Well aren't they stupid.

http://www.theregister.co.uk/2006/03/23/br...a_nature_study/

This guy is really obsessed!

Darn it. We must have been too slow providing backup info. There I was thinking we had like a week or so.

In summary, he is saying that Nature were paid by Wikipedia to provide good press for them. Or if not paid, then encouraged somehow...

At first glance, this seems to be a fairly scathing attack on Nature, which is odd considering their real target is Wikipedia.

Some of EB's claims here seem to be valid however (for example, minor inaccuracies were given the same weight as major ones, although some people would say an inaccuracy is an inaccuracy is an inaccuracy), and as such a number of Wikipedians (myself included, this has been discussed on Wikien-l) would welcome another study into the relative accuracy of WP and EB. The more information and analysis the better, because that provides more opportunity for WP to improve.

On the other hand, in many cases EB has simply made comments to the effect of "we don't agree that this is wrong", and have left it at that. WP corrected all the errors identified by the Nature reviewers: Wikipedia:External peer review/Nature December 2005/Errors.

In more detail:

• EB asserts that some of the comparisons used text that did not appear in the main EB, rather in the Britannica yearbooks and the student version of EB. However they don't say which entries they are referring to.
• EB notes that they have no separate entry for "punctuated equilibrium", and that the EB text compared with WP's article was drawn from a longer entry on the theory of evolution. As such, they claim that it is unfair to criticise the EB entry for having critical omissions. But surely that's their problem for dealing with the topic too briefly?
• EB have quibbled with the qualifications of the reviewers Nature engaged, saying specifically that the reviewers "were scientists, not encyclopedists" - so who does EB get to peer review their articles? Surely for science articles, they must get scientists to do peer review, or do they leave it exclusively to "editors who have extensive experience in publishing an encyclopedia"?

Finally, in the appendices, EB addresses all of the errors and omissions identified by the Nature reviewers. They're quite long, so I'll let you read them yourself, but as a sample of the general character, EB's first three responses start with "We do not accept this criticism, which only reflects the point of view of the
reviewer", "We do not accept the validity of this review", and "We do not accept these criticisms." This is the most interesting part for me, when it is compared with the way WP editors responded to the review: Wikipedia:External peer review/Nature December 2005/Errors.

In summary, another review would be great. As I said on the mailing list:


For reference, here is the original list of errors and omissions identified by the Nature reviewers, have a look yourself: supplementary_information.pdf

From the article:

I don't think Orlowski remotely understands the notion of futures markets. The "terror market" and Wikipedia have zero in common.


I don't think he thinks they're that respected if they're this easily swayed, so this sentence comes across as a sarcastic jab.


That's your problem, EB. Make an article!


Now that is damning, and Nature should answer this accusation. And that's not the only damning thing in all this. The PDF is quite good, Britannica seems to be examing each error individually and setting the record straight. I'd love to see Nature's response to this. And there could be some useful information pulled from this PDF to improve Wikipedia.

what if Nature was wrong and you "fixed" wikipedia articles with false information provided by nature?


for example



Britannica has the Library of Congress backing its claim, what does Nature have? According to Britannica, Nature was not even willing to disclose the raw data they used to perform the study.

Well, its not totally unbelievable for Wikipedia to have paid Nature a sum of money to make them look good.

I think that this article is simply one of a series, and he was asking for feedback on a future article which he intends to write. Of course, I could be absolutely wrong, but that's what it looks like to me.

I think Andrew Orlowski's theory is more likely



Jim Giles is probably just another Wikipedia fan boy.

Ok.


Let's talk to EB about sponsoring another review then, or really any other journal, institute, publisher or any other body that wants to do a review. It's all better for WP, we can just use the results to improve the articles.

Very good point, Hushthis, and it is probably what Andrew Orlowski is saying. However, in reality, I think its equally likely that they were actually paid hard cash to do it. Wikipedia certainly needed it, and were on the way down beforehand. Of course, whether Wikipedia would have gone way down we don't know. But this certainly helped them out. Note that of the what 10 different studies of Wikipedia articles and their quality, this was the first one that put Wikipedia in any kind of a good light. Interesting that this would happen amidst Seigenthaler scandals and the like.

Of course, it might just be that Nature happened to do their survey and not check things out well, or that they happened to select an area which Wikipedia was good at, or that Wikipedia tried to portray what was overall pretty bad for them as if it was a lot better than it was.

Anyway, I think that if you automatically assume based on this tiny thing that Wikipedia is better than Brittanica (when the survey clearly stated the opposite) then you are naive.

WP:EPR lists the external peer reviews that have been conducted on WP. Note that the Nature review is by far the most comprehensive of all conducted so far, and there is only one other review that was conducted in a formal process (the other evaluates the Featured Article process and concludes that it "is not ideal, but it does seem relatively rigorous").

Most of the other ones (the informal ones) were conducted by newspapers. You can see the results and responses for yourself with the link I gave above. They're all pretty interesting, if people haven't seen them already.

Yes, well, I wonder if those that gave Wikipedia very low marks (e.g. "0" on articles) would have changed their mind had they given a more "exhaustive" review. All of them bar Nature put Wikipedia in a very, very bad light.

Yet Wikipedia now says that Nature is the only one that matters. Interesting that.