Thu 23rd March 2006, 2:52pm
At first glance, this seems to be a fairly scathing attack on Nature, which is odd considering their real target is Wikipedia.
Some of EB's claims here seem to be valid however (for example, minor inaccuracies were given the same weight as major ones, although some people would say an inaccuracy is an inaccuracy is an inaccuracy), and as such a number of Wikipedians (myself included, this has been discussed on Wikien-l) would welcome another study into the relative accuracy of WP and EB. The more information and analysis the better, because that provides more opportunity for WP to improve.
On the other hand, in many cases EB has simply made comments to the effect of "we don't agree that this is wrong", and have left it at that. WP corrected all the errors identified by the Nature reviewers: Wikipedia:External peer review/Nature December 2005/Errors
In more detail:
- EB asserts that some of the comparisons used text that did not appear in the main EB, rather in the Britannica yearbooks and the student version of EB. However they don't say which entries they are referring to.
- EB notes that they have no separate entry for "punctuated equilibrium", and that the EB text compared with WP's article was drawn from a longer entry on the theory of evolution. As such, they claim that it is unfair to criticise the EB entry for having critical omissions. But surely that's their problem for dealing with the topic too briefly?
- EB have quibbled with the qualifications of the reviewers Nature engaged, saying specifically that the reviewers "were scientists, not encyclopedists" - so who does EB get to peer review their articles? Surely for science articles, they must get scientists to do peer review, or do they leave it exclusively to "editors who have extensive experience in publishing an encyclopedia"?
Finally, in the appendices, EB addresses all of the errors and omissions identified by the Nature reviewers. They're quite long, so I'll let you read them yourself, but as a sample of the general character, EB's first three responses start with "We do not accept this criticism, which only reflects the point of view of the
reviewer", "We do not accept the validity of this review", and "We do not accept these criticisms." This is the most interesting part for me, when it is compared with the way WP editors responded to the review: Wikipedia:External peer review/Nature December 2005/Errors
In summary, another review would be great. As I said on the mailing list:
...we should invite EB to organise another study, to be conducted by a journal of their choice. Perhaps they can even contribute to the method (multiple reviewers for each comparison would be a good inclusion), on the condition that the results of the study are published at the same time as the list of errors. Then we can fix them within days, just like with the Nature review, and put that fact out in a press release.
For reference, here is the original list of errors and omissions identified by the Nature reviewers, have a look yourself: supplementary_information.pdf