[wordup] Nature Magazine Compares: Wikipedia vs. Britannica

[Adam Shand]

Considering how long Wikipedia has been around I'd consider this a  
ringing endorsement for it.   Way to go guys!

Adam.

From: http://www.nature.com/nature/journal/v438/n7070/full/438900a.html

Nature 438, 900-901 (15 December 2005) | doi:10.1038/438900a

Special Report
Internet encyclopaedias go head to head

Jim Giles

Jimmy Wales' Wikipedia comes close to Britannica in terms of the  
accuracy of its science entries, a Nature investigation finds.  
UPDATE: see details of how the data were collected for this article  
in the supplementary information.

One of the extraordinary stories of the Internet age is that of  
Wikipedia, a free online encyclopaedia that anyone can edit. This  
radical and rapidly growing publication, which includes close to 4  
million entries, is now a much-used resource. But it is also  
controversial: if anyone can edit entries, how do users know if  
Wikipedia is as accurate as established sources such as Encyclopaedia  
Britannica?

Several recent cases have highlighted the potential problems. One  
article was revealed as falsely suggesting that a former assistant to  
US Senator Robert Kennedy may have been involved in his  
assassination. And podcasting pioneer Adam Curry has been accused of  
editing the entry on podcasting to remove references to competitors'  
work. Curry says he merely thought he was making the entry more  
accurate.

However, an expert-led investigation carried out by Nature — the  
first to use peer review to compare Wikipedia and Britannica's  
coverage of science — suggests that such high-profile examples are  
the exception rather than the rule.

The exercise revealed numerous errors in both encyclopaedias, but  
among 42 entries tested, the difference in accuracy was not  
particularly great: the average science entry in Wikipedia contained  
around four inaccuracies; Britannica, about three.

Considering how Wikipedia articles are written, that result might  
seem surprising. A solar physicist could, for example, work on the  
entry on the Sun, but would have the same status as a contributor  
without an academic background. Disputes about content are usually  
resolved by discussion among users.

But Jimmy Wales, co-founder of Wikipedia and president of the  
encyclopaedia's parent organization, the Wikimedia Foundation of St  
Petersburg, Florida, says the finding shows the potential of  
Wikipedia. "I'm pleased," he says. "Our goal is to get to Britannica  
quality, or better."

Wikipedia is growing fast. The encyclopaedia has added 3.7 million  
articles in 200 languages since it was founded in 2001. The English  
version has more than 45,000 registered users, and added about 1,500  
new articles every day of October 2005. Wikipedia has become the 37th  
most visited website, according to Alexa, a web ranking service.

But critics have raised concerns about the site's increasing  
influence, questioning whether multiple, unpaid editors can match  
paid professionals for accuracy. Writing in the online magazine TCS  
last year, former Britannica editor Robert McHenry declared one  
Wikipedia entry — on US founding father Alexander Hamilton — as "what  
might be expected of a high-school student". Opening up the editing  
process to all, regardless of expertise, means that reliability can  
never be ensured, he concluded.

Yet Nature's investigation suggests that Britannica's advantage may  
not be great, at least when it comes to science entries. In the  
study, entries were chosen from the websites of Wikipedia and  
Encyclopaedia Britannica on a broad range of scientific disciplines  
and sent to a relevant expert for peer review. Each reviewer examined  
the entry on a single subject from the two encyclopaedias; they were  
not told which article came from which encyclopaedia. A total of 42  
usable reviews were returned out of 50 sent out, and were then  
examined by Nature's news team.

Only eight serious errors, such as misinterpretations of important  
concepts, were detected in the pairs of articles reviewed, four from  
each encyclopaedia. But reviewers also found many factual errors,  
omissions or misleading statements: 162 and 123 in Wikipedia and  
Britannica, respectively.

Editors at Britannica would not discuss the findings, but say their  
own studies of Wikipedia have uncovered numerous flaws. "We have  
nothing against Wikipedia," says Tom Panelas, director of corporate  
communications at the company's headquarters in Chicago. "But it is  
not the case that errors creep in on an occasional basis or that a  
couple of articles are poorly written. There are lots of articles in  
that condition. They need a good editor."

Several Nature reviewers agreed with Panelas' point on readability,  
commenting that the Wikipedia article they reviewed was poorly  
structured and confusing. This criticism is common among information  
scientists, who also point to other problems with article quality,  
such as undue prominence given to controversial scientific theories.  
But Michael Twidale, an information scientist at the University of  
Illinois at Urbana-Champaign, says that Wikipedia's strongest suit is  
the speed at which it can updated, a factor not considered by  
Nature's reviewers.

"People will find it shocking to see how many errors there are in  
Britannica," Twidale adds. "Print encyclopaedias are often set up as  
the gold standards of information quality against which the failings  
of faster or cheaper resources can be compared. These findings remind  
us that we have an 18-carat standard, not a 24-carat one."

The most error-strewn article, that on Dmitry Mendeleev, co-creator  
of the periodic table, illustrates this. Michael Gordin, a science  
historian at Princeton University who wrote a 2004 book on Mendeleev,  
identified 19 errors in Wikipedia and 8 in Britannica. These range  
from minor mistakes, such as describing Mendeleev as the 14th child  
in his family when he was the 13th, to more significant inaccuracies.  
Wikipedia, for example, incorrectly describes how Mendeleev's work  
relates to that of British chemist John Dalton. "Who wrote this  
stuff?" asked another reviewer. "Do they bother to check with experts?"

But to improve Wikipedia, Wales is not so much interested in checking  
articles with experts as getting them to write the articles in the  
first place.

As well as comparing the two encyclopaedias, Nature surveyed more  
than 1,000 Nature authors and found that although more than 70% had  
heard of Wikipedia and 17% of those consulted it on a weekly basis,  
less than 10% help to update it. The steady trickle of scientists who  
have contributed to articles describe the experience as rewarding, if  
occasionally frustrating (see 'Challenges of being a Wikipedian').

Greater involvement by scientists would lead to a "multiplier  
effect", says Wales. Most entries are edited by enthusiasts, and the  
addition of a researcher can boost article quality hugely. "Experts  
can help write specifics in a nuanced way," he says.

Wales also plans to introduce a 'stable' version of each entry. Once  
an article reaches a specific quality threshold it will be tagged as  
stable. Further edits will be made to a separate 'live' version that  
would replace the stable version when deemed to be a significant  
improvement. One method for determining that threshold, where users  
rate article quality, will be trialled early next year.

Additional research by Declan Butler, Jenny Hogan, Michael Hopkin,  
Mark Peplow and Tom Simonite.