Last week I learned about a fascinating study published by Nature in December 2005 that compared the Wikipedia to the Encyclopedia Brittanica, in order to gauge the relative accuracy of the two sources. For a sample of 42 science topics, the study found that the number of serious errors (i.e., misinterpretation of critical concepts) were identical in both sources. Wikipedia averaged four less-fatal factual errors per article, while the Britannica had three factual errors on average. Go here to read the full report.
Nature did not come up with this idea; Ed Felten did a brief "Wikipedia Quality Check"in September 2004. I hope that such investigations are extended, with much larger samples, in all major scholarly disciplines. It is important to understand if the accuracy of the Wikipedia is always comparable to traditional sources, or if it varies depending on the discipline.
One amusing quote from the Nature report: "Editors at Britannica would not discuss the findings, but say their own studies of Wikipedia have uncovered numerous flaws." Hmm--I wish I had been a fly on the wall as those studies were designed.
Wikipedia represents grass-roots encyclopedia development; anyone can log on and fire away about any topic they wish. Britannica maintains the established approach of developing reference sources, namely contracting with experts to write about various topics. Nature reports that Wikipedia's founders have moved past their impassioned claims about Wikipedia's inherent superiority to a more mature understanding that it also has flaws. They have taken steps to make sure that controversial articles cannot be edited endlessly, which should prevent political foes from sitting at their keyboards and erasing each other work. They are also--horror of horrors!--seeking out experts for some topics. Less than 10% of the scientists surveyed by Nature write Wikipedia articles, even though over 70% of them have heard of it.
Wikipedia has moved through an initial "messianic" phase into a more realistic "coming down to Earth" phase. Their model is both promising and beset with potential problems. Although Wikipedia is maturing, Britannica's pseudo-studies show that the old guard has not yet appreciated that a new era in encyclopedia development has arrived.
Brittanica has just realised a very forthright rejection of the analysis, which, if confirmed suggests it is Nature which should be examining its own procedures and output. he eagerness to embrace 'citizen media' seems to make us dangerously tolerant of its basic weaknesses.
Posted by: Gus swan | March 23, 2006 at 03:58 AM