How Wikipedia Fought COVID Misinformation

While social media sites have been criticized for helping spread COVID misinformation and conspiracy theories, Wikipedia strives to be accurate and to actively fight misinformation.

Dr James Heilman. Source: Victor Grigos

To learn more, Medscape interviewed James Heilman, MD, clinical assistant professor in the Department of Emergency Medicine at the University of British Columbia and a prominent Wikipedian.

This interview has been edited for length and clarity.

Was COVID information treated differently from other medical information on Wikipedia?

The quick answer is no. Topics are treated differently depending on how much interest they garner. We saw a lot of editors start to work on the COVID topic because it was getting such global attention.

Famously, anyone can edit a Wikipedia page. Is that true for medical information?

It is sort of true, but it’s more complicated than the simple phrase “anybody can edit” would imply. Anybody can edit parts of Wikipedia, at least initially. If you do good work, then your ability to edit will expand. However, if you do poor work, your ability to edit will be limited.

Who is allowed to edit medical information? Do you set standards in terms of qualifications, such as a science or medical background?

The key to editing Wikipedia is the sources you use. Your background in that way is much less important, because the sources really reign supreme.

We reference our articles more densely than one would see in a traditional journal or textbook. And the reason is because we have no way to verify that our editors are experts, and thus the material really needs to stand upon the sources entirely.

You provide guidance for editors on what you consider good sources. The randomized controlled trial is usually considered the be-all and end-all, but at Wikipedia you prefer secondary sources, right?

It depends on the entire body of literature. A randomized controlled trial isn’t necessarily the be-all and end-all, even within medicine. The BMJ, for their Christmas issue, published a now-famous randomized trial looking at parachutes and people jumping out of an airplane, which found that the parachute didn’t make any difference. The reason why was that the airplane was parked on the ground. It illustrates that if you have a randomized controlled trial that doesn’t ask the right question, you do not end up with a useful answer.

We saw that with the mask trial that came out of Denmark. The question they asked [whether a mask recommendation prevented SARS-CoV-2 infection] wasn’t the question that we were looking to answer. There’s a part of medicine that is science and there’s a part that is art. You often don’t have the perfect evidence to answer the question, and thus you need to use some common sense. Literature reviews’ secondary sources tackle those issues. They take all the evidence out there and synthesize it.

Another good example is what we see with SSRIs and depression. There’s been a great deal of publication bias in that area, where randomized controlled trials that are negative don’t get published and those that are positive do get published. Unless you take all the available randomized trials, published and unpublished, and synthesize them, you do not reach the best available position on the topic.

What were the particular challenges with COVID? Was it the sheer volume of publications?

With COVID, initially most of the publications were in preprint format. There was a great deal of exuberance, and some of that infiltrated Wikipedia. Some of our material briefly was overly positive about certain areas of COVID. Wikipedia as a whole has done better than many in remaining skeptical, with respect to the literature.

We never had a positive position on hydroxychloroquine that wasn’t rapidly reversed within a few seconds, because there’s never been any high-quality evidence for it in COVID. Yet, if you were to read the popular press, you could potentially have come to a different conclusion

Speaking of preprints, how do you handle them? Do you just consider them not good enough sources?

Yeah. They’re not good enough.

How do you handle the issue of data that are published but then retracted, or it’s questionable, like the Surgisphere debacle ? Do your processes put you in a good position to avoid these problems?

Yes. Our goal is not to necessarily include the most breaking information on every topic; we take a slower approach. We give the scientific process a little more time to work before we present those conclusions as solidly grounded.

And, of course, the Surgisphere stuff was published as a primary source, but once people started digging into it further, the issues came to light and it never made it into a high-quality secondary source.

Do you have statisticians on your team? Even if you read the literature, you might not understand what the authors did with the statistics.

We generally do not. Typically, we do not critique the methods of the literature reviews or the systematic reviews ourselves. We expect that high-quality publications, like The Lancet, The BMJ, or the Cochrane systematic reviews have experts that are doing the heavy lifting around the statistics and that their conclusions can be presented as found.

Of course, for active or controversial areas of medicine, different organizations might come to different conclusions; and as Wikipedia, we simply paraphrase the major positions on a topic.

For example, the Cochrane Collaboration finds that the evidence around breast cancer screening for women of any age is questionable, and the US Preventive Services Task Force recommends breast cancer screening between the ages of 50 and 75 every 2 years, and then the American College of Radiology recommends that women start at 40 and do it every year.

Those are three very different opinions. We as Wikipedia have no position on which of these is right, but all three are well-respected opinions on the topic in question, and we simply report that these are the opinions out there.

Did you take that same approach with the coronavirus — with masks, for example, when the early recommendations were all over the place?

Yes. We included the CDC position on masks when the CDC was less favorable toward them in the first few months of the pandemic. We included the World Health Organization’s position on masks as well. And as their positions became stronger, Wikipedia reflected that.

Of course, certain opinions have no weight at all — for example, the opinions of anti-masking groups like [the self-described] America’s Frontline doctors.

Positions that aren’t recognized by the medical establishment as being reliable don’t have any weight. We have an amazing page on all the falsehoods that have been spread around coronavirus.

How do you keep out, say, anti-vaxxers or people who might try to trick you by editing some pages, behaving well, and then going wild?

Those folks aren’t coming with high-quality sources. We have a continuous active review process for topics that are of great interest.

For example, take the article on the COVID-19 pandemic. If you click the link called “Page information,” you’ll see that there are 1775 people keeping an eye on it. As soon as anyone makes a change to that article, it will be rapidly reviewed by people who will dig into the sources for the changes. They’ll critique you, they’ll revert you, they’ll adjust the wording if they don’t think it reflects the source, or if it’s not supported by the source in question.

How many people worked on the COVID-19 articles?

The 1775 figure is the number who are watching it and who are notified of edits. The article has seen 21,000 edits since it began (about 1700 a month), and 3191 different editors worked on it.

That’s just in English.

How does this compare with other outbreaks, like Ebola or Zika?

Ebola was one of the first articles I worked on that received massive attention (other than the one on Rorschach inkblots).

During the Ebola outbreak, that content got nearly 100 million pageviews. Our COVID content is in 188 languages, it’s received 983,000 edits, is being worked on by 97,000 editors, and it’s received 579 million pageviews.

We have articles about coronavirus in basically every individual country, every state, every province, where we go into detail in each geographical area.

I assume it’s unprecedented in Wikipedia history as the most visited, most read.

It’s the most everything.

Why did Wikipedia start a collaboration with the World Health Organization in 2020?

Actually, we’ve worked with WHO since 2012. These less formal collaborations included working with the vaccine division and the International Classification of Diseases division, and we had a Wikipedian resident who worked at the WHO in Geneva.

The most recent collaboration is mainly around copyright. Organizations like the US government have been using open licenses for years, and that has made collaborating with institutions like the CDC and NIH really easy. However, most international organizations are only beginning to adopt it. With WHO taking a step in this direction, it will make it easier for us to provide a more global perspective on many health topics.

For example, WHO has graphs and maps showing the prevalence of diseases within various areas of the world. This will make it easier for us to reuse that material without having to re-create it ourselves to avoid copyright issues.

If there’s something that WHO recommends but other agencies don’t, will Wikipedia still give all the positions?

Yes, for sure. Our collaboration with WHO is not exclusive.

How did you get involved in Wikipedia?

I stumbled across it almost 14 years ago. I was working a night shift in the emergency department and looking around the internet, and I came across this website that wasn’t very good. Then I noticed that there was an edit button and I realized that I could fix the internet.

Is it true that you cut back on ER shifts to stay on top of the COVID page?

I cut back on my clinical work years ago to spend more time working on public-facing health information, including health information on Wikipedia.

With COVID, I’ve recently been pulled back to working full-time. In the past few months I’ve been less active on Wikipedia, partly because of that.

You’re not paid for your Wikipedia work.

Correct; I’ve always been a volunteer.

You think the medical community should get involved in Wikipedia. Why is that?

Wikipedia is a frequently used source of healthcare information for people around the world.

One of the most rewarding collaborations I’ve worked on was developing short summaries of diseases in English that were translated into the language Odia. There’s a retired orthopedic surgeon by the name of Subas Chandra Rout, who has spent the past 10 years creating medical content in his language, which is spoken by nearly 40 million individuals.

Before his work, a Google search for topics like malaria or dengue fever would return zero results in Odia. There is no machine translation for Odia. This single individual, through the Wikimedia movement, has created thousands of medical articles for speakers of this language. That’s one of the powers of this movement.

Another exciting effort is developing content for offline use. In much of the world, the internet exists intermittently or not at all. One of the solutions we’re working on is offline apps, such that you can download all of Wikipedia’s medical content, or slices of other content, to your phone. Even if the cell phone towers go down, you still have access to the content that you pre-downloaded.

The fully offline solution we’re working on is basically miniature computers we manufacture and mail around the world at cost. When you turn one on, it generates a Wi-Fi signal and everybody who logs onto that Wi-Fi signal can browse Wikipedia, because Wikipedia exists within the device itself. It’s like we’re mailing encyclopedias

Just like the old Encyclopedia Britannica door-to-door salesmen.

Exactly. There’s a huge amount of work yet to be done, so come join us. It’s a big movement; there’s room for everybody.

If our readers wanted to join, do they just click an edit button?

Yes. If you’re browsing Wikipedia and you see a spelling mistake, fix it. It’s no harder than using a word processor. Then you too can fix the internet.

Tricia Ward is an executive editor at Medscape who primarily covers cardiology and nephrology. She has made personal donations to the Wikipedia Foundation. Follow her on Twitter @_triciaward.

For more news, follow Medscape on Facebook, Twitter, Instagram, and YouTube.

Source: Read Full Article