Is fact-checking enough?

Media Industry Discussion
1

Hey Olasians!!

How’s everyone doing? I was just checking out Olas’ latest post on X when, by chance, I came across an interesting article about the limits of fact-checking (call it serendipity, if you will). The article basically argues that while fact-checking is essential for quality control in media and reducing the spread of fake news, it doesn’t always get the job done.

This is especially true when it comes to scientific facts. Fact-checkers aren’t scientists, and sometimes they overstate the consensus on issues like climate science or present a narrow view of the debate, making it seem like the science is “settled” when it might not be. The article has a point. One example I always talk about is related to climate science. We often read these doomsday scenarios about climate change (especially if you follow The Guardian, lol), but a lot of those are based on the RCP 8.5 scenario from the IPCC, which is the most extreme. Other scenarios are much milder and more likely, considering how quickly we’re decarbonizing. If you check out Roger Pielke’s newsletter (really interesting guy, highly recommend), he breaks down this bias pretty well.

Back to the article—fact-checking is definitely a key tool for media literacy, keeping politicians in check and cleaning up the journalistic record. But it’s clearly limited when it comes to explaining complex issues where there isn’t settled consensus and multiple perspectives are in play. In these cases, instead of straight-up fact-checking, it might be better to present the full debate and why some people disagree with the “official” view. I wonder if Olas has any thoughts or solutions on this. It would be amazing to have a platform that not only fact-checks politicians and journalists but also highlights ongoing debates on controversial topics, instead of reducing everything to “true” or “false.” That’s what we info-seekers really want, right?

3 Likes
2

I totally get what you’re saying about fact-checkers not being the same as scientists. It’s a huge problem worldwide. I’d add that people being kinda lazy when it comes to really digging into what they’re reading is a major problem as well. Most folks don’t bother to look deeper into the sources or check out the author’s background, which is super important if you want to understand why a story might be slanted a certain way.

Now, I’d also add a follow-up question. How can we trust these fact-checking outfits when even the studies they’re citing might be biased because of who’s funding them? It’s tricky.

Here’s the thing: even when a study follows all the proper scientific methods, the funding source can still influence the results in sneaky ways. They might choose specific research questions, design the study in a certain way, interpret the data with a particular slant, or only report the results that fit their agenda.

Fact-checkers have their work cut out for them. They often lack expertise in super specialized fields, they’re usually working under tight deadlines, and it can be tough to get access to all the nitty-gritty details of a study. Plus, as you say in your post, there might be conflicts of interest within the scientific community itself, meaning that there is no consensus in what they are technically fact checking.

So, what can we do about it? Well, for starters, I think fact-checking organizations should be totally upfront about where their own funding comes from and how they do their work (yes, including the sources they have at hand. I’m sure not all fact checking companies have connections to the same academic cources or scientific labs doing research). It’d be great if they could consult with a bunch of different experts to help balance out any individual biases. They should also do some digging into who’s funding the studies they’re citing and make it part of the disclamer.

For me, the bottom line is: while fact-checkers are definitely important in fighting misinformation, we shouldn’t treat them like they’re infallible. It’s on readers to keep their critical thinking skills sharp, do your own research, and always approach information with a healthy dose of skepticism.

I’ll leave a question that pops up as I finish reviewing this answer

How much will Olas Protocol users be able to “see” the fact checking sources and the sources or their sources (study/funding) ?

2 Likes
3

Yeah, this is maybe a noob question but—who are they then? Is there a world-wide regulating body that names fact-checkers? How does one become a fact-checker anyways? Never thought about it before honestly.

Good one. There are layers upon layers of interpretations of the same thing. There is essentially no way to know the ultimate truth, but we can only strive for it.

1 Like
4

To close up on this - a perfect example is this case perfectly portrayed by Nic Carter (x.com) on this post.

image
image1290×1587 178 KB

You can clearly understand the tone and urgency that this had at the moment. This is coming from the supposed most powerful man on earth, with a supposed team of top scientists advising him on what is best for a country - and ultimately the world as everyone saw how US policies were echoed around the world during this period of COVID.

Additionally, the Dr. Fauci, a scientist of decades of experience and knowledge in the area of viruses, lied hundreds on time on interviews and declarations under oath. Again, all of this transmited by hundreds of media sites in all available channels.

What’s the manual say we do now? Furthermore what are we supposed to do if the source of research is known to be completely rotten?

E.g.

  1. ‘The situation has become appalling’: fake scientific papers push research credibility to crisis point | Peer review and scientific publishing | The Guardian
    "Last year the annual number of papers retracted by research journals topped 10,000 for the first time. Most analysts believe the figure is only the tip of an iceberg of scientific fraud.

“The situation has become appalling,” said Professor Dorothy Bishop of Oxford University. “The level of publishing of fraudulent papers is creating serious problems for science. In many fields it is becoming difficult to build up a cumulative approach to a subject, because we lack a solid foundation of trustworthy findings. And it’s getting worse and worse.”"

  1. https://www.cnn.com/2024/01/22/business/harvard-dana-farber-cancer-institute-data-manipulation-claims/index.html
    “Six manuscripts have retractions underway, 31 have been “identified as warranting corrections” and another one with a reported error “remains under examination,” according to Rollins.”
5

Hi @Biduido

So I think it’s important not to conflate poor quality fact checking with fact checking in general. A lot of fact checking we see today falls foul of the very same biases that are inherent to the writing in the media in the first place.

However fact checking done well is still extremely valuable. To use the example you gave, the reason why the RCP 8.5 debacle was exposed in climate science was because people like Roger Pielke exposed it. They exposed entire industries (academia and media) misrespresenting it on a large scale. They were so successful that in the last IPCC climate report, the issue was dealt with.

These fact checkers didn’t just present one side of a debate. They corrected wrongs and included important context to give people a broader picture of the science - of what was high confidence and what isn’t. That’s what good fact checkers do. The key is to design a system that produces high quality fact checking like this as opposed to the biased fact checking we see in media today. We think we’ve designed such a system.

2 Likes
6

Hi @ciaran

I think you are right. The issue, IMHO, also comes from the background of fact-checkers. I believe that journalists are, on average, terrible fact-checkers. They barely understand the nuances of many topics, and they are willing to jump to conclusions very fast. They have turned fact-checking into another entertainment trick, like in the presidential debates in the US. They fact-check candidates to create audience engagement and spice up the debates. That’s it. There is no real deep discussion about the topics, no context, no nuance. Done that way, fact-checking is garbage.

On the other side, the best fact-checkers are experts, who did the homework and are willing to explain that, just like Pielke did. Unfortunately, not many of those find the time or the incentives to do that. For academics, it’s mostly a waste of time as sharing their knowledge publicly is not incentivized, and it might even be a distraction from posting papers and attending conferences.

So, getting the right voices in the debate is a challenge, sometimes. I’m curious about your ideas on this. Are you implementing some incentives in Olas to get the right people to join as fact-checkers? Is that anywhere in your roadmap?

1 Like
7

@ciaran and @Biduido make some excellent points. The challenge really lies in who is doing the fact-checking and how it’s being done. Fact-checking isn’t inherently flawed, but the execution often is, especially when it’s reduced to quick soundbites or lacks the necessary context and expertise.

One of the biggest issues is incentivizing true experts to participate in the process. As @ciaran mentioned, fact-checking by seasoned experts like Pielke adds depth and balance, but it’s hard to get experts to invest their time. I think creating proper incentives could help bridge that gap by encouraging deeper, more accurate fact-checking, and ensuring transparency in sources and funding.

It’s not just about calling something ‘true’ or ‘false.’ We need a system that promotes contextual accuracy, acknowledges debates, and is transparent about both the sources and potential biases. Curious to see how Olas plans to address this!

1 Like
8

That’s a really insightful reflection on fact-checking and how it’s applied to complex issues like climate science. Fact-checking, while crucial in combatting misinformation, does indeed have its limits, especially when dealing with scientific debates where consensus is evolving or nuanced. It’s interesting that you mentioned Roger Pielke’s work—he’s one of the voices emphasizing the problems with relying too heavily on the most extreme scenarios, such as RCP 8.5 in climate projections.

To build on your points:

  1. Fact-Checking Limitations: Fact-checkers are primarily trained to verify claims based on existing evidence, but when it comes to scientific or complex policy issues, their understanding may be limited by the scope of their expertise. Scientific consensus can shift over time as more evidence comes to light. Presenting a dynamic and ongoing debate, as you suggested, is often more informative than simply labeling a claim as “true” or “false.” A report from the American Press Institute emphasizes that fact-checking should ideally involve experts in the field to provide context on controversial or complicated topics.
  2. Climate Science and RCP Scenarios: You’re spot on about the emphasis placed on extreme climate scenarios. The RCP 8.5 scenario has been widely criticized for being outdated and overly pessimistic given the current trajectory of decarbonization. In contrast, other scenarios, like RCP 4.5 or 2.6, represent more moderate outcomes. Pielke and other researchers argue that much of the media’s framing on climate is biased towards worst-case projections, leading to sensationalized reporting. Studies published in journals like Nature have called for a more balanced approach to climate modeling to avoid such distortions.
  3. Media Literacy and Complex Debates: You raise an excellent point about the need for platforms that highlight ongoing debates rather than just offering binary fact-checks. In fact, the Oxford Internet Institute and others have been exploring ways to enhance fact-checking tools by incorporating nuance and context, especially in controversial areas. This is important because, as noted by experts at the Poynter Institute, focusing solely on binary “true/false” outcomes can oversimplify public understanding, especially when scientific, legal, or political issues are more layered.

It would be fascinating if Olas could take these ideas forward, helping to facilitate discussions on complex issues by presenting multiple perspectives in a structured way, maybe even allowing for real-time contributions from experts or the community. That would be a great evolution of the traditional fact-checking model!

What do you think

2 Likes
9

Yes the plan is for journalists to post a bounty for fact checkers as part of their stake. At the moment we’re thinking 20% of the stake but this isn’t set in stone.

Fact checkers can then try and claim the bounty. However there is a reputational cost to raising issues that are later deemed invalid. If this happens enough times in a certain period, the user will be disbarred from fact checking for a period.

The competing claims between journalists and fact checkers are settled by a randomly chosen panel of judges.

1 Like
10

Yes agreed. We don’t plan to give a binary true or false to entire articles. Each article will get a score based on its content. If specific things that are said are shown to be inaccurate, the article will be marked down. Readers will be able to see the parts it was marked down on. It doesn’t mean there isn’t other valuable info in the article and the reader will be able to see that.

2 Likes