The Surprising Outcome of Community Moderation in Reddit Politics Discussions (2024)

How can democratic societies host hard conversations about our conflicts in ways that keep important conversations going rather than shut them down? If you’ve been online at all this year, you’ve probably witnessed a heated conversation with deep disagreement, whether it’s about wars, injustice, or even just the question of who gets to speak.

While some disagreement may be productive, it’s not uncommon to see conversations slip into rule-violating behavior, such as flaming or personal attacks. On Reddit, volunteer moderators are tasked with reducing this kind of behavior. However, they have limited tools at their disposal, relying mostly on punitive measures, such as removing content and banning users. While these tools can help stop violations in their tracks, they may not help users learn how to participate productively in a community in the future.

• How can we improve users’ experience of bans and suspensions?
• What can online communities do to improve diverse dialog on contentious topics?

Few communities know this challenge better than r/politics, one of the world’s largest conversation spaces about American politics. With nearly 8.5 million subscribers, the subreddit is one of the most active and popular communities for political discussions. It has hosted conversation threads for major US events, such as Supreme Court decisions, crisis events, and political deaths, as well as AMAs (Ask Me Anything) with presidential candidates, senators, and entertainers with a variety of political views.

r/politics wants to be a place where political discourse happens; however, because it’s one of the largest spaces to discuss politics in America, it sees a lot of egregious behavior and bad faith contribution. But because the issues at hand are often contentious, the community also gets out-of-bounds contributions from people who were caught up in the moment or didn’t understand the norms. Mods deal with bad faith contributions by issuing bans that permanently prevent people from participating in the future. But to deal with lesser violations, the community uses temporary suspensions to inform people about how serious their non-normative behavior has been, and give them time to cool off.

However, open questions remain about how effective temporary suspensions are in mitigating future violations and helping people come back to be meaningful contributors to the community. We want to help them with this through field experiments.

r/politics, with 39 moderators and millions of comments per month, is one of the largest spaces anywhere for people to discuss US political news

Since 2022, we’ve been working with r/politics mods to better understand how to effectively leverage the tools at their disposal to help rule-violating community members participate successfully in their community in the future. As the first step toward designing new research together, the moderators securely shared their mod log which we have analyzed alongside public records from the subreddit between Jan 2020 and the end of December 2021. To protect people’s information, we analyzed the data on secure, access-controlled servers with encrypted data stores, as part of a study that received ethics review from Cornell’s review board.

Why r/politics is so important and conversations can be so fraught

It’s important to understand the scale of r/politics, which is one of the largest single spaces for conversations about US politics in the world—a community with 8.5 million subscribers —more than MSNBC, USA Today have on YouTube. In the U.S., 8% of Americans regularly get news from Reddit.

r/politics, with 39 moderators and millions of comments per month, is one of the largest spaces anywhere for people to discuss US political news, especially as news publishers continue to disable comments sections. In 2021, a non-election year in the US, over 200 thousand reddit users posted nearly 1.5 million comments per month on 20 thousand posts. Election years see even higher activity. In the 30 days prior to the 2020 election, r/politics received 3.6 million comments on 49 thousand posts by 374 thousand unique user accounts.

Because r/politics is one of the only places where the average person’s observations about politics could be seen by millions of people, it attracts a fierce competition over opinions and agendas. Consequently, many comments in r/politics get removed for violating community norms — for example rules encouraging civility, discouraging doxxing, and encouraging recent news only.

During non-election months, the community removed roughly 124 thousand comments per month (8.5%). In the lead-up to an election, that increased to 274 thousand comments (7.6%). It’s possible that the true content removal rates might be even higher, since Reddit’s own systems sometimes remove especially-awful comments before community moderators and community bots notice them.

How does that translate to the experience of commenters? Out of the 51 thousand people who participated in an average month roughly 24% experienced at least one kind of post or comment removal —even if just for submitting a duplicate link. Nearly 4 thousand accounts received a temporary or permanent ban on average, 1.9% of all posters and commenters per month. In the 30 days before an election, twice as many accounts receive a removal, but that’s largely because the community has 80% more commenters, a portion of which would be bots and spammers; according to moderators, removed content from human accounts experience is typically due to civility violations. Since these rates depend on moderator capacity, it’s possible that moderators might not notice all norm violations in these high volume periods.

Who Gets Temporary and Permanent Bans in r/politics and What Happens Next?

In an ideal world, people who do get suspended or banned are able return and participate substantively in a respectful and productive way. To inform future research that tests interventions to support this kind of successful community participation, we decided to analyze differences in behavior for accounts that temporary or permanent bans—and what happens after their accounts are restored.

To study this, we created a dataset that analyzed participation in the six months before and after an account received their first temporary suspension. We also created two other comparison groups: (a) accounts whose first ban was permanent and (b) accounts that were active on the same days as those that received temporary suspensions (Table 3).

what happens after a suspended account is restored? We found that 81% of people in r/politics do come back and participate.

Accounts that receive their first temporary suspension do have key differences from accounts that never received bans in our dataset. Temporarily suspended accounts comment much more and the median % of comments removed is 13 percentage points higher than non-banned accounts—a rate of removed comments that is over 5 times higher. Accounts that receive permanent bans behave in ways that moderators approve even less often, with a median of 40% of their comments being removed.

What happens after a temporarily suspended account is restored? We found that 81% of people do come back and participate after temporary suspensions. Those that return tend to follow the rules at higher rates than all suspended accounts in the period before suspension — though their median rate of comment removal is still 4 times higher than accounts with no bans (Table 3). Furthermore, a quarter of accounts that return do get another ban of some kind (20% of accounts with temporary suspensions), so there’s still huge room for improvement in how the community supports the experience of temporary suspensions.

Table 3: The vast majority of r/politics commenters do not receive bans. Among those with temporary suspensions, 81% participate in some way after their account is restored, and the median % of comments removed is 9 percentage points lower for restore accounts. Nevertheless, roughly a quarter of all accounts that receive a temporary suspension (20% / 81%) receive a second ban within six months after their account is restored.

Directions for Future Research with r/politics

Based on this research, we’re excited about the potential to develop research with r/politics on ways to improve people’s experience and re-integration to the community after experiencing a ban. We think there is potential for several kinds of studies and experiments, if the community and moderators consent:

• Surveys and interviews:
  • Asking people who have experienced suspensions to tell us more about their experience and how to improve it, building on prior research about the topic
• Norms interventions:
  • Messages that explain what a temporary suspension means
  • Messages with guides on how to contribute productively to the community
  • Messages from different political standpoints that describe the value of a functional shared conversation
  • Messages that help people productively manage their moral outrage, which is one possible driving force behind heated conversations that get out of hand
• (Your ideas here)

Supporting Other Subreddits to Study Temporary Suspensions with CAT Lab

If you are a moderator of a community that is interested in fielding experiments to test ideas to help people learn to contribute meaningfully to a community after a temporary suspension, please reach out to Sarah Gilbert either through Reddit (/u/SarahAGilbert) or email (sarah.gilbert@cornell.edu). We have rough draft software that could help generate a similar report for your community, as well as support future experimentation this year.

When you write to us, please check the following details. If you’re unsure, we can talk through it with you:

• Are you making at least dozens of temporary suspensions a month?
• Do you have at least a year of archived mod logs, for any period covered by the Pushshift dataset? (Reddit’s founding up to April 2023)
• If your mod log data is more recent, do you have an alternative source of log data that we could work with?