For years, Twitter, like other social media platforms, has freely given its platform data to independent researchers so that they can analyze everything from online trolling to the spread of misinformation. Those studies have been critical to understanding the tactics used by scammers, foreign influence campaigns and other malicious actors trying to manipulate social media.
But now researchers fear the future of that work is under threat, after Twitter abruptly announced plans last week to restrict access to this data and charge money for it.
The plan provided few details before it was delayed Wednesday amid a public backlash, but the company nonetheless intends to move forward with it. Twitter now says it will end free access to its application programming interface, or API, the software enabling third parties to tap into Twitter’s systems, on Feb. 13, and plans to replace it with paid access starting at $100 a month for “basic access” granting a “low level of API usage.”
Such a change would chill civic research on Twitter manipulation and have devastating consequences for transparency, accountability and the public good, said Rebekah Tromble, director of George Washington University’s Institute for Data, Democracy and Politics.
“It’s just abundantly clear that the impact is going to be profound on the research community, profound on civil society organizations that rely on free access to the API in order to provide a vitally important public service,” Tromble told CNN. “Twitter essentially has gone, in a blink of an eye, from being an industry leader in transparency to the bottom of the barrel.”
The plan highlights the tension between new owner Elon Musk’s vows to bolster transparency on Twitter’s platform and the pressure he faces to shore up revenue through subscription products in order to improve the company’s bottom line, particularly in the face of an advertiser revolt.
Twitter’s plan to restrict access to data also comes as it boasted to European Union authorities, in a report made public Thursday, that it provides “industry-leading access to data.”
“Since 2006, academic researchers have used data from the public conversation to study topics as diverse as the conversation on Twitter itself,” the report said, “from state-backed efforts to disrupt the public conversation to floods and climate change, from attitudes and perceptions about COVID-19 to efforts to promote healthy conversation online.”
“Today, academic researchers are one of the largest groups of people using the Twitter API,” the report added. But Twitter’s changes erecting a paywall for API access may jeopardize all that, Tromble and other researchers said.
The company didn’t immediately respond to a request for comment.
Undermining the fight against bots
Musk has framed Twitter’s API restrictions as part of his wider effort to stamp out spammy, automated accounts. The data interface “is being abused badly right now by bot scammers and opinion manipulators,” he tweeted last week. “There’s no verification process or cost, so easy to spin up 100k bots to do bad things.”
But Musk’s proposed cure — putting a price on platform access — triggered a backlash from third-party software developers behind so-called “good” Twitter bots that also could be harmed by the change. Two days later, Musk seemed to soften his stance, but his concession only appeared to address the concerns of developers who use the API to write automated tweets, said Tromble, not those of researchers who use the API to collect and analyze large volumes of platform data.
Musk’s allegations about the prevalence of bots on Twitter were a centerpiece of his attempt last year to back out of buying Twitter, an ultimately unsuccessful effort. Ironically, however, Musk’s looming changes to the Twitter API might make it harder to study bot behavior on Twitter.
One of the third-party applications that depends on Twitter’s API is Botometer, a tool developed by Indiana University researchers and that was heavily cited by Musk’s team in court to defend his claims that Twitter was ridden with “bad” bots.
Kaicheng Yang, one of the creators of Botometer, told CNN Thursday that Twitter’s coming API changes “would definitely make my work harder” and raise the project’s costs. Yang said he would attempt to find funding to keep the project going, but that users could ultimately have to bear the costs of Botometer’s use of Twitter’s API, which he said could be “prohibitive” and drive people away from the tool.
“We also teach students in the classroom about how to use our tools to study social media and understand online manipulation. With the paid API, we won’t be able to do that anymore,” Yang said. He also added that even if Botometer can somehow continue, other researchers with smaller budgets, particularly in developing countries, likely won’t be able to afford Twitter’s API.
‘Not affordable’ to researchers
While it’s no surprise that Twitter might ask researchers to chip in for data access, the company has made little effort to understand researchers’ constraints and whether paying $100 a month is even feasible, said Tromble.
“There’s been no conversation with the affected community,” she said. “$100 a month is affordable to scholars at Ivy League institutions; it’s not affordable to under-resourced academic and civil-society researchers. If Twitter had that community in mind, it would be absolutely appropriate for them to reach out and have some conversations and perhaps introduce a sliding scale approach to this.”
Given the breadth of research worldwide based on Twitter data, dealing with topics ranging from elections to the Covid-19 pandemic, the impact on smaller research organizations could be substantial. Last week, more than 100 civil society groups and more than 500 individuals signed an open letter urging Twitter to keep its API easily accessible to researchers.
“Twitter’s new CEO Elon Musk has promised to make the platform more transparent and to reduce the prevalence of spam and manipulative accounts,” the letter said. “In fact, the independent research community has developed many of the most cutting-edge techniques used to manage bots. API access has provided a critical resource for that work. Twitter’s new barriers to data access will reduce the very transparency that both the platform and our societies desperately need.”
The impending API paywall also follows Twitter’s decision to eliminate a partnership with researchers known as the Twitter Moderation Research Consortium. The initiative was launched as recently as last year and had just gotten off the ground when Musk took over the company and axed the program, said Tromble, who described the consortium as an industry-leading push for transparency.
“The team is gone,” she said. “The researchers who were signed up to the program have heard nothing from the company in months, since Musk’s acquisition.”
Adding to Twitter’s scrutiny by regulators
Beyond the immediate impact to researchers, Twitter’s transparency issues could land the company in hot water with policymakers.
Last week, Massachusetts Democratic Rep. Lori Trahan called Twitter’s API plans “the latest in a series of bad moves from Twitter under Elon Musk’s leadership.”
“Twitter should be making it easier to study what’s happening on its platform, not harder,” Trahan said in a statement, citing a briefing her staff received from Twitter in December in which Twitter committed not to limit researcher access to the platform.
US policymakers have broadly called for expanding researcher access to social media data as a way of holding platforms accountable. Several other platforms that filed reports to the European Union — reports that were also made public Thursday alongside Twitter’s — indicated a commitment to growing that access, said Tromble.
“Meta was able to point to new programs they were developing,” Tromble said. “TikTok was able to point to specific, new APIs they’re developing. Twitter simply had blanks. There was nothing they could say, and in fact what they did say was, ‘We’re already better than everyone else,’ which was true until a week ago.”
The reports to the EU are seen as an important baseline marker for compliance with the Code of Practice on Disinformation, a voluntary set of self-regulatory commitments that can reduce a tech company’s potential liability under the Digital Services Act, a recently passed EU content moderation law that will be enforced beginning in 2024. Violations of the Digital Services Act can lead to fines of up to 6% of a platform’s global annual revenue.
“More work is needed when it comes to providing access to data for researchers,” said Věra Jourová, the European Union’s vice-president for values and transparency, in a statement. “We must have more transparency and cannot rely on the online platforms alone for the quality of information. They need to be independently verifiable. I am disappointed to see that Twitter’s report lags behind others and I expect a more serious commitment to their obligations stemming from the Code.”
