The problem of Ilona Maska’s Twitter bot is fake news

Perhaps more worrying is the fact that the results of an investigation published in Science in 2018, which looked at about 126,000 stories – true and false – shared on Twitter from 2006 to 2017, show that people are more likely to spread false news than robots.

Darius Kazemi, a programmer who has been creating and studying bots for ten years, describes the problems with identifying bots as twofold: first, bots are measured through internal metrics that are usually inaccessible to the public; second, machine learning algorithms designed to identify bots are informed through challenges of human judgment that are often erroneous.

This may be at least better than what Mr. Musk suggested: last week he said his team would try to determine a more accurate percentage of Twitter bots by making a “random sample of 100 @twitter subscribers”. He invited his followers to also take part in the exercise. Mr. Musk recently estimated that fake users make up at least 20% of all Twitter accounts. In securities statements, Twitter has long estimated that fake or spam accounts make up less than 5% of the total number of active users, but also said the actual number “may be higher than we think”.

When Twitter CEO Parag Agraval tried to demonstrate problems with both internal and external identification of bots on Twitter, Mr. Musk cut short the discussion with emoticons. It is not even clear that the platform without boots will not suffer in any sense. Mr Musk has compared Twitter bots to termites in the house, but they can also be useful: for example, a newspaper bot that automatically publishes headlines every day is an obvious government service.

According to Mr. Kazemi’s research, people mistakenly identify bots for a variety of reasons. Active stalkers who spend a lot of time on Twitter browsing the news, but who rarely tweet, are often called bots, as are the accounts of non-English-speaking people who mistranslate a few words. Especially avid fans, who often and quickly publish messages, can look like bots. And there will always be accounts owned by real people so that they can anonymously engage in unpleasant behavior, such as harassment. As a self-proclaimed freedom of speech absolutist, Mr Musk says he wants more freedom on Twitter; in this context he seems to be fighting for less.

Mr. Musk wonders if advertisers can know what they’re getting for their money without realizing how many accounts are actually human. This is a legitimate concern, but certainly not specific to Twitter. Mr Kazemi described Twitter as one of the most open platforms for data, making it possible for extensive academic study compared to other platforms.

“Twitter is in bad shape,” Mr Kazemi said. “It puts it in the top 10% of all platforms.”

Twitter has historically not been afraid to look bad to get better. Back in 2019, Twitter shared the tide after the company said it needed additional investment to clear its user base. He also said he is moving from distributing monthly users to daily active users who can be monetized, or to those who he believes can really view ads.

Instead of trying to compete with people like Facebook and Instagram, in huge numbers, Twitter, if anything, presented a low rating to real users in an attempt to more accurately show advertisers its value. Last month, the company publicly reviewed several-quarters of the user data below, acknowledging that in some cases it failed to link multiple individual accounts.

Meta Platforms said in its annual statement that in the fourth quarter, duplicate accounts for its Facebook app could represent about 11% of global monthly active users, and fake accounts potentially accounted for about 5% of the global monthly user base. He called duplicate and false accounts “very difficult to measure” on their scale, noting that “it is possible that the actual figures … may differ significantly from our estimates, perhaps beyond our estimates.”

Conclusions: No basic numbers of platform users are useful for anything other than arbitrary benchmarks used to compare themselves.

Last week on Twitter, Mr. Kazemi revealed that Mask’s own Twitter has been labeled Botometer as a potential bot, assigning him a bot probability of 3.5 out of a possible 5.

Mr Kazemi said his work on online platforms trying to pinpoint the exact number of users was so futile that he quit his career. At least in his case it was a legitimate reason to walk.

Write Laura Foreman at laura.forman@wsj.com