Twitter has released the code that chooses which tweets are shown on your timeline to GitHub and has put up a blog post explaining the decision. It breaks down what the algorithm looks for when determining which Tweets to display in the timeline for you and how it ranks and filters them.
According to Twitter’s blog post, “the recommendation pipeline is made up of three main stages.” First, it collects the “best tweets from various recommendation sources”, then it ranks those tweets with a “machine learning model”.
Finally, it filters out Tweets from people you’ve blocked, Tweets you’ve already seen, or Tweets that aren’t safe for work before putting them on your timeline.
The post further explains each step of the process. For example, it notes that the first phase looks at about 1,500 Tweets and aims to make your For You timeline about 50 percent Tweets from people you follow (called “in-networks”). And called “in-network”.
out-of-network” accounts that you don’t follow. It also says that ranking is meant to “optimize for positive engagement (eg, likes, retweets and replies)” and that this last step will try to make sure you’re not seeing too many tweets from the same person.
Of course, the most detail will be available upon sifting through the code, which the researchers are already doing. CEO Elon Musk has been promising this move for some time — on March 24, 2022, before the site took ownership, he polled his followers about whether Twitter’s algorithm should be open source, and received about 83 percent of the responses. Said “yes”. In February, he promised it would happen within a week, before pushing the deadline to March 31 earlier this month.
Elon Musk tweeted that Friday’s release was “most of the recommendation algorithms” and said the rest would be released in the future. He also said that the hope is that “independent third parties should be able to determine with reasonable accuracy what users will likely be shown.” In a space discussing Algorithm’s release, he said the plan was to make it “the least gameable system on the Internet” and to make it as robust as Linux.
Perhaps the most famous and successful open-source project. “The overall goal is to maximize irrelevant user minutes,” he said. Elon Musk is preparing his audience to be disappointed in the algorithm when they see it (which of course is making a huge assumption that people will actually understand complex code). He added that it is “overly complex and not fully understood internally” and that people will “discover many silly things” but has promised to fix issues if they are discovered.
“Providing code transparency will be incredibly embarrassing at first, but this should lead to a rapid improvement in recommendation quality,” he tweeted. There is a difference between code transparency, where users will be able to see the mechanisms that choose tweets for their timeline, and code being open source, where the community can actually submit their code for consideration and algorithms in other projects.
While Musk has said it will be open source, Twitter really has to work if it wants to earn that label. This includes putting in place systems for governance that decide which requests to approve, attention to user-raised issues, and how to prevent bad actors from sabotaging the code for their own purposes.
Twitter says that people can submit pull requests that may eventually end up in its codebase. The company says that it is working on it. The readme for GitHub states, “We invite the community to submit GitHub issues and pull requests with suggestions on improving the recommendation algorithms.” However, it is said that Twitter is still in the process of building “tools to manage these suggestions and sync changes to our internal repository”.
There are many things that Musk’s Twitter has promised to do (like polling users before making major decisions) that it hasn’t stuck to, so the proof will be whether it actually adopts a community code.
The decision to increase transparency around its recommendations is not happening in a bubble. Musk has been openly critical of how Twitter’s previous management handled moderation and recommendation and launched a barrage of stories that he claimed would expose the platform’s “free speech suppression”.
@CrastNet