Twitter open-sourced recommendation algorithm – Software Craftsmanship Weekly vol. 127

1. Twitter on the path to transparency: recommendation algorithm becomes open-source, and source code leaks.

Elon Musk is certainly a man of his word.

In a move that causes nervous giggles in some and head scratching in others, however, Twitter has decided to release its algorithm recommendation and its ML portion as open-source. The whole thing, incidentally, was also described on the Twitter blog. Elon Musk, the company’s CEO, when taking over the organization made several promises about how transparent the platform would become under his reign, and now we have the opportunity to observe one step of the process. Admittedly, he prepared the public in advance that they might be disappointed because the code, as he claimed, is “too complex and not fully understood even internally”, and people will “discover a lot of stupid things.” Nonetheless, Musk believes that simply revealing the rules of the game publicly will lead to rapid improvements in the quality of recommendations.

Open-Source indeed has many faces, as we have seen in these reviews many times before. There is a fundamental difference between code that anyone can read and open-source code like Linux. The current movement is just the first step – if Twitter wants to prove that it is genuinely open-source, it must start accepting changes from the community (and implement a clear process for its governance). The company claims to be working on this, but only time will tell if it’s not just a smoke screen.

Twitter’s decision on transparency is not taking place in a vacuum. Other open-source projects, such as Mastodon and the Jack Dorsey-backed Bluesky, are challenging the social media giant. And while the initial frenzy over Mastodon seems to be over, and Twitter has defended itself, Musk’s image as a visionary has suffered heavily from the whole acquisition. The public release of the algorithm – one of the pillars of his promises when he took over the company – is expected to allow him to gain some extra points.

While Twitter is just beginning its open-source journey, someone has decided to speed it up a bit. That’s because parts of Twitter’s (proprietary) source code were recently leaked on GitHub, as reported by the New York Times. The leak was quickly cleaned up, but could potentially expose vulnerabilities in the company’s software and give competitors an unfair advantage. It is suspected that a former employee may be behind the leak, which is not surprising given the massive layoffs that followed Elon Musk’s takeover of the company.

To track down the responsible party, Twitter has requested that GitHub disclose user data for those who may have both uploaded and… downloaded the data. GitHub has not yet responded to questions about whether it will comply with Twitter’s request.

Ironically, the news of the source code leak came just a few days before Twitter announced the recommendation algorithm being made open-source. However, it is worth noting that opening up the recommendation algorithm will probably reveal much less of the company’s proprietary code than the leak from GitHub did (I didn’t see the earlier leak and I’m not in a hurry to let Elon Musk know about Artur Skowroński’s existence).

Sources

• twitter/the-algorithm
• twitter/the-algorithm-ml
• Twitter reveals some of its source code, including its recommendation algorithm
• Twitter Says Parts of Its Source Code Were Leaked Online

Discover more IT content selected for you

In Vived, you will find articles handpicked by devs. Download the app and read the good stuff!

2 Alibaba is restructuring: Is it the influence of the Chinese government or an adjustment to international scrutiny?

It’s been a long time since we’ve had the Middle Kingdom with us, hasn’t it? Well, buckle up now, because just like in the good old days, we’re about to go on a wild ride through the technological and political landscape. Indeed, Alibaba Group, China’s e-commerce juggernaut, is splitting into six units and exploring fundraising or IPO opportunities for most of them. Is this a simple corporate decision, or is it a case of “Made in China” government influence?

With Alibaba’s founder Jack Ma returning from his one-year absence abroad, the timing of this restructuring seems almost perfect. Is it a coincidence or a strategic move? It appears that the Chinese government is somewhat easing its stance towards private enterprises, but at the same time, its influence on the decision to restructure Alibaba is certainly significant. It breaks up the company’s monopolistic capabilities to influence the financial market, weakening its political environment.

The new lineup of groups includes the Cloud Intelligence Group, Taobao Tmall Commerce Group, Local Services Group, Cainiao Smart Logistics Group, Global Digital Commerce Group, and Digital Media and Entertainment Group. Each of them will have their CEO, management team, and the ability to raise capital and pursue an IPO – except Taobao Tmall Commerce Group, which will remain under the Alibaba umbrella.

So, is this restructuring really the result of behind-the-scenes maneuvering by the Chinese government? Or is it Alibaba’s way of adapting to international scrutiny, especially from the US, which has expressed concerns about the national security implications of Chinese tech firms? After all, the US is accelerating efforts to either take over or ban the operations of TikTok, which was initiated during the Donald Trump administration.

Last week, there was five hours of testimony before Congress by TikTok CEO, Shou Zi Chew, who tried to assuage concerns about the platform’s Chinese parent company, ByteDance, and its ties to the Chinese Communist Party. Chew insisted that TikTok has been building a “firewall” to protect U.S. user data from unauthorized access from abroad for more than two years, stressing that U.S. data is stored on U.S. servers and supervised by U.S. personnel. But lawmakers remained unconvinced, finding Chew’s answers evasive and expressing concerns about the app’s impact on American children, further fueling calls for a nationwide ban on the platform. A nuanced article on the topic was published by the invaluable The New Yorker.

Regardless of the motives, Alibaba’s strategic move can be seen as a signal to the global community that it is ready to adapt and play by the rules in the ever-evolving game of technology and politics. And I feel like I’m a year younger – instead of AI, at least for a moment, we’re talking about Elon Musk and China again.

Sources

• China’s Alibaba to break up empire into six units as Jack Ma returns home
• TikTok congressional hearing: CEO Shou Zi Chew grilled by US lawmakers
• The TikTok Hearings Inspired Little Faith in Social Media or in Congress

Discover more IT content selected for you

In Vived, you will find articles handpicked by devs. Download the app and read the good stuff!

3. Amazon Sidewalk: The Future of IoT?

Have you ever dreamed of living in a world where your lost dog sends a constant signal via its collar, even if it’s running around in fields out of Wi-Fi range? Well, welcome to the future!

Amazon has opened its Sidewalk IoT network to developers, claiming to cover more than 90% of the US population. The low-power, long-range network is expected to provide free, high-speed data for IoT devices that don’t require high-bandwidth connectivity or expensive LTE or 5G. Amazon has already formed partnerships with companies such as Ring and CareBand, but now wants other developers to build devices that use the network. The network uses Bluetooth Low Energy (BLE) and LoRa and allows users to connect to the Internet via nearby Sidewalk gateways, such as Echo and Ring devices.

Potential use cases for Sidewalk network include pet trackers (we have that!), package trackers, weather stations, and other devices that don’t require high bandwidth. The dynamic range of Sidewalk is also seen as a benefit, as moving devices such as dog trackers (the subject that won’t die) could be monitored while moving around the city. While Amazon has faced privacy concerns related to using its devices to create a social network, the adoption of Sidewalk has been enthusiastically received by users who own Ring and Echo devices.

Developers can already request a test kit today to start building Sidewalk-enabled products.

Unfortunately, the solution currently only applies to the United States, based on Amazon’s hardware saturation in the market. To become a true revolution, the network of connected devices still needs to expand beyond the borders of the USA. However, I hope that if Sidewalk does impact various industries and daily life, over time the world may witness how seamlessly Sidewalk connectivity transcends geographical boundaries.

I for one am looking forward to it, especially since lately in the IoT world there has been rather more talk about delays in deployment (Hue) or even the withdrawal (Belkin) of individual providers from the Matter standard. So it’s nice to see some positive changes for the ecosystem for a change.

Sources

• Amazon just opened up its Sidewalk network for anyone to build connected gadgets on

PS: Yes, I know that Italy has blocked the use of ChatGPT – but we’ll probably talk about that as other countries take up the topic. I had to take a little break from generative AI today.