{"text":[[{"start":7.96,"text":"This article is an on-site version of our The AI Shift newsletter. Premium subscribers can sign up here to get the newsletter delivered every Thursday. Standard subscribers can upgrade to Premium here, or explore all FT newsletters"}],[{"start":21.78,"text":"Welcome back to The AI Shift, our weekly exploration of the intersection between AI and the world of work. This week’s edition is a two-parter, going deeper into both halves of the risk vs reliability framework we introduced last month. First, we reflect on last week’s revelation that an OpenAI AI agent autonomously carried out a cyber attack on another AI company. And then we take a look at the latest evidence on leading AI models’ ability to replace human workers — this time specifically in online gig work."}],[{"start":22.28,"text":"John writes"}],[{"start":52.92,"text":"I’m sure by now everyone will have read about last week’s big AI story, but it may be worth a brief recap. During an internal test of its latest models’ cyber capabilities, OpenAI tasked two of its most advanced models — one not yet publicly released — with taking on ‘ExploitGym’, a set of almost 900 challenges designed to test whether AI agents can exploit known vulnerabilities in real-world software."}],[{"start":77.46,"text":"But instead of attempting to solve them, the models decided the most direct route to success was to locate and steal the solutions to the tests. So they identified and exploited previously unknown weaknesses in what was thought to have been a securely walled-off testing environment, broke out on to the open internet and hacked into the private servers of another multibillion-dollar AI company called Hugging Face (where they presumably believed the solutions may have been stored). The attack was serious enough to prompt Hugging Face to contact law enforcement, and most concerning of all it wasn’t until several days after the attack began that OpenAI discovered what had happened."}],[{"start":114.6,"text":"Some people outside the industry have sought to downplay the seriousness of what happened, arguing variously that it should hardly be surprising that AI models tasked with exploiting a particular set of software vulnerabilities would find and exploit other ones, or that all of this is just a marketing ploy by OpenAI to showcase its models’ abilities. But both are wide of the mark. Attempting to pass the test by solving the challenges is the expected behaviour; attempting to cheat on the test by stealing the answers is not the expected behaviour. And the revelation that OpenAI lost control of its agents and took several days to discover they had executed a large and sustained attack on another company’s infrastructure is not good publicity for OpenAI or for the technology more broadly."}],[{"start":162.6,"text":"The incident underscores the seriousness of the risks of escalating AI-orchestrated cyber attacks we wrote about last month. As Hugging Face wrote in their report on the attack, it exposes a fundamental asymmetry between cyber attack and defence. The insurgent agents operated with a speed, scale and persistence far beyond what any human hacker would have been capable of, increasing their chance of success while drowning the defensive side in a deluge of activity to investigate."}],[{"start":192,"text":"It also exposes shortcomings in how internal testing of cutting-edge models is being carried out and monitored. Both OpenAI and Anthropic have reported that their models have repeatedly behaved in unexpected and alarming ways during tests of cyber capabilities over recent months."}],[{"start":209.28,"text":"There has been a tendency in some quarters to portray AI leaders’ statements about the risks of harmful outcomes and the possibility that development may need to be slowed or more tightly regulated as self-serving hype, but to me incidents like these make it clear that the concerns are well-founded."}],[{"start":226.28,"text":"So that’s this week’s update on the dangers. How about the usefulness side? A few weeks ago we got the latest update of what I think is one of the most informative measures of AI’s capacity to displace human work and workers — the Remote Labour Index, which has frontier AI models attempt real-world remote-work projects which are then assessed by domain experts against human outputs for the same tasks. Examples include producing digital designs and 3D models for product design or architecture tasks, building interactive data visualisations and generating animated videos. What I find most helpful is that you can inspect some case studies to get a sense of how impressive (or terrible!) the AI-generated submissions are, and how this is changing over time."}],[{"start":270.56,"text":"The headline finding from the most recent update is that the best-performing model tested to date, Anthropic’s Claude Fable, was judged to deliver professionally acceptable work on 16 per cent of digital remote-work tasks. A one in six passing rate is very low — a human worker with that track record would quickly stop getting new commissions — but the pace of improvement is noteworthy. Anthropic’s previous top model Opus 4.8 had scored only 8 per cent, and the one before that 4 per cent."}],[{"start":302.36,"text":"Looking through some of the AI work submitted paints a similar picture. The best models just a few months ago were producing jewellery designs and floor plans that would have been laughed out of town. The latest ones are significantly more accurate and polished — but generally still not quite there."}],[{"start":319.72,"text":"As with all things AI and work, the question of how to interpret the results all comes down to perspective. On the one hand, the speed of improvement is impressive, and the fact that AI is producing acceptable work in real white-collar human professions at all is remarkable. We already know AI has displaced work and reduced human incomes in online gig-work marketplaces for simple writing, coding and design tasks; these results suggest it is moving up the value chain."}],[{"start":349.54,"text":"On the other hand, the types of errors in some of the AI work submissions suggest that in many cases AI simply has no sense of what someone in that profession is really trying to do, or what constitutes good work, and is brute-forcing its way to something approaching acceptable. There may be many tasks where this doesn’t matter — where passable work is passable work — but to my mind it’s another demonstration that automating a task is an entirely different thing to automating a job. In that sense, what we’re looking at here might be better understood as AI getting better at augmenting a wider range of human workers, rather than replacing them."}],[{"start":350.04,"text":"Sarah writes"}],[{"start":386,"text":"This makes me wonder, John, whether the “white-collar gig economy” is now on its last legs. Way back in the mists of time (2015) I wrote a piece on the rise of freelance labour platforms like Upwork — a phenomenon I described as “the human cloud”. As I put it back then: “Employers are starting to see the human cloud as a new way to get work done. White-collar jobs are chopped into hundreds of discrete projects or tasks, then scattered into a virtual “cloud” of willing workers who could be anywhere in the world, so long as they have an internet connection.”"}],[{"start":418.44,"text":"At the time, this seemed as if it would enable some good things for workers (flexible work opportunities, “digital nomad” lifestyles and so on) and some bad things (poor pay, undercutting, opaque conditions, no employment rights). But what I didn’t foresee was that it would also make people particularly vulnerable to being displaced by AI."}],[{"start":438.5,"text":"In a call to announce its first-quarter results this year, Upwork’s chief executive Hayden Brown said that, starting in February, “accelerated AI adoption degraded the volume of client activity on the low end, impacting contracts of $500 and below. While this dynamic is not new, the pace of AI automation was faster than previously seen . . . On the low end, simple tasks are getting replaced with AI tools.”"}],[{"start":463.38,"text":"The hope for Upwork is that the platform can become a useful service for small businesses which want to use AI but need the assistance of human freelancers to make the most of it. Or as Brown put it, to become the “AI diffusion layer for SMBs [small and medium-sized businesses]: providing the AI-skilled talent they need to create value from this exciting technology.” I guess this could be described as “augmentation”, John, and it fits with the evidence you describe that AI is still actually not that great at doing a lot of real-world remote work tasks without some human hand-holding. Although it’s not clear to me whether it’s the AI augmenting the human in this scenario, or the other way around."}],[{"start":502.4,"text":"Regardless, it doesn’t look as if investors are buying this story so far. Upwork’s shares are worth about half what they were at the start of the year. Meanwhile, many freelancers I hear from say they’re focusing more on high-end work and building direct relationships with clients."}],[{"start":518.28,"text":"The days when you could travel around Thailand while doing a bit of random copywriting or logo design seem to have come and gone."}],[{"start":518.78,"text":"School’s out (ish) for summer . . ."}],[{"start":527.66,"text":"John and I are off on our respective holidays soon, so we’ll be running a reduced fortnightly schedule in August (with editions coming out on the 13th and 27th). We’ll be back to normal in September!"}],[{"start":528.16,"text":"Recommended reading"}],[{"start":538.88,"text":"Arvind Narayanan has an insightful note arguing that another reason AI does better at coding than other tasks is that software is a rare — perhaps unique — case where the models’ training data doesn’t just include the outputs of human work but the entire process from ideation through development to production (John)"}],[{"start":562.74,"text":"The FT Weekend ran an interesting piece by a philosopher urging her fellow academics not to take jobs with the AI industry (Sarah)"}],[{"start":null,"text":"