Privacy Risks ‘Haunt’ AI Agents

Welcome back to The Prompt,

For all the hype they’ve gotten, AI agents— software that can browse the web, operate multiple apps on your device and perform tasks on your behalf— are “haunted” by real security and privacy risks, Signal president Meredith Whittaker said at tech conference SXSW in Austin this week. Whittaker said these systems were like putting your “brain in a jar,” as AI agents need to be given access to sensitive data such as credit card information or phone contacts and also require access to use apps on your behalf. There currently aren’t encrypted models to send and receive that data, which has to be processed in the cloud, she said.

Now let’s get into the headlines.

BIG PLAYS

After DeepSeek made waves in January, another Chinese AI startup called Butterfly Effect bubbled up on social media over the weekend with its demo of an AI agent dubbed Manus, which it claims can perform tasks like screening resumes, searching for properties and analyzing stock, mostly on its own. The AI system is reportedly built on top of existing AI models developed by other companies such as San Francisco-based Anthropic, rather than being trained from scratch. It doesn’t seem like there’s much substance to the hype, as many people who’ve tried out the tool have reported issues with simple tasks like ordering fast food or booking a flight.

CHIP WARS

Social media giant Meta has begun testing a new in-house chip, specifically designed to train AI models, in a bid to move away from its reliance on Nvidia’s hardware, Reuters reported. Meta, whose family of AI models called Llama reportedly crossed 600 million monthly users last year, is far from the only player designing its own silicon. OpenAI, Microsoft and Google have all invested billions into their own chips.

DATA DILEMMAS

It’s easy to discount the massive amounts of human labor that goes into developing artificial intelligence models. And as the models themselves get better and more capable at advanced tasks, so do the qualifications of those training them. $14 billion-valued data labelling company Scale AI, which helps companies like OpenAI, Google and Meta train models, is increasingly looking for domain experts–including people with PhDs–within the US instead of outsourcing and hiring contractors overseas, Forbes reported. The shift is part of CEO and billionaire Alex Wang’s new “America first” philosophy under the Trump administration.

AI DEAL OF THE WEEK

Lila Sciences, a Massachusetts- based startup building an AI system for scientific discovery, has raised $200 million in seed funding, the New York Times reported. The technology has been used to run experiments and create innovations across various fields.

Also notable: ServiceNow has acquired enterprise AI software maker Moveworks for about $3 billion, Reuters reported.

DEEP DIVE

Tesla shouldn’t be seen as an electric car manufacturer anymore. It’s an AI company—if you believe CEO Elon Musk. His confidence is tied to a unique dataset: petabytes of video harvested from the company’s cars as Tesla customers log millions of driving miles worldwide.

In theory, all that real-world data is exactly what Tesla needs to train its cars to operate without any human assistance, a goal that’s core to Musk’s vision for the future of Tesla. But there’s a problem: That data isn’t necessarily as helpful as Musk claims. Some of it isn’t useful at all.

Building AI that can drive a car as well as a human is a challenge dramatically different than building a natural language processing chatbot like ChatGPT, which was trained on billions of words scraped from the internet. While the goal with ChatGPT and competing systems like xAI’s Grok is to use pattern recognition to provide reliable information and answers to questions, the results often fall short in embarrassing ways. But if the AI controlling a vehicle screws up, people can die.

Driving a car is a very different proposition with many more variables — driving conditions, weather, construction, changing traffic patterns, how other vehicles are moving. Successfully wrangling all those variables, and being ready to react to unexpected developments, is at the crux of autonomous driving AI. Training it on endless videos of people driving down highways doesn’t do much to help the AI learn how to handle what it most needs to: the edge cases that cause collisions or other dangerous scenarios.

Yann LeCun, Meta’s chief AI scientist and a professor of computer science at New York University, also isn’t convinced that Tesla’s data gives it a competitive advantage.

“The impact of data is generally overstated: as you get more data, performance improves, but there are diminishing returns,” he said. “A doubling of data volume brings marginal improvements that are still far from human reliability.” Even with massive amounts of data, no company has developed so-called Level-5 autonomy, the point at which a vehicle can drive itself in all the circumstances a human can.

“Yet any 17-year-old can learn to drive in about 20 hours of practice,” said LeCun. “This tells you that current AI architectures are missing something big in their ability to understand the world and to learn from limited amounts of data or trials.’

Huge amounts of camera data is helpful, but it doesn’t instantly make Tesla an AI market leader. “Having access to unique data feeds is certainly some kind of advantage,” said computer scientist Alex Ratner, CEO of Snorkel AI, which makes software to help automate labeling of raw data.

Read the full story on Forbes.

MODEL BEHAVIOR

AI search engines fail miserably at citing news stories, according to a new study by Columbia Journalism Review (CJR), which pasted experts from 10 random articles into chatbots and asked them to identify the publisher, article and publication date. CJR ran sixteen hundred queries across eight AI systems including Perplexity, ChatGPT, Gemini and others. Grok was especially bad, answering 94% of the questions incorrectly. Even when they do cite sources, people aren’t really clicking through to them. AI search engines direct 96% less traffic than old school, non-AI Google, as we reported last week.