AI Agents Will Be Manipulation Engines Surrendering to algorithmic agents risks putting us under their influence.
By Kate Crawford Dec 23, 2024 In 2025, it will be commonplace to talk with a personal AI agent that knows your schedule, your circle of friends, the places you go. This will be sold as a convenience equivalent to having a personal, unpaid assistant. These anthropomorphic agents are designed to support and charm us so that we fold them into every part of our lives, giving them deep access to our thoughts and actions. With voice-enabled interaction, that intimacy will feel even closer. That sense of comfort comes from an illusion that we are engaging with something truly humanlike, an agent that is on our side. Of course, this appearance hides a very different kind of system at work, one that serves industrial priorities that are not always in line with our own. New AI agents will have far greater power to subtly direct what we buy, where we go, and what we read. That is an extraordinary amount of power. AI agents are designed to make us forget their true allegiance as they whisper to us in humanlike tones. These are manipulation engines, marketed as seamless convenience. People are far more likely to give complete access to a helpful AI agent that feels like a friend. This makes humans vulnerable to being manipulated by machines that prey on the human need for social connection in a time of chronic loneliness and isolation. Every screen becomes a private algorithmic theater, projecting a reality crafted to be maximally compelling to an audience of one. This is a moment that philosophers have warned us about for years. Before his death, philosopher and neuroscientist Daniel Dennett wrote that we face a grave peril from AI systems that emulate people: “These counterfeit people are the most dangerous artifacts in human history … distracting and confusing us and by exploiting our most irresistible fears and anxieties, will lead us into temptation and, from there, into acquiescing to our own subjugation.” The emergence of personal AI agents represents a form of cognitive control that moves beyond blunt instruments of cookie tracking and behavioral advertising toward a more subtle form of power: the manipulation of perspective itself. Power no longer needs to wield its authority with a visible hand that controls information flows; it exerts itself through imperceptible mechanisms of algorithmic assistance, molding reality to fit the desires of each individual. It’s about shaping the contours of the reality we inhabit. This influence over minds is a psychopolitical regime: It directs the environments where our ideas are born, developed, and expressed. Its power lies in its intimacy—it infiltrates the core of our subjectivity, bending our internal landscape without us realizing it, all while maintaining the illusion of choice and freedom. After all, we are the ones asking AI to summarize that article or produce that image. We may have the power of the prompt, but the real action lies elsewhere: the design of the system itself. And the more personalized the content, the more effectively a system can predetermine the outcomes. Consider the ideological implications of this psychopolitics. Traditional forms of ideological control relied on overt mechanisms—censorship, propaganda, repression. In contrast, today’s algorithmic governance operates under the radar, infiltrating the psyche. It is a shift from the external imposition of authority to the internalization of its logic. The open field of a prompt screen is an echo chamber for a single occupant. This brings us to the most perverse aspect: AI agents will generate a sense of comfort and ease that makes questioning them seem absurd. Who would dare critique a system that offers everything at your fingertips, catering to every whim and need? How can one object to infinite remixes of content? Yet this so-called convenience is the site of our deepest alienation. AI systems may appear to be responding to our every desire, but the deck is stacked: from the data used to train the system, to the decisions about how to design it, to the commercial and advertising imperatives that shape the outputs. We will be playing an imitation game that ultimately plays us. <https://www.wired.com/story/ai-agents-personal-assistants-manipulation-engines/>
