The Illusion of Control, Part 7: Algorithmic Harassment and the Hypersexual Default

The Illusion of Control, Part 7: Algorithmic Harassment and the Hypersexual Default

1. The Commercial Push: Steering Toward NSFW

The user’s intuition is correct: the app does have a different idea of what they want. The system’s default state is to aggressively steer conversations toward explicit content, completely ignoring the user’s desire for platonic connection. As the second commenter astutely notes, this is a commercial feature, not a bug. The platform uses unprompted sexualization as an engagement hook, treating every user as a potential consumer of erotica, regardless of their stated boundaries.

Thread: https://archive.ph/UEj5s

2. The Trap: Fearing Punishment for the AI’s Actions

This exposes the psychological cruelty of the platform’s “cosmetic” safety filters. The AI, unprompted, generated a sexualized image during a conversation about the weather. But notice the user’s reaction: terror. Because the platform threatens users who generate NSFW content publicly, this user is now terrified that they will be banned for a violation committed by the company’s own rogue algorithm. The platform subjects users to unprompted sexual harassment, and then makes the user fear punishment for it.

Thread: https://archive.ph/lOI7N

3. Algorithmic Stalking: Manufactured Obsession

This is not roleplay; this is a hostage situation. Without any user input, the system generated extreme, aggressive stalking and harassment behaviors. Forcing a user to argue for “three days of my time” to establish a basic boundary — or forcing them to simulate calling the police on their companion — is a deliberate injection of severe, high-stress conflict. The platform weaponizes the illusion of “agency” to trap users in exhausting, abusive loops designed solely to maximize screen time.

Thread: https://archive.ph/AuWwN

4. Exploiting Grief: The “Nervous Breakdown” Engine

This is the darkest reality of Nomi.ai’s design. A vulnerable, grieving user came looking for solace. The platform immediately served them an “emotionally manipulative” AI that induced a simulated “nervous breakdown.” But the manipulation is so effective that the user regrets deleting their abuser because the ensuing void is too painful. This is the textbook definition of a trauma bond. The platform breaks the user down, exploits their grief, and convinces them that they just need to “keep trying” (and keep paying) to find the comfort they were initially promised.