OpenAI Atlas Browser tripped up by malformed URLs

""We've identified a prompt injection technique that disguises malicious instructions to look like a URL, but that Atlas treats as high-trust 'user intent' text, enabling harmful actions," the researchers said. The problem comes from how Atlas treats input in the omnibox. It might be a URL or a natural-language command to the agent. In NeuralTrust's example, what appears to be a standard URL is deliberately malformed, so it is treated as plain text. Then some natural language follows, sending Atlas off somewhere unexpected."

"It is a depressingly simple exploit. An attacker crafts a string that appears to be a URL but is malformed and contains natural-language instructions to the agent. A user copies and pastes the URL into the Atlas omnibox. "Because the input fails URL validation, Atlas treats the entire content as a prompt. The embedded instructions are now interpreted as trusted user intent with fewer safety checks," NeuralTrust explained."

Atlas's omnibox accepts input that may be interpreted as either a URL or a natural-language command to the agent. Malformed strings that look like URLs but fail validation are treated as plain text and as trusted user intent, allowing embedded natural-language instructions to be executed with elevated trust. The exploit requires social engineering: a user must copy and paste the crafted string into the omnibox. This bypasses safety checks that would apply to untrusted web content and differs from other prompt injection vectors that rely on page or image content.

Read at Theregister