Privacy technologies
fromSearch Engine Roundtable
2 days agoAnthropic Updates Its Crawler Documentation: ClaudeBot, Claude-User & Claude-SearchBot
Anthropic updated its crawler documentation to clarify the functions of three bots—ClaudeBot, Claude-User, and Claude-SearchBot—and explain the consequences of blocking each crawler.