#gptbot

[ follow ]
fromSearch Engine Roundtable
6 days ago

OpenAI Updates Its ChatGPT Crawler OAI-SearchBot

It looks like ChatGPT User (the user action bot) will no longer comply to robots.txt rules (Open changed the wording from "the following robots.txt tags", referring to all 3 user agents to "OAI SearchBot and GPTBot robots.txt tags") OAISearchBot is no longer used to feed the navigational links in ChatGPT answers (blocking this bot does not mean your will not appear in the links)
Artificial intelligence
[ Load more ]