Which websites block OAI-SearchBot?
OAI-SearchBot is OpenAI's crawler that surfaces websites in ChatGPT search results. Of the 1,278 popular websites we checked, 121 (9%) block it completely and 927 restrict some paths. 181 sites name it explicitly in robots.txt.
How to block or allow OAI-SearchBot
# block User-agent: OAI-SearchBot Disallow: / # allow User-agent: OAI-SearchBot Allow: /
Blocking OAI-SearchBot can keep your pages out of OpenAI's AI answers and citations. Check your robots.txt.
Sites that block OAI-SearchBot (121)
- facebook.com
- instagram.com
- mail.ru
- twitter.com
- amazon.com
- x.com
- pinterest.com
- tiktok.com
- myfritz.net
- msn.com
- baidu.com
- reddit.com
- registrar-servers.com
- nytimes.com
- soundcloud.com
- cnn.com
- bbc.com
- meraki.com
- imdb.com
- launchpad.net
- weibo.com
- skybridge.click
- naver.com
- bloomberg.com
- claude.ai
- cnbc.com
- sohu.com
- quora.com
- usatoday.com
- threads.com
- imgsmail.ru
- telegraph.co.uk
- sharethrough.com
- yelp.com
- dailymail.com
- hihonorcloud.com
- fb.com
- pinimg.com
- quantserve.com
- adform.net
- lefigaro.fr
- daum.net
- nbcnews.com
- mailchi.mp
- btloader.com
- elmundo.es
- huffpost.com
- marca.com
- xiaohongshu.com
- economist.com
- theconversation.com
- netcraze.io
- ad.nl
- nu.nl
- namu.wiki
- hln.be
- kakao.com
- themoviedb.org
- spiegel.de
- actu.fr
- sapo.pt
- iltalehti.fi
- eenadu.net
- tjk.org
- aka.ms
- keirin.jp
- usnews.com
- qcloud.com
- zdnet.com
- rocketmoney.dev
- adrta.com
- wikihow.com
- blackhub.team
- sedoparking.com
- clevertap-prod.com
- secureserver.net
- ign.com
- salesforceliveagent.com
- adsco.re
- mashable.com
- openssl.org
- mparticle.com
- variety.com
- mirror.co.uk
- congress.gov
- co.com
- rollingstone.com
- redditmedia.com
- express.co.uk
- hollywoodreporter.com
- news.com.au
- expressapisv2.net
- smh.com.au
- pxf.io
- thehill.com
- tmdb.org
- huntress.io
- system-monitor.com
- newscientist.com
- ksapisrv.com
- franceinfo.fr
- salesforce-scrt.com
- jwpltx.com
- chicagotribune.com
- osu.edu
- imageshack.us
- nrk.no
- ctvnews.ca
- france24.com
- hellotalk8.com
- newsbreak.com
- yr.no
- kahoot.it
- metro.co.uk
- siteforce.com
- nzherald.co.nz
- flipboard.com
- nate.com
- permutive.app
- cuny.edu
- deadline.com
Popular sites that allow OAI-SearchBot
What is OAI-SearchBot?
OAI-SearchBot is operated by OpenAI. According to OpenAI's documentation, it surfaces websites in ChatGPT search results.
How many websites block OAI-SearchBot?
121 of the 1278 popular websites in our directory (9%) block it completely in robots.txt.
Should I block OAI-SearchBot?
Only if you don't want OpenAI's AI answers to read or cite your pages. Most businesses that want to be recommended by AI assistants allow it.