Which websites block Google-Extended?
Google-Extended is Google's crawler that controls use of content for Gemini training and grounding (does not affect Google Search). Of the 1,278 popular websites we checked, 205 (16%) block it completely and 860 restrict some paths. 234 sites name it explicitly in robots.txt.
How to block or allow Google-Extended
# block User-agent: Google-Extended Disallow: / # allow User-agent: Google-Extended Allow: /
Google-Extended is a training crawler: blocking it opts out of model training but doesn't by itself remove you from AI answers. Check your robots.txt.
Sites that block Google-Extended (205)
- instagram.com
- mail.ru
- amazon.com
- whatsapp.net
- pinterest.com
- tiktok.com
- yahoo.com
- myfritz.net
- msn.com
- baidu.com
- chatgpt.com
- vimeo.com
- reddit.com
- registrar-servers.com
- snapchat.com
- gravatar.com
- nytimes.com
- flickr.com
- soundcloud.com
- cnn.com
- bbc.com
- meraki.com
- imdb.com
- launchpad.net
- reuters.com
- nature.com
- weibo.com
- alibaba.com
- skybridge.click
- naver.com
- bloomberg.com
- wsj.com
- espn.com
- globo.com
- go.com
- claude.ai
- aol.com
- pixabay.com
- cnbc.com
- statista.com
- sohu.com
- behance.net
- ft.com
- wired.com
- uol.com.br
- usatoday.com
- threads.com
- imgsmail.ru
- sharethrough.com
- techcrunch.com
- yelp.com
- dailymail.com
- hihonorcloud.com
- lemonde.fr
- fb.com
- pinimg.com
- bandcamp.com
- quantserve.com
- ted.com
- people.com
- adform.net
- lefigaro.fr
- academia.edu
- pexels.com
- daum.net
- figma.com
- hbr.org
- nbcnews.com
- genius.com
- mailchi.mp
- repubblica.it
- corriere.it
- btloader.com
- investopedia.com
- elmundo.es
- theatlantic.com
- letterboxd.com
- n-tv.de
- huffpost.com
- marca.com
- tabelog.com
- xiaohongshu.com
- aftonbladet.se
- cookpad.com
- nypost.com
- economist.com
- theconversation.com
- netcraze.io
- ad.nl
- nu.nl
- namu.wiki
- vnexpress.net
- vg.no
- fmkorea.com
- dcinside.com
- pandora.com
- gazzetta.it
- hln.be
- kakao.com
- abc.net.au
- themoviedb.org
- ladepeche.fr
- mydramalist.com
- actu.fr
- jstor.org
- sapo.pt
- nikkansports.com
- dagbladet.no
- iltalehti.fi
- eenadu.net
- tjk.org
- aka.ms
- keirin.jp
- usnews.com
- vinted.fr
- qcloud.com
- newyorker.com
- merkur.de
- rocketmoney.dev
- adrta.com
- marketwatch.com
- meb.gov.tr
- sciencemag.org
- euronews.com
- blackhub.team
- lnkd.in
- strava.com
- sedoparking.com
- clevertap-prod.com
- science.org
- secureserver.net
- salesforceliveagent.com
- adsco.re
- thesun.co.uk
- openssl.org
- mparticle.com
- variety.com
- sendbird.com
- disney.com
- instacart.com
- umd.edu
- congress.gov
- co.com
- nejm.org
- rollingstone.com
- nba.com
- redditmedia.com
- slate.com
- abcnews.com
- tidal.com
- hollywoodreporter.com
- news.com.au
- zeit.de
- expressapisv2.net
- smh.com.au
- pxf.io
- kuaishou.com
- thehill.com
- tmdb.org
- grok.com
- huntress.io
- cell.com
- system-monitor.com
- newscientist.com
- ksapisrv.com
- franceinfo.fr
- salesforce-scrt.com
- sueddeutsche.de
- glassdoor.com
- idnes.cz
- jwpltx.com
- chicagotribune.com
- osu.edu
- imageshack.us
- medscape.com
- nrk.no
- france24.com
- thetimes.co.uk
- yumpu.com
- hellotalk8.com
- theregister.com
- tagesschau.de
- newsbreak.com
- yr.no
- kahoot.it
- crunchbase.com
- vogue.com
- myanimelist.net
- metro.co.uk
- siteforce.com
- unep.org
- allrecipes.com
- nzherald.co.nz
- worldcat.org
- zalando.de
- nate.com
- permutive.app
- cuny.edu
- 123rf.com
- nymag.com
- ebsco.com
- deadline.com
- vietnam.vn
- orf.at
- ansa.it
Popular sites that allow Google-Extended
What is Google-Extended?
Google-Extended is operated by Google. According to Google's documentation, it controls use of content for Gemini training and grounding (does not affect Google Search).
How many websites block Google-Extended?
205 of the 1278 popular websites in our directory (16%) block it completely in robots.txt.
Should I block Google-Extended?
It depends on whether you're comfortable with your content being used for AI training. Blocking it doesn't remove you from AI answers, which use separate crawlers.