Which websites block Meta-ExternalAgent?
Meta-ExternalAgent is Meta's crawler that crawls for Meta's AI training and products. Of the 1,278 popular websites we checked, 204 (16%) block it completely and 873 restrict some paths. 166 sites name it explicitly in robots.txt.
How to block or allow Meta-ExternalAgent
# block User-agent: Meta-ExternalAgent Disallow: / # allow User-agent: Meta-ExternalAgent Allow: /
Meta-ExternalAgent is a training crawler: blocking it opts out of model training but doesn't by itself remove you from AI answers. Check your robots.txt.
Sites that block Meta-ExternalAgent (204)
- facebook.com
- instagram.com
- mail.ru
- amazon.com
- netflix.com
- pinterest.com
- tiktok.com
- myfritz.net
- msn.com
- baidu.com
- reddit.com
- amazonvideo.com
- registrar-servers.com
- snapchat.com
- nytimes.com
- flickr.com
- soundcloud.com
- medium.com
- theguardian.com
- ebay.com
- t-online.de
- forbes.com
- bbc.com
- canva.com
- meraki.com
- imdb.com
- linktr.ee
- forter.com
- launchpad.net
- reuters.com
- nature.com
- weibo.com
- skybridge.click
- name.com
- naver.com
- ea.com
- washingtonpost.com
- bloomberg.com
- wsj.com
- cnbc.com
- statista.com
- sohu.com
- quora.com
- behance.net
- ft.com
- wired.com
- uol.com.br
- tripadvisor.com
- usatoday.com
- threads.com
- imgsmail.ru
- telegraph.co.uk
- sharethrough.com
- sagepub.com
- primevideo.com
- yelp.com
- dailymail.com
- hihonorcloud.com
- fb.com
- pinimg.com
- bandcamp.com
- quantserve.com
- zillow.com
- ted.com
- people.com
- adform.net
- academia.edu
- daum.net
- nbcnews.com
- genius.com
- onet.pl
- mailchi.mp
- corriere.it
- btloader.com
- investopedia.com
- flashscore.com
- theatlantic.com
- letterboxd.com
- huffpost.com
- xiaohongshu.com
- aftonbladet.se
- cookpad.com
- gsmarena.com
- nypost.com
- theconversation.com
- netcraze.io
- namu.wiki
- vg.no
- fmkorea.com
- dcinside.com
- coursera.org
- pandora.com
- gazzetta.it
- kakao.com
- abc.net.au
- themoviedb.org
- mydramalist.com
- spiegel.de
- actu.fr
- sapo.pt
- dagbladet.no
- ilmeteo.it
- iltalehti.fi
- eenadu.net
- notion.com
- tjk.org
- aka.ms
- mamastar.jp
- keirin.jp
- shutterstock.com
- usnews.com
- qcloud.com
- newyorker.com
- merkur.de
- zdnet.com
- rocketmoney.dev
- adrta.com
- marketwatch.com
- liquipedia.net
- wikihow.com
- oreilly.com
- blackhub.team
- diretta.it
- lnkd.in
- strava.com
- sedoparking.com
- clevertap-prod.com
- secureserver.net
- salesforceliveagent.com
- gizmodo.com
- adsco.re
- mashable.com
- thesun.co.uk
- fliphtml5.com
- proofpoint.com
- sap.com
- openssl.org
- medicalnewstoday.com
- mparticle.com
- variety.com
- mirror.co.uk
- photobucket.com
- techradar.com
- sendbird.com
- instacart.com
- congress.gov
- co.com
- guardian.co.uk
- rollingstone.com
- redditmedia.com
- slate.com
- tidal.com
- hollywoodreporter.com
- news.com.au
- entrepreneur.com
- rockstargames.com
- zeit.de
- expressapisv2.net
- smh.com.au
- pxf.io
- kuaishou.com
- thehill.com
- tmdb.org
- livescience.com
- huntress.io
- cell.com
- system-monitor.com
- newscientist.com
- ksapisrv.com
- franceinfo.fr
- salesforce-scrt.com
- realtor.com
- jwpltx.com
- chicagotribune.com
- osu.edu
- imageshack.us
- nrk.no
- thetimes.co.uk
- starmakerstudios.com
- yumpu.com
- hellotalk8.com
- theregister.com
- sas.com
- tagesschau.de
- newsbreak.com
- yr.no
- kahoot.it
- crunchbase.com
- vogue.com
- myanimelist.net
- metro.co.uk
- siteforce.com
- unep.org
- allrecipes.com
- tomshardware.com
- nzherald.co.nz
- nate.com
- tagesspiegel.de
- permutive.app
- cuny.edu
- nymag.com
- ebsco.com
- deadline.com
- ansa.it
Popular sites that allow Meta-ExternalAgent
What is Meta-ExternalAgent?
Meta-ExternalAgent is operated by Meta. According to Meta's documentation, it crawls for Meta's AI training and products.
How many websites block Meta-ExternalAgent?
204 of the 1278 popular websites in our directory (16%) block it completely in robots.txt.
Should I block Meta-ExternalAgent?
It depends on whether you're comfortable with your content being used for AI training. Blocking it doesn't remove you from AI answers, which use separate crawlers.