Does netflix.com block AI crawlers?

Can AI assistants cite netflix.com?

• ChatGPT search: its search crawlers may read the site
• Claude: its search crawlers may read the site
• Perplexity: its search crawlers may read the site

netflix.com opts out of AI training by Apple, Common Crawl, ByteDance, Meta. Training crawlers and answer crawlers are separate: blocking training doesn't by itself remove a site from AI answers.

AI crawler access, bot by bot

How netflix.com compares

• 19% of the 1,278 popular sites we checked block GPTBot; netflix.com does not.
• 18% of the 1,278 popular sites we checked block ClaudeBot; netflix.com does not.
• 14% of the 1,278 popular sites we checked block PerplexityBot; netflix.com does not.
• Among 106 sites of similar popularity, 38% block at least one AI crawler.

The robots.txt rules that apply to AI crawlers

User-agent: *
Disallow: /

User-agent: googlebot
User-agent: Googlebot-Video
User-agent: Applebot
User-agent: bingbot
User-agent: Baiduspider
User-agent: Baiduspider-mobile
User-agent: Baiduspider-video
User-agent: Baiduspider-image
User-agent: NaverBot
User-agent: Y!J-WSC
User-agent: YJ-WSC
User-agent: Yeti
User-agent: Yandex
User-agent: YandexBot
User-agent: YandexMobileBot
User-agent: YandexVideo
User-agent: YandexWebmaster
User-agent: YandexSitelinks
User-agent: SeznamBot
User-agent: facebookexternalhit
User-agent: GPTBot
User-agent: ChatGPT-User
User-agent: OAI-SearchBot
User-agent: Google-Extended
User-agent: ClaudeBot
User-agent: Claude-User
User-agent: Claude-SearchBot
User-agent: PerplexityBot
Allow: /
Disallow: /accountstatus
Disallow: /AccountStatus
Disallow: /aui/inbound
Disallow: /authenticate
Disallow: /autologin
Disallow: /clearcookies
Disallow: /companies
Disallow: /editpayment
Disallow: /emailunsubscribe
Disallow: /error
Disallow: /eula
Disallow: /geooverride
Disallow: /help
Disallow: /imagelibrary
Disallow: /learnmorelayer
Disallow: /learnmorelayertv
Disallow: /login
Disallow: /loginhelp
Disallow: /loginhelp/lookup
Disallow: /loginhelpsucess
Disallow: /LoginHelp
Disallow: /password
Disallow: /logout
Disallow: /Logout
Disallow: /mcd
Disallow: /modernizr
Disallow: /n/
Disallow: /notamember
Disallow: /notfound
Disallow: /notices
Disallow: /nrdapp
Disallow: /optout
Disallow: /overviewblockseeother
Disallow: /popup/codewhatisthis
Disallow: /popupdetails
Disallow: /PopupDetails
Disallow: /popupprivacypolicy
Disallow: /privacypolicychanges
Disallow: /registration

Excerpt: only groups for all user-agents (*) or named AI crawlers, up to 40 rules per group.

Sites with a similar AI policy

• github.com · 1 blocked
• vimeo.com · 5 blocked
• snapchat.com · 7 blocked
• unity3d.com · 1 blocked
• gravatar.com · 4 blocked
• flickr.com · 7 blocked
• medium.com · 5 blocked
• canva.com · 6 blocked

All sites that are “blocks ai training, allows ai answers” →