Does flickr.com block AI crawlers?

Can AI assistants cite flickr.com?

• ChatGPT search: its search crawlers may read the site
• Claude: its search crawlers may read the site
• Perplexity: its search crawlers may read the site

flickr.com opts out of AI training by OpenAI, Anthropic, Google, Apple, Common Crawl, ByteDance, Meta. Training crawlers and answer crawlers are separate: blocking training doesn't by itself remove a site from AI answers.

AI crawler access, bot by bot

How flickr.com compares

• 19% of the 1,278 popular sites we checked block GPTBot; flickr.com is one of them.
• 18% of the 1,278 popular sites we checked block ClaudeBot; flickr.com is one of them.
• 14% of the 1,278 popular sites we checked block PerplexityBot; flickr.com does not.
• Among 171 sites of similar popularity, 34% block at least one AI crawler.

The robots.txt rules that apply to AI crawlers

User-agent: Adsbot-Google
User-agent: Alexabot
User-agent: AppleBot
User-agent: archive.org_bot
User-agent: Assaybot
User-agent: baiduspider
User-agent: bitlybot
User-agent: bingbot
User-agent: BingPreview
User-agent: ChatGPT-User
User-agent: Claude-SearchBot
User-agent: Claude-User
User-agent: coccocbot
User-agent: Discordbot
User-agent: DuckDuckBot
User-agent: facebookexternalhit
User-agent: Google-Site-Verification
User-agent: Google-Sitemaps
User-agent: Googlebot
User-agent: Googlebot-Image
User-agent: Googlebot-Mobile
User-agent: Googlebot-News
User-agent: Googlebot-Video
User-agent: gsa-crawler
User-agent: HatenaBlog-bot
User-agent: ia_archiver
User-agent: Mediapartners-Google
User-agent: msnbot
User-agent: NAVER
User-agent: OAI-SearchBot
User-agent: PerplexityBot
User-agent: PetalBot
User-agent: Pingdom
User-agent: Pinterest
User-agent: redditbot
User-agent: Slackbot
User-agent: Slurp
User-agent: snapchat
User-agent: Twitterbot
User-agent: wp.com
User-agent: Yandex
User-agent: YandexImages
User-agent: YandexVideoParser
User-agent: Yeti
User-agent: ZoomBot
Disallow: /gp/
Disallow: /report_abuse.gne
Disallow: /abuse
Disallow: /images/*
Disallow: /apps/*
Disallow: /tools/demos/*
Disallow: /search
Disallow: /services/oauth
Disallow: /groups/10millionphotos/
Disallow: /photos/youpy/
Disallow: /photos/i_love_u_get_away_from_me/
Disallow: /faves-i_love_u_get_away_from_me/
Disallow: /photos/gbachelie/
Disallow: /photos/archivesact/6011019532/nearby/
Disallow: /yss_fragment.gne
Disallow: /photos/tags/*/page*
Disallow: /photos/*/lightbox/
Disallow: /groups/*/lightbox/
Disallow: /explore/*/lightbox/

User-agent: *
Disallow: /

Excerpt: only groups for all user-agents (*) or named AI crawlers, up to 40 rules per group.

Sites with a similar AI policy

• medium.com · 5 blocked
• gravatar.com · 4 blocked
• unity3d.com · 1 blocked
• canva.com · 6 blocked
• snapchat.com · 7 blocked
• who.int · 1 blocked
• linktr.ee · 2 blocked
• forter.com · 3 blocked

All sites that are “blocks ai training, allows ai answers” →