Does fliphtml5.com block AI crawlers?

Can AI assistants cite fliphtml5.com?

• ChatGPT search: its search crawlers may read the site
• Claude: its search crawlers may read the site
• Perplexity: its search crawlers may read the site

fliphtml5.com opts out of AI training by Common Crawl, ByteDance, Meta. Training crawlers and answer crawlers are separate: blocking training doesn't by itself remove a site from AI answers.

AI crawler access, bot by bot

How fliphtml5.com compares

• 19% of the 1,278 popular sites we checked block GPTBot; fliphtml5.com does not.
• 18% of the 1,278 popular sites we checked block ClaudeBot; fliphtml5.com does not.
• 14% of the 1,278 popular sites we checked block PerplexityBot; fliphtml5.com does not.
• Among 169 sites of similar popularity, 21% block at least one AI crawler.

The robots.txt rules that apply to AI crawlers

User-agent: meta-externalagent
Disallow: /

User-agent: Meta-ExternalAgent
Disallow: /

User-agent: Bytespider
Disallow: /

User-agent: CCBot
Disallow: /

User-agent: *
Disallow: */help/
Disallow: */edit-book/*
Disallow: */404.php
Disallow: */access-statistics.php
Disallow: */accountinfo.php
Disallow: */bookinfo.php
Disallow: */buyvip.php
Disallow: */forget-password.php
Disallow: */invoice.php
Disallow: */left.php
Disallow: */paymentinfo.php
Disallow: */payvipinfo.php
Disallow: */register-successfully.php
Disallow: */reset-password.php
Disallow: /pass/
Disallow: /bookvisitinfo.html
Disallow: /statistic-server/
Disallow: /api/
Disallow: /check/
Disallow: /learning-center/*/tags/
Disallow: /*/learning-center/tags/
Disallow: /ebooks/
Disallow: */homepage/
Disallow: /*/*/templates/
Disallow: *?search=
Disallow: /blog/page/
Disallow: /blog/*/page/
Disallow: /blog/*/*/page/
Disallow: /blog/*/*/*/page/
Disallow: /blog/*/*/*/*/page/
Disallow: /*/javascript/config.js
Disallow: /*/slide_javascript/slideJS.js
Disallow: /*/files/pageEditor.js
Disallow: /*/javascript/plugin.json
Disallow: /*/getui.json
Disallow: /*/user/*.json
Disallow: /*config.js*
Disallow: /*slideJS.js*
Disallow: /*pageEditor.js*
Disallow: /*plugin.json*

Excerpt: only groups for all user-agents (*) or named AI crawlers, up to 40 rules per group.

Sites with a similar AI policy

• proofpoint.com · 1 blocked
• sap.com · 3 blocked
• unity.com · 1 blocked
• medicalnewstoday.com · 6 blocked
• gizmodo.com · 3 blocked
• photobucket.com · 5 blocked
• techradar.com · 2 blocked
• sendbird.com · 7 blocked

All sites that are “blocks ai training, allows ai answers” →