Does n-able.com block AI crawlers?

Can AI assistants cite n-able.com?

• ChatGPT search: its search crawlers may read the site
• Claude: its search crawlers may read the site
• Perplexity: its search crawlers may read the site

n-able.com opts out of AI training by Common Crawl. Training crawlers and answer crawlers are separate: blocking training doesn't by itself remove a site from AI answers.

AI crawler access, bot by bot

How n-able.com compares

• 19% of the 1,278 popular sites we checked block GPTBot; n-able.com does not.
• 18% of the 1,278 popular sites we checked block ClaudeBot; n-able.com does not.
• 14% of the 1,278 popular sites we checked block PerplexityBot; n-able.com does not.
• Among 182 sites of similar popularity, 25% block at least one AI crawler.

The robots.txt rules that apply to AI crawlers

User-agent: *
Disallow: /wp-admin/
Disallow: /wp-login.php
Disallow: /wp-content/uploads/resources/
Allow: /wp-admin/admin-ajax.php

User-agent: aiHitBot
User-agent: Barkrowler
User-agent: BDCbot
User-agent: BLEXBot
User-agent: BLP_bbot
User-agent: brokenlinkcheck.com
User-agent: Buck
User-agent: CCBot
User-agent: Cliqzbot
User-agent: cyencebot
User-agent: DomainCrawler
User-agent: Dow Jones Searchbot
User-agent: Exabot
User-agent: ExtLinksBot
User-agent: FemtosearchBot
User-agent: Fever
User-agent: GarlikCrawler
User-agent: Gigabot
User-agent: gobuster
User-agent: GrapeshotCrawler
User-agent: heritrix
User-agent: istellabot
User-agent: Jersey
User-agent: Jobkicks
User-agent: libwww-perl
User-agent: linkdexbot
User-agent: LinkpadBot
User-agent: ltx71 - (http://ltx71.com/)
User-agent: lua-resty-http
User-agent: LumtelBot
User-agent: magpie-crawler
User-agent: Magus bot
User-agent: Mail.RU_Bot
User-agent: Megaindex.ru
User-agent: Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; Trident/6.0) LinkCheck by Siteimprove.com
User-agent: Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; Trident/6.0) SiteCheck-sitecrawl by Siteimprove.com
User-agent: NL-Crawler
User-agent: OnPageBot
User-agent: Riddler
User-agent: ScoutJet
User-agent: Scrapy
User-agent: Seekport
User-agent: SeznamBot
User-agent: Siteimprove
User-agent: SMTBot
User-agent: UptimeRobot
User-agent: VelenPublicWebCrawler
User-agent: Wget
User-agent: yacybot
User-agent: Yeti
User-agent: YisouSpider
User-agent: YunSecurityBot
User-agent: ZoominfoBot
Disallow: /

Excerpt: only groups for all user-agents (*) or named AI crawlers, up to 40 rules per group.

Sites with a similar AI policy

• tagesschau.de · 7 blocked
• cbssports.com · 1 blocked
• loom.com · 1 blocked
• sas.com · 2 blocked
• theregister.com · 7 blocked
• crunchbase.com · 7 blocked
• nicovideo.jp · 1 blocked
• myanimelist.net · 7 blocked

All sites that are “blocks ai training, allows ai answers” →