Does rustore.ru block AI crawlers?

Can AI assistants cite rustore.ru?

• ChatGPT search: its search crawlers may read the site
• Claude: its search crawlers may read the site
• Perplexity: its search crawlers may read the site

rustore.ru doesn't opt out of AI model training through robots.txt. Training crawlers and answer crawlers are separate: blocking training doesn't by itself remove a site from AI answers.

AI crawler access, bot by bot

How rustore.ru compares

• 19% of the 1,278 popular sites we checked block GPTBot; rustore.ru does not.
• 18% of the 1,278 popular sites we checked block ClaudeBot; rustore.ru does not.
• 14% of the 1,278 popular sites we checked block PerplexityBot; rustore.ru does not.
• Among 201 sites of similar popularity, 31% block at least one AI crawler.

The robots.txt rules that apply to AI crawlers

User-agent: *
Allow: /*.css
Allow: /*.js
Allow: /*.jpg
Allow: /*.png
Allow: /*.svg
Allow: /*.webp
Allow: /prostore/rss
Disallow: /api/
Disallow: /catalog/auth*
Disallow: /catalog/app/ru.vk.store
Disallow: /catalog/search/page-
Disallow: /challenge.html
Disallow: *?errorCode=
Disallow: *?errorText=
Disallow: /developer/api/*
Disallow: /external/help*
Disallow: /external/simple-selection/buttons/config
Disallow: /external/simple-selection/api/product-metrics
Disallow: /help/en/
Disallow: /help/zh/
Disallow: /instruction?
Disallow: /offers/ad_info*
Disallow: /offers/qr
Disallow: */permissions
Disallow: */preload
Disallow: *product-metrics
Disallow: */rating
Disallow: /set_cookie/*
Disallow: /tracking/
Disallow: */versions

Excerpt: only groups for all user-agents (*) or named AI crawlers, up to 40 rules per group.

Sites with a similar AI policy

• usercentrics.eu · 0 blocked
• superhosting.bg · 0 blocked
• lge.com · 0 blocked
• lanacion.com.ar · 0 blocked
• gofundme.com · 0 blocked
• auth0.com · 0 blocked
• finn.no · 0 blocked
• huggingface.co · 0 blocked

All sites that are “open to ai crawlers” →