Cloudflare Flags Perplexity AI for Using Stealth Crawlers to Bypass Website No-Crawl Directives
Cloudflare has accused Perplexity AI, an AI-powered answer engine, of engaging in stealth web crawling to evade robots.txt no-crawl directives and network blocks. According to...