Cloudflare and GoDaddy have announced a strategic partnership aimed at shaping the future of the “agentic web”—an emerging internet model where AI agents actively interact, transact, and retrieve data on behalf of users. The collaboration focuses on giving website owners greater transparency and control over how AI systems access and use their content, while also introducing open standards to establish trust and identity in AI-driven environments.
Enabling Control Over AI Crawlers
As part of this initiative, GoDaddy will integrate Cloudflare’s AI Crawl Control into its hosting platform. This feature allows website owners—including small businesses and independent creators—to manage how AI-powered crawlers interact with their sites. With this integration, users can permit, restrict, or monetize access to their content, ensuring better protection against unauthorized data scraping while maintaining flexibility in how content is shared.
Introducing Identity Standards for AI Agents
A major challenge in the evolving AI ecosystem is verifying the identity and intent of automated agents. To address this, GoDaddy has introduced the Agent Name Service (ANS), an open standard designed to provide consistent naming, authentication, and discovery of AI agents. Built on established technologies like DNS and public key infrastructure (PKI), ANS enables website owners to distinguish between legitimate AI agents and potentially malicious or spoofed entities.
Cloudflare complements this effort with its Web Bot Auth system, which uses cryptographic verification to authenticate bot and agent traffic. Additionally, its Signature Agent Card framework allows developers to transparently disclose an agent’s identity and purpose, further strengthening accountability across the web.
Building Trust in the AI-Driven Internet
Together, these technologies aim to establish a secure and transparent foundation for the next phase of the internet. The partnership focuses on three key pillars:
- Permission-based access: Website owners can define clear rules for AI interaction, including blocking, allowing, or requiring compensation for data usage.
- Trusted AI commerce: Verified agent identities enable secure transactions, allowing AI systems to perform tasks like data requests or purchases with cryptographic assurance.
- Fair value exchange: As AI-generated answers replace traditional search traffic, these tools help ensure content creators retain control and potential revenue from their work.
Supporting the Future of the Open Web
Industry leaders emphasize that as the internet transitions from human-centric browsing to AI-driven interactions, new infrastructure is essential to maintain trust and sustainability. By combining AI Crawl Control, ANS, and Web Bot Auth, Cloudflare and GoDaddy are laying the groundwork for a more transparent and accountable digital ecosystem.
This partnership reflects a broader shift toward an AI-first web, where identity, verification, and consent are central to how data is accessed and used. For developers, businesses, and creators, these advancements offer critical tools to navigate and thrive in the rapidly evolving landscape of AI-powered internet services.
