Summary:
- Fetcher bots play a significant role in retrieving real-time content for AI assistants, with OpenAI’s ChatGPT leading in generating fetcher bot requests.
- AI agents generate higher traffic volumes per interaction than browser-based users, with Fastly’s network handling this traffic for content delivery.
- Fastly is witnessing AI workloads running directly on its edge compute platform, with a shift in customer strategy towards optimizing AI agent traffic rather than blocking it.
The use of fetcher bots, AI agents, and edge computing is revolutionizing the way content is retrieved and processed in real time. OpenAI’s ChatGPT and related bots are at the forefront of this transformation, generating a significant portion of fetcher bot requests. These bots are capable of generating a high volume of requests per minute to individual sites, showcasing their efficiency in retrieving content for users interacting with AI assistants.
In addition to fetcher bots, AI agents are also playing a crucial role in processing queries by checking multiple websites, resulting in higher traffic volumes per interaction compared to browser-based users. Fastly’s network is instrumental in handling this increased traffic for customers using the platform for content delivery, ensuring a seamless experience for users accessing content through AI agents.
Moreover, Fastly is witnessing a shift in customer strategy towards optimizing AI agent traffic rather than blocking it. Large media companies are now focusing on how to make their content more relevant to AI models and chatbots, while also managing access and enforcing content licensing agreements. Fastly has developed AI bot mitigation technology to address this need, allowing customers to permit beneficial bots while blocking harmful ones. Additionally, the company is one of the first edge providers to support the Really Simple Licensing (RSL) protocol, working with large media customers to refine its implementation at the edge. This protocol aims to enforce content rights agreements related to AI models, reflecting the evolving landscape of AI and edge computing in content delivery.
