Cloudflare has started with OpenAI a pilot research project to test whether data from its network can make AI-based search more accurate. The company, which is located in front of over a fifth of the webobserve in real time how the pages change and how the traffic behaves: the test conveys these signals, such as the freshness of the contents and the quality of the traffic, into the OpenAI search and response system.
The declared objective concerns the content discovery. The responses generated by the models are only as good as the pages that feed them: when the index lags behind, the chatbot returns old or incorrect information. OpenAI aims to leverage Cloudflare’s live view of the web to help ChatGPT discover new content more quickly.
“Having up-to-date information is important to provide accurate responses to people who use ChatGPT,” he said Nick RyderOpenAI’s VP of research, framing the trial as a way to discover content more efficiently.
The pilot moves on different terrain from the content agreements already signed by OpenAI to power ChatGPT: beyond the licenses, what is at stake here is the discoverythat is, how and how quickly new material is reached.
A reversal from last year
The move appears to be in dissonance with Cloudflare’s recent path. Over the past year, the company has allowed sites to block AI crawlers unless companies pay publishers, and has built a protocol to distinguish humans from bots. The slogan was clear: your contents, your rules.
This project goes in the opposite direction. Instead of blocking access to content, Cloudflare now helps an artificial intelligence engine reach it. The CEO Matthew Prince he presented it as a matter of efficiency, a way to “make AI research more efficient and help people get quality answers faster.”
The search continues to move from links to answers. Publishers already fear losing traffic as chatbots summarize their work, a concern that has prompted regulators to impose opt-out mechanisms. Cloudflare now finds itself on both sides: selling the tools to block the crawlers and experimenting with the rig to power them.
