Your page can look finished to you and still arrive broken to the machines that decide whether you get found. That’s the premise of this season of Get Discovered.
In our latest episode, host Joe Walsh sat down with someone who has spent the year inside that problem: Venessa Chege, Senior Product Manager at Prerender and product owner of AI Insights.
It started with a sentence customers kept repeating: “I’m disappearing from search.” They’d typed a prompt into ChatGPT, expected to show up, and didn’t. Their competitors did. As Venessa explains, “search” no longer just means Google. It means AI Overviews, ChatGPT, Claude, amongst others, and most teams had no idea how to show up there.
First, what is rendering?
Tell people you work at Prerender and the follow-up is always: what’s rendering? Venessa borrows from architecture: before a house is built, you have blueprints, measurements and specs. Combine those instructions and you get the finished picture or render.
Web pages work the same way. Content, styling, reviews and product data are all instructions waiting to be assembled. Rendering executes them and produces the final version a machine can actually read. Skip that step, and the machine gets the blueprints, not the house.
Why we didn’t build an AI prompt-tracking tool
The obvious answer to “am I visible in ChatGPT?” is a prompt-tracking tool. So that’s what the team prototyped first, after interviewing in-house SEOs, agency leads, ecommerce teams and even CFOs. Three worries kept surfacing: we showed up once, then vanished. Competitors are getting surfaced instead. And even when we’re missing, we don’t know what to fix.
Customers loved the prototype. Then technical discovery began. Prompt tracking meant choosing between API calls and UI scraping, each with real trade-offs, in a market already crowded with tools doing the same thing.
Meanwhile, Prerender already held something rarer: raw crawl logs showing every request machines make to a site.
Machine access is the floor, not the party
If the machine that decides your visibility can’t reach your page, nothing else matters. Not your keywords, not your content structure, not your FAQ section. The front door is shut.
Joe is careful about the limits of that claim. A closed door means your own pages can’t be cited, though third-party mentions are a different story. But the reverse isn’t proven: nobody has shown that fixing access gets you cited more. Access puts you in the running. It doesn’t win the race.
For ecommerce, that distinction is costly. A shopper asks an AI assistant for running shoes and wants a direct link to buy. If your product page fails for that crawler, you might still be mentioned. But the link that drives the sale is gone.
Half the sites checked were blocking machine access
The episode’s most striking number: about half of the sites that ran diagnostics in AI Insights had at least one major AI crawler blocked. Most didn’t know.
The culprit is rarely a strategy decision. It’s usually a firewall or CDN rule set years ago to keep out unwanted traffic and never revisited, while CDN providers keep adding new blocking defaults. Blocking can be a valid choice. A publisher protecting content from AI training may happily accept the trade-off. The point is that it should be a choice.
The telling detail is that many sites unblocked access the same day they saw their results.
How AI agents raise the stakes for machine access
Today, AI platforms mostly read your content. Next, they’ll act on it. Google’s AI Mode already handles accommodation booking inside the platform, and bookings through Claude using an MCP server are already happening, with no website involved.
Neither expects fully delegated buying tomorrow. But when research, comparison and purchase all happen inside one AI platform, a closed gate removes you from the entire buyer journey. Start tracking access now, and you’ll have the data to see how site changes affect it before the stakes rise.
The takeaway
Venessa’s advice is blunt. Before you add another FAQ block because a LinkedIn post said so, confirm the gate is open. Access is binary, allowed or not, which makes it the easiest place to start.
Check your server logs. See which crawlers are actually arriving. Audit your CDN and firewall rules. Then optimize.
AI Insights is included with Prerender at no extra cost, and it’s available from day one of your free trial.
Tune in to the full conversation
Listen to the full episode of Get Discovered to hear how it was built, and follow Venessa on LinkedIn for more on AI visibility. Subscribe across YouTube, Spotify, and Apple Podcasts so you don’t miss any conversation on the fundamentals of online discovery and machine experience.
About AI Insights
AI Insights from Prerender.io shows you whether AI crawlers can actually reach your pages. Firewall rules, CDN defaults and outdated robots.txt files often block them without anyone noticing. Pages these crawlers can’t reach don’t show up in AI search. AI Insights gives you a daily access health status for major AI crawlers. It also shows which pages are affected, ranked by priority, with step-by-step fixes and an exportable report for your dev team. It’s included on current Prerender.io plans at no extra cost. See AI Insights in action. Understand AI Insights in your Prerender.io dashboard.
