Bot proxy for dynamic routes.

Serve fresh content to search bots without edge SSR. Prerender covers static routes. The proxy renders dynamic pages on request.

Why proxy?

Build-time misses

Paginated routes, search results, dated content - too many URLs to prerender. Proxy renders on-request.

Freshness

Content changed since last build? Proxy re-renders and caches. Bots see current pages.

Bots only

Humans get static HTML from prerender. Only bots hitting uncached dynamic URLs hit proxy.

How it works

# Bot hits /blog/page/5
# Not prerendered? → Proxy renders

# 1. Detect botUserAgent
# 2. Fetch targetUrl with Puppeteer
# 3. Cache HTML to KV or disk
# 4. Return with fresh meta

# Config in ssr.config.js
proxy: {
  url: 'https://my-proxy.example.com',
  botList: ['googlebot', 'bingbot', 'claudebot'],
  cacheTTL: 3600,  // seconds
}

Implementation

VPS (Node + Puppeteer)

Use scripts/proxy.js. Runs Express with Puppeteer. Works on any VPS with Node 18+.

Cloudflare Worker

Use scripts/proxy.worker.js with Browser Rendering API. Requires Workers Paid plan.

Cache refresh

Send X-Prestruct-Refresh header with secret to bust cache for a path.

Enable proxy

ssr.config.js

export default {
  routes: [...],
  proxy: {
    url: 'https://your-proxy.workers.dev',
    // or null to disable
    botList: ['googlebot', 'bingbot', 'claudebot', 'perplexitybot'],
    targetUrl: process.env.PRESTRUCT_TARGET_URL,
  },
}