Back to portfolio

Production web extraction

AI Web Scraping Agent

A URL plus a sentence becomes validated JSON. With actual security.

Live demo Source

Demo limits

  • ·5 scrapes per session, 10 requests per minute, 5 concurrent browsers
  • ·200 calls per UTC day on the shared key. Paste your own and the cap does not apply
  • ·Kill switch that stops paid calls without a redeploy

A public AI demo is a spending endpoint. Cost control is part of what I ship.

Problem

Most "AI scraper" demos are 30 lines of code that any client could run themselves. The hard parts (SSRF defense, DNS rebinding, prompt injection from scraped content, per-key cache scoping, BYOK across providers) are exactly the parts demos skip.

Approach

  • →SSRF v2: IDNA + IPv6 unwrap + IP allowlist computed before the request, re-checked on every subresource via Playwright route hooks (DNS-rebinding guard).
  • →Untrusted scraped content is wrapped in a system-prompt cage before reaching the LLM, prompt injection isolation.
  • →BYOK across 6 providers (Groq, OpenAI, Anthropic, DeepSeek, Google Gemini, xAI Grok), 16 models. Showcase mode uses a shared free Groq key with per-IP rate limits.
  • →SQLite cache keyed by sha256(url + actions + prompt + model + sha256(api_key)): scoped per user, no cross-tenant leak.
  • →Backend lives on an internal-only Docker bridge; only the Next.js frontend can reach it.

Stack

FastAPINext.js 16Playwright Chromium (stealth)BeautifulSoupSQLiteTraefik