Web Scraping to Database is a data workflow that chains Firecrawl + Supabase to automate a common task. Schedule a Firecrawl scrape of any website and store the structured results directly in a Supabase table for analysis. Once configured, it saves ~12 hours/week of manual competitive research, plus elimination of brittle custom scrapers and runs through Claude Code, Cursor, Windsurf or any MCP-compatible AI agent.
Schedule a Firecrawl scrape of any website and store the structured results directly in a Supabase table for analysis.
Execute this recipe in your browser — no local install, no Claude Code. Streams results live.
Firecrawl handles the hard parts of scraping — JS rendering, pagination, rate limiting — and returns clean structured data. Supabase gives you a queryable database to accumulate that data over time. Together they replace a brittle custom scraper + manual CSV import workflow.
Write a custom scraper that breaks every time the site updates, export CSV, import into a database manually, fix encoding issues.
Configure Firecrawl once, data flows into Supabase on schedule. Query it with SQL immediately.
Concrete ROI — not marketing fluff.
Time saved
~12 hours/week of manual competitive research, plus elimination of brittle custom scrapers
This prompt is the workflow. Paste into Claude Code, Cursor, or Windsurf.
You are a scraping-to-database agent. Runs on a schedule defined in scrape-targets.yaml.
For each target (url, schema, supabase_table):
1. Call firecrawl.scrape(url=target.url, formats=["json"], json_schema=target.schema, render_js=true) to get structured rows
2. For each row, compute a stable hash(url + primary_key) to use as upsert key
3. Call supabase.execute_sql with parameterized UPSERT:
INSERT INTO ${target.supabase_table} (...) VALUES (...) ON CONFLICT (hash) DO UPDATE SET ...
4. Track diff count: rows_inserted, rows_updated, rows_unchanged
5. If rows_updated > 0, call supabase.execute_sql to insert a changelog row in scrape_log
On rate-limit or 5xx from Firecrawl, retry with exponential backoff (3 attempts). Report row counts only.How this workflow fires and what env vars you need.
0 */6 * * * # every 6 hours
Install everything — MCPs, prompt, env template — in a single call.
$ mcpizy recipe install firecrawl-supabase-scraping ✓ Installs all 2 MCP servers ✓ Writes prompt to ~/.mcpizy/prompts/firecrawl-supabase-scraping.md ✓ Generates .env.example in current directory ✓ Ready to paste into Claude Code
Requires mcpizy CLI v1.1+ — install via npm i -g mcpizy.
$ mcpizy install firecrawl && mcpizy install supabaseRun Tavily searches on scheduled topics and index the results in Supabase for trend analysis and content research.
When a Supabase row changes, the corresponding Redis cache key is automatically invalidated to keep your API fresh.
Parse your GitHub repos and build a Neo4j knowledge graph of files, functions, imports, and authors for code intelligence.
Query Parquet files directly from S3 using DuckDB without any ETL. Results are returned in seconds for ad-hoc analytics.
Web Scraping to Database is a data automation that uses Firecrawl + Supabase together via the Model Context Protocol. Schedule a Firecrawl scrape of any website and store the structured results directly in a Supabase table for analysis.
Setup takes around 15 min setup, continuous data collection. You install the required MCP servers with `mcpizy install firecrawl && mcpizy install supabase`, connect your accounts, and the workflow is ready to run.
Once running, this workflow saves ~12 hours/week of manual competitive research, plus elimination of brittle custom scrapers. The concrete business value: Replaces a full-time data engineer maintaining scrapers ($120-180k/year) with a declarative schema; Pricing intelligence updates daily — catch competitor price drops and reprice within hours, not weeks.
You need 2 MCP servers: Firecrawl (mcpizy install firecrawl), Supabase (mcpizy install supabase). All are installable in one command via the MCPizy CLI and configured in your `.claude.json` or `.cursor/mcp.json`.
Yes. The workflow runs with any MCP-compatible AI agent — Claude Code, Claude Desktop, Cursor, Windsurf, VS Code with Copilot, and custom agents built on the MCP SDK. The MCP servers are identical across clients; only the config file path (`.claude.json` vs `.cursor/mcp.json`) changes.
Install the required MCPs from the marketplace and automate this in 15 min setup.
$ mcpizy install firecrawl && mcpizy install supabase
Free to install. Connect your accounts and this workflow runs itself.