I Crawled 65,000 Pages of My Own Site Without Parsing a Single Sitemap
Somewhere between talks on day one, I think it was during a hallway chat that I decided to run an aggressive crawl experiment using direct link scraping and BFS.
SEO Architect and developer building pipelines that bridge search, large language models, and web crawlers. Trusted by 100+ brands and startups worldwide.
Autonomous CLI and Streamlit dashboard to audit visual layout, redirect loops, and Core Web Vitals.
LLM-driven browser agents that perform automated website QA, layout regression testing, and security scans.
Model Context Protocol (MCP) server allowing LLMs to run semantic clustering, queries, and GSC audits.
Model Context Protocol connection server designed to integrate Adobe Analytics queries with AI assistants.
Somewhere between talks on day one, I think it was during a hallway chat that I decided to run an aggressive crawl experiment using direct link scraping and BFS.
I took Meta AI's TRIBE v2 model and turned it into an automated image-to-intent classifier, enabling highly precise visual SEO auditing.
I had a simple question: what if I could talk directly to my search metrics? Here is how I loaded 16 months of queries and clicks into ChromaDB.
How AI visibility tools collect data: API-based collection vs real browser-based UI execution. A technical breakdown of their trade-offs.
All 30 AEO and GEO agencies compared side-by-side, evaluating their LLM citation performance, pricing, and optimization strategies.
Step-by-step guide to tracking brand visibility in AI search with specialized agents, evaluating citation shares and brand mentions.