Crawler.sh

Local AEO and SEO spider with markdown content extraction for technical SEO.

Crawl any website in seconds. Find SEO issues, extract clean content, and export to JSON, Sitemap XML, or CSV — from the terminal or a desktop app.