Claude Code
Cursor
Codex
WindsurfThe SEO crawler
built for the AI era
Crawl, audit and fix SEO issues - without leaving your terminal.
Claude Code v2.1.81
Sonnet 4.6 · Claude Team
~/Documents/Websites/Crawly
Features
Everything you need in one crawl
Issues
Know what's broken before Google does
- 19 issue types detected automatically - nothing to configure
- Non-indexable pages with the exact reason explained
- Broken links, redirect chains, missing titles, duplicate H1s, mixed content
Pages
Your entire site. One table.
- Every URL with status code, title, H1 and meta at a glance
- Filter by indexability, depth, word count or response time
- Click any page to see inbound links and full on-page detail
Crawl Comparison
Catch regressions before clients do
- Diff any two crawls side by side
- Field-level changes: title, H1, status code, indexability
- Run post-deploy QA in minutes, not hours
How it works
From URL to insight in minutes
Download Crawly
Free native macOS app. No licence, no page cap, no Java. Works on Apple Silicon and Intel.
Download freeEnter a URL and crawl
Crawly spiders the site, follows every internal link, and records every SEO signal automatically.
Read the guideReview issues and data
19 issue types detected and grouped by severity. No manual filtering - every problem is already categorised.
See all featuresQuery with Claude Code
Connect Crawly's MCP server to Claude Code and ask questions about your crawl data in plain English.
MCP setup guideStop paying for an SEO crawler.
Every major SEO crawler charges a yearly licence or monthly subscription. Crawly does the same job - for free, with no page cap.
Free SEO Tools
Free tools for every SEO job
No signup, no limits. Use them alongside Crawly or on their own.
Link Analysis
Backlink Checker
See which domains link to any site
DA Checker
Check the authority score of any domain
Bulk DA Checker
Check authority for multiple domains at once
Bulk Backlink Checker
Check backlink profiles for up to 100 domains
Spam Score Checker
Identify potentially harmful backlink domains
Authority Comparison
Compare domain authority side by side
Toxic Backlink Checker
Find toxic domains and export a disavow file
Disavow File Generator
Generate a Google disavow file from any domain list
Technical SEO
HTTP Status Checker
Check status codes for any URL instantly
Site Crawler
Crawl and audit any website online
Redirect Checker
Trace redirect chains and spot loops
Canonical Checker
Check canonical tags on any page
Robots.txt Generator
Build a robots.txt file for your site
hreflang Generator
Generate hreflang tags for international SEO
On-Page SEO
Learn
SEO guides
Plain-English guides to technical SEO, crawling, and everything in between.
What is Technical SEO? The Complete Guide
Technical SEO is the foundation every other SEO effort builds on. Here is what it covers, why it matters, and how to audit it properly.
What is SEO Spider Software? (And How to Use It)
SEO spider software crawls your site like Googlebot, recording titles, status codes, and issues across every page. Here is how it works and what to look for.
What Is Crawl Budget and Why Does It Matter?
Crawl budget determines how much of your site Google actually crawls. Here is when it matters, what wastes it, and how to audit it.
How to Find (and Fix) Broken Links on Any Website
Broken links waste crawl budget, harm user experience, and lose link equity. Here is how to find them all and track down which pages are causing the problem.
What is On-Page SEO? The Complete Guide
On-page SEO covers title tags, headings, content, images, and schema. Here is what each one does and how to audit them.
How to Crawl Any Website with Claude Code
Crawly's built-in MCP server connects your crawl data directly to Claude Code. Here is how to set it up and what you can ask it.
Start crawling smarter
Download Crawly for free. Connect to Claude Code via MCP and start auditing your site in minutes.
Download freemacOS · Apple Silicon & Intel · v1.5.3
Compare
How Crawly compares
The only free, native Mac crawler that works with Claude Code, Codex, Cursor and more.
| Tool | Price | Native Mac | Page limit | MCP | Crawl compare |
|---|---|---|---|---|---|
| FREE | UNLIMITED | |||
| Free | 500 URLs | ||||
| £199 / yr | Unlimited | ||||
| £13.50 / mo | Unlimited | ||||
| Free | Unlimited | ||||
| Free | Unlimited | ||||
| Free | Unlimited | ||||
| Free | Unlimited |
Also comparing? Crawly vs Screaming Frog · Crawly vs Sitebulb · Crawly vs Ahrefs · Crawly vs Semrush · Crawly vs Sitechecker · Crawly vs Seobility
Looking for a specific use case? SEO Tool for Agencies · SEO Tool for Developers · SEO Audit Tool