Crawl Analyzer
Paths
URLs
Dead-ends
Frequency
Orphans
Budget
Verify
Tools
Session
WordPress-Pingback · AutomaticBarrier · Apr 16 03:09 · 15 hits
Scrape pages
Crawl path
⊕ Fit
2xx OK
3xx Redirect
4xx Error
5xx Server error
— edge = referer · dashed = no referer
? what do these mean
2xx OK
Page was successfully fetched. Bot read the content and may have followed links from it.
200
Standard success — full content returned.
304
Not Modified — bot already has this page cached, server skipped sending content. Saves bandwidth but counts as a crawl hit.
Graph edges & nodes
— solid edge
— URL was reached via a referer link from another page in this session.
- - dashed edge
— No referer. Bot fetched this URL from its own crawl queue, not via a link.
dashed border node
— Entry point. No other page in this session linked to it.
grey node
— External referer. Not part of this session — it's the page that triggered the crawl.
3xx Redirect
Bot was redirected to another URL. Each redirect costs one crawl budget hit.
301
Permanent — passes link equity to destination. Use for page moves.
302
Temporary — does not reliably pass link equity. Avoid for permanent moves.
307
Temporary (method-safe) — same SEO implications as 302.
308
Permanent (method-safe) — like 301 but preserves POST method.
4xx Error
Bot hit a dead end — no content returned, no links followed. Wastes crawl budget.
400
Bad Request — malformed URL or request.
401
Unauthorised — login required, bot can't access.
403
Forbidden — server blocked the bot (firewall, IP, robots).
404
Not Found — page doesn't exist. High volume = broken internal links.
410
Gone — page permanently removed. Tells Google to deindex faster than 404.
429
Too Many Requests — bot rate-limited. May reduce crawl frequency.
5xx Server error
Server failed to respond. Bot may retry but will deprioritise the site if errors persist.
500
Internal Server Error — unhandled exception on the server.
502
Bad Gateway — upstream server (e.g. PHP-FPM) returned an invalid response.
503
Service Unavailable — server overloaded or in maintenance mode.
504
Gateway Timeout — upstream server took too long to respond.
Crawl Sequence
[500]
/wp-cron.php?doing_wp_cron=1776337763.5017750263214111328125
+0s
[500]
/wp-cron.php?doing_wp_cron=1776338022.2510249614715576171875
+259s
[500]
/wp-cron.php?doing_wp_cron=1776339086.3769469261169433593750
+1323s
[500]
/wp-cron.php?doing_wp_cron=1776339383.3197441101074218750000
+1620s
[500]
/wp-cron.php?doing_wp_cron=1776339535.1772119998931884765625
+1772s
[500]
/wp-cron.php?doing_wp_cron=1776339686.3360929489135742187500
+1923s
[500]
/wp-cron.php?doing_wp_cron=1776339873.9252328872680664062500
+2110s
[500]
/wp-cron.php?doing_wp_cron=1776340286.9635970592498779296875
+2524s
[500]
/wp-cron.php?doing_wp_cron=1776340887.1530950069427490234375
+3124s
[500]
/wp-cron.php?doing_wp_cron=1776341170.8406100273132324218750
+3407s
[500]
/wp-cron.php?doing_wp_cron=1776341487.5220189094543457031250
+3724s
[500]
/wp-cron.php?doing_wp_cron=1776342133.2331769466400146484375
+4370s
[500]
/wp-cron.php?doing_wp_cron=1776343141.5732390880584716796875
+5378s
[500]
/wp-cron.php?doing_wp_cron=1776343586.3685309886932373046875
+5823s
[500]
/wp-cron.php?doing_wp_cron=1776344058.5412878990173339843750
+6295s
0s
6295s
Link attributes
Click a node in the graph to see its outgoing links
This page hasn't been scraped yet — no link data available.
Run scraper →
▶ show
#
Target URL
Text
Role
Placement
Rel
↗