Crawl Analyzer
Paths
URLs
Dead-ends
Frequency
Orphans
Budget
Verify
Tools
Session
WordPress-Pingback · AutomaticBarrier · Apr 14 17:26 · 20 hits
Scrape pages
Crawl path
⊕ Fit
2xx OK
3xx Redirect
4xx Error
5xx Server error
— edge = referer · dashed = no referer
? what do these mean
2xx OK
Page was successfully fetched. Bot read the content and may have followed links from it.
200
Standard success — full content returned.
304
Not Modified — bot already has this page cached, server skipped sending content. Saves bandwidth but counts as a crawl hit.
Graph edges & nodes
— solid edge
— URL was reached via a referer link from another page in this session.
- - dashed edge
— No referer. Bot fetched this URL from its own crawl queue, not via a link.
dashed border node
— Entry point. No other page in this session linked to it.
grey node
— External referer. Not part of this session — it's the page that triggered the crawl.
3xx Redirect
Bot was redirected to another URL. Each redirect costs one crawl budget hit.
301
Permanent — passes link equity to destination. Use for page moves.
302
Temporary — does not reliably pass link equity. Avoid for permanent moves.
307
Temporary (method-safe) — same SEO implications as 302.
308
Permanent (method-safe) — like 301 but preserves POST method.
4xx Error
Bot hit a dead end — no content returned, no links followed. Wastes crawl budget.
400
Bad Request — malformed URL or request.
401
Unauthorised — login required, bot can't access.
403
Forbidden — server blocked the bot (firewall, IP, robots).
404
Not Found — page doesn't exist. High volume = broken internal links.
410
Gone — page permanently removed. Tells Google to deindex faster than 404.
429
Too Many Requests — bot rate-limited. May reduce crawl frequency.
5xx Server error
Server failed to respond. Bot may retry but will deprioritise the site if errors persist.
500
Internal Server Error — unhandled exception on the server.
502
Bad Gateway — upstream server (e.g. PHP-FPM) returned an invalid response.
503
Service Unavailable — server overloaded or in maintenance mode.
504
Gateway Timeout — upstream server took too long to respond.
Crawl Sequence
[500]
/wp-cron.php?doing_wp_cron=1776216385.7479801177978515625000
+0s
[500]
/wp-cron.php?doing_wp_cron=1776217391.1505939960479736328125
+1006s
[500]
/wp-cron.php?doing_wp_cron=1776218240.3979580402374267578125
+1855s
[500]
/wp-cron.php?doing_wp_cron=1776219259.0182199478149414062500
+2874s
[500]
/wp-cron.php?doing_wp_cron=1776219338.7558441162109375000000
+2953s
[500]
/wp-cron.php?doing_wp_cron=1776219555.9735031127929687500000
+3171s
[500]
/wp-cron.php?doing_wp_cron=1776220964.8038320541381835937500
+4579s
[500]
/wp-cron.php?doing_wp_cron=1776222685.7072570323944091796875
+6300s
[500]
/wp-cron.php?doing_wp_cron=1776223565.1281960010528564453125
+7180s
[500]
/wp-cron.php?doing_wp_cron=1776224358.4244930744171142578125
+7973s
[500]
/wp-cron.php?doing_wp_cron=1776225036.0800559520721435546875
+8651s
[500]
/wp-cron.php?doing_wp_cron=1776225121.4708180427551269531250
+8736s
[500]
/wp-cron.php?doing_wp_cron=1776225537.1376121044158935546875
+9152s
[500]
/wp-cron.php?doing_wp_cron=1776226161.0995540618896484375000
+9776s
[500]
/wp-cron.php?doing_wp_cron=1776226376.4103980064392089843750
+9991s
[500]
/wp-cron.php?doing_wp_cron=1776226492.8331789970397949218750
+10107s
[500]
/wp-cron.php?doing_wp_cron=1776226572.4492490291595458984375
+10187s
[500]
/wp-cron.php?doing_wp_cron=1776227043.3958818912506103515625
+10658s
[500]
/wp-cron.php?doing_wp_cron=1776227789.0499999523162841796875
+11404s
[500]
/wp-cron.php?doing_wp_cron=1776227908.7946879863739013671875
+11523s
0s
11523s
Link attributes
Click a node in the graph to see its outgoing links
This page hasn't been scraped yet — no link data available.
Run scraper →
▶ show
#
Target URL
Text
Role
Placement
Rel
↗