Session — AutomaticBarrier

Crawl path

2xx OK 3xx Redirect 4xx Error 5xx Server error — edge = referer · dashed = no referer ? what do these mean

2xx OK

Page was successfully fetched. Bot read the content and may have followed links from it.

200 Standard success — full content returned.

304 Not Modified — bot already has this page cached, server skipped sending content. Saves bandwidth but counts as a crawl hit.

Graph edges & nodes

— solid edge — URL was reached via a referer link from another page in this session.

- - dashed edge — No referer. Bot fetched this URL from its own crawl queue, not via a link.

dashed border node — Entry point. No other page in this session linked to it.

grey node — External referer. Not part of this session — it's the page that triggered the crawl.

3xx Redirect

Bot was redirected to another URL. Each redirect costs one crawl budget hit.

301 Permanent — passes link equity to destination. Use for page moves.

302 Temporary — does not reliably pass link equity. Avoid for permanent moves.

307 Temporary (method-safe) — same SEO implications as 302.

308 Permanent (method-safe) — like 301 but preserves POST method.

4xx Error

Bot hit a dead end — no content returned, no links followed. Wastes crawl budget.

400 Bad Request — malformed URL or request.

401 Unauthorised — login required, bot can't access.

403 Forbidden — server blocked the bot (firewall, IP, robots).

404 Not Found — page doesn't exist. High volume = broken internal links.

410 Gone — page permanently removed. Tells Google to deindex faster than 404.

429 Too Many Requests — bot rate-limited. May reduce crawl frequency.

5xx Server error

Server failed to respond. Bot may retry but will deprioritise the site if errors persist.

500 Internal Server Error — unhandled exception on the server.

502 Bad Gateway — upstream server (e.g. PHP-FPM) returned an invalid response.

503 Service Unavailable — server overloaded or in maintenance mode.

504 Gateway Timeout — upstream server took too long to respond.

Crawl Sequence

[500] /wp-cron.php?doing_wp_cron=1776216385.7479801177978515625000

+0s

[500] /wp-cron.php?doing_wp_cron=1776217391.1505939960479736328125

+1006s

[500] /wp-cron.php?doing_wp_cron=1776218240.3979580402374267578125

+1855s

[500] /wp-cron.php?doing_wp_cron=1776219259.0182199478149414062500

+2874s

[500] /wp-cron.php?doing_wp_cron=1776219338.7558441162109375000000

+2953s

[500] /wp-cron.php?doing_wp_cron=1776219555.9735031127929687500000

+3171s

[500] /wp-cron.php?doing_wp_cron=1776220964.8038320541381835937500

+4579s

[500] /wp-cron.php?doing_wp_cron=1776222685.7072570323944091796875

+6300s

[500] /wp-cron.php?doing_wp_cron=1776223565.1281960010528564453125

+7180s

[500] /wp-cron.php?doing_wp_cron=1776224358.4244930744171142578125

+7973s

[500] /wp-cron.php?doing_wp_cron=1776225036.0800559520721435546875

+8651s

[500] /wp-cron.php?doing_wp_cron=1776225121.4708180427551269531250

+8736s

[500] /wp-cron.php?doing_wp_cron=1776225537.1376121044158935546875

+9152s

[500] /wp-cron.php?doing_wp_cron=1776226161.0995540618896484375000

+9776s

[500] /wp-cron.php?doing_wp_cron=1776226376.4103980064392089843750

+9991s

[500] /wp-cron.php?doing_wp_cron=1776226492.8331789970397949218750

+10107s

[500] /wp-cron.php?doing_wp_cron=1776226572.4492490291595458984375

+10187s

[500] /wp-cron.php?doing_wp_cron=1776227043.3958818912506103515625

+10658s

[500] /wp-cron.php?doing_wp_cron=1776227789.0499999523162841796875

+11404s

[500] /wp-cron.php?doing_wp_cron=1776227908.7946879863739013671875

+11523s

0s11523s

Link attributes

Click a node in the graph to see its outgoing links