cd ../tools
public cyber tool / webmasters / browser-side
Robots.txt Security Review
Robots.txt is public and protects nothing. This tool helps spot signals that may reveal sensitive paths or create SEO issues.
review --robots.txt
local analysis / no remote fetch
review --robots.txt
Core rule
Do not use robots.txt to hide private areas: anyone can read it.
Protect admin, backup, staging and sensitive files with authentication, server permissions and correct configuration.
Keep robots.txt simple: crawler guidance, sitemap and only the blocks that are really necessary.
references