While working on blocking AI crawlers, I added a whole bunch related to SEO spiders. Then it struck me: how come a never blocked SEO crawlers on company websites? It probably impacts using those SEO tools to evaluate your own sites, but it hurts competitors when they are trying to get intel on your sites. Or at a minimum block spiders from services you aren’t using.