Search engines are great when they bring free audience to a site through organic search results. But some search spiders are simply terrible.
They are terrible because they come from a product that doesn’t provide any audience.
They are especially terrible when they pour through a site, consume vast amounts of bandwidth and come back again for more, ever hungry.
SEO might stand for not just Search Engine Optimization but also Search Engine Opposition.
Obviously every site wants to attract the most audience possible from Google, Bing and other good search engines.
But sometimes blocking search engines is a good thing. (more…)