Tag: robots.txt

Robots.txt Guide for Popular CMS and Shopping Carts

Posted by Mark Thompson - January 21, 2010 - SEO

It can be difficult to know which directories should be restricted from search engines and which should be allowed.  I thought it would be a good idea to create a robots.txt reference guide for popular content management systems and shopping carts.  Depending on what backend you are using, all you have to do is copy […]

Read More