Comments on: Robots.txt Guide for Popular CMS and Shopping Carts

By: Setting Up WordPress for SEO - Wordpress Tutorials

Setting Up WordPress for SEO - Wordpress Tutorials — Mon, 05 Sep 2011 11:18:04 +0000

[...] thаt уου ѕhουld block frοm search engines. Check out thе Robots.txt guide I wrote thаt wіll ѕhοw уου exactly whісh directories [...]

By: Mark Thompson

Mark Thompson — Wed, 08 Jun 2011 14:06:43 +0000

Good question.

Sorry, the [plain] is just a formatting issue with the page…please disregard that.

No you don't need to add Allow: /

I would add the no slash and slash, as search engines treat each of those differently.

Thanks for the questions
Mark

By: Jay K

Jay K — Wed, 08 Jun 2011 07:11:10 +0000

WordPress questions:

1) What is the [plain] on the first line for?

2) do you need to add Allow: /?

3) do you need to duplicate the list for both no slash or a slash "/"?

By: Setting Up WordPress for SEO - Bali Web Design Tips | Best Web Design Bali Blog- The Butterfly Web Design Bali

Mon, 06 Jun 2011 02:49:02 +0000

[...] are certain directories that you should block from search engines. Check out the Robots.txt guide I wrote that will show you exactly which directories you should [...]

By: Brock

Brock — Wed, 08 Dec 2010 21:16:31 +0000

Thank you, Magento has blocked out a lot of there forums from the public. What is [Plain][/Plain]?

By: cochinshop

cochinshop — Thu, 28 Oct 2010 09:44:14 +0000

Nice Posts Thanks

By: Setting Up Wordpress for SEO | My Free Design Area - A Webpage for Joomla Free & Commercial Templates, Wordpress Free & Commerciak Themes, and Drupla Free and Commercial templates, Flash Templates and animation, Html templates

Tue, 06 Jul 2010 07:36:17 +0000

[...] are certain directories that you should block from search engines. Check out the Robots.txt guide I wrote that will show you exactly which directories you should [...]

By: moncler outlet store

moncler outlet store — Wed, 09 Jun 2010 04:22:06 +0000

Here elaborates the matter not only extensively but also detailly .I support the write's unique point.It is useful and benefit to your daily life.You can go those http://conditions-encountered.com/ " >wslmart.net sits to know more relate things.They are strongly recommended by friends.Personally

By: Setting Up Wordpress for SEO

Setting Up Wordpress for SEO — Sun, 28 Feb 2010 01:40:50 +0000

[...] are certain directories that you should block from search engines. Check out the Robots.txt guide I wrote that will show you exactly which directories you should [...]

By: Mark Thompson

Mark Thompson — Tue, 16 Feb 2010 22:32:03 +0000

It is best to block both the non / and the /. This will ensure that the search engines do not index the directories, given that they treat the non and slash versions as two separate urls.