http://barrysaunders.com/2009/11/short-thoughts-robots-txt-and-paywalls/?utm_source=rss&utm_medium=rss&utm_campaign=short-thoughts-robots-txt-and-paywalls For every complex problem, there is a solution that is simple, neat, and wrong. Sat, 04 Feb 2012 15:36:38 +0000 hourly 1 http://wordpress.org/?v=3.3.1 http://barrysaunders.com/2009/11/short-thoughts-robots-txt-and-paywalls/#comment-48 Custom Motorcycle Mon, 29 Mar 2010 03:40:40 +0000 http://barrysaunders.com/?p=281#comment-48 It's all a move to get another search engine (ahem... Bing) to bid on the rights to list them exclusively. It’s all a move to get another search engine (ahem… Bing) to bid on the rights to list them exclusively.

]]> http://barrysaunders.com/2009/11/short-thoughts-robots-txt-and-paywalls/#comment-47 tigtog Wed, 11 Nov 2009 02:28:32 +0000 http://barrysaunders.com/?p=281#comment-47 That's a very good thought. Not all search engines respect robots.txt restrictions, but I've never heard that Google doesn't abide by them. That’s a very good thought. Not all search engines respect robots.txt restrictions, but I’ve never heard that Google doesn’t abide by them.

]]>