Are you stopping msn from crawling your website

What’s new from msn, are you stopping msn from crawling your website.

Msn and yahoo and google all at one place. Is it so yes for what ?

Even though i am posting it late but it doesn’t mean it value less.For making better compatibility with results and to deliver the more and related search results.

As we all know that we use the Robots Exclusion Protocol (REP) to control how search engines access and display their content including the archive,index and other matters.

The REP offers an easy and efficient way to communicate with search engines, and is currently used by millions of publishers worldwide.They say that they have working with yahoo and google just to make sure that there exists a common and relaible way through which webmasters can submit their content to them.

What are the spiders of the msn which i should allow/include in robots.txt

The complete list is as follows:

MSNBot Main web crawler (www.live.com)

MSNBot-Media Images & all other media (images.live.com)

MSNBot-NewsBlogs News and blogs (search.live.com/news)

MSNBot-Products Products & shopping (products.live.com)

MSNBot-Academic Academic search (academic.live.com)

Our tech tags: Are you stopping msn from crawling your website, msn seo