does anyone have a tutorial on how to setup a robots.txt file onto your server...Here is a good one. http://www.searchengineworld.com/robots ... torial.htmIf you can specify your exact needs, then i can provide content for your rebots.txtGenerally, you create file with .txt extension (user notepad or ms words to do that), you name file 'robots', then you write rules for search engine spiders and then you upload it to the root of your server.Yep. That's the one I learned from. Make sure you upload it in your root.Hello,Thanks guys. Can you just give me some hint of "Rules".thank youNickyHello,I have a page called error.aspx i do not want bot to visit it so can i write something like this? User-agent: *Disallow: error.aspx At the same time i want it to visit my sitemap.xml and other pages of my site.Is this ok guys?Leme knowChouHi Shrek,Check if this is correct http://www.jnbsilver.com/robots.txtchouIt should beUser-agent: * Disallow: /error.aspx/Slash is very important!
Robots.txt
- Thread starter holtbrass
- Start date