ques:Under what circumstances would you look to exclude pages from search engines using robots.txt vs meta robots tag?
----------------------------------- Its clear. When you want some pages of your website not to be indexed by search engines you use robotx.txt for preventing them from crawling those pages. Robots.txt is the way to go. That's what google recommends. Quote: Originally Posted by Mobile Development
ques:Under what circumstances would you look to exclude pages from search engines using robots.txt vs meta robots tag?
----------------------------------- Hi,
I think Robots meta tag would usually be applicable to page level while robots.txt would be used at catalog or site level and must appear at the root directory of your sites.
all the best, i think robots.txt is correct way... Using meta tags is also ok. But that way Google will crawl your site but not index it.
And using robots.txt Google will keep away from the page and not even crawl it.
I normally use meta tags , But Google suggests to use robotx.txt using robots.txt is best way
----------------------------------- Its clear. When you want some pages of your website not to be indexed by search engines you use robotx.txt for preventing them from crawling those pages. Robots.txt is the way to go. That's what google recommends. Quote: Originally Posted by Mobile Development
ques:Under what circumstances would you look to exclude pages from search engines using robots.txt vs meta robots tag?----------------------------------- Hi,
I think Robots meta tag would usually be applicable to page level while robots.txt would be used at catalog or site level and must appear at the root directory of your sites.
all the best, i think robots.txt is correct way... Using meta tags is also ok. But that way Google will crawl your site but not index it.
And using robots.txt Google will keep away from the page and not even crawl it.
I normally use meta tags , But Google suggests to use robotx.txt using robots.txt is best way