D Decept New Member Sep 16, 2012 #1 i have a home page with some links and mail ids i need to stop scraping my urls and mail-ids form that web page...i have used robots.txt but most of the bad crawlers wont respect that....
i have a home page with some links and mail ids i need to stop scraping my urls and mail-ids form that web page...i have used robots.txt but most of the bad crawlers wont respect that....
