Ceantyintedge
New Member
Hey..Where can I tjek how often I get crawled ?..Can I change my META code, to get crawled more often, and will a robot.txt help me ?.. If so, then how do I make a robot.txt, that helps me ?..robot.txt doesn't help you get crawled more frequently. You use robot.txt to deny unwanted bots access to your site or deny any/all bots access to specific files/directories.You can't control how frequently you get crawled. Don't waste your time trying to figure it out.Frequent upates and more inbound links increases the crawl frequency.