Recently I discover that some variants of a google proxy visits my sites. I doubt these are legal google crawlers because these crawlers are NOT always behind a proxy (like the hostname describes) and identify itself as a browser. The hostname is formatted similar/like google bot but with the string 'proxy' added to it. My PHP blocking class blocks these crawlers, but is it correct to block these ones? What are they and are these from google or is it fake?Here some info about one of these crawlers:\[code\]BlockedIp Notifier Report - IP:66.249.81.131:: has been blockedTicket ID : {EVNT_136877_2013040520130402_33147_10348} Event type : Access blocked Event date : 04/05/2013 - 19:17:47 (server date-time) Event counter : First occurring Processed url : http://streambutler.net/ From url : http://www.google.com/search Domain : streambutler.net Domain IP : 95.170.70.213 Visitor IP : 66.249.81.131 Proxy IP : 66.249.81.131 Critical : Yes Action required : No Additional informationProblem : Bad Proxy - via 66.249.81.131 Hostname : google-proxy-66-249-81-131.google.com Block : Yes Refferer : http://www.google.com/search AgentString : Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.4 (KHTML, like G... Browser : Chrome 22.0.1229 Platform : Linux Robot : No Mobile : No Tablet : No Console : No Crawler : No Agent_type : browser Agent_name : chrome Agent_version : 22.0.1229 Os_type : linux Os_name : linux Agent_languagetag : en Status : ok Request : 66.249.81.131 Languagecode : us Country : United States Region : California City : Mountain View Zipcode : 94043 Latitude : 37.406 Longitude : -122.079 Timezone : -07:00 Available from : \'http Areacode : 0 Dmacode : 0 Continentcode : na Currencycode : USD Currencysymbol : $ Currencysymbol_utf8 : $ Currencyconverter : 1 Extended : 1 Organization : NULL \[/code\]other variants found
- google-proxy-66-249-81-131.google.com (identifies itself as Firefox6.0 ???)
- google-proxy-66-249-81-148.google.com (tries to access a javascript file)
- google-proxy-66-249-81-131.google.com
- google-proxy-66-249-81-111.google.com (tries to access a javascriptfile)
- google-proxy-66-249-81-164.google.com