Headers and Screen Scrapes

[frosch68]

Hi Scott. You wrote an excellent article on doing screen<BR>scrapes in .NET. This works great, except when there is security involved. The web site I need to connect to requires user-agent and extra header info. I know that this stuff is all there, but I am having a hell of a time deciphering it.<BR><BR>We currently use SmartObjects ASPHTTP to scrape. Our code<BR>looks like this:<BR><BR>HttpObj.UserAgent="AutoSender/1.1"<BR>HttpObj.AddExtraHeader "Cookie:" & MyCookie=ID=" &<BR>Session("MyCookie")<BR><BR>In order to do the same using the ASP.NET scrape, I need<BR>something like this:<BR><BR>objWebClient.Headers.Add("ID", Session("MyCookie"))<BR><BR>I'm not sure if that is right, and I'm not sure how to add<BR>the useragent - I know it exists, I just can't seem to get it working. Can you help? And if so, maybe you can update the article?<BR>