C# - Crawling XML RESTful webservice

pacolo

New Member
I need to get a lot of xml data from a webservice.I send async requests using Task like this snippet\[code\]LimitedConcurrencyLevelTaskScheduler lcts = new LimitedConcurrencyLevelTaskScheduler(10);TaskFactory factory = new TaskFactory(lcts);List<Task> tasks = new List<Task>();...tasks.Add(factory.StartNew(() => GetRecords(country, names)));...Task.WaitAll(tasks.ToArray());\[/code\]The webservice has a lot of work to do in backend and so a lot of requests go in timeout or I receive a generic error from the webservice.I am trying to solve this problem firstly tuning the number of LimitedConcurrencyLevelTaskScheduler.Some XML have a size of 2Mb, so I read the stream with XmlReader and I use \[code\]XmlNode node = new XmlDocument().ReadNode(reader);\[/code\] for the data I need, but sometimes the program crashes or goes in deadloop (I suppose the cause is a not proper xml response by the webservice).I think that creating an XDoc directly from the entire response is worse. Could you suggest me a possibile reliable solution?Thanks in advance :)
 
Back
Top