PHP application running out of Memory

KevinC

New Member
I am writing a set of classes for a crawler, it crawls a start page, pulls three links based on parameters (found using Simple Html Dom Parser allowing use of jquery like selectors), crawls those pages, then goes to page 2, picks the next 3 pages. Current max pages is 57 times.Needless to say I am getting: Allowed memory size of 50331648 bytes exhausted error message. Is there any way I can avoid running out of memmory.To let you know, after pulling in the contents of the first page, I run a go() function, which continuosly pulls in the pages until $this->maxpages is reached. I suppose I could run the loop when instantiating the classes, but would this help.
 
Back
Top