Parse huge XML - remember last successfully processed node to set offset on next run

kurdishboy

New Member
I have some pretty big xml files which used for scheduled import. I use cron to parse them.The problem is that processing takes too much time and always exceeds the php "max_execution_time". Since I use XMLReader, that allows to read xml "line by line", the only one solution I see: track current working "node", memorize it and set node offset on next cron run.Now I have\[code\] $xml = new XMLReader; $xml->open($file); $pointer = 0; while($xml->read()) { if ($xml->nodeType == XMLReader::ELEMENT && $xml->localName == 'Product') { $chunk = array(); $chunk['ProductID'] = $xml->getAttribute('ProductID'); $chunk['ProductName'] = $xml->getAttribute('ProductName'); process_import($chunk); // Process received date save_current_node_in_BD($pointer++); // insert current position in BD } } $xml->close();}\[/code\]Is it good idea to use $pointer++ to count processed nodes?How to set an offset for next cron run?
 
Back
Top