From: gilbert dot musnik at fr dot adp dot com
Operating system: Windows
PHP version: 5.2.6
PHP Bug Type: SimpleXML related
Bug description: Memory failure in very large XML file
Description:
------------
Hello,
It's been days I am trying to sort out a problem which pops up when I
process a huge XML file. In "reproduce code" windows is a trivial script
which allocates memory by making an array very big. Assuming a
memory_limit of 1024M in <php.ini>, if I don't load the huge XML file, the
script crashes as expected with a "memory exhausted" fatal error diagnostic
(see expected results window).
However, if I do load my huge XML file with simplexml_load_file(around 300
MB!), the script crashes with a "Out of memory" fatal error diagnostic. By
the way, it doesn't matter if memory_limit is set to 512M or 2048M, the
script stumbles on the same "allocated" figure of 332 MB (see actual
results windows).
The strangest thing is that before I start swelling the array, the memory
usage is almost the same whether ot not
simplexml_load_file("VERY_BIG_FILE.XML") is invoked or not.
It looks as if memory allocated by simplexml_load_file() routine is NOT
taken into account in overall PHP memory usage and memory limit
management.
Thanks in advance for your feedback on that,
Gilbert Musnik
Reproduce code:
---------------
<?php
$flag_xml = true;
$now = Date("Y/m/d - H:i:s");
if ($flag_xml) {
print ("[ $now ] Before simplexml_load_file\n");
$f_inp = "VERY_BIG_FILE.xml"; // mine is 300 MB
$ll0 = time();
$xml = simplexml_load_file ($f_inp);
$ll1 = time() - $ll0;
$now = Date("Y/m/D - H:i:s");
print ("[ $now ] After simplexml_load_file: $ll1 secondes\n");
}
else {
print ("[ $now ] NO simplexml_load_file\n");
}
$bufrs = array();
$nmax = 150000000;
for ($ii = 0; $ii < $nmax; $ii ++) {
$bufr = str_repeat ("ABCDEFGHIJ", 1);
array_push ($bufrs, $bufr);
$jj = $ii % 1000000;
if ($jj == 0) {
$xx = number_format ($ii);
$qq = number_format (memory_get_usage());
$memory_limit = ini_get('memory_limit');
print ("$xx - memory usage=$qq -
memory_limit=$memory_limit)\n");
}
};
?>
Expected result:
----------------
N:\>\php-5.2.6-Win32\php.exe -c \MD_config\php.ini \MD_config\test_mem.php
[ 2008/06/16 - 18:06:10 ] No simplexml_load_file
0 - memory usage=64,552 - memory_limit=1024M)
1,000,000 - memory usage=100,259,664 - memory_limit=1024M)
2,000,000 - memory usage=200,454,016 - memory_limit=1024M)
3,000,000 - memory usage=304,842,624 - memory_limit=1024M)
4,000,000 - memory usage=400,842,624 - memory_limit=1024M)
5,000,000 - memory usage=513,619,840 - memory_limit=1024M)
6,000,000 - memory usage=609,619,840 - memory_limit=1024M)
7,000,000 - memory usage=705,619,840 - memory_limit=1024M)
8,000,000 - memory usage=801,619,840 - memory_limit=1024M)
9,000,000 - memory usage=931,174,272 - memory_limit=1024M)
10,000,000 - memory usage=1,027,174,272 - memory_limit=1024M)
Fatal error: Allowed memory size of 1073741824 bytes exhausted (tried to
allocate 35 bytes) in N:\md_config\test_mem.php on line 23
Actual result:
--------------
N:\>\php-5.2.6-Win32\php.exe -c \MD_config\php.ini \MD_config\test_mem.php
[ 2008/06/16 - 18:05:06 ] Before simplexml_load_file
[ 2008/06/Mon - 18:05:33 ] After simplexml_load_file: 27 secondes
0 - memory usage=65,472 - memory_limit=1024M)
1,000,000 - memory usage=100,260,376 - memory_limit=1024M)
2,000,000 - memory usage=200,454,728 - memory_limit=1024M)
3,000,000 - memory usage=304,843,336 - memory_limit=1024M)
Fatal error: Out of memory (allocated 332660736) (tried to allocate 35
bytes) in N:\md_config\test_mem.php on line 23
--
Edit bug report at http://bugs.php.net/?id=45291&edit=1
--
Try a CVS snapshot (PHP 5.2):
http://bugs.php.net/fix.php?id=45291&r=trysnapshot52
Try a CVS snapshot (PHP 5.3):
http://bugs.php.net/fix.php?id=45291&r=trysnapshot53
Try a CVS snapshot (PHP 6.0):
http://bugs.php.net/fix.php?id=45291&r=trysnapshot60
Fixed in CVS: http://bugs.php.net/fix.php?id=45291&r=fixedcvs
Fixed in release:
http://bugs.php.net/fix.php?id=45291&r=alreadyfixed
Need backtrace: http://bugs.php.net/fix.php?id=45291&r=needtrace
Need Reproduce Script: http://bugs.php.net/fix.php?id=45291&r=needscript
Try newer version: http://bugs.php.net/fix.php?id=45291&r=oldversion
Not developer issue: http://bugs.php.net/fix.php?id=45291&r=support
Expected behavior: http://bugs.php.net/fix.php?id=45291&r=notwrong
Not enough info:
http://bugs.php.net/fix.php?id=45291&r=notenoughinfo
Submitted twice:
http://bugs.php.net/fix.php?id=45291&r=submittedtwice
register_globals: http://bugs.php.net/fix.php?id=45291&r=globals
PHP 4 support discontinued: http://bugs.php.net/fix.php?id=45291&r=php4
Daylight Savings: http://bugs.php.net/fix.php?id=45291&r=dst
IIS Stability: http://bugs.php.net/fix.php?id=45291&r=isapi
Install GNU Sed: http://bugs.php.net/fix.php?id=45291&r=gnused
Floating point limitations: http://bugs.php.net/fix.php?id=45291&r=float
No Zend Extensions: http://bugs.php.net/fix.php?id=45291&r=nozend
MySQL Configuration Error: http://bugs.php.net/fix.php?id=45291&r=mysqlcfg
