ID: 40261
User updated by: thuejk at gmail dot com
Reported By: thuejk at gmail dot com
Status: Open
Bug Type: Performance problem
-Operating System: Linux
+Operating System: All
PHP Version: 5.2.0
New Comment:
I added a few printf's and found out that the $num generated "holes" in
the memory are 384 big.
Zend has a special system for small allocations, but that only works
for holes below ZEND_MM_SMALL_SIZE=280.
The $num allocations at the end, which cause the problems, and 40 or 88
long.
Note that these numbers are from a 64-bit machine, which of course has
8-byte pointers, and so a larger MM overhead. (The bug does also occur
on 32bit machines)
One temporary solution could be to raise
#define ZEND_MM_NUM_BUCKETS 32
from which ZEND_MM_SMALL_SIZE is defined, so that ZEND_MM_SMALL_SIZE is
made twice or perhaps 4 times as big. (I got a segfault when I tried
that, haven't looked into why)
A better solution would be to organize the free blocks in a balanced
tree, instead of a linear list which has to be traversed.
Previous Comments:
------------------------------------------------------------------------
[2007-01-29 01:42:28] thuejk at gmail dot com
Ok, after a good deal of work I actually tracked down the problem to
fragmentation in your memory management.
This understanding enabled me to construct a simpler test case. Note
that the problem is unrelated to database function, and this new
testcase does not require any database.
<?php
//According to my calculations, setting $num=18000000 should make this
script take ~one year.
$num = 100000;
$a = Array();
for ($i=0; $i<$num; $i++) {
$a[$i] = Array(1);
}
/* The underlying memory now looks something like
* row structure
* row value
* row structure
* ...
*/
$b = Array();
for ($i=0; $i<$num; $i++) {
$b[$i] = $a[$i][0];
}
/* The b rows are allocated
*
* Though I haven't checked it in the code, I have reason to believe
* that the values inserted into the b rows are pointers to the same
* memory allocated for the values in the a rows
*/
unset($a);
/* The a rows are unallocated, but the values are still references from
$b, so the memory map looks something like
* row value
* free_block
* row_value
* free_block
* ...
*
* repeated $num times.
*/
/* Now, for each memory allocation for a row, PHP runs through all
* free blocks checking for a best fit. Since there are $num free
* blocks, this takes time. This is done by the function
* _zend_mm_alloc_int in PHP 5.2.0 Zend/zend_alloc.c :
*
* end = &heap->free_buckets[0];
* for (p = end->next_free_block; p != end; p = p->next_free_block) {
* size_t s = ZEND_MM_FREE_BLOCK_SIZE(p);
* if (s > true_size) {
* if (s < best_size) { // better fit
* best_fit = p;
* best_size = s;
* }
* } else if (s == true_size) {
* // Found "big" free block of exactly the same size
* best_fit = p;
* goto zend_mm_finished_searching_for_block;
* }
* }
*/
$c = Array();
for ($i=0; $i<$num; $i++) {
$c[$i] = 1;
}
?>
------------------------------------------------------------------------
[2007-01-28 01:52:37] thuejk at gmail dot com
On second though, black boxes are not good for testing. A simpler timer
added...
------------------------------------------------------------------------
[2007-01-28 01:47:01] thuejk at gmail dot com
I have made it a bit shorter.
I left the DebugStats class in, it is useful. Just treat it as a black
box. I can garantie that the two calls into it does not account for the
8 seconds performance problem...
------------------------------------------------------------------------
[2007-01-28 01:30:33] [EMAIL PROTECTED]
Could please modify the script, so that it would be _short_ but
complete?
------------------------------------------------------------------------
[2007-01-28 01:25:13] thuejk at gmail dot com
I added the requested changes to the linked file
http://thuejk.dk/test.php.txt
------------------------------------------------------------------------
The remainder of the comments for this report are too long. To view
the rest of the comments, please view the bug report online at
http://bugs.php.net/40261
--
Edit this bug report at http://bugs.php.net/?id=40261&edit=1
