There are two different issues IMO:
- the segfault. I think I have identified the origin of the problem, it
comes from the GarbageCollector implementation
- the leak. I also noticed this increase in memory consumption, however it's
no more the case if the size of the vert list is constant. If you replace,
for instance, the random loop with:
for x in range(50):
for y in range(50):
[...]
Then memory consumption does not increase anymore (can you check that
point?). That means that the pythonocc garbage collector does the job as
expected and I suspect rather OCC memory manager to be the cause of this
behavior. To be sure about that, this example should first be ported to C++
and included into the OCE testing suite.
Thomas
2011/9/29 István Csanády <istvancsan...@gmail.com>
> The program you sent is still leaking even if I comment out
> gcurve.GetObject() but it seems to be better then it was. If I leave the
> gcurve.GetObject() line it also crashes.
> Here is a memory consumption log:
>
> 76.015625, 76.015625, 76.19921875, 76.19921875, 76.19921875, 83.55859375,
> 82.9140625, 82.9140625, 82.9140625, 84.13671875, 85.296875, 84.99609375,
> 84.99609375, 84.99609375, 83.24609375, 84.234375, 83.24609375, 84.140625,
> 89.64453125, 89.4140625, 87.4140625, 87.4140625, 87.1640625, 87.1640625,
> 88.3125, 88.1640625, 88.1640625, 88.73046875, 88.1640625, 88.1640625,
> 87.21484375, 86.1640625, 86.1640625, 86.1640625, 89.1328125, 89.10546875,
> 88.16796875, 92.15234375, 91.16796875, 90.16796875, 90.16796875,
> 90.16796875, 95.42578125, 95.07421875, 94.71875, 93.76953125, 93.32421875,
> 92.07421875, 92.921875, 92.10546875, 91.49609375, 91.07421875, 91.07421875,
> 91.07421875, 91.07421875, 94.8515625, 94.07421875, 94.07421875,
> 93.82421875, 93.82421875, 93.82421875, 93.82421875, 92.82421875,
> 92.82421875, 93.38671875, 93.1328125, 93.1328125, 93.1328125, 93.1328125,
> 94.26171875, 92.625, 92.625, 93.69921875, 94.5078125, 93.578125,
> 92.578125, 92.578125, 92.078125, 91.078125, 91.078125, 93.2109375,
> 93.078125, 92.078125, 92.078125, 94.12109375, 93.4921875, 93.078125,
> 95.7890625, 93.078125, 93.45703125, 93.078125, 93.078125, 94.41796875,
> 93.578125, 94.27734375, 93.578125, 93.578125, 93.578125, 93.59375,
> 93.66796875, 93.328125, 93.41796875, 93.98828125, 96.0625, 95.5703125,
> 95.328125, 95.44140625, 95.328125, 95.328125, 95.078125, 95.078125,
> 93.92578125, 93.078125, 93.1796875, 93.078125, 92.078125, 92.078125,
> 93.4921875, 93.4921875, 93.078125, 93.078125, 93.078125, 93.4296875,
> 93.078125, 94.2109375, 93.48828125, 94.1640625, 93.578125, 93.26953125,
> 92.578125, 91.578125, 90.828125, 90.828125, 90.828125, 90.828125,
> 97.32421875, 96.7421875, 97.63671875, 96.7421875, 96.7421875, 96.7421875,
> 96.7421875, 96.7421875, 96.7421875, 96.7421875, 96.7421875, 95.7421875,
> 95.7421875, 95.7421875, 95.7421875, 98.7890625, 97.33203125, 97.59375,
> 97.33203125, 97.08203125, 99.3828125, 97.33203125, 96.4453125, 96.08203125,
> 95.08203125, 95.08203125, 95.08203125, 95.08203125, 96.890625, 96.08203125,
> 96.08203125, 97.80078125, 97.08203125, 98.56640625, 97.46875, 97.46875,
> 97.46875, 96.46875, 96.08203125, 100.66015625, 100.33203125
>
> On Thu, Sep 29, 2011 at 7:00 AM, Thomas Paviot <tpav...@gmail.com> wrote:
>
>> The issue comes from the line curve_object = gcurve.GetObject(). Comment
>> out this line to check that memory consumption is constant over time.
>>
>> Attached another implementation of your program, can you please test it
>> and report.
>>
>>
>> Thomas
>>
>> 2011/9/28 István Csanády <istvancsan...@gmail.com>
>>
>>>
>>> The corrected code does not crash, but it leaks like sieve (0-3MB/s for
>>> me)... And if you try to call smart_purge() after pop_context() it
>>> segfaults.
>>>
>>>
>>>
>>> On Wed, Sep 28, 2011 at 10:22 PM, Thomas Paviot <tpav...@gmail.com>wrote:
>>>
>>>> 2011/9/28 István Csanády <istvancsan...@gmail.com>
>>>>
>>>>> Hi Thomas,
>>>>>
>>>>> On Wed, Sep 28, 2011 at 9:19 PM, Thomas Paviot <tpav...@gmail.com>wrote:
>>>>>
>>>>>> Hi Istvan,
>>>>>>
>>>>>> Not yet. Trying to run your example actually helped me to figure
>>>>>> out/fix a serious regression in the current pythonocc master branch.
>>>>>> There
>>>>>> is still another bug to fix. I'm currently running pythonocc master (
>>>>>> https://github.com/tpaviot/pythonocc)/OCE-0.6.0-rc3.
>>>>>
>>>>>
>>>>>
>>>>>> Do you run pythonocc-0.5/OCC630?
>>>>>>
>>>>>
>>>>> Yes.
>>>>>
>>>>> So you say that my code does not crash for you? If I remember correctly
>>>>> you are a Mac user to, aren't you? Are you using 10.6.7 too with Python
>>>>> 2.6.1 ? May I try to recompile pythonocc with OCE? Can it fix this error?
>>>>> This bug is incredibly hard to track down since the python debugger does
>>>>> not
>>>>> stop when the C++ code crashes... Do you use any special tools to debug
>>>>> such
>>>>> errors when developing pythonocc?
>>>>>
>>>>
>>>> I didn't say it crashes, I was running the test.
>>>>
>>>> It's done now, I can reproduce the crash, the program segfaults after a
>>>> few loops are performed (I run OSX 10.6.8). I can't explain so far what
>>>> happens, I need to dive into the example (make it even simpler) and check
>>>> what's going on.
>>>>
>>>> I also suggest you use garbage.push_context() and pop_context(): only
>>>> objects created between push/pop will be killed by the smart_purge method.
>>>> The corrected program attached works properly.
>>>>
>>>
>>>
>>>
>>>>
>>>>
>>>>>
>>>>> István
>>>>>
>>>>
>>>> Thomas
>>>>
>>>>
>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>> All the best,
>>>>>>
>>>>>> Thomas
>>>>>>
>>>>>>
>>>>>> 2011/9/28 István Csanády <istvancsan...@gmail.com>
>>>>>>
>>>>>>> Hi Thomas,
>>>>>>>
>>>>>>> Have you managed to reproduce the error? I've been trying to track
>>>>>>> down the bug with valgrind - without any success... Probably I should
>>>>>>> recompile python with the debug and --without-pymalloc flags to make
>>>>>>> valgrind work with it. I am using Mac OS X 10.6.7.
>>>>>>>
>>>>>>> István
>>>>>>>
>>>>>>> On Wed, Sep 28, 2011 at 11:21 AM, Thomas Paviot
>>>>>>> <tpav...@gmail.com>wrote:
>>>>>>>
>>>>>>>> 2011/9/27 István Csanády <istvancsan...@gmail.com>
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>
>>>>>>>> Hi Istvan,
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Last time when I wrote about this bug I thought it was caused by
>>>>>>>>> some numpy buffer overflow. Unfortunately it was not. Finally I
>>>>>>>>> managed to
>>>>>>>>> create some code that can reproduce this bug. I have attached the
>>>>>>>>> code and
>>>>>>>>> some crash logs.
>>>>>>>>
>>>>>>>>
>>>>>>>> I've been trying to reproduce the issue.
>>>>>>>>
>>>>>>>>
>>>>>>>>> I think the problem is probably with OCC's Standard_MMgrOpt class
>>>>>>>>> memory recycling for small blocks (Is it possible that smart_purge
>>>>>>>>> frees
>>>>>>>>> memory that Standard_MMgrOpt want to reuse?).
>>>>>>>>
>>>>>>>>
>>>>>>>> No.
>>>>>>>>
>>>>>>>>
>>>>>>>>> Note that sometimes it takes a long time until the error occurs,
>>>>>>>>> but usually after the 50th-100th iteration the code crashes for me.
>>>>>>>>> (The
>>>>>>>>> code is totally pointless, I have removed every unneccessary parts)
>>>>>>>>>
>>>>>>>>> István
>>>>>>>>>
>>>>>>>>
>>>>>>>> Thomas
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Pythonocc-users mailing list
>>>>>>>> Pythonocc-users@gna.org
>>>>>>>> https://mail.gna.org/listinfo/pythonocc-users
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Pythonocc-users mailing list
>>>>>>> Pythonocc-users@gna.org
>>>>>>> https://mail.gna.org/listinfo/pythonocc-users
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Pythonocc-users mailing list
>>>>>> Pythonocc-users@gna.org
>>>>>> https://mail.gna.org/listinfo/pythonocc-users
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Pythonocc-users mailing list
>>>>> Pythonocc-users@gna.org
>>>>> https://mail.gna.org/listinfo/pythonocc-users
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> Pythonocc-users mailing list
>>>> Pythonocc-users@gna.org
>>>> https://mail.gna.org/listinfo/pythonocc-users
>>>>
>>>>
>>>
>>> _______________________________________________
>>> Pythonocc-users mailing list
>>> Pythonocc-users@gna.org
>>> https://mail.gna.org/listinfo/pythonocc-users
>>>
>>>
>>
>> _______________________________________________
>> Pythonocc-users mailing list
>> Pythonocc-users@gna.org
>> https://mail.gna.org/listinfo/pythonocc-users
>>
>>
>
> _______________________________________________
> Pythonocc-users mailing list
> Pythonocc-users@gna.org
> https://mail.gna.org/listinfo/pythonocc-users
>
>
_______________________________________________
Pythonocc-users mailing list
Pythonocc-users@gna.org
https://mail.gna.org/listinfo/pythonocc-users
