Dave, Frank,
Thank you for reporting your feelings and needs for a better
pythonOCC. Maybe it's time to discuss with you the future of this
library, and the paths we should The project was born 2 years ago, and
my initial goal is reached: it's now possible to do the same things
with pythonOCC as with OCC, but from the Python language: all
algorithms posted on the OCC forum can be easily ported to python, and
they work the same way.
When starting the project, I wanted pythonOCC to be as close as
possible to the original C++ API. In my mind, python was just a
convenient way to prototype C++ applications, so it was really
important to me that the move from python to C++ (or the opposite) was
as simple as possible: same classes/method names, same algorithms etc.
However, when developing with pythonOCC, Jelle and I realized that the
OCC API is not so convenient and provide very low level objects. A
more 'pythonic' pythonOCC then raised in our minds. We then started to
implement what we called a 'high level API' (but we're currently
discussing the name of this additional layer): this layer provides
useful and *pythonic* classes/methods/functions on top of OCC. The
'Parametric Application Framework' is the best example for this API,
and is the work I'm the most proud of: I think Jelle and I achieved a
very elegant and pythonic way to use and extend the features provided
by both OCC/SGEOM libraries (see for instance
http://code.google.com/p/pythonocc/source/browse/trunk/src/addons/PAF/Context.py).
There are also geometric/topological 'high level' functions very
similar to the Frank's MakeEdge function. They are available from the
Utils.Construct module
(http://api.pythonocc.org/OCC.Utils.Construct-module.html), but we
never posted anything or documented these functions.
I agree with both of you Dave, Frank (and I know Jelle also agrees)
that the API needs now to be more pythonic. However, making the
library more pythonic will make us reach an irreversible point: it
will not be possible anymore to move from python code to C++ code. But
I think that pythonOCC has now to live it's own life, and may have to
move away from its C++ 'mother' (or 'father', don't know what is the
gender of OCC).
In order to achieve that point, here are the points we could focus on
from now:
- move TCollection*, TColgp* etc. to python lists. We developed ugly
hacks to transform such classes to python lists, but's it's not
perfect and should be done at the SWIG level,
- implement more 'high level api' functions/methods (as suggested by
Frank). Many of these functions have already been developed (the high
level API source code is here:
http://code.google.com/p/pythonocc/source/browse/#svn/trunk/src/addons).
Maybe they should be more documented,
- for the topics related with memory management, Frank, let me think
about your suggestions the coming two days. Point (1) is closely
related to the way SWIG handles C++ references. OpenCascades makes an
extensive use of these awful '&' symbol. At the asme time, both OCC
and python use a ref counting mechanism for object deletion
management: for each class, the SWIG wrapper provides a python proxy
to a C++ object (this C++ object can be accessed by the 'this'
attribute of any pythonOCC object). When deleting a python class, you
have to take care about the deletion of the proxy (python ref count -
gc) and the deletion of the pointed object (OCC ref counting). I spent
*many* hours studying this mechanism, and I can help you to save time
if you ever decide to dive into SWIG. Point (2) is interesting, never
thought about that. I don't know yet whether it's better to do it from
the Python or the SWIG/C++ level.
Best Regards,
Thomas
2010/3/17 Dave Cowden <dave.cow...@gmail.com
<mailto:dave.cow...@gmail.com>>
Yes I understand, and agree that it would be better if the memory
management was a bit easier
I've also found that there is a fairly blurry line between
'pythonicness' and 'poor/ugly API'. Having written about 2000+
lines of code, i've written a lot of wrapper classes.
Many of these wrappers i'd wager are nearly identical to those any
other python developer writes as soon as they deal with
openCascade. for example generators to convert topexp iteration
to a for/in loop, edge wrappers to return points convieniently,
convienience methods to created edges and curves ( I have a
function nearly the same as yours in my code btw , case in point ).
I guess what i'm saying is that, though garbage collection helps,
its also true that your MakeEdge method is somewhat compensating
for a klunky api that doesnt just return Shape() and throw an
exception instead of doing Perform() then IsDone() and such. Its
clear that the solution to this is to let people contribute helper
and wrappers over time so that we all benefit.
I think what we need from Thomas/Jelle is a place that these can
be checked in and then apidoc is autogenerated. I'd be more than
happy to commit some of my utilty/wrapper code, but i'd want to
know where i can look to see the apidoc for the other such code is
before I add it...
On Wed, Mar 17, 2010 at 1:09 PM, Frank Conradie <fr...@qfin.net
<mailto:fr...@qfin.net>> wrote:
Hi Dave
Thanks for the suggestion - closures would certainly help.
However, I am currently less interested in ways to work
*around* the "un-pythonic" behavior of the wrapper, but would
rather like to discuss the technicalities and politics(!) of
making the wrapper more "pythonic". Here is what I would
ideally like to do:
* Clear up exactly what "un-pythonic" behavior I have run into
while using the library for a fairly complicated algorithm
(two modules, 3000+ lines of code).
* Discuss whether it is *technically* possible to change any
of this behavior to be more "pythonic"
* If it is indeed *technically* possible, discuss whether it
is feasible and worthwhile to actually implement
Here is a summary of the current "unexpected" behavior as I
see it:
(1). When using OCC objects in Python, an unsuspecting
programmer would expect them to behave like Python objects,
i.e. use Python ref counting and garbage collection behavior.
E.g. a BRepBuilderAPI object has a TopoDS_Shape member, so
this "myShape" has a refcount of 1 initially, and if it is
retrieved via the Shape() method and put into, say, a list,
its refcount must be 2, etc. So even if the BRepBuilderAPI
object goes out of scope, its internal shape is expected to
still have a refcount of 1 and "survive" a garbage collector
purge. However, this refcounting obviously does not happen on
the C++ side, so the BRepBuilderAPI object would have to be
kept "alive" as long as its internal shape is needed.
(2). Getting the "internal" shape from a BRepBuilderAPI,
BRepPrimAPI, BRepAlgoAPI, etc. object:
* C++: calling Shape() always returns the *same* myShape
instance
* Python: calling Shape() returns a *new* SWIG wrapper
object every time, each with the same __hash__ value so they
"look" like the same instance (at least to sets and dicts)
* The same unexpected behavior occurs when "exploring" a
shape, i.e. new SWIG wrapper objects are instantiated for
every returned shape
* The fact that different SWIG wrapper objects hash to the
same value at least allows us to identify unique shapes when
put in sets and dicts, but it confuses the GarbageCollector,
as the same C++ object can be "collected" multiple times, once
for each SWIG object wrapping it
Now, I can live with point (1), as I can see how this would be
near impossible to implement to work like it would in Python.
However, I really would like to discuss whether point (2) can
be addressed, so that there is always a 1-to-1 correspondence
between a C++ object and its SWIG wrapper object. The SWIG
wrapper already generates a single unique __hash__ value for a
single C++ object, so would it not be feasible to keep a map
of C++ object --> hash value --> SWIG wrapper object?
In pseudo-code, when the wrapper is asked for an OCC object:
h = hash(occobject)
if h in occ2swig_map:
swig_wrapper = occ2swig_map[h]
else:
swig_wrapper = new SwigWrapper(occobject)
occ2swig_map[h] = swig_wrapper
return swig_wrapper
I have never used SWIG myself, so excuse any ignorance on my
side (I will try and spend some time looking deeper into SWIG
over the next few days).
Hopefully we can have a fruitful and open discussion about
this. I can certainly work with the library as-is, and am
indeed doing so successfully at the moment. I just find that
it is no fun having to deal with the very kinds of C++ issues
I am trying to avoid by using Python in the 1st place ;-)
Cheers,
Frank
On 17/03/2010 4:22 AM, Dave Cowden wrote:
Hi, Frank:
I am not sure, but I think you could implement your utility
function using a python closure. This would allow the
original object to remain in scope until the nested function
has completed. so for example:
To do closures in python, you have to use lamda, which
creates a new function having access to the outer scope, even
after the function returns. So
def MakeEdge(p1, p2):
gp1,gp2 = [gp_Pnt(x,y,z) for x,y,z in (p1,p2)]
mkedge = BRepBuilderAPI_MakeEdge(gp1, gp2)
return mkedge.Shape()
Becomes:
def MakeEdge(p1,p2):
gp1,gp2 = [gp_Pnt(x,y,z) for x,y,z in (p1,p2)]
mkedge = BRepBuilderAPI_MakeEdge(gp1, gp2)
return lambda e: mkedge.Shape();
now you can do
me = MakeEdge(p1,p2);
me.Shape() //return the generated shape
me.Shape() //return the generated shap again. It will be the
same object because the underlying brepbuilder is still in a
closure
Now if you kill me, the closure will be discarded.
Dont know if this helps any or not.
On Tue, Mar 16, 2010 at 7:53 PM, Frank Conradie
<fr...@qfin.net <mailto:fr...@qfin.net>> wrote:
Hi Thomas
After much investigation today, I have now added a flag
to GarbageCollecter to disable it for our use, as it
really does not work as intended for relatively complex
code (I now manually track which objects I create and
need to call _kill_pointed for). There are two main
issues that we run into with the current GarbageCollector:
1. As already discussed, classes like the BRepPrimAPI and
BRepAlgoAPI have an internal "myShape" data member that
gets destroyed when the parent object's _kill_pointed
method is called, causing all kinds of issues. As an
example, it makes utility functions like the one below
unusable, as the local BRepPrimAPI object goes out of
scope and is thus collected by GarbageCollector for
purging, even though the caller of the function may want
to use the resulting shape for some time to come, after
the parent is "purged":
def MakeEdge(p1, p2):
gp1,gp2 = [gp_Pnt(x,y,z) for x,y,z in (p1,p2)]
mkedge = BRepBuilderAPI_MakeEdge(gp1, gp2)
return mkedge.Shape()
2. Every time a C++ OCC object is retrieved from Python
in any way, a brand new SWIG wrapper object is
instantiated for it, very easily/quickly leading to
multiple wrapper objects on the Python side for the same
underlying C++ object (I see this especially when using
OCC shape containers repeatedly). When any one of these
wrapper objects goes out of scope and is collected and
purged, the underlying C++ object is destroyed (via
_kill_pointed), even though there may well be other
Python wrapper objects still needing access to it.
I guess the ideal solution to this issue would be to
keep a mapping of C++ object -> SWIG wrapper object on
the C++ side, so that there is always a 1-to-1
correspondence between the two, but I'm not sure how
plausible this really is from an implementation
point-of-view. As an example of what I mean, consider
this: every time you call "mkedge.Shape()" in my example
above, you get a new/different Python object, wrapping
the same underlying myShape instance, which is totally
unexpected and unintuitive from a Python perspective. A
"pure" Python programmer would expect to get the exact
same Python object back every time, and not different
wrapper objects with just an "artificial" identical hash
value.
I realize that this is a very difficult issue, and that
as users of the PythonOCC wrapper library we will need to
take responsibility for memory management issues, but I
can see lots of confusion for Python programmers using
the library and expecting "Python-like" behaviour out of
the box.
I wish I new more about the SWIG wrapper object
instantiation process so that I could make more concrete
suggestions, instead of just complaining ;-) What is
your gut-feeling about this Thomas - do you think these
two issues can be addressed in the wrapper, or is the
current way the only viable implementation? If you can
give me a short description of the SWIG wrapper
instantiation process, I may be able to delve into it myself.
Thanks for your help with this thus far,
Frank
On 16/03/2010 9:04 AM, Thomas Paviot wrote:
Hi Frank,
I also suggest you use the push_context() and
pop_context() methods at the beginning and the end of
your algorithm. garbage.push_context() creates a new
collector. Objects collected will go to this new list.
Then, at the end of your algorithm, just call
.smart_purge() and then pop_context(). Otherwise, when
calling the smart_purge() method, you'll erase all other
objects that might have been stored in the gc thus
causing strange things.
def my_high_memory_consuming_algorithm():
GarbageCollector.garbage.push_context()
... do whatever you want ...
GarbageCollector.garbage.smart_purge() #only the
objects created since the call to push_context are
erased. Use MMGT_OPT=0 to make sure freeing #memory.
GarbageCollector.pop_context()
return data
There's also a garbage.prevent_object(object) method,
that will take care to not delete an object that it's
important to keep reference.
These three functions are quite new (but validated with
a unittest). I'm curious to have your feedback!
Regards,
Thomas
These two functions are quite new,
2010/3/16 Frank Conradie <fr...@qfin.net
<mailto:fr...@qfin.net>>
Hi Thomas
Well, I'm glad it's not a bug then! I can think of a
few ways to handle this situation in my algorithm,
so it should not be a problem. I'm just wondering
how many other classes behave in this "unexpected"
way :O
Cheers,
Frank
On 15/03/2010 10:26 PM, Thomas Paviot wrote:
Hi Frank,
This behaviour does not come from SWIG but from
OpenCASCADE itself. Let me explain you in a few
words what happens here.
The BRepPrimAPI_MakeBox class inherits from the
BRepBuilderAPI_MakeShape one. This class provides
the Shape() method you can find in
BRepPrimAPI_Make* basic geometry constructors. If
you have a look at the Shape() method in the file
BRepBuilderAPI_MakeShape.cxx, you will read:
"""
//=======================================================================
//function : Shape
//purpose :
//=======================================================================
const TopoDS_Shape&
BRepBuilderAPI_MakeShape::Shape() const
{
if (!IsDone()) {
// the following is const cast away
((BRepBuilderAPI_MakeShape*) (void*)
this)->Build();
Check();
}
return myShape;
}
"""
The Shape() method then returns a reference to the
protected attribute 'myShape' of the class
BRepBuilderAPI_MakeBox. In other words, if you
delete the BRepPrimAPI_MakeBox instance, you loose
the reference to the shape.
For instance:
>>> from OCC.BRepPrimAPI_MakeBox import *
>>> from OCC.TopoDS import *
>>> box = BRepPrimAPI_MakeBox(10,10,10)
>>> shp = box.Shape()
>>> print shp.IsNull()
0
>>> del box #object is garbage collected, not
deleted yet
>>> print shp.IsNull()
0
>>> GarbageCollector.garbage.smart_purge() # all
objects are deleted, loose the reference to myShape
attribute
>>> print shp.IsNull()
1
Then force objects deletion only if you're sure you
don't need the shapes anymore. I don't know what
you're trying to do exactly, so it's difficult to
suggest any best practice.
Regards,
Thomas
2010/3/15 Frank Conradie <fr...@qfin.net
<mailto:fr...@qfin.net>>
Hi Thomas
I am getting crash behaviour in the interim 0.5
PythonOCC release,
related to the new GarbageCollector
implementation. The code below
crashes on my PC:
----------------------------------------
from OCC.gp import *
from OCC.BRepPrimAPI import *
from OCC.BRepBuilderAPI import *
from OCC.TopExp import *
from OCC.TopoDS import *
o = 0.0
w = 0.1
fp1 = (o,o,o)
fp2 = (w,w,w)
mkbox =
BRepPrimAPI_MakeBox(gp_Pnt(fp1[0],fp1[1],fp1[2]),
gp_Pnt(fp2[0],fp2[1],fp2[2]))
s = mkbox.Shape()
del mkbox
print s.ShapeType()
GarbageCollector.garbage.smart_purge()
# The next line crashes my system
print s.ShapeType()
----------------------------------------
It is almost as if calling _kill_pointed for
the mkbox causes shape "s"
to become "corrupted" in some way. I dug
through the SWIG code a bit,
but cannot see anything obviously wrong, so I
was hoping you could shed
some light on the issue.
By the way, I did get the SVN compiled myself
as well, and get the same
crash with my compiled lib as well as the one
you made available on your
website.
Thanks,
Frank
_______________________________________________
Pythonocc-users mailing list
Pythonocc-users@gna.org
<mailto:Pythonocc-users@gna.org>
https://mail.gna.org/listinfo/pythonocc-users
_______________________________________________
Pythonocc-users mailing list
Pythonocc-users@gna.org <mailto:Pythonocc-users@gna.org>
https://mail.gna.org/listinfo/pythonocc-users
_______________________________________________
Pythonocc-users mailing list
Pythonocc-users@gna.org <mailto:Pythonocc-users@gna.org>
https://mail.gna.org/listinfo/pythonocc-users
_______________________________________________
Pythonocc-users mailing list
Pythonocc-users@gna.org <mailto:Pythonocc-users@gna.org>
https://mail.gna.org/listinfo/pythonocc-users
_______________________________________________
Pythonocc-users mailing list
Pythonocc-users@gna.org <mailto:Pythonocc-users@gna.org>
https://mail.gna.org/listinfo/pythonocc-users
_______________________________________________
Pythonocc-users mailing list
Pythonocc-users@gna.org <mailto:Pythonocc-users@gna.org>
https://mail.gna.org/listinfo/pythonocc-users
_______________________________________________
Pythonocc-users mailing list
Pythonocc-users@gna.org <mailto:Pythonocc-users@gna.org>
https://mail.gna.org/listinfo/pythonocc-users
_______________________________________________
Pythonocc-users mailing list
Pythonocc-users@gna.org <mailto:Pythonocc-users@gna.org>
https://mail.gna.org/listinfo/pythonocc-users
_______________________________________________
Pythonocc-users mailing list
Pythonocc-users@gna.org
https://mail.gna.org/listinfo/pythonocc-users
