Hi Frank, According to me, you points (1) and (2) are parts of the low-level API (i.e. the SWIG level). Python proxies are created by SWIG (for instance Standard.py), at the same time than the C++ code (when compiled, _Standard.so). I guess the pseudo code you inserted in your previous message should go to the proxy module. And maybe we would have to write our own python module to handle with OCC objects as we want.
The strange behaviour you mention might be Windows specific. For instance, I noticed that the following code crashes under Windows: s = Standard_Transient() hash(s) hash(s) hash(s) hash(s) hash(s) # crashes the 5th time you call hash(s) - Windows XP/Py25/32bit No problem under MacOSX or Linux. Thomas 2010/3/18 Frank Conradie <fr...@qfin.net> > Hi Thomas > > Thanks for your thoughts. I completely agree on keeping the low-level API > as close as possible to the C++ API, which is why I am happy dealing with > the issues raised in my point (1) by writing my own higher-level wrappers > for now. My main point is just that the current GarbageCollector does not > work well at all with the low-level API, and maybe such a GC should not even > be part of the low-level API. > > However, I would argue that the issue of getting multiple *different* proxy > objects for the *same* underlying C++ object, my point (2), should be > addressed even in the low-level Python API. Otherwise all higher level > wrappers will have to deal with this really odd behavior, which believe me > is not that easy. > > By the way, in trying to use a dictionary to keep track of all the proxy > objects created per underlying C++ object, i.e. a dict that maps hash(o) -> > [o1,o2,...], I found that trying to put objects of certain classes in a set > or dict crashes OCC. The classes I saw this from included > BRepBuilderAPI_Sewing and BRepTools_ReShape - very strange. > > Cheers, > Frank > > > On 17/03/2010 3:37 PM, Thomas Paviot wrote: > > Dave, Frank, > > Thank you for reporting your feelings and needs for a better pythonOCC. > Maybe it's time to discuss with you the future of this library, and the > paths we should The project was born 2 years ago, and my initial goal is > reached: it's now possible to do the same things with pythonOCC as with OCC, > but from the Python language: all algorithms posted on the OCC forum can be > easily ported to python, and they work the same way. > > When starting the project, I wanted pythonOCC to be as close as possible > to the original C++ API. In my mind, python was just a convenient way to > prototype C++ applications, so it was really important to me that the > move from python to C++ (or the opposite) was as simple as possible: same > classes/method names, same algorithms etc. > > However, when developing with pythonOCC, Jelle and I realized that the > OCC API is not so convenient and provide very low level objects. A more > 'pythonic' pythonOCC then raised in our minds. We then started to implement > what we called a 'high level API' (but we're currently discussing the name > of this additional layer): this layer provides useful and *pythonic* > classes/methods/functions on top of OCC. The 'Parametric Application > Framework' is the best example for this API, and is the work I'm the most > proud of: I think Jelle and I achieved a very elegant and pythonic way to > use and extend the features provided by both OCC/SGEOM libraries (see for > instance > http://code.google.com/p/pythonocc/source/browse/trunk/src/addons/PAF/Context.py). > There are also geometric/topological 'high level' functions very similar to > the Frank's MakeEdge function. They are available from the Utils.Construct > module (http://api.pythonocc.org/OCC.Utils.Construct-module.html), but we > never posted anything or documented these functions. > > I agree with both of you Dave, Frank (and I know Jelle also agrees) that > the API needs now to be more pythonic. However, making the library more > pythonic will make us reach an irreversible point: it will not be possible > anymore to move from python code to C++ code. But I think that pythonOCC has > now to live it's own life, and may have to move away from its C++ 'mother' > (or 'father', don't know what is the gender of OCC). > > In order to achieve that point, here are the points we could focus on > from now: > - move TCollection*, TColgp* etc. to python lists. We developed ugly hacks > to transform such classes to python lists, but's it's not perfect and should > be done at the SWIG level, > - implement more 'high level api' functions/methods (as suggested by > Frank). Many of these functions have already been developed (the high level > API source code is here: > http://code.google.com/p/pythonocc/source/browse/#svn/trunk/src/addons). > Maybe they should be more documented, > - for the topics related with memory management, Frank, let me think about > your suggestions the coming two days. Point (1) is closely related to the > way SWIG handles C++ references. OpenCascades makes an extensive use of > these awful '&' symbol. At the asme time, both OCC and python use a ref > counting mechanism for object deletion management: for each class, the SWIG > wrapper provides a python proxy to a C++ object (this C++ object can be > accessed by the 'this' attribute of any pythonOCC object). When deleting a > python class, you have to take care about the deletion of the proxy (python > ref count - gc) and the deletion of the pointed object (OCC ref counting). I > spent *many* hours studying this mechanism, and I can help you to save time > if you ever decide to dive into SWIG. Point (2) is interesting, never > thought about that. I don't know yet whether it's better to do it from the > Python or the SWIG/C++ level. > > Best Regards, > > Thomas > > 2010/3/17 Dave Cowden <dave.cow...@gmail.com> > >> Yes I understand, and agree that it would be better if the memory >> management was a bit easier >> >> I've also found that there is a fairly blurry line between >> 'pythonicness' and 'poor/ugly API'. Having written about 2000+ lines of >> code, i've written a lot of wrapper classes. >> >> Many of these wrappers i'd wager are nearly identical to those any other >> python developer writes as soon as they deal with openCascade. for example >> generators to convert topexp iteration to a for/in loop, edge wrappers to >> return points convieniently, convienience methods to created edges and >> curves ( I have a function nearly the same as yours in my code btw , case >> in point ). >> >> I guess what i'm saying is that, though garbage collection helps, its >> also true that your MakeEdge method is somewhat compensating for a klunky >> api that doesnt just return Shape() and throw an exception instead of doing >> Perform() then IsDone() and such. Its clear that the solution to this is to >> let people contribute helper and wrappers over time so that we all benefit. >> >> I think what we need from Thomas/Jelle is a place that these can be >> checked in and then apidoc is autogenerated. I'd be more than happy to >> commit some of my utilty/wrapper code, but i'd want to know where i can look >> to see the apidoc for the other such code is before I add it... >> >> >> On Wed, Mar 17, 2010 at 1:09 PM, Frank Conradie <fr...@qfin.net> wrote: >> >>> Hi Dave >>> >>> Thanks for the suggestion - closures would certainly help. However, I am >>> currently less interested in ways to work *around* the "un-pythonic" >>> behavior of the wrapper, but would rather like to discuss the technicalities >>> and politics(!) of making the wrapper more "pythonic". Here is what I would >>> ideally like to do: >>> >>> * Clear up exactly what "un-pythonic" behavior I have run into while >>> using the library for a fairly complicated algorithm (two modules, 3000+ >>> lines of code). >>> * Discuss whether it is *technically* possible to change any of this >>> behavior to be more "pythonic" >>> * If it is indeed *technically* possible, discuss whether it is feasible >>> and worthwhile to actually implement >>> >>> Here is a summary of the current "unexpected" behavior as I see it: >>> >>> (1). When using OCC objects in Python, an unsuspecting programmer would >>> expect them to behave like Python objects, i.e. use Python ref counting and >>> garbage collection behavior. E.g. a BRepBuilderAPI object has a TopoDS_Shape >>> member, so this "myShape" has a refcount of 1 initially, and if it is >>> retrieved via the Shape() method and put into, say, a list, its refcount >>> must be 2, etc. So even if the BRepBuilderAPI object goes out of scope, its >>> internal shape is expected to still have a refcount of 1 and "survive" a >>> garbage collector purge. However, this refcounting obviously does not happen >>> on the C++ side, so the BRepBuilderAPI object would have to be kept "alive" >>> as long as its internal shape is needed. >>> >>> (2). Getting the "internal" shape from a BRepBuilderAPI, BRepPrimAPI, >>> BRepAlgoAPI, etc. object: >>> * C++: calling Shape() always returns the *same* myShape instance >>> * Python: calling Shape() returns a *new* SWIG wrapper object every >>> time, each with the same __hash__ value so they "look" like the same >>> instance (at least to sets and dicts) >>> * The same unexpected behavior occurs when "exploring" a shape, i.e. >>> new SWIG wrapper objects are instantiated for every returned shape >>> * The fact that different SWIG wrapper objects hash to the same value >>> at least allows us to identify unique shapes when put in sets and dicts, but >>> it confuses the GarbageCollector, as the same C++ object can be "collected" >>> multiple times, once for each SWIG object wrapping it >>> >>> Now, I can live with point (1), as I can see how this would be near >>> impossible to implement to work like it would in Python. However, I really >>> would like to discuss whether point (2) can be addressed, so that there is >>> always a 1-to-1 correspondence between a C++ object and its SWIG wrapper >>> object. The SWIG wrapper already generates a single unique __hash__ value >>> for a single C++ object, so would it not be feasible to keep a map of C++ >>> object --> hash value --> SWIG wrapper object? >>> >>> In pseudo-code, when the wrapper is asked for an OCC object: >>> >>> h = hash(occobject) >>> if h in occ2swig_map: >>> swig_wrapper = occ2swig_map[h] >>> else: >>> swig_wrapper = new SwigWrapper(occobject) >>> occ2swig_map[h] = swig_wrapper >>> return swig_wrapper >>> >>> I have never used SWIG myself, so excuse any ignorance on my side (I will >>> try and spend some time looking deeper into SWIG over the next few days). >>> >>> Hopefully we can have a fruitful and open discussion about this. I can >>> certainly work with the library as-is, and am indeed doing so successfully >>> at the moment. I just find that it is no fun having to deal with the very >>> kinds of C++ issues I am trying to avoid by using Python in the 1st place >>> ;-) >>> >>> Cheers, >>> Frank >>> >>> >>> >>> >>> On 17/03/2010 4:22 AM, Dave Cowden wrote: >>> >>> Hi, Frank: >>> >>> I am not sure, but I think you could implement your utility function >>> using a python closure. This would allow the original object to remain in >>> scope until the nested function has completed. so for example: >>> >>> To do closures in python, you have to use lamda, which creates a new >>> function having access to the outer scope, even after the function returns. >>> So >>> >>> def MakeEdge(p1, p2): >>> gp1,gp2 = [gp_Pnt(x,y,z) for x,y,z in (p1,p2)] >>> mkedge = BRepBuilderAPI_MakeEdge(gp1, gp2) >>> return mkedge.Shape() >>> >>> Becomes: >>> >>> def MakeEdge(p1,p2): >>> gp1,gp2 = [gp_Pnt(x,y,z) for x,y,z in (p1,p2)] >>> mkedge = BRepBuilderAPI_MakeEdge(gp1, gp2) >>> return lambda e: mkedge.Shape(); >>> >>> now you can do >>> me = MakeEdge(p1,p2); >>> >>> me.Shape() //return the generated shape >>> me.Shape() //return the generated shap again. It will be the same object >>> because the underlying brepbuilder is still in a closure >>> >>> Now if you kill me, the closure will be discarded. >>> >>> Dont know if this helps any or not. >>> >>> On Tue, Mar 16, 2010 at 7:53 PM, Frank Conradie <fr...@qfin.net> wrote: >>> >>>> Hi Thomas >>>> >>>> After much investigation today, I have now added a flag to >>>> GarbageCollecter to disable it for our use, as it really does not work as >>>> intended for relatively complex code (I now manually track which objects I >>>> create and need to call _kill_pointed for). There are two main issues that >>>> we run into with the current GarbageCollector: >>>> >>>> 1. As already discussed, classes like the BRepPrimAPI and BRepAlgoAPI >>>> have an internal "myShape" data member that gets destroyed when the parent >>>> object's _kill_pointed method is called, causing all kinds of issues. As an >>>> example, it makes utility functions like the one below unusable, as the >>>> local BRepPrimAPI object goes out of scope and is thus collected by >>>> GarbageCollector for purging, even though the caller of the function may >>>> want to use the resulting shape for some time to come, after the parent is >>>> "purged": >>>> >>>> def MakeEdge(p1, p2): >>>> gp1,gp2 = [gp_Pnt(x,y,z) for x,y,z in (p1,p2)] >>>> mkedge = BRepBuilderAPI_MakeEdge(gp1, gp2) >>>> return mkedge.Shape() >>>> >>>> 2. Every time a C++ OCC object is retrieved from Python in any way, a >>>> brand new SWIG wrapper object is instantiated for it, very easily/quickly >>>> leading to multiple wrapper objects on the Python side for the same >>>> underlying C++ object (I see this especially when using OCC shape >>>> containers >>>> repeatedly). When any one of these wrapper objects goes out of scope and is >>>> collected and purged, the underlying C++ object is destroyed (via >>>> _kill_pointed), even though there may well be other Python wrapper objects >>>> still needing access to it. >>>> I guess the ideal solution to this issue would be to keep a mapping >>>> of C++ object -> SWIG wrapper object on the C++ side, so that there is >>>> always a 1-to-1 correspondence between the two, but I'm not sure how >>>> plausible this really is from an implementation point-of-view. As an >>>> example >>>> of what I mean, consider this: every time you call "mkedge.Shape()" in my >>>> example above, you get a new/different Python object, wrapping the same >>>> underlying myShape instance, which is totally unexpected and unintuitive >>>> from a Python perspective. A "pure" Python programmer would expect to get >>>> the exact same Python object back every time, and not different wrapper >>>> objects with just an "artificial" identical hash value. >>>> >>>> I realize that this is a very difficult issue, and that as users of the >>>> PythonOCC wrapper library we will need to take responsibility for memory >>>> management issues, but I can see lots of confusion for Python programmers >>>> using the library and expecting "Python-like" behaviour out of the box. >>>> >>>> I wish I new more about the SWIG wrapper object instantiation process so >>>> that I could make more concrete suggestions, instead of just complaining >>>> ;-) What is your gut-feeling about this Thomas - do you think these two >>>> issues can be addressed in the wrapper, or is the current way the only >>>> viable implementation? If you can give me a short description of the SWIG >>>> wrapper instantiation process, I may be able to delve into it myself. >>>> >>>> Thanks for your help with this thus far, >>>> Frank >>>> >>>> >>>> >>>> >>>> On 16/03/2010 9:04 AM, Thomas Paviot wrote: >>>> >>>> Hi Frank, >>>> >>>> I also suggest you use the push_context() and pop_context() methods at >>>> the beginning and the end of your algorithm. garbage.push_context() creates >>>> a new collector. Objects collected will go to this new list. Then, at the >>>> end of your algorithm, just call .smart_purge() and then pop_context(). >>>> Otherwise, when calling the smart_purge() method, you'll erase all other >>>> objects that might have been stored in the gc thus causing strange things. >>>> >>>> def my_high_memory_consuming_algorithm(): >>>> GarbageCollector.garbage.push_context() >>>> >>>> ... do whatever you want ... >>>> >>>> GarbageCollector.garbage.smart_purge() #only the objects created >>>> since the call to push_context are erased. Use MMGT_OPT=0 to make sure >>>> freeing #memory. >>>> GarbageCollector.pop_context() >>>> return data >>>> >>>> There's also a garbage.prevent_object(object) method, that will take >>>> care to not delete an object that it's important to keep reference. >>>> >>>> These three functions are quite new (but validated with a unittest). >>>> I'm curious to have your feedback! >>>> >>>> Regards, >>>> >>>> Thomas >>>> >>>> These two functions are quite new, >>>> 2010/3/16 Frank Conradie <fr...@qfin.net> >>>> >>>>> Hi Thomas >>>>> >>>>> Well, I'm glad it's not a bug then! I can think of a few ways to handle >>>>> this situation in my algorithm, so it should not be a problem. I'm just >>>>> wondering how many other classes behave in this "unexpected" way :O >>>>> >>>>> Cheers, >>>>> Frank >>>>> >>>>> >>>>> On 15/03/2010 10:26 PM, Thomas Paviot wrote: >>>>> >>>>> Hi Frank, >>>>> >>>>> This behaviour does not come from SWIG but from OpenCASCADE itself. >>>>> Let me explain you in a few words what happens here. >>>>> >>>>> The BRepPrimAPI_MakeBox class inherits from the >>>>> BRepBuilderAPI_MakeShape one. This class provides the Shape() method you >>>>> can >>>>> find in BRepPrimAPI_Make* basic geometry constructors. If you have a look >>>>> at >>>>> the Shape() method in the file BRepBuilderAPI_MakeShape.cxx, you will >>>>> read: >>>>> >>>>> """ >>>>> >>>>> //======================================================================= >>>>> //function : Shape >>>>> //purpose : >>>>> >>>>> //======================================================================= >>>>> >>>>> const TopoDS_Shape& BRepBuilderAPI_MakeShape::Shape() const >>>>> { >>>>> if (!IsDone()) { >>>>> // the following is const cast away >>>>> ((BRepBuilderAPI_MakeShape*) (void*) this)->Build(); >>>>> Check(); >>>>> } >>>>> return myShape; >>>>> } >>>>> """ >>>>> >>>>> The Shape() method then returns a reference to the protected >>>>> attribute 'myShape' of the class BRepBuilderAPI_MakeBox. In other words, >>>>> if >>>>> you delete the BRepPrimAPI_MakeBox instance, you loose the reference to >>>>> the >>>>> shape. >>>>> >>>>> For instance: >>>>> >>>>> >>> from OCC.BRepPrimAPI_MakeBox import * >>>>> >>> from OCC.TopoDS import * >>>>> >>> box = BRepPrimAPI_MakeBox(10,10,10) >>>>> >>> shp = box.Shape() >>>>> >>> print shp.IsNull() >>>>> 0 >>>>> >>> del box #object is garbage collected, not deleted yet >>>>> >>> print shp.IsNull() >>>>> 0 >>>>> >>> GarbageCollector.garbage.smart_purge() # all objects are deleted, >>>>> loose the reference to myShape attribute >>>>> >>> print shp.IsNull() >>>>> 1 >>>>> >>>>> Then force objects deletion only if you're sure you don't need the >>>>> shapes anymore. I don't know what you're trying to do exactly, so it's >>>>> difficult to suggest any best practice. >>>>> >>>>> Regards, >>>>> >>>>> Thomas >>>>> >>>>> 2010/3/15 Frank Conradie <fr...@qfin.net> >>>>> >>>>>> Hi Thomas >>>>>> >>>>>> I am getting crash behaviour in the interim 0.5 PythonOCC release, >>>>>> related to the new GarbageCollector implementation. The code below >>>>>> crashes on my PC: >>>>>> >>>>>> ---------------------------------------- >>>>>> from OCC.gp import * >>>>>> from OCC.BRepPrimAPI import * >>>>>> from OCC.BRepBuilderAPI import * >>>>>> from OCC.TopExp import * >>>>>> from OCC.TopoDS import * >>>>>> >>>>>> o = 0.0 >>>>>> w = 0.1 >>>>>> fp1 = (o,o,o) >>>>>> fp2 = (w,w,w) >>>>>> >>>>>> mkbox = BRepPrimAPI_MakeBox(gp_Pnt(fp1[0],fp1[1],fp1[2]), >>>>>> gp_Pnt(fp2[0],fp2[1],fp2[2])) >>>>>> s = mkbox.Shape() >>>>>> del mkbox >>>>>> >>>>>> print s.ShapeType() >>>>>> GarbageCollector.garbage.smart_purge() >>>>>> # The next line crashes my system >>>>>> print s.ShapeType() >>>>>> ---------------------------------------- >>>>>> >>>>>> It is almost as if calling _kill_pointed for the mkbox causes shape >>>>>> "s" >>>>>> to become "corrupted" in some way. I dug through the SWIG code a bit, >>>>>> but cannot see anything obviously wrong, so I was hoping you could >>>>>> shed >>>>>> some light on the issue. >>>>>> >>>>>> By the way, I did get the SVN compiled myself as well, and get the >>>>>> same >>>>>> crash with my compiled lib as well as the one you made available on >>>>>> your >>>>>> website. >>>>>> >>>>>> Thanks, >>>>>> Frank >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Pythonocc-users mailing list >>>>>> Pythonocc-users@gna.org >>>>>> https://mail.gna.org/listinfo/pythonocc-users >>>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Pythonocc-users mailing >>>>> listpythonocc-us...@gna.orghttps://mail.gna.org/listinfo/pythonocc-users >>>>> >>>>> >>>>> _______________________________________________ >>>>> Pythonocc-users mailing list >>>>> Pythonocc-users@gna.org >>>>> https://mail.gna.org/listinfo/pythonocc-users >>>>> >>>>> >>>> >>>> _______________________________________________ >>>> Pythonocc-users mailing >>>> listpythonocc-us...@gna.orghttps://mail.gna.org/listinfo/pythonocc-users >>>> >>>> >>>> _______________________________________________ >>>> Pythonocc-users mailing list >>>> Pythonocc-users@gna.org >>>> https://mail.gna.org/listinfo/pythonocc-users >>>> >>>> >>> >>> _______________________________________________ >>> Pythonocc-users mailing >>> listpythonocc-us...@gna.orghttps://mail.gna.org/listinfo/pythonocc-users >>> >>> >>> _______________________________________________ >>> Pythonocc-users mailing list >>> Pythonocc-users@gna.org >>> https://mail.gna.org/listinfo/pythonocc-users >>> >>> >> >> _______________________________________________ >> Pythonocc-users mailing list >> Pythonocc-users@gna.org >> https://mail.gna.org/listinfo/pythonocc-users >> >> > > _______________________________________________ > Pythonocc-users mailing > listpythonocc-us...@gna.orghttps://mail.gna.org/listinfo/pythonocc-users > > > _______________________________________________ > Pythonocc-users mailing list > Pythonocc-users@gna.org > https://mail.gna.org/listinfo/pythonocc-users > >
_______________________________________________ Pythonocc-users mailing list Pythonocc-users@gna.org https://mail.gna.org/listinfo/pythonocc-users
