Hi thomas, thanks as always for weighing in. You point about c++ to python portability is interesting. There is another strong argument that supports retaining a set of objects that map exactly to the occ ones. Doing so enables using code and suggestions from the opencascade forums and documentation to be relevant and accurate.
However all of this is solved by ensuring a fairly strict separation between the 'high level' api and the core one. I'll simply decide which I want based on the needs at the time. I think you would see a decent amount of activity in a 'high level' api if it was visible and documented. I think this would happen because there are lots of people like me: I know little c++, and little swig, but hey I can bang out some pure python! On 3/17/10, Thomas Paviot <tpav...@gmail.com> wrote: > Dave, Frank, > > Thank you for reporting your feelings and needs for a better pythonOCC. > Maybe it's time to discuss with you the future of this library, and the > paths we should The project was born 2 years ago, and my initial goal is > reached: it's now possible to do the same things with pythonOCC as with OCC, > but from the Python language: all algorithms posted on the OCC forum can be > easily ported to python, and they work the same way. > > When starting the project, I wanted pythonOCC to be as close as possible to > the original C++ API. In my mind, python was just a convenient way to > prototype C++ applications, so it was really important to me that the > move from python to C++ (or the opposite) was as simple as possible: same > classes/method names, same algorithms etc. > > However, when developing with pythonOCC, Jelle and I realized that the OCC > API is not so convenient and provide very low level objects. A more > 'pythonic' pythonOCC then raised in our minds. We then started to implement > what we called a 'high level API' (but we're currently discussing the name > of this additional layer): this layer provides useful and *pythonic* > classes/methods/functions on top of OCC. The 'Parametric Application > Framework' is the best example for this API, and is the work I'm the most > proud of: I think Jelle and I achieved a very elegant and pythonic way to > use and extend the features provided by both OCC/SGEOM libraries (see for > instance > http://code.google.com/p/pythonocc/source/browse/trunk/src/addons/PAF/Context.py). > There are also geometric/topological 'high level' functions very similar to > the Frank's MakeEdge function. They are available from the Utils.Construct > module (http://api.pythonocc.org/OCC.Utils.Construct-module.html), but we > never posted anything or documented these functions. > > I agree with both of you Dave, Frank (and I know Jelle also agrees) that the > API needs now to be more pythonic. However, making the library more pythonic > will make us reach an irreversible point: it will not be possible anymore to > move from python code to C++ code. But I think that pythonOCC has now to > live it's own life, and may have to move away from its C++ 'mother' (or > 'father', don't know what is the gender of OCC). > > In order to achieve that point, here are the points we could focus on from > now: > - move TCollection*, TColgp* etc. to python lists. We developed ugly hacks > to transform such classes to python lists, but's it's not perfect and should > be done at the SWIG level, > - implement more 'high level api' functions/methods (as suggested by Frank). > Many of these functions have already been developed (the high level API > source code is here: > http://code.google.com/p/pythonocc/source/browse/#svn/trunk/src/addons). > Maybe they should be more documented, > - for the topics related with memory management, Frank, let me think about > your suggestions the coming two days. Point (1) is closely related to the > way SWIG handles C++ references. OpenCascades makes an extensive use of > these awful '&' symbol. At the asme time, both OCC and python use a ref > counting mechanism for object deletion management: for each class, the SWIG > wrapper provides a python proxy to a C++ object (this C++ object can be > accessed by the 'this' attribute of any pythonOCC object). When deleting a > python class, you have to take care about the deletion of the proxy (python > ref count - gc) and the deletion of the pointed object (OCC ref counting). I > spent *many* hours studying this mechanism, and I can help you to save time > if you ever decide to dive into SWIG. Point (2) is interesting, never > thought about that. I don't know yet whether it's better to do it from the > Python or the SWIG/C++ level. > > Best Regards, > > Thomas > > 2010/3/17 Dave Cowden <dave.cow...@gmail.com> > >> Yes I understand, and agree that it would be better if the memory >> management was a bit easier >> >> I've also found that there is a fairly blurry line between 'pythonicness' >> and 'poor/ugly API'. Having written about 2000+ lines of code, i've >> written >> a lot of wrapper classes. >> >> Many of these wrappers i'd wager are nearly identical to those any other >> python developer writes as soon as they deal with openCascade. for >> example >> generators to convert topexp iteration to a for/in loop, edge wrappers to >> return points convieniently, convienience methods to created edges and >> curves ( I have a function nearly the same as yours in my code btw , case >> in point ). >> >> I guess what i'm saying is that, though garbage collection helps, its also >> true that your MakeEdge method is somewhat compensating for a klunky api >> that doesnt just return Shape() and throw an exception instead of doing >> Perform() then IsDone() and such. Its clear that the solution to this is >> to >> let people contribute helper and wrappers over time so that we all >> benefit. >> >> I think what we need from Thomas/Jelle is a place that these can be >> checked >> in and then apidoc is autogenerated. I'd be more than happy to commit some >> of my utilty/wrapper code, but i'd want to know where i can look to see >> the >> apidoc for the other such code is before I add it... >> >> >> On Wed, Mar 17, 2010 at 1:09 PM, Frank Conradie <fr...@qfin.net> wrote: >> >>> Hi Dave >>> >>> Thanks for the suggestion - closures would certainly help. However, I am >>> currently less interested in ways to work *around* the "un-pythonic" >>> behavior of the wrapper, but would rather like to discuss the >>> technicalities >>> and politics(!) of making the wrapper more "pythonic". Here is what I >>> would >>> ideally like to do: >>> >>> * Clear up exactly what "un-pythonic" behavior I have run into while >>> using >>> the library for a fairly complicated algorithm (two modules, 3000+ lines >>> of >>> code). >>> * Discuss whether it is *technically* possible to change any of this >>> behavior to be more "pythonic" >>> * If it is indeed *technically* possible, discuss whether it is feasible >>> and worthwhile to actually implement >>> >>> Here is a summary of the current "unexpected" behavior as I see it: >>> >>> (1). When using OCC objects in Python, an unsuspecting programmer would >>> expect them to behave like Python objects, i.e. use Python ref counting >>> and >>> garbage collection behavior. E.g. a BRepBuilderAPI object has a >>> TopoDS_Shape >>> member, so this "myShape" has a refcount of 1 initially, and if it is >>> retrieved via the Shape() method and put into, say, a list, its refcount >>> must be 2, etc. So even if the BRepBuilderAPI object goes out of scope, >>> its >>> internal shape is expected to still have a refcount of 1 and "survive" a >>> garbage collector purge. However, this refcounting obviously does not >>> happen >>> on the C++ side, so the BRepBuilderAPI object would have to be kept >>> "alive" >>> as long as its internal shape is needed. >>> >>> (2). Getting the "internal" shape from a BRepBuilderAPI, BRepPrimAPI, >>> BRepAlgoAPI, etc. object: >>> * C++: calling Shape() always returns the *same* myShape instance >>> * Python: calling Shape() returns a *new* SWIG wrapper object every >>> time, each with the same __hash__ value so they "look" like the same >>> instance (at least to sets and dicts) >>> * The same unexpected behavior occurs when "exploring" a shape, i.e. >>> new SWIG wrapper objects are instantiated for every returned shape >>> * The fact that different SWIG wrapper objects hash to the same value >>> at least allows us to identify unique shapes when put in sets and dicts, >>> but >>> it confuses the GarbageCollector, as the same C++ object can be >>> "collected" >>> multiple times, once for each SWIG object wrapping it >>> >>> Now, I can live with point (1), as I can see how this would be near >>> impossible to implement to work like it would in Python. However, I >>> really >>> would like to discuss whether point (2) can be addressed, so that there >>> is >>> always a 1-to-1 correspondence between a C++ object and its SWIG wrapper >>> object. The SWIG wrapper already generates a single unique __hash__ value >>> for a single C++ object, so would it not be feasible to keep a map of C++ >>> object --> hash value --> SWIG wrapper object? >>> >>> In pseudo-code, when the wrapper is asked for an OCC object: >>> >>> h = hash(occobject) >>> if h in occ2swig_map: >>> swig_wrapper = occ2swig_map[h] >>> else: >>> swig_wrapper = new SwigWrapper(occobject) >>> occ2swig_map[h] = swig_wrapper >>> return swig_wrapper >>> >>> I have never used SWIG myself, so excuse any ignorance on my side (I will >>> try and spend some time looking deeper into SWIG over the next few days). >>> >>> Hopefully we can have a fruitful and open discussion about this. I can >>> certainly work with the library as-is, and am indeed doing so >>> successfully >>> at the moment. I just find that it is no fun having to deal with the very >>> kinds of C++ issues I am trying to avoid by using Python in the 1st place >>> ;-) >>> >>> Cheers, >>> Frank >>> >>> >>> >>> >>> On 17/03/2010 4:22 AM, Dave Cowden wrote: >>> >>> Hi, Frank: >>> >>> I am not sure, but I think you could implement your utility function >>> using a python closure. This would allow the original object to remain in >>> scope until the nested function has completed. so for example: >>> >>> To do closures in python, you have to use lamda, which creates a new >>> function having access to the outer scope, even after the function >>> returns. >>> So >>> >>> def MakeEdge(p1, p2): >>> gp1,gp2 = [gp_Pnt(x,y,z) for x,y,z in (p1,p2)] >>> mkedge = BRepBuilderAPI_MakeEdge(gp1, gp2) >>> return mkedge.Shape() >>> >>> Becomes: >>> >>> def MakeEdge(p1,p2): >>> gp1,gp2 = [gp_Pnt(x,y,z) for x,y,z in (p1,p2)] >>> mkedge = BRepBuilderAPI_MakeEdge(gp1, gp2) >>> return lambda e: mkedge.Shape(); >>> >>> now you can do >>> me = MakeEdge(p1,p2); >>> >>> me.Shape() //return the generated shape >>> me.Shape() //return the generated shap again. It will be the same object >>> because the underlying brepbuilder is still in a closure >>> >>> Now if you kill me, the closure will be discarded. >>> >>> Dont know if this helps any or not. >>> >>> On Tue, Mar 16, 2010 at 7:53 PM, Frank Conradie <fr...@qfin.net> wrote: >>> >>>> Hi Thomas >>>> >>>> After much investigation today, I have now added a flag to >>>> GarbageCollecter to disable it for our use, as it really does not work >>>> as >>>> intended for relatively complex code (I now manually track which objects >>>> I >>>> create and need to call _kill_pointed for). There are two main issues >>>> that >>>> we run into with the current GarbageCollector: >>>> >>>> 1. As already discussed, classes like the BRepPrimAPI and BRepAlgoAPI >>>> have an internal "myShape" data member that gets destroyed when the >>>> parent >>>> object's _kill_pointed method is called, causing all kinds of issues. As >>>> an >>>> example, it makes utility functions like the one below unusable, as the >>>> local BRepPrimAPI object goes out of scope and is thus collected by >>>> GarbageCollector for purging, even though the caller of the function may >>>> want to use the resulting shape for some time to come, after the parent >>>> is >>>> "purged": >>>> >>>> def MakeEdge(p1, p2): >>>> gp1,gp2 = [gp_Pnt(x,y,z) for x,y,z in (p1,p2)] >>>> mkedge = BRepBuilderAPI_MakeEdge(gp1, gp2) >>>> return mkedge.Shape() >>>> >>>> 2. Every time a C++ OCC object is retrieved from Python in any way, a >>>> brand new SWIG wrapper object is instantiated for it, very >>>> easily/quickly >>>> leading to multiple wrapper objects on the Python side for the same >>>> underlying C++ object (I see this especially when using OCC shape >>>> containers >>>> repeatedly). When any one of these wrapper objects goes out of scope and >>>> is >>>> collected and purged, the underlying C++ object is destroyed (via >>>> _kill_pointed), even though there may well be other Python wrapper >>>> objects >>>> still needing access to it. >>>> I guess the ideal solution to this issue would be to keep a mapping >>>> of C++ object -> SWIG wrapper object on the C++ side, so that there is >>>> always a 1-to-1 correspondence between the two, but I'm not sure how >>>> plausible this really is from an implementation point-of-view. As an >>>> example >>>> of what I mean, consider this: every time you call "mkedge.Shape()" in >>>> my >>>> example above, you get a new/different Python object, wrapping the same >>>> underlying myShape instance, which is totally unexpected and unintuitive >>>> from a Python perspective. A "pure" Python programmer would expect to >>>> get >>>> the exact same Python object back every time, and not different wrapper >>>> objects with just an "artificial" identical hash value. >>>> >>>> I realize that this is a very difficult issue, and that as users of the >>>> PythonOCC wrapper library we will need to take responsibility for memory >>>> management issues, but I can see lots of confusion for Python >>>> programmers >>>> using the library and expecting "Python-like" behaviour out of the box. >>>> >>>> I wish I new more about the SWIG wrapper object instantiation process so >>>> that I could make more concrete suggestions, instead of just complaining >>>> ;-) What is your gut-feeling about this Thomas - do you think these two >>>> issues can be addressed in the wrapper, or is the current way the only >>>> viable implementation? If you can give me a short description of the >>>> SWIG >>>> wrapper instantiation process, I may be able to delve into it myself. >>>> >>>> Thanks for your help with this thus far, >>>> Frank >>>> >>>> >>>> >>>> >>>> On 16/03/2010 9:04 AM, Thomas Paviot wrote: >>>> >>>> Hi Frank, >>>> >>>> I also suggest you use the push_context() and pop_context() methods at >>>> the beginning and the end of your algorithm. garbage.push_context() >>>> creates >>>> a new collector. Objects collected will go to this new list. Then, at >>>> the >>>> end of your algorithm, just call .smart_purge() and then pop_context(). >>>> Otherwise, when calling the smart_purge() method, you'll erase all other >>>> objects that might have been stored in the gc thus causing strange >>>> things. >>>> >>>> def my_high_memory_consuming_algorithm(): >>>> GarbageCollector.garbage.push_context() >>>> >>>> ... do whatever you want ... >>>> >>>> GarbageCollector.garbage.smart_purge() #only the objects created >>>> since the call to push_context are erased. Use MMGT_OPT=0 to make sure >>>> freeing #memory. >>>> GarbageCollector.pop_context() >>>> return data >>>> >>>> There's also a garbage.prevent_object(object) method, that will take >>>> care to not delete an object that it's important to keep reference. >>>> >>>> These three functions are quite new (but validated with a unittest). >>>> I'm curious to have your feedback! >>>> >>>> Regards, >>>> >>>> Thomas >>>> >>>> These two functions are quite new, >>>> 2010/3/16 Frank Conradie <fr...@qfin.net> >>>> >>>>> Hi Thomas >>>>> >>>>> Well, I'm glad it's not a bug then! I can think of a few ways to handle >>>>> this situation in my algorithm, so it should not be a problem. I'm just >>>>> wondering how many other classes behave in this "unexpected" way :O >>>>> >>>>> Cheers, >>>>> Frank >>>>> >>>>> >>>>> On 15/03/2010 10:26 PM, Thomas Paviot wrote: >>>>> >>>>> Hi Frank, >>>>> >>>>> This behaviour does not come from SWIG but from OpenCASCADE itself. >>>>> Let me explain you in a few words what happens here. >>>>> >>>>> The BRepPrimAPI_MakeBox class inherits from the >>>>> BRepBuilderAPI_MakeShape one. This class provides the Shape() method >>>>> you can >>>>> find in BRepPrimAPI_Make* basic geometry constructors. If you have a >>>>> look at >>>>> the Shape() method in the file BRepBuilderAPI_MakeShape.cxx, you will >>>>> read: >>>>> >>>>> """ >>>>> >>>>> //======================================================================= >>>>> //function : Shape >>>>> //purpose : >>>>> >>>>> //======================================================================= >>>>> >>>>> const TopoDS_Shape& BRepBuilderAPI_MakeShape::Shape() const >>>>> { >>>>> if (!IsDone()) { >>>>> // the following is const cast away >>>>> ((BRepBuilderAPI_MakeShape*) (void*) this)->Build(); >>>>> Check(); >>>>> } >>>>> return myShape; >>>>> } >>>>> """ >>>>> >>>>> The Shape() method then returns a reference to the protected attribute >>>>> 'myShape' of the class BRepBuilderAPI_MakeBox. In other words, if you >>>>> delete >>>>> the BRepPrimAPI_MakeBox instance, you loose the reference to the shape. >>>>> >>>>> For instance: >>>>> >>>>> >>> from OCC.BRepPrimAPI_MakeBox import * >>>>> >>> from OCC.TopoDS import * >>>>> >>> box = BRepPrimAPI_MakeBox(10,10,10) >>>>> >>> shp = box.Shape() >>>>> >>> print shp.IsNull() >>>>> 0 >>>>> >>> del box #object is garbage collected, not deleted yet >>>>> >>> print shp.IsNull() >>>>> 0 >>>>> >>> GarbageCollector.garbage.smart_purge() # all objects are deleted, >>>>> loose the reference to myShape attribute >>>>> >>> print shp.IsNull() >>>>> 1 >>>>> >>>>> Then force objects deletion only if you're sure you don't need the >>>>> shapes anymore. I don't know what you're trying to do exactly, so it's >>>>> difficult to suggest any best practice. >>>>> >>>>> Regards, >>>>> >>>>> Thomas >>>>> >>>>> 2010/3/15 Frank Conradie <fr...@qfin.net> >>>>> >>>>>> Hi Thomas >>>>>> >>>>>> I am getting crash behaviour in the interim 0.5 PythonOCC release, >>>>>> related to the new GarbageCollector implementation. The code below >>>>>> crashes on my PC: >>>>>> >>>>>> ---------------------------------------- >>>>>> from OCC.gp import * >>>>>> from OCC.BRepPrimAPI import * >>>>>> from OCC.BRepBuilderAPI import * >>>>>> from OCC.TopExp import * >>>>>> from OCC.TopoDS import * >>>>>> >>>>>> o = 0.0 >>>>>> w = 0.1 >>>>>> fp1 = (o,o,o) >>>>>> fp2 = (w,w,w) >>>>>> >>>>>> mkbox = BRepPrimAPI_MakeBox(gp_Pnt(fp1[0],fp1[1],fp1[2]), >>>>>> gp_Pnt(fp2[0],fp2[1],fp2[2])) >>>>>> s = mkbox.Shape() >>>>>> del mkbox >>>>>> >>>>>> print s.ShapeType() >>>>>> GarbageCollector.garbage.smart_purge() >>>>>> # The next line crashes my system >>>>>> print s.ShapeType() >>>>>> ---------------------------------------- >>>>>> >>>>>> It is almost as if calling _kill_pointed for the mkbox causes shape >>>>>> "s" >>>>>> to become "corrupted" in some way. I dug through the SWIG code a bit, >>>>>> but cannot see anything obviously wrong, so I was hoping you could >>>>>> shed >>>>>> some light on the issue. >>>>>> >>>>>> By the way, I did get the SVN compiled myself as well, and get the >>>>>> same >>>>>> crash with my compiled lib as well as the one you made available on >>>>>> your >>>>>> website. >>>>>> >>>>>> Thanks, >>>>>> Frank >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Pythonocc-users mailing list >>>>>> Pythonocc-users@gna.org >>>>>> https://mail.gna.org/listinfo/pythonocc-users >>>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Pythonocc-users mailing >>>>> listpythonocc-us...@gna.orghttps://mail.gna.org/listinfo/pythonocc-users >>>>> >>>>> >>>>> _______________________________________________ >>>>> Pythonocc-users mailing list >>>>> Pythonocc-users@gna.org >>>>> https://mail.gna.org/listinfo/pythonocc-users >>>>> >>>>> >>>> >>>> _______________________________________________ >>>> Pythonocc-users mailing >>>> listpythonocc-us...@gna.orghttps://mail.gna.org/listinfo/pythonocc-users >>>> >>>> >>>> _______________________________________________ >>>> Pythonocc-users mailing list >>>> Pythonocc-users@gna.org >>>> https://mail.gna.org/listinfo/pythonocc-users >>>> >>>> >>> >>> _______________________________________________ >>> Pythonocc-users mailing >>> listpythonocc-us...@gna.orghttps://mail.gna.org/listinfo/pythonocc-users >>> >>> >>> _______________________________________________ >>> Pythonocc-users mailing list >>> Pythonocc-users@gna.org >>> https://mail.gna.org/listinfo/pythonocc-users >>> >>> >> >> _______________________________________________ >> Pythonocc-users mailing list >> Pythonocc-users@gna.org >> https://mail.gna.org/listinfo/pythonocc-users >> >> > _______________________________________________ Pythonocc-users mailing list Pythonocc-users@gna.org https://mail.gna.org/listinfo/pythonocc-users
