Hi Dave, You're right, there was also the idea to benefit from the OpenCascade documentation. The deal was: 'just go and read occ documentation to get started, I take care of python development'. I agree it's not suitable anymore.
The 'strict separation' you're dealing with is currently the object of a discussion we have with Jelle. Rather than 'strict', I prefer 'functional separation'. From a functional analysis viewpoint, these different levels (or layers) in the API serve specific user needs. Thomas 2010/3/17 Dave Cowden <dave.cow...@gmail.com> > Hi thomas, thanks as always for weighing in. > > You point about c++ to python portability is interesting. There is > another strong argument that supports retaining a set of objects that > map exactly to the occ ones. Doing so enables using code and > suggestions from the opencascade forums and documentation to be > relevant and accurate. > > However all of this is solved by ensuring a fairly strict separation > between the 'high level' api and the core one. I'll simply decide > which I want based on the needs at the time. > > I think you would see a decent amount of activity in a 'high level' > api if it was visible and documented. I think this would happen > because there are lots of people like me: I know little c++, and > little swig, but hey I can bang out some pure python! > > On 3/17/10, Thomas Paviot <tpav...@gmail.com> wrote: > > Dave, Frank, > > > > Thank you for reporting your feelings and needs for a better pythonOCC. > > Maybe it's time to discuss with you the future of this library, and the > > paths we should The project was born 2 years ago, and my initial goal is > > reached: it's now possible to do the same things with pythonOCC as with > OCC, > > but from the Python language: all algorithms posted on the OCC forum can > be > > easily ported to python, and they work the same way. > > > > When starting the project, I wanted pythonOCC to be as close as possible > to > > the original C++ API. In my mind, python was just a convenient way to > > prototype C++ applications, so it was really important to me that the > > move from python to C++ (or the opposite) was as simple as possible: same > > classes/method names, same algorithms etc. > > > > However, when developing with pythonOCC, Jelle and I realized that the > OCC > > API is not so convenient and provide very low level objects. A more > > 'pythonic' pythonOCC then raised in our minds. We then started to > implement > > what we called a 'high level API' (but we're currently discussing the > name > > of this additional layer): this layer provides useful and *pythonic* > > classes/methods/functions on top of OCC. The 'Parametric Application > > Framework' is the best example for this API, and is the work I'm the most > > proud of: I think Jelle and I achieved a very elegant and pythonic way to > > use and extend the features provided by both OCC/SGEOM libraries (see for > > instance > > > http://code.google.com/p/pythonocc/source/browse/trunk/src/addons/PAF/Context.py > ). > > There are also geometric/topological 'high level' functions very similar > to > > the Frank's MakeEdge function. They are available from the > Utils.Construct > > module (http://api.pythonocc.org/OCC.Utils.Construct-module.html), but > we > > never posted anything or documented these functions. > > > > I agree with both of you Dave, Frank (and I know Jelle also agrees) that > the > > API needs now to be more pythonic. However, making the library more > pythonic > > will make us reach an irreversible point: it will not be possible anymore > to > > move from python code to C++ code. But I think that pythonOCC has now to > > live it's own life, and may have to move away from its C++ 'mother' (or > > 'father', don't know what is the gender of OCC). > > > > In order to achieve that point, here are the points we could focus on > from > > now: > > - move TCollection*, TColgp* etc. to python lists. We developed ugly > hacks > > to transform such classes to python lists, but's it's not perfect and > should > > be done at the SWIG level, > > - implement more 'high level api' functions/methods (as suggested by > Frank). > > Many of these functions have already been developed (the high level API > > source code is here: > > http://code.google.com/p/pythonocc/source/browse/#svn/trunk/src/addons). > > Maybe they should be more documented, > > - for the topics related with memory management, Frank, let me think > about > > your suggestions the coming two days. Point (1) is closely related to the > > way SWIG handles C++ references. OpenCascades makes an extensive use of > > these awful '&' symbol. At the asme time, both OCC and python use a ref > > counting mechanism for object deletion management: for each class, the > SWIG > > wrapper provides a python proxy to a C++ object (this C++ object can be > > accessed by the 'this' attribute of any pythonOCC object). When deleting > a > > python class, you have to take care about the deletion of the proxy > (python > > ref count - gc) and the deletion of the pointed object (OCC ref > counting). I > > spent *many* hours studying this mechanism, and I can help you to save > time > > if you ever decide to dive into SWIG. Point (2) is interesting, never > > thought about that. I don't know yet whether it's better to do it from > the > > Python or the SWIG/C++ level. > > > > Best Regards, > > > > Thomas > > > > 2010/3/17 Dave Cowden <dave.cow...@gmail.com> > > > >> Yes I understand, and agree that it would be better if the memory > >> management was a bit easier > >> > >> I've also found that there is a fairly blurry line between > 'pythonicness' > >> and 'poor/ugly API'. Having written about 2000+ lines of code, i've > >> written > >> a lot of wrapper classes. > >> > >> Many of these wrappers i'd wager are nearly identical to those any other > >> python developer writes as soon as they deal with openCascade. for > >> example > >> generators to convert topexp iteration to a for/in loop, edge wrappers > to > >> return points convieniently, convienience methods to created edges and > >> curves ( I have a function nearly the same as yours in my code btw , > case > >> in point ). > >> > >> I guess what i'm saying is that, though garbage collection helps, its > also > >> true that your MakeEdge method is somewhat compensating for a klunky > api > >> that doesnt just return Shape() and throw an exception instead of doing > >> Perform() then IsDone() and such. Its clear that the solution to this > is > >> to > >> let people contribute helper and wrappers over time so that we all > >> benefit. > >> > >> I think what we need from Thomas/Jelle is a place that these can be > >> checked > >> in and then apidoc is autogenerated. I'd be more than happy to commit > some > >> of my utilty/wrapper code, but i'd want to know where i can look to see > >> the > >> apidoc for the other such code is before I add it... > >> > >> > >> On Wed, Mar 17, 2010 at 1:09 PM, Frank Conradie <fr...@qfin.net> wrote: > >> > >>> Hi Dave > >>> > >>> Thanks for the suggestion - closures would certainly help. However, I > am > >>> currently less interested in ways to work *around* the "un-pythonic" > >>> behavior of the wrapper, but would rather like to discuss the > >>> technicalities > >>> and politics(!) of making the wrapper more "pythonic". Here is what I > >>> would > >>> ideally like to do: > >>> > >>> * Clear up exactly what "un-pythonic" behavior I have run into while > >>> using > >>> the library for a fairly complicated algorithm (two modules, 3000+ > lines > >>> of > >>> code). > >>> * Discuss whether it is *technically* possible to change any of this > >>> behavior to be more "pythonic" > >>> * If it is indeed *technically* possible, discuss whether it is > feasible > >>> and worthwhile to actually implement > >>> > >>> Here is a summary of the current "unexpected" behavior as I see it: > >>> > >>> (1). When using OCC objects in Python, an unsuspecting programmer would > >>> expect them to behave like Python objects, i.e. use Python ref counting > >>> and > >>> garbage collection behavior. E.g. a BRepBuilderAPI object has a > >>> TopoDS_Shape > >>> member, so this "myShape" has a refcount of 1 initially, and if it is > >>> retrieved via the Shape() method and put into, say, a list, its > refcount > >>> must be 2, etc. So even if the BRepBuilderAPI object goes out of scope, > >>> its > >>> internal shape is expected to still have a refcount of 1 and "survive" > a > >>> garbage collector purge. However, this refcounting obviously does not > >>> happen > >>> on the C++ side, so the BRepBuilderAPI object would have to be kept > >>> "alive" > >>> as long as its internal shape is needed. > >>> > >>> (2). Getting the "internal" shape from a BRepBuilderAPI, BRepPrimAPI, > >>> BRepAlgoAPI, etc. object: > >>> * C++: calling Shape() always returns the *same* myShape instance > >>> * Python: calling Shape() returns a *new* SWIG wrapper object every > >>> time, each with the same __hash__ value so they "look" like the same > >>> instance (at least to sets and dicts) > >>> * The same unexpected behavior occurs when "exploring" a shape, > i.e. > >>> new SWIG wrapper objects are instantiated for every returned shape > >>> * The fact that different SWIG wrapper objects hash to the same > value > >>> at least allows us to identify unique shapes when put in sets and > dicts, > >>> but > >>> it confuses the GarbageCollector, as the same C++ object can be > >>> "collected" > >>> multiple times, once for each SWIG object wrapping it > >>> > >>> Now, I can live with point (1), as I can see how this would be near > >>> impossible to implement to work like it would in Python. However, I > >>> really > >>> would like to discuss whether point (2) can be addressed, so that there > >>> is > >>> always a 1-to-1 correspondence between a C++ object and its SWIG > wrapper > >>> object. The SWIG wrapper already generates a single unique __hash__ > value > >>> for a single C++ object, so would it not be feasible to keep a map of > C++ > >>> object --> hash value --> SWIG wrapper object? > >>> > >>> In pseudo-code, when the wrapper is asked for an OCC object: > >>> > >>> h = hash(occobject) > >>> if h in occ2swig_map: > >>> swig_wrapper = occ2swig_map[h] > >>> else: > >>> swig_wrapper = new SwigWrapper(occobject) > >>> occ2swig_map[h] = swig_wrapper > >>> return swig_wrapper > >>> > >>> I have never used SWIG myself, so excuse any ignorance on my side (I > will > >>> try and spend some time looking deeper into SWIG over the next few > days). > >>> > >>> Hopefully we can have a fruitful and open discussion about this. I can > >>> certainly work with the library as-is, and am indeed doing so > >>> successfully > >>> at the moment. I just find that it is no fun having to deal with the > very > >>> kinds of C++ issues I am trying to avoid by using Python in the 1st > place > >>> ;-) > >>> > >>> Cheers, > >>> Frank > >>> > >>> > >>> > >>> > >>> On 17/03/2010 4:22 AM, Dave Cowden wrote: > >>> > >>> Hi, Frank: > >>> > >>> I am not sure, but I think you could implement your utility function > >>> using a python closure. This would allow the original object to remain > in > >>> scope until the nested function has completed. so for example: > >>> > >>> To do closures in python, you have to use lamda, which creates a new > >>> function having access to the outer scope, even after the function > >>> returns. > >>> So > >>> > >>> def MakeEdge(p1, p2): > >>> gp1,gp2 = [gp_Pnt(x,y,z) for x,y,z in (p1,p2)] > >>> mkedge = BRepBuilderAPI_MakeEdge(gp1, gp2) > >>> return mkedge.Shape() > >>> > >>> Becomes: > >>> > >>> def MakeEdge(p1,p2): > >>> gp1,gp2 = [gp_Pnt(x,y,z) for x,y,z in (p1,p2)] > >>> mkedge = BRepBuilderAPI_MakeEdge(gp1, gp2) > >>> return lambda e: mkedge.Shape(); > >>> > >>> now you can do > >>> me = MakeEdge(p1,p2); > >>> > >>> me.Shape() //return the generated shape > >>> me.Shape() //return the generated shap again. It will be the same > object > >>> because the underlying brepbuilder is still in a closure > >>> > >>> Now if you kill me, the closure will be discarded. > >>> > >>> Dont know if this helps any or not. > >>> > >>> On Tue, Mar 16, 2010 at 7:53 PM, Frank Conradie <fr...@qfin.net> > wrote: > >>> > >>>> Hi Thomas > >>>> > >>>> After much investigation today, I have now added a flag to > >>>> GarbageCollecter to disable it for our use, as it really does not work > >>>> as > >>>> intended for relatively complex code (I now manually track which > objects > >>>> I > >>>> create and need to call _kill_pointed for). There are two main issues > >>>> that > >>>> we run into with the current GarbageCollector: > >>>> > >>>> 1. As already discussed, classes like the BRepPrimAPI and BRepAlgoAPI > >>>> have an internal "myShape" data member that gets destroyed when the > >>>> parent > >>>> object's _kill_pointed method is called, causing all kinds of issues. > As > >>>> an > >>>> example, it makes utility functions like the one below unusable, as > the > >>>> local BRepPrimAPI object goes out of scope and is thus collected by > >>>> GarbageCollector for purging, even though the caller of the function > may > >>>> want to use the resulting shape for some time to come, after the > parent > >>>> is > >>>> "purged": > >>>> > >>>> def MakeEdge(p1, p2): > >>>> gp1,gp2 = [gp_Pnt(x,y,z) for x,y,z in (p1,p2)] > >>>> mkedge = BRepBuilderAPI_MakeEdge(gp1, gp2) > >>>> return mkedge.Shape() > >>>> > >>>> 2. Every time a C++ OCC object is retrieved from Python in any way, a > >>>> brand new SWIG wrapper object is instantiated for it, very > >>>> easily/quickly > >>>> leading to multiple wrapper objects on the Python side for the same > >>>> underlying C++ object (I see this especially when using OCC shape > >>>> containers > >>>> repeatedly). When any one of these wrapper objects goes out of scope > and > >>>> is > >>>> collected and purged, the underlying C++ object is destroyed (via > >>>> _kill_pointed), even though there may well be other Python wrapper > >>>> objects > >>>> still needing access to it. > >>>> I guess the ideal solution to this issue would be to keep a > mapping > >>>> of C++ object -> SWIG wrapper object on the C++ side, so that there is > >>>> always a 1-to-1 correspondence between the two, but I'm not sure how > >>>> plausible this really is from an implementation point-of-view. As an > >>>> example > >>>> of what I mean, consider this: every time you call "mkedge.Shape()" in > >>>> my > >>>> example above, you get a new/different Python object, wrapping the > same > >>>> underlying myShape instance, which is totally unexpected and > unintuitive > >>>> from a Python perspective. A "pure" Python programmer would expect to > >>>> get > >>>> the exact same Python object back every time, and not different > wrapper > >>>> objects with just an "artificial" identical hash value. > >>>> > >>>> I realize that this is a very difficult issue, and that as users of > the > >>>> PythonOCC wrapper library we will need to take responsibility for > memory > >>>> management issues, but I can see lots of confusion for Python > >>>> programmers > >>>> using the library and expecting "Python-like" behaviour out of the > box. > >>>> > >>>> I wish I new more about the SWIG wrapper object instantiation process > so > >>>> that I could make more concrete suggestions, instead of just > complaining > >>>> ;-) What is your gut-feeling about this Thomas - do you think these > two > >>>> issues can be addressed in the wrapper, or is the current way the only > >>>> viable implementation? If you can give me a short description of the > >>>> SWIG > >>>> wrapper instantiation process, I may be able to delve into it myself. > >>>> > >>>> Thanks for your help with this thus far, > >>>> Frank > >>>> > >>>> > >>>> > >>>> > >>>> On 16/03/2010 9:04 AM, Thomas Paviot wrote: > >>>> > >>>> Hi Frank, > >>>> > >>>> I also suggest you use the push_context() and pop_context() methods > at > >>>> the beginning and the end of your algorithm. garbage.push_context() > >>>> creates > >>>> a new collector. Objects collected will go to this new list. Then, at > >>>> the > >>>> end of your algorithm, just call .smart_purge() and then > pop_context(). > >>>> Otherwise, when calling the smart_purge() method, you'll erase all > other > >>>> objects that might have been stored in the gc thus causing strange > >>>> things. > >>>> > >>>> def my_high_memory_consuming_algorithm(): > >>>> GarbageCollector.garbage.push_context() > >>>> > >>>> ... do whatever you want ... > >>>> > >>>> GarbageCollector.garbage.smart_purge() #only the objects created > >>>> since the call to push_context are erased. Use MMGT_OPT=0 to make sure > >>>> freeing #memory. > >>>> GarbageCollector.pop_context() > >>>> return data > >>>> > >>>> There's also a garbage.prevent_object(object) method, that will take > >>>> care to not delete an object that it's important to keep reference. > >>>> > >>>> These three functions are quite new (but validated with a unittest). > >>>> I'm curious to have your feedback! > >>>> > >>>> Regards, > >>>> > >>>> Thomas > >>>> > >>>> These two functions are quite new, > >>>> 2010/3/16 Frank Conradie <fr...@qfin.net> > >>>> > >>>>> Hi Thomas > >>>>> > >>>>> Well, I'm glad it's not a bug then! I can think of a few ways to > handle > >>>>> this situation in my algorithm, so it should not be a problem. I'm > just > >>>>> wondering how many other classes behave in this "unexpected" way :O > >>>>> > >>>>> Cheers, > >>>>> Frank > >>>>> > >>>>> > >>>>> On 15/03/2010 10:26 PM, Thomas Paviot wrote: > >>>>> > >>>>> Hi Frank, > >>>>> > >>>>> This behaviour does not come from SWIG but from OpenCASCADE itself. > >>>>> Let me explain you in a few words what happens here. > >>>>> > >>>>> The BRepPrimAPI_MakeBox class inherits from the > >>>>> BRepBuilderAPI_MakeShape one. This class provides the Shape() method > >>>>> you can > >>>>> find in BRepPrimAPI_Make* basic geometry constructors. If you have a > >>>>> look at > >>>>> the Shape() method in the file BRepBuilderAPI_MakeShape.cxx, you > will > >>>>> read: > >>>>> > >>>>> """ > >>>>> > >>>>> > //======================================================================= > >>>>> //function : Shape > >>>>> //purpose : > >>>>> > >>>>> > //======================================================================= > >>>>> > >>>>> const TopoDS_Shape& BRepBuilderAPI_MakeShape::Shape() const > >>>>> { > >>>>> if (!IsDone()) { > >>>>> // the following is const cast away > >>>>> ((BRepBuilderAPI_MakeShape*) (void*) this)->Build(); > >>>>> Check(); > >>>>> } > >>>>> return myShape; > >>>>> } > >>>>> """ > >>>>> > >>>>> The Shape() method then returns a reference to the protected > attribute > >>>>> 'myShape' of the class BRepBuilderAPI_MakeBox. In other words, if you > >>>>> delete > >>>>> the BRepPrimAPI_MakeBox instance, you loose the reference to the > shape. > >>>>> > >>>>> For instance: > >>>>> > >>>>> >>> from OCC.BRepPrimAPI_MakeBox import * > >>>>> >>> from OCC.TopoDS import * > >>>>> >>> box = BRepPrimAPI_MakeBox(10,10,10) > >>>>> >>> shp = box.Shape() > >>>>> >>> print shp.IsNull() > >>>>> 0 > >>>>> >>> del box #object is garbage collected, not deleted yet > >>>>> >>> print shp.IsNull() > >>>>> 0 > >>>>> >>> GarbageCollector.garbage.smart_purge() # all objects are deleted, > >>>>> loose the reference to myShape attribute > >>>>> >>> print shp.IsNull() > >>>>> 1 > >>>>> > >>>>> Then force objects deletion only if you're sure you don't need the > >>>>> shapes anymore. I don't know what you're trying to do exactly, so > it's > >>>>> difficult to suggest any best practice. > >>>>> > >>>>> Regards, > >>>>> > >>>>> Thomas > >>>>> > >>>>> 2010/3/15 Frank Conradie <fr...@qfin.net> > >>>>> > >>>>>> Hi Thomas > >>>>>> > >>>>>> I am getting crash behaviour in the interim 0.5 PythonOCC release, > >>>>>> related to the new GarbageCollector implementation. The code below > >>>>>> crashes on my PC: > >>>>>> > >>>>>> ---------------------------------------- > >>>>>> from OCC.gp import * > >>>>>> from OCC.BRepPrimAPI import * > >>>>>> from OCC.BRepBuilderAPI import * > >>>>>> from OCC.TopExp import * > >>>>>> from OCC.TopoDS import * > >>>>>> > >>>>>> o = 0.0 > >>>>>> w = 0.1 > >>>>>> fp1 = (o,o,o) > >>>>>> fp2 = (w,w,w) > >>>>>> > >>>>>> mkbox = BRepPrimAPI_MakeBox(gp_Pnt(fp1[0],fp1[1],fp1[2]), > >>>>>> gp_Pnt(fp2[0],fp2[1],fp2[2])) > >>>>>> s = mkbox.Shape() > >>>>>> del mkbox > >>>>>> > >>>>>> print s.ShapeType() > >>>>>> GarbageCollector.garbage.smart_purge() > >>>>>> # The next line crashes my system > >>>>>> print s.ShapeType() > >>>>>> ---------------------------------------- > >>>>>> > >>>>>> It is almost as if calling _kill_pointed for the mkbox causes shape > >>>>>> "s" > >>>>>> to become "corrupted" in some way. I dug through the SWIG code a > bit, > >>>>>> but cannot see anything obviously wrong, so I was hoping you could > >>>>>> shed > >>>>>> some light on the issue. > >>>>>> > >>>>>> By the way, I did get the SVN compiled myself as well, and get the > >>>>>> same > >>>>>> crash with my compiled lib as well as the one you made available on > >>>>>> your > >>>>>> website. > >>>>>> > >>>>>> Thanks, > >>>>>> Frank > >>>>>> > >>>>>> > >>>>>> _______________________________________________ > >>>>>> Pythonocc-users mailing list > >>>>>> Pythonocc-users@gna.org > >>>>>> https://mail.gna.org/listinfo/pythonocc-users > >>>>>> > >>>>> > >>>>> > >>>>> _______________________________________________ > >>>>> Pythonocc-users mailing > >>>>> listpythonocc-us...@gna.orghttps:// > mail.gna.org/listinfo/pythonocc-users > >>>>> > >>>>> > >>>>> _______________________________________________ > >>>>> Pythonocc-users mailing list > >>>>> Pythonocc-users@gna.org > >>>>> https://mail.gna.org/listinfo/pythonocc-users > >>>>> > >>>>> > >>>> > >>>> _______________________________________________ > >>>> Pythonocc-users mailing > >>>> listpythonocc-us...@gna.orghttps:// > mail.gna.org/listinfo/pythonocc-users > >>>> > >>>> > >>>> _______________________________________________ > >>>> Pythonocc-users mailing list > >>>> Pythonocc-users@gna.org > >>>> https://mail.gna.org/listinfo/pythonocc-users > >>>> > >>>> > >>> > >>> _______________________________________________ > >>> Pythonocc-users mailing > >>> listpythonocc-us...@gna.orghttps:// > mail.gna.org/listinfo/pythonocc-users > >>> > >>> > >>> _______________________________________________ > >>> Pythonocc-users mailing list > >>> Pythonocc-users@gna.org > >>> https://mail.gna.org/listinfo/pythonocc-users > >>> > >>> > >> > >> _______________________________________________ > >> Pythonocc-users mailing list > >> Pythonocc-users@gna.org > >> https://mail.gna.org/listinfo/pythonocc-users > >> > >> > > > > _______________________________________________ > Pythonocc-users mailing list > Pythonocc-users@gna.org > https://mail.gna.org/listinfo/pythonocc-users >
_______________________________________________ Pythonocc-users mailing list Pythonocc-users@gna.org https://mail.gna.org/listinfo/pythonocc-users
