> On May 28, 2015, at 5:50 AM, Phil Rosenberg <p.d.rosenb...@gmail.com> wrote: > > Hi Jim > Sorry, when I suggested a plmalloc and plfree I meant for these to > call free and malloc, plus do some extra bookkeeping for us to keep > track of our garbage collection needs, not as functions to interact > directly with the OS to bypass free and malloc > > I think the following could be a useful model. If in plstream we have > variables something like > void **tempMem; > PLINT nTempMemArrays; > void **nextTempMemArray; > > then a function plmalloc, which would malloc the needed memory, assign > it to nextTempMemArray, increment nextTempMemArray ready for the next > allocation and if needed increase the size of tempMem. > plfree would free the memory, search tempMem and set that element to > NULL, possibly rolling back nextTempMemArray to the point after the > last non-null pointer. > Then when we return from an API function we can call a function > something like plCleanMem which would free all memory from the tempMem > array. > In fact there would really be little need to call plfree unless the > memory used is large and we are worried about resources - before > returning to the calling program everything would be cleaned up > anyway. > > Even without a setjmp/longjmp error handling mechanism, this seems > like a useful way for us to manage memory in plplot and avoid memory > leaks. I have heard about using memory pools before, but what happens > when the pool runs out, does the whole pool need reallocating? I think > the method above does what we need with minimal fuss, but am very open > to other suggestions - this is all new to me in a C context. Comments > anyone? > > The only issue I can think of is rentrancy. If we call an API function > internally then it will free all the memory before returning, some of > which is likely to still be in use. So something like this would need > us to separate out the public API from internal functions which do the > same job - the public API would just call the internal function. Does > anyone know if this would affect the other language bindings? >
I would caution against this approach on rolling our own garbage collection. It really is a hard problem to do correctly (let alone in a thread safe fashion). We are far better off calling free() when the memory is not utilized. I would advocate using the atexit() function, which appears to be available on many (if not all) the platforms we support. > With this in place, using setjmp/longjmp seems to give us a trivial > way to remove all our plexit calls. > > Phil > > On 26 May 2015 at 21:17, Jim Dishaw <j...@dishaw.org> wrote: >> >> >> >> >>> On May 26, 2015, at 3:53 PM, Phil Rosenberg <p.d.rosenb...@gmail.com> wrote: >>> >>> Okay, well it was an option I thought I would put out there as I think >>> it was worth considering. I will try to answer the questions people >>> had anyway in case people are interested. >>> >>> Regarding domestic bindings, the C front end would remain. Although >>> the API would have to change if we wanted to return an error code. but >>> this is about plplot internal code. What is the maximum number of >>> levels back which we might need to pass an error code? A dozen or more >>> probably. The idea is to avoid all that internal book keeping. >>> Especially if 10 layers down a memory allocation fails. Every layer up >>> from this needs to have code to deal with checking error codes for >>> every function it calls and then free resources and propagate that >>> error code. With the C++ model a throw does all that automatically. >>> >> >> I was looking at consolidating all the error/warning messages routines into >> one or two functions and create a new file in the src directory. One of the >> motivations was to cleanup some of the mallocs and static char array >> allocations that are scattered throughout the code. I have some code that I >> have been using for awhile that I was going to offer. The other advantage is >> that it makes is easier to implement different translations. >> >>> Regarding efficiency. There would almost certainly be no change. The >>> compiler would probably optimise away anything superfluous. >>> >>> As far as I know C++ is ubiquitous. The earliest C++ compilers >>> actually rewrote the C++ to C and passed it to a C compiler! >>> >>> Compilation speed for C++ can be slower if there is a lot of use of >>> Templates. Templates are functions or classes which can have a version >>> for any variable type, so instead of writing >>> int round( int var, int basis); >>> float round( float var, float basis); >>> double round( double var, doublebasis); >>> ... >>> >>> you have >>> template<class T> >>> T round( T var, T basis); >>> >>> Then the user can call round with any type and it will work, providing >>> the code is compilable when you replace T with your type or class. But >>> there is a compiletime cost associated with the extra work required by >>> the compiler to do this. >>> >>> Regarding name mangling we can use extern C to give C name mangling to >>> the API, then the library, both static and dynamic as I understand it, >>> behaves just like a C library. >>> >>> As far as using a C++ compiler is concerned I am afraid that is >>> something I have never worried about as I only write C++ programs so I >>> always link against C runtimes. But as Alan says we already use C++ >>> code in PLPlot so this should already be taken care of. >>> >>> If we are not interested in moving to C++ then we still need a method >>> to propagate errors up through the multiple layers of code in the >>> PLPlot library and do so in a way which minimises the risk of memory >>> or other resource leaks. Because I work in C++ I'm not necessarily the >>> best person to know best practice for this. But here are some ideas. I >>> would be very interested to hear comments and more suggestions. >>> >>> 1) We make every internal function in PLPlot return an error code and >>> check these codes religiously at every call. Even simple functions >>> would probably have to do this because in the future simple functions >>> may grow to more complex ones and changing a function that previously >>> returned a meaningful value to one which returns an error code is >>> likely to be error prone so best get them all out of the way. >>> >>> 2) We include an error code in the PLStream struct. We then check this >>> after every function call to check if it has changed. Again this just >>> requires us to be religious about checking the current error code. >>> >>> 3)I just found out about setjmp and longjmp. These are C macros. As >>> far as I can tell setjmp saves the state of the program stack and >>> creates a jump point. Calling longjump jumps back to the jump point >>> and restores the state. However, any memory malloc'd in the meantime >>> will leak, unless we have some way to free it later. This might be >>> conceivable by having an array of pointers in the PLStream to all >>> memory allocated (maybe create a function plmalloc and plfree to deal >>> with this?) which we can deal with at the end. Does anyone have any >>> experience with setjmp and longjmp or any advice? I also don't know >>> how it deals with re-entrancy or nesting of setjmp calls. Or how to >>> deal with our C++ code calling our C code - does our C++ code only >>> call the public API or does it have access to internal routines? >>> >> >> I have used the setjmp/longjmp approach. You definitely want to minimize the >> mallocs and the "distance" you jump. I prefer to do big mallocs and then >> partition the chunk of memory as needed rather than calling malloc multiple >> times. That helps when unwinding to cleanup on an error condition. Wrapping >> malloc/free with a plmalloc/plfree approach can be useful if the intent is >> to help cleanup. I do not recommend trying to manage memory because it is >> hard job to do correctly. >> >>> 4) Any other suggestions? >>> >>> All these methods require us to have some strategy for dealing with >>> freeing memory although it is often trivial, in some cases this can be >>> complex and a strong candidate for future bugs and memory leaks. >>> >>> I'm really keen to hear people's thoughts on this. As I said I work in >>> C++, so I don't know best practice in C for dealing with this, but we >>> definitely need to make a call and have a definite strategy for this >>> otherwise it will be a nightmare. >>> >>> Phil >>> >>> >>>> On 25 May 2015 at 21:06, Alan W. Irwin <ir...@beluga.phys.uvic.ca> wrote: >>>>> On 2015-05-25 17:29-0000 Arjen Markus wrote: >>>>> >>>>> An issue related to the use of C++ that has not been raised yet, but >>>> which surfaced recently in my comprehensive testing efforts is the >>>> fact that linking a C++ program requires a C++-enabled linker. Thus >>>> introducing C++ as the language in which PLplot is to be implemented >>>> would complicate the use of static builds. That may not be the most >>>> common option nowadays, but I think we need to take a conscious >>>> decision: do we want to continue to support static builds or not? One >>>> pro for static builds is that they make deployment, especially of >>>> binary-only distributions much easier (and safer). >>>> >>>> Hi Arjen: >>>> >>>> Yes, I think we should keep supporting static builds which work >>>> virtually perfectly now with our CMake-based build systems for the >>>> build of PLplot from source and the build of the installed examples. >>>> >>>> I assume what you mean by a C++-enabled linker is that extra libraries >>>> have to be linked in for that case. Our CMake build handles this >>>> situation with ease both for the core build and installed examples >>>> build. So static linking is already completely supported in that case. >>>> >>>> Of course, there is currently a limitation on our traditional >>>> (Makefile + pkg-config) build for e.g., Fortran examples where >>>> pkg-config does not automatically know the name/location of the >>>> special C++ library that needs to be linked in for a given C++ >>>> compiler so the user would have to add that link himself to the >>>> pkg-config results to successfully build the Fortran examples using >>>> our traditional build system for the installed examples. >>>> >>>> Currently this problem occurs if C++ code is included in libplplot >>>> from our C++ device drivers, psttf, qt, and wxwidgets. But that is a >>>> very common case (or should be) to have those device drivers enabled >>>> so if we adopt C++ for the core library this limitation in our >>>> traditional build will essentially just stay the same in most cases. >>>> So I don't see this limitation of our traditional build system for the >>>> installed examples as a big concern with switching our core library to >>>> C++ in all cases. >>>> >>>> Don't get me wrong, I would like this limitation to be resolved so >>>> that our traditional build of the installed examples works as well as >>>> the CMake build of those. When discussing this with Andrew I >>>> mentioned one possibility for implementing a fix for this issue, but >>>> that is a lot of work which I am going to leave to others if they want >>>> to make our Makefile+pkg-config approach as high quality as the >>>> CMake-based one for building our installed examples. >>>> >>>> Alan >>>> >>>> __________________________ >>>> Alan W. Irwin >>>> >>>> Astronomical research affiliation with Department of Physics and Astronomy, >>>> University of Victoria (astrowww.phys.uvic.ca). >>>> >>>> Programming affiliations with the FreeEOS equation-of-state >>>> implementation for stellar interiors (freeeos.sf.net); the Time >>>> Ephemerides project (timeephem.sf.net); PLplot scientific plotting >>>> software package (plplot.sf.net); the libLASi project >>>> (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); >>>> and the Linux Brochure Project (lbproject.sf.net). >>>> __________________________ >>>> >>>> Linux-powered Science >>>> __________________________ >>>> >>>> ------------------------------------------------------------------------------ >>>> One dashboard for servers and applications across Physical-Virtual-Cloud >>>> Widest out-of-the-box monitoring support with 50+ applications >>>> Performance metrics, stats and reports that give you Actionable Insights >>>> Deep dive visibility with transaction tracing using APM Insight. >>>> http://ad.doubleclick.net/ddm/clk/290420510;117567292;y >>>> _______________________________________________ >>>> Plplot-devel mailing list >>>> Plplot-devel@lists.sourceforge.net >>>> https://lists.sourceforge.net/lists/listinfo/plplot-devel >>> >>> ------------------------------------------------------------------------------ >>> _______________________________________________ >>> Plplot-devel mailing list >>> Plplot-devel@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/plplot-devel ------------------------------------------------------------------------------ _______________________________________________ Plplot-devel mailing list Plplot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/plplot-devel
