Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-10-05 Thread Adriano dos Santos Fernandes
On 21/09/2021 10:15, Alex Peshkoff via Firebird-devel wrote: > On 9/21/21 2:50 PM, Adriano dos Santos Fernandes wrote: >> I think we can improve things here. >> >> We have our GlobalPtr, InitInstance and we may continue using them. And >> we have FB_NEW which allocates memory from the global

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-09-21 Thread Adriano dos Santos Fernandes
On 21/09/2021 09:24, Adriano dos Santos Fernandes wrote: > On 21/09/2021 09:11, Dimitry Sibiryakov wrote: >> Adriano dos Santos Fernandes wrote 21.09.2021 13:50: >>> We may split that global default pool in two, so we'll have another >>> default pool for external (not controlled by us) libraries.

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-09-21 Thread Alex Peshkoff via Firebird-devel
On 9/21/21 4:13 PM, Dimitry Sibiryakov wrote: Alex Peshkoff via Firebird-devel wrote 21.09.2021 15:06: Valgring makes execution many times slower (that's from my own experience).   Does it really matter if test suite isn't run in production anyway?.. Memory leaks detector does run in

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-09-21 Thread Alex Peshkoff via Firebird-devel
On 9/21/21 2:50 PM, Adriano dos Santos Fernandes wrote: I think we can improve things here. We have our GlobalPtr, InitInstance and we may continue using them. And we have FB_NEW which allocates memory from the global default pool. We may split that global default pool in two, so we'll have

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-09-21 Thread Dimitry Sibiryakov
Alex Peshkoff via Firebird-devel wrote 21.09.2021 15:06: Valgring makes execution many times slower (that's from my own experience). Does it really matter if test suite isn't run in production anyway?.. Can you check that for performance? No. They are not available for Windows as well

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-09-21 Thread Alex Peshkoff via Firebird-devel
On 9/21/21 4:02 PM, Dimitry Sibiryakov wrote: Alex Peshkoff via Firebird-devel wrote 21.09.2021 14:51: That's why this solution is usable only if we can't do something else.   https://github.com/google/sanitizers can detect much wider set of problems that our pools. Its support is built into

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-09-21 Thread Dimitry Sibiryakov
Alex Peshkoff via Firebird-devel wrote 21.09.2021 14:51: That's why this solution is usable only if we can't do something else. https://github.com/google/sanitizers can detect much wider set of problems that our pools. Its support is built into GCC. -- WBR, SD. Firebird-Devel mailing

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-09-21 Thread Alex Peshkoff via Firebird-devel
On 9/21/21 3:31 PM, Dimitry Sibiryakov wrote: Adriano dos Santos Fernandes wrote 21.09.2021 14:24: Though we will lose way to monitor leaks.   Valgrind, drMemory and sanitizers will work in this case. Valgring makes execution many times slower (that's from my own experience). I've never

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-09-21 Thread Alex Peshkoff via Firebird-devel
On 9/21/21 3:24 PM, Adriano dos Santos Fernandes wrote: On 21/09/2021 09:11, Dimitry Sibiryakov wrote: Adriano dos Santos Fernandes wrote 21.09.2021 13:50: We may split that global default pool in two, so we'll have another default pool for external (not controlled by us) libraries. All memory

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-09-21 Thread Dimitry Sibiryakov
Adriano dos Santos Fernandes wrote 21.09.2021 14:24: Though we will lose way to monitor leaks. Valgrind, drMemory and sanitizers will work in this case. -- WBR, SD. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-09-21 Thread Adriano dos Santos Fernandes
On 21/09/2021 09:11, Dimitry Sibiryakov wrote: > Adriano dos Santos Fernandes wrote 21.09.2021 13:50: >> We may split that global default pool in two, so we'll have another >> default pool for external (not controlled by us) libraries. All memory >> allocated directly with "operator new" (not

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-09-21 Thread Dimitry Sibiryakov
Adriano dos Santos Fernandes wrote 21.09.2021 13:50: We may split that global default pool in two, so we'll have another default pool for external (not controlled by us) libraries. All memory allocated directly with "operator new" (not FB_NEW) will go to external pool. Wouldn't it be simpler

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-09-21 Thread Adriano dos Santos Fernandes
On 17/09/2021 11:13, Alex Peshkoff via Firebird-devel wrote: > On 9/17/21 12:44 PM, Dimitry Sibiryakov wrote: >> Alex Peshkoff via Firebird-devel wrote 17.09.2021 9:08: >>> Is it about a library dynamically (dlopen, LoadLibrary) loaded or >>> linked to the exacutable? >> >>   Both. The library

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-09-19 Thread Alex Peshkoff via Firebird-devel
On 9/17/21 6:15 PM, Dimitry Sibiryakov wrote: Alex Peshkoff via Firebird-devel wrote 17.09.2021 17:03: as an additional cherry on a cake we have very fast, efficient and (last but not least) familiar for developers support of hunting for memory leaks.   Is it more familiar than DrMemory and

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-09-17 Thread Dimitry Sibiryakov
Alex Peshkoff via Firebird-devel wrote 17.09.2021 17:03: as an additional cherry on a cake we have very fast, efficient and (last but not least) familiar for developers support of hunting for memory leaks. Is it more familiar than DrMemory and valgrind? Former is currently useless because

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-09-17 Thread Alex Peshkoff via Firebird-devel
On 9/17/21 5:53 PM, Dimitry Sibiryakov wrote: Alex Peshkoff via Firebird-devel wrote 17.09.2021 16:13: First of all dynamically loaded libraries are unloaded in reverse order, next invoken executable dtors/atexits, and only after it - dtors of linked libraries.   Ah, ok. But returning to the

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-09-17 Thread Dimitry Sibiryakov
Alex Peshkoff via Firebird-devel wrote 17.09.2021 16:13: First of all dynamically loaded libraries are unloaded in reverse order, next invoken executable dtors/atexits, and only after it - dtors of linked libraries. Ah, ok. But returning to the topic: which library must provide pooled memory

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-09-17 Thread Alex Peshkoff via Firebird-devel
On 9/17/21 12:44 PM, Dimitry Sibiryakov wrote: Alex Peshkoff via Firebird-devel wrote 17.09.2021 9:08: Is it about a library dynamically (dlopen, LoadLibrary) loaded or linked to the exacutable?   Both. The library tree is the same (at least on Linux). Tree is the same - but cleanup order

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-09-17 Thread Dimitry Sibiryakov
Alex Peshkoff via Firebird-devel wrote 17.09.2021 9:08: Is it about a library dynamically (dlopen, LoadLibrary) loaded or linked to the exacutable? Both. The library tree is the same (at least on Linux). -- WBR, SD. Firebird-Devel mailing list, web interface at

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-09-17 Thread Alex Peshkoff via Firebird-devel
On 9/16/21 10:58 PM, Dimitry Sibiryakov wrote: Unfortunately atexit of executable is called after atexit of the library causing the executable to call the routine(s) from the library with already destructed globals. Is it about a library dynamically (dlopen, LoadLibrary) loaded or linked

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-09-16 Thread Dimitry Sibiryakov
Adriano dos Santos Fernandes wrote 16.09.2021 21:43: And it should happen if you called it before de constructors. Or it was not done in this way? I didn't call it at all. Firebird engine did it for me. Are there any published test? Did it was compiled with c++11? Look at whole thread:

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-09-16 Thread Adriano dos Santos Fernandes
On 16/09/2021 16:13, Dimitry Sibiryakov wrote: > Adriano dos Santos Fernandes wrote 16.09.2021 21:08: >> And this has 0 importance for the matter, considering that tools >> vendors implement what the standard describes. > >   We already had a discussion here that shown that atexit (and called >

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-09-16 Thread Adriano dos Santos Fernandes
On 16/09/2021 16:27, Dimitry Sibiryakov wrote: > Adriano dos Santos Fernandes wrote 16.09.2021 20:39: >> I wanted to control the order of constructors mostly to control the >> order of destructors (which should be inverse). > >   KISS. Just gather them all into single CPP file. > Ok. Let's

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-09-16 Thread Dimitry Sibiryakov
Adriano dos Santos Fernandes wrote 16.09.2021 20:39: I wanted to control the order of constructors mostly to control the order of destructors (which should be inverse). KISS. Just gather them all into single CPP file. -- WBR, SD. Firebird-Devel mailing list, web interface at

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-09-16 Thread Dimitry Sibiryakov
Adriano dos Santos Fernandes wrote 16.09.2021 21:08: And this has 0 importance for the matter, considering that tools vendors implement what the standard describes. We already had a discussion here that shown that atexit (and called from it PluginModule::doCleanup()) is fired after global

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-09-16 Thread Adriano dos Santos Fernandes
On Thu, Sep 16, 2021 at 4:04 PM Dimitry Sibiryakov wrote: > Adriano dos Santos Fernandes wrote 16.09.2021 20:39: > > It looks like there is another way to make that. > > > > std::atexit (https://en.cppreference.com/w/cpp/utility/program/atexit) > docs says: > >No matter what docs say,

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-09-16 Thread Dimitry Sibiryakov
Adriano dos Santos Fernandes wrote 16.09.2021 20:39: It looks like there is another way to make that. std::atexit (https://en.cppreference.com/w/cpp/utility/program/atexit) docs says: No matter what docs say, std::atexit is just a synonym for atexit. -- WBR, SD. Firebird-Devel mailing

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-09-16 Thread Adriano dos Santos Fernandes
Let's recap that. I wanted to control the order of constructors mostly to control the order of destructors (which should be inverse). Then a non-GlobalPtr/-InitInstance object allocating memory would not cause a crash if the memory manager is destroyed before it. It looks like there is another

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-09-16 Thread Alex Peshkoff via Firebird-devel
On 9/16/21 4:38 PM, Adriano dos Santos Fernandes wrote: Hi! Unfortunately init_priority do not work as we intended in MacOS. clang generate correct assembly/object file but the MacOS linker do not sort and merge the sections of different files. Pity. But typical for Mac :-(

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-09-16 Thread Adriano dos Santos Fernandes
Hi! Unfortunately init_priority do not work as we intended in MacOS. clang generate correct assembly/object file but the MacOS linker do not sort and merge the sections of different files. The result is that it reorder correctly only per cpp file, not across different cpp files. Adriano

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-04-19 Thread Adriano dos Santos Fernandes
On Thu, Apr 15, 2021 at 3:28 PM Adriano dos Santos Fernandes < adrian...@gmail.com> wrote: > On 15/04/2021 15:05, Alex Peshkoff via Firebird-devel wrote: > > On 4/15/21 8:21 PM, Adriano dos Santos Fernandes wrote: > > > >>> Can you send me full test (or patch)? I want ot test on Android. > >>> >

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-04-15 Thread Adriano dos Santos Fernandes
On 15/04/2021 15:05, Alex Peshkoff via Firebird-devel wrote: > On 4/15/21 8:21 PM, Adriano dos Santos Fernandes wrote: > >>> Can you send me full test (or patch)? I want ot test on Android. >>> >> Attached test. >> >> Here is results (init.cpp is showed many times because it's included in >> many

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-04-15 Thread Alex Peshkoff via Firebird-devel
On 4/15/21 8:21 PM, Adriano dos Santos Fernandes wrote: Can you send me full test (or patch)? I want ot test on Android. Attached test. Here is results (init.cpp is showed many times because it's included in many libraries): Same result on android: eneric_arm64:/data/firebird #

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-04-15 Thread Adriano dos Santos Fernandes
On 15/04/2021 14:27, Dimitry Sibiryakov wrote: > 15.04.2021 19:21, Adriano dos Santos Fernandes wrote: >> CtorDtorDebug::CtorDtorDebug - utl.cpp 6000 >> CtorDtorDebug::CtorDtorDebug - init.cpp 6000 >> CtorDtorDebug::CtorDtorDebug - init.cpp 5000 >> CtorDtorDebug::CtorDtorDebug - init.cpp 5400 > >

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-04-15 Thread Alex Peshkoff via Firebird-devel
On 4/15/21 8:27 PM, Dimitry Sibiryakov wrote: 15.04.2021 19:21, Adriano dos Santos Fernandes wrote: CtorDtorDebug::CtorDtorDebug - utl.cpp 6000 CtorDtorDebug::CtorDtorDebug - init.cpp 6000 CtorDtorDebug::CtorDtorDebug - init.cpp 5000 CtorDtorDebug::CtorDtorDebug - init.cpp 5400  This piece

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-04-15 Thread Dimitry Sibiryakov
15.04.2021 19:21, Adriano dos Santos Fernandes wrote: CtorDtorDebug::CtorDtorDebug - utl.cpp 6000 CtorDtorDebug::CtorDtorDebug - init.cpp 6000 CtorDtorDebug::CtorDtorDebug - init.cpp 5000 CtorDtorDebug::CtorDtorDebug - init.cpp 5400 This piece doesn't look right. -- WBR, SD.

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-04-15 Thread Adriano dos Santos Fernandes
On 15/04/2021 11:40, Alex Peshkoff via Firebird-devel wrote: > On 4/15/21 2:33 PM, Adriano dos Santos Fernandes wrote: > I have tested with common/classes/init.cpp and this solved the specific problem I had with Boost.Test: Cleanup global __attribute__ ((init_priority

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-04-15 Thread Alex Peshkoff via Firebird-devel
On 4/15/21 2:33 PM, Adriano dos Santos Fernandes wrote: I have tested with common/classes/init.cpp and this solved the specific problem I had with Boost.Test: Cleanup global __attribute__ ((init_priority (2000))); As far as I can see from documentation init_priority works only inside simgle

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-04-15 Thread Adriano dos Santos Fernandes
On 15/04/2021 08:33, Adriano dos Santos Fernandes wrote: > On 15/04/2021 07:55, Alex Peshkoff via Firebird-devel wrote: >>> >>> It uses global std objects which in this case have theirs destructors >>> called after Firebird default pool are already cleaned up. >> >> Yes, that's really a problem.

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-04-15 Thread Adriano dos Santos Fernandes
On 15/04/2021 07:55, Alex Peshkoff via Firebird-devel wrote: >> >> It uses global std objects which in this case have theirs destructors >> called after Firebird default pool are already cleaned up. > > Yes, that's really a problem. And must say that even forgetting about > MemPool implementation

Re: [Firebird-devel] Memory manager cleanup / external libraries

2021-04-15 Thread Alex Peshkoff via Firebird-devel
On 4/14/21 8:25 PM, Adriano dos Santos Fernandes wrote: Hi! C++ guarantees the execution order of constructors only inside the same translation unit (say, a .cpp file) for global objects. It also guarantees inverse order of destructors in relation to constructors order. But constructor order

[Firebird-devel] Memory manager cleanup / external libraries

2021-04-14 Thread Adriano dos Santos Fernandes
Hi! C++ guarantees the execution order of constructors only inside the same translation unit (say, a .cpp file) for global objects. It also guarantees inverse order of destructors in relation to constructors order. But constructor order across translation units are undefined. We used to have