Ok, so this is simple jemalloc creating its background thread by default. Not a problem per se.
Regards Antoine. On Fri, 13 Aug 2021 20:25:22 -0700 James Van Alstine <[email protected]> wrote: > I ended up just rebuilding arrow with debug symbols. > > It is indeed a duplicate of > https://issues.apache.org/jira/browse/ARROW-9530 > > ==13317== Memcheck, a memory error detector > ==13317== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. > ==13317== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info > ==13317== Command: ./app/example/example > ==13317== > Hello World! > ==13317== > ==13317== HEAP SUMMARY: > ==13317== in use at exit: 304 bytes in 1 blocks > ==13317== total heap usage: 13 allocs, 12 frees, 75,181 bytes allocated > ==13317== > ==13317== 304 bytes in 1 blocks are possibly lost in loss record 1 of 1 > ==13317== at 0x483DD99: calloc (in > /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so) > ==13317== by 0x40149CA: allocate_dtv (dl-tls.c:286) > ==13317== by 0x40149CA: _dl_allocate_tls (dl-tls.c:532) > ==13317== by 0x5A70322: allocate_stack (allocatestack.c:622) > ==13317== by 0x5A70322: pthread_create@@GLIBC_2.2.5 > (pthread_create.c:660) > ==13317== by 0x554B443: je_arrow_private_je_pthread_create_wrapper > (background_thread.c:48) > ==13317== by 0x554B443: background_thread_create_signals_masked > (background_thread.c:365) > ==13317== by 0x554B443: background_thread_create_locked > (background_thread.c:573) > ==13317== by 0x554B63B: je_arrow_private_je_background_thread_create > (background_thread.c:598) > ==13317== by 0x4011B89: call_init.part.0 (dl-init.c:72) > ==13317== by 0x4011C90: call_init (dl-init.c:30) > ==13317== by 0x4011C90: _dl_init (dl-init.c:119) > ==13317== by 0x4001139: ??? (in /usr/lib/x86_64-linux-gnu/ld-2.31.so) > ==13317== > ==13317== LEAK SUMMARY: > ==13317== definitely lost: 0 bytes in 0 blocks > ==13317== indirectly lost: 0 bytes in 0 blocks > ==13317== possibly lost: 304 bytes in 1 blocks > ==13317== still reachable: 0 bytes in 0 blocks > ==13317== suppressed: 0 bytes in 0 blocks > ==13317== > ==13317== For lists of detected and suppressed errors, rerun with: -s > ==13317== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0) > > On Fri, Aug 13, 2021 at 6:32 PM James Van Alstine <[email protected]> > wrote: > > > Is there a libarrow deb with debug symbols so I can resolve the (???)s > > valgrind returns? > > > > On Fri, Aug 13, 2021 at 5:34 PM James Van Alstine <[email protected]> > > wrote: > > > >> Yes, > >> > >> For you can repro: > >> > >> Program: > >> ``` > >> #include <iostream> > >> > >> int main() { > >> std::cout << "Hello World!" << std::endl; > >> > >> return 0; > >> } > >> ``` > >> Executable: > >> ``` > >> clang++-10 -g -Wall -O2 -l:libarrow.so main.cc -o hello > >> ``` > >> > >> with --leak-check=full > >> ``` > >> ind --leak-check=full ./hello > >> ==6456== Memcheck, a memory error detector > >> ==6456== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. > >> ==6456== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright > >> info > >> ==6456== Command: ./hello > >> ==6456== > >> Hello World! > >> ==6456== > >> ==6456== HEAP SUMMARY: > >> ==6456== in use at exit: 7,094 bytes in 100 blocks > >> ==6456== total heap usage: 3,550 allocs, 3,450 frees, 347,875 bytes > >> allocated > >> ==6456== > >> ==6456== 368 bytes in 1 blocks are possibly lost in loss record 12 of 15 > >> ==6456== at 0x483DD99: calloc (in > >> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so) > >> ==6456== by 0x40149CA: allocate_dtv (dl-tls.c:286) > >> ==6456== by 0x40149CA: _dl_allocate_tls (dl-tls.c:532) > >> ==6456== by 0x6A1D322: allocate_stack (allocatestack.c:622) > >> ==6456== by 0x6A1D322: pthread_create@@GLIBC_2.2.5 > >> (pthread_create.c:660) > >> ==6456== by 0x58E5AB5: ??? (in > >> /usr/lib/x86_64-linux-gnu/libarrow.so.500.0.0) > >> ==6456== by 0x58E6B62: ??? (in > >> /usr/lib/x86_64-linux-gnu/libarrow.so.500.0.0) > >> ==6456== by 0x58E6CEB: ??? (in > >> /usr/lib/x86_64-linux-gnu/libarrow.so.500.0.0) > >> ==6456== by 0x4011B89: call_init.part.0 (dl-init.c:72) > >> ==6456== by 0x4011C90: call_init (dl-init.c:30) > >> ==6456== by 0x4011C90: _dl_init (dl-init.c:119) > >> ==6456== by 0x4001139: ??? (in /usr/lib/x86_64-linux-gnu/ld-2.31.so) > >> ==6456== > >> ==6456== LEAK SUMMARY: > >> ==6456== definitely lost: 0 bytes in 0 blocks > >> ==6456== indirectly lost: 0 bytes in 0 blocks > >> ==6456== possibly lost: 368 bytes in 1 blocks > >> ==6456== still reachable: 6,726 bytes in 99 blocks > >> ==6456== suppressed: 0 bytes in 0 blocks > >> ==6456== Reachable blocks (those to which a pointer was found) are not > >> shown. > >> ==6456== To see them, rerun with: --leak-check=full --show-leak-kinds=all > >> ==6456== > >> ==6456== For lists of detected and suppressed errors, rerun with: -s > >> ==6456== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0) > >> ``` > >> > >> If by `ARROW_DEFAULT_MEMORY_POOL=system` you mean setting that as an env > >> var and then running the same executable, then > >> the valgrind output is the same. > >> > >> I think what is happening here is glibc is running _dl_init() when > >> linking, which is kicking off the constructor of some global object in > >> Arrow. > >> > >> > >> On Fri, Aug 13, 2021 at 5:20 PM Weston Pace <[email protected]> > >> wrote: > >> > >>> Can you try running your program with ARROW_DEFAULT_MEMORY_POOL=system > >>> > >>> I am wondering if you are encountering > >>> https://issues.apache.org/jira/browse/ARROW-9530 > >>> > >>> Otherwise, can you run with `--leak-check=full` which should give us a > >>> hint where the possibly lost memory is coming from? > >>> > >>> On Fri, Aug 13, 2021 at 11:52 AM James Van Alstine > >>> <[email protected]> wrote: > >>> > > >>> > Greetings, > >>> > > >>> > When running a simple C++ hello world program, simply linking against > >>> Arrow produces a valgrind report that looks like the one below. Obviously, > >>> there's some constructors allocating memory before main(). Is this > >>> intentional like the allocators in STL? If it is, is there some way to > >>> free > >>> whatever is being allocated? If not, then how do people typically profile > >>> their program's allocations when using Arrow? > >>> > > >>> > ==1768== Memcheck, a memory error detector > >>> > ==1768== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et > >>> al. > >>> > ==1768== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright > >>> info > >>> > ==1768== Command: ./hello > >>> > ==1768== > >>> > Hello World! > >>> > ==1768== > >>> > ==1768== HEAP SUMMARY: > >>> > ==1768== in use at exit: 7,094 bytes in 100 blocks > >>> > ==1768== total heap usage: 3,550 allocs, 3,450 frees, 347,875 bytes > >>> allocated > >>> > ==1768== > >>> > ==1768== LEAK SUMMARY: > >>> > ==1768== definitely lost: 0 bytes in 0 blocks > >>> > ==1768== indirectly lost: 0 bytes in 0 blocks > >>> > ==1768== possibly lost: 368 bytes in 1 blocks > >>> > ==1768== still reachable: 6,726 bytes in 99 blocks > >>> > ==1768== suppressed: 0 bytes in 0 blocks > >>> > ==1768== Rerun with --leak-check=full to see details of leaked memory > >>> > ==1768== > >>> > ==1768== For lists of detected and suppressed errors, rerun with: -s > >>> > ==1768== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0) > >>> > > >>> > >> >
