Yes. I’m comfortable using a valgrind suppression file to get rid of it now that I know what it is.
Cheers, James On Sat, Aug 14, 2021 at 1:19 PM Antoine Pitrou <[email protected]> wrote: > > Ok, so this is simple jemalloc creating its background thread by > default. Not a problem per se. > > Regards > > Antoine. > > > On Fri, 13 Aug 2021 20:25:22 -0700 > James Van Alstine <[email protected]> wrote: > > > I ended up just rebuilding arrow with debug symbols. > > > > It is indeed a duplicate of > > https://issues.apache.org/jira/browse/ARROW-9530 > > > > ==13317== Memcheck, a memory error detector > > ==13317== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. > > ==13317== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright > info > > ==13317== Command: ./app/example/example > > ==13317== > > Hello World! > > ==13317== > > ==13317== HEAP SUMMARY: > > ==13317== in use at exit: 304 bytes in 1 blocks > > ==13317== total heap usage: 13 allocs, 12 frees, 75,181 bytes allocated > > ==13317== > > ==13317== 304 bytes in 1 blocks are possibly lost in loss record 1 of 1 > > ==13317== at 0x483DD99: calloc (in > > /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so) > > ==13317== by 0x40149CA: allocate_dtv (dl-tls.c:286) > > ==13317== by 0x40149CA: _dl_allocate_tls (dl-tls.c:532) > > ==13317== by 0x5A70322: allocate_stack (allocatestack.c:622) > > ==13317== by 0x5A70322: pthread_create@@GLIBC_2.2.5 > > (pthread_create.c:660) > > ==13317== by 0x554B443: je_arrow_private_je_pthread_create_wrapper > > (background_thread.c:48) > > ==13317== by 0x554B443: background_thread_create_signals_masked > > (background_thread.c:365) > > ==13317== by 0x554B443: background_thread_create_locked > > (background_thread.c:573) > > ==13317== by 0x554B63B: je_arrow_private_je_background_thread_create > > (background_thread.c:598) > > ==13317== by 0x4011B89: call_init.part.0 (dl-init.c:72) > > ==13317== by 0x4011C90: call_init (dl-init.c:30) > > ==13317== by 0x4011C90: _dl_init (dl-init.c:119) > > ==13317== by 0x4001139: ??? (in /usr/lib/x86_64-linux-gnu/ld-2.31.so) > > ==13317== > > ==13317== LEAK SUMMARY: > > ==13317== definitely lost: 0 bytes in 0 blocks > > ==13317== indirectly lost: 0 bytes in 0 blocks > > ==13317== possibly lost: 304 bytes in 1 blocks > > ==13317== still reachable: 0 bytes in 0 blocks > > ==13317== suppressed: 0 bytes in 0 blocks > > ==13317== > > ==13317== For lists of detected and suppressed errors, rerun with: -s > > ==13317== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0) > > > > On Fri, Aug 13, 2021 at 6:32 PM James Van Alstine < > [email protected]> > > wrote: > > > > > Is there a libarrow deb with debug symbols so I can resolve the (???)s > > > valgrind returns? > > > > > > On Fri, Aug 13, 2021 at 5:34 PM James Van Alstine < > [email protected]> > > > wrote: > > > > > >> Yes, > > >> > > >> For you can repro: > > >> > > >> Program: > > >> ``` > > >> #include <iostream> > > >> > > >> int main() { > > >> std::cout << "Hello World!" << std::endl; > > >> > > >> return 0; > > >> } > > >> ``` > > >> Executable: > > >> ``` > > >> clang++-10 -g -Wall -O2 -l:libarrow.so main.cc -o hello > > >> ``` > > >> > > >> with --leak-check=full > > >> ``` > > >> ind --leak-check=full ./hello > > >> ==6456== Memcheck, a memory error detector > > >> ==6456== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et > al. > > >> ==6456== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright > > >> info > > >> ==6456== Command: ./hello > > >> ==6456== > > >> Hello World! > > >> ==6456== > > >> ==6456== HEAP SUMMARY: > > >> ==6456== in use at exit: 7,094 bytes in 100 blocks > > >> ==6456== total heap usage: 3,550 allocs, 3,450 frees, 347,875 bytes > > >> allocated > > >> ==6456== > > >> ==6456== 368 bytes in 1 blocks are possibly lost in loss record 12 of > 15 > > >> ==6456== at 0x483DD99: calloc (in > > >> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so) > > >> ==6456== by 0x40149CA: allocate_dtv (dl-tls.c:286) > > >> ==6456== by 0x40149CA: _dl_allocate_tls (dl-tls.c:532) > > >> ==6456== by 0x6A1D322: allocate_stack (allocatestack.c:622) > > >> ==6456== by 0x6A1D322: pthread_create@@GLIBC_2.2.5 > > >> (pthread_create.c:660) > > >> ==6456== by 0x58E5AB5: ??? (in > > >> /usr/lib/x86_64-linux-gnu/libarrow.so.500.0.0) > > >> ==6456== by 0x58E6B62: ??? (in > > >> /usr/lib/x86_64-linux-gnu/libarrow.so.500.0.0) > > >> ==6456== by 0x58E6CEB: ??? (in > > >> /usr/lib/x86_64-linux-gnu/libarrow.so.500.0.0) > > >> ==6456== by 0x4011B89: call_init.part.0 (dl-init.c:72) > > >> ==6456== by 0x4011C90: call_init (dl-init.c:30) > > >> ==6456== by 0x4011C90: _dl_init (dl-init.c:119) > > >> ==6456== by 0x4001139: ??? (in /usr/lib/x86_64-linux-gnu/ > ld-2.31.so) > > >> ==6456== > > >> ==6456== LEAK SUMMARY: > > >> ==6456== definitely lost: 0 bytes in 0 blocks > > >> ==6456== indirectly lost: 0 bytes in 0 blocks > > >> ==6456== possibly lost: 368 bytes in 1 blocks > > >> ==6456== still reachable: 6,726 bytes in 99 blocks > > >> ==6456== suppressed: 0 bytes in 0 blocks > > >> ==6456== Reachable blocks (those to which a pointer was found) are not > > >> shown. > > >> ==6456== To see them, rerun with: --leak-check=full > --show-leak-kinds=all > > >> ==6456== > > >> ==6456== For lists of detected and suppressed errors, rerun with: -s > > >> ==6456== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from > 0) > > >> ``` > > >> > > >> If by `ARROW_DEFAULT_MEMORY_POOL=system` you mean setting that as an > env > > >> var and then running the same executable, then > > >> the valgrind output is the same. > > >> > > >> I think what is happening here is glibc is running _dl_init() when > > >> linking, which is kicking off the constructor of some global object in > > >> Arrow. > > >> > > >> > > >> On Fri, Aug 13, 2021 at 5:20 PM Weston Pace <[email protected]> > > >> wrote: > > >> > > >>> Can you try running your program with > ARROW_DEFAULT_MEMORY_POOL=system > > >>> > > >>> I am wondering if you are encountering > > >>> https://issues.apache.org/jira/browse/ARROW-9530 > > >>> > > >>> Otherwise, can you run with `--leak-check=full` which should give us > a > > >>> hint where the possibly lost memory is coming from? > > >>> > > >>> On Fri, Aug 13, 2021 at 11:52 AM James Van Alstine > > >>> <[email protected]> wrote: > > >>> > > > >>> > Greetings, > > >>> > > > >>> > When running a simple C++ hello world program, simply linking > against > > >>> Arrow produces a valgrind report that looks like the one below. > Obviously, > > >>> there's some constructors allocating memory before main(). Is this > > >>> intentional like the allocators in STL? If it is, is there some way > to free > > >>> whatever is being allocated? If not, then how do people typically > profile > > >>> their program's allocations when using Arrow? > > >>> > > > >>> > ==1768== Memcheck, a memory error detector > > >>> > ==1768== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward > et > > >>> al. > > >>> > ==1768== Using Valgrind-3.15.0 and LibVEX; rerun with -h for > copyright > > >>> info > > >>> > ==1768== Command: ./hello > > >>> > ==1768== > > >>> > Hello World! > > >>> > ==1768== > > >>> > ==1768== HEAP SUMMARY: > > >>> > ==1768== in use at exit: 7,094 bytes in 100 blocks > > >>> > ==1768== total heap usage: 3,550 allocs, 3,450 frees, 347,875 > bytes > > >>> allocated > > >>> > ==1768== > > >>> > ==1768== LEAK SUMMARY: > > >>> > ==1768== definitely lost: 0 bytes in 0 blocks > > >>> > ==1768== indirectly lost: 0 bytes in 0 blocks > > >>> > ==1768== possibly lost: 368 bytes in 1 blocks > > >>> > ==1768== still reachable: 6,726 bytes in 99 blocks > > >>> > ==1768== suppressed: 0 bytes in 0 blocks > > >>> > ==1768== Rerun with --leak-check=full to see details of leaked > memory > > >>> > ==1768== > > >>> > ==1768== For lists of detected and suppressed errors, rerun with: > -s > > >>> > ==1768== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 > from 0) > > >>> > > >> > > > > > >
