This is the same memory leak that I found using XCode Instruments and later verified it using valgrind. My apologies for not following up with a better report (like the valgrind log) at the time. I completely forgot about it until I saw Petr's message.
Michael On Thu, Sep 20, 2012 at 8:12 AM, Petr Velan <[email protected]> wrote: > Hi John, > > I just updated to latest SVN revision and tried the reorder function > on read-only partition. The reordering is now performed exactly as > described in documentation, so the problem no longer exists. > > However, I run into a problem with some memory leaks in the new > revision. My previous version was 3.0.10 (SVN 532) and there were no > leaks in my usecase. Now I'm loosing some memory. To simulate the > memory loss, you only need to create a partition in a simple program: > #include <fastbit/ibis.h> > #include <iostream> > > int main(int argc, char *argv[]) { > ibis::gVerbose = 10; > ibis::part part(argv[1], false); > return 0; > } > > Attached below is the program log with verbose set to 10. You might > notice that the leak is 40 bytes long, and that it is caused by > fileManager::storage class created in array_t constructor not being > deleted. In my case, the first constucted array_t's actual is probably > not removed. I've managed to trace the problem to "ibis::bitvector > amask" of ibis::part, but I cannot seem to find the problem. Most > likely the fileManager is replaced somewhere without proper > deallocation, but I cannot find it. > > Yours sincerely, > Petr Velan > > velan@wall:~/Documents/devel/tmp/test> valgrind --leak-check=full > ./test2 ../../data/000000000001/1/ > ==25287== Memcheck, a memory error detector > ==25287== Copyright (C) 2002-2010, and GNU GPL'd, by Julian Seward et al. > ==25287== Using Valgrind-3.6.1 and LibVEX; rerun with -h for copyright info > ==25287== Command: ./test2 ../../data/000000000001/1/ > ==25287== > > FastBit ibis1.3.2.6 > Log messages started on Thu Sep 20 17:05:57 2012 > fileManager::storage(0x775b8e0, 0) initialization completed > array_t<j> constructed at 0x7fefffb20 with actual=0x775b8e0, m_begin=0 > and actual->size()=0 > bitvector (0x7fefffb10) constructed with m_vec at 0x7fefffb20 > fileManager::ctor found the physical memory size to be 4083470336 bytes > fileManager initialization complete -- maxBytes=2041735168, maxOpenFiles=768 > part::readMetaData -- opened ../../data/000000000001/1/-part.txt for reading > Name = "1" > > Description = "Generated by ipfixcol fastbit plugin" > > Number_of_columns = 11 > > Number_of_rows = 499992 > > Timestamp = 1329386614 > > State = 1 > > END HEADER > > fileManager::storage(0x775f3e0, 0) initialization completed > array_t<j> constructed at 0x775f2e0 with actual=0x775f3e0, m_begin=0 > and actual->size()=0 > bitvector (0x775f2d0) constructed with m_vec at 0x775f2e0 > read info about column 1.e0id1 (ULONG) > part::readMetaData -- got column e0id1 from > ../../data/000000000001/1/-part.txt > fileManager::storage(0x77605a0, 0) initialization completed > array_t<j> constructed at 0x77604a0 with actual=0x77605a0, m_begin=0 > and actual->size()=0 > bitvector (0x7760490) constructed with m_vec at 0x77604a0 > read info about column 1.e0id11 (USHORT) > part::readMetaData -- got column e0id11 from > ../../data/000000000001/1/-part.txt > fileManager::storage(0x7761770, 0) initialization completed > array_t<j> constructed at 0x7761670 with actual=0x7761770, m_begin=0 > and actual->size()=0 > bitvector (0x7761660) constructed with m_vec at 0x7761670 > read info about column 1.e0id12 (UINT) > part::readMetaData -- got column e0id12 from > ../../data/000000000001/1/-part.txt > fileManager::storage(0x7762930, 0) initialization completed > array_t<j> constructed at 0x7762830 with actual=0x7762930, m_begin=0 > and actual->size()=0 > bitvector (0x7762820) constructed with m_vec at 0x7762830 > read info about column 1.e0id152 (ULONG) > part::readMetaData -- got column e0id152 from > ../../data/000000000001/1/-part.txt > fileManager::storage(0x7763b00, 0) initialization completed > array_t<j> constructed at 0x7763a00 with actual=0x7763b00, m_begin=0 > and actual->size()=0 > bitvector (0x77639f0) constructed with m_vec at 0x7763a00 > read info about column 1.e0id153 (ULONG) > part::readMetaData -- got column e0id153 from > ../../data/000000000001/1/-part.txt > fileManager::storage(0x7764cd0, 0) initialization completed > array_t<j> constructed at 0x7764bd0 with actual=0x7764cd0, m_begin=0 > and actual->size()=0 > bitvector (0x7764bc0) constructed with m_vec at 0x7764bd0 > read info about column 1.e0id2 (ULONG) > part::readMetaData -- got column e0id2 from > ../../data/000000000001/1/-part.txt > fileManager::storage(0x7765e90, 0) initialization completed > array_t<j> constructed at 0x7765d90 with actual=0x7765e90, m_begin=0 > and actual->size()=0 > bitvector (0x7765d80) constructed with m_vec at 0x7765d90 > read info about column 1.e0id4 (UBYTE) > part::readMetaData -- got column e0id4 from > ../../data/000000000001/1/-part.txt > fileManager::storage(0x7767050, 0) initialization completed > array_t<j> constructed at 0x7766f50 with actual=0x7767050, m_begin=0 > and actual->size()=0 > bitvector (0x7766f40) constructed with m_vec at 0x7766f50 > read info about column 1.e0id5 (UBYTE) > part::readMetaData -- got column e0id5 from > ../../data/000000000001/1/-part.txt > fileManager::storage(0x7768210, 0) initialization completed > array_t<j> constructed at 0x7768110 with actual=0x7768210, m_begin=0 > and actual->size()=0 > bitvector (0x7768100) constructed with m_vec at 0x7768110 > read info about column 1.e0id6 (UBYTE) > part::readMetaData -- got column e0id6 from > ../../data/000000000001/1/-part.txt > fileManager::storage(0x77693d0, 0) initialization completed > array_t<j> constructed at 0x77692d0 with actual=0x77693d0, m_begin=0 > and actual->size()=0 > bitvector (0x77692c0) constructed with m_vec at 0x77692d0 > read info about column 1.e0id7 (USHORT) > part::readMetaData -- got column e0id7 from > ../../data/000000000001/1/-part.txt > fileManager::storage(0x776a590, 0) initialization completed > array_t<j> constructed at 0x776a490 with actual=0x776a590, m_begin=0 > and actual->size()=0 > bitvector (0x776a480) constructed with m_vec at 0x776a490 > read info about column 1.e0id8 (UINT) > part::readMetaData -- got column e0id8 from > ../../data/000000000001/1/-part.txt > part[1]::gainReadAccess -- pthread_rwlock_rdlock(0x7fefffbb8) for readRIDs > fileManager::storage(0x776b9a0, 0) initialization completed > array_t<N4ibis5rid_tE> constructed at 0x776b940 with actual=0x776b9a0, > m_begin=0 and actual->size()=0 > part[1]::readRIDs -- the file manager failed to read file > "../../data/000000000001/1/-rids". There is no RIDs. > part[1]::releaseAccess -- pthread_rwlock_unlock(0x7fefffbb8) for readRIDs > Warning -- failed to read the content of > ../../data/000000000001/1/-part.msk, fileManager::getFile returned > -101 > fileManager::storage(0x776d040, 0x776d0b0) added 12 bytes to increase > totalBytes to 12 > fileManager::storage(0x776d040, 0x776d0b0) initialization completed > with 12 elements > fileManager::flushFile will do nothing because > "../../data/000000000001/1/-part.msk" is not tracked by the file > manager > part::init -- mask for partition 1 has 499992 set bits out of 499992 > Constructed a part named 1 > activeDir = "../../data/000000000001/1" > part: 1 (Generated by ipfixcol fastbit plugin) with 499992 rows, 11 columns > Column list: > e0id1: (ULONG) [28, 1.44872e+09] > e0id11: (USHORT) [0, 65535] > e0id12: (UINT) [1.0466e+08, 4.02607e+09] > e0id152: (ULONG) [1.26981e+12, 1.26982e+12] > e0id153: (ULONG) [1.26981e+12, 1.26982e+12] > e0id2: (ULONG) [1, 1.3616e+06] > e0id4: (UBYTE) [1, 41] > e0id5: (UBYTE) [0, 0] > e0id6: (UBYTE) [0, 31] > e0id7: (USHORT) [0, 65535] > e0id8: (UINT) [1.0466e+08, 3.75763e+09] > > part[1]::gainWriteAccess -- pthread_rwlock_wrlock(0x7fefffbb8) for ~part > clearing data partition 1 > column[1.e0id1]::writeLock -- pthread_rwlock_wrlock(0x775f340) for ~column > clearing column 1.e0id1 > column[1.e0id1]::writeLock -- pthread_rwlock_unlock(0x775f340) for ~column > bitvector (0x775f2d0) clear the content of bitvector with m_vec at 0x775f2e0 > array_t<j>::freeMemory this=0x775f2e0 actual=0x775f3e0 and m_begin=0 > (active references: 0, past references: 1) > fileManager::storage(0x775f3e0, 0) cleared > column[1.e0id11]::writeLock -- pthread_rwlock_wrlock(0x7760500) for ~column > clearing column 1.e0id11 > column[1.e0id11]::writeLock -- pthread_rwlock_unlock(0x7760500) for ~column > bitvector (0x7760490) clear the content of bitvector with m_vec at 0x77604a0 > array_t<j>::freeMemory this=0x77604a0 actual=0x77605a0 and m_begin=0 > (active references: 0, past references: 1) > fileManager::storage(0x77605a0, 0) cleared > column[1.e0id12]::writeLock -- pthread_rwlock_wrlock(0x77616d0) for ~column > clearing column 1.e0id12 > column[1.e0id12]::writeLock -- pthread_rwlock_unlock(0x77616d0) for ~column > bitvector (0x7761660) clear the content of bitvector with m_vec at 0x7761670 > array_t<j>::freeMemory this=0x7761670 actual=0x7761770 and m_begin=0 > (active references: 0, past references: 1) > fileManager::storage(0x7761770, 0) cleared > column[1.e0id152]::writeLock -- pthread_rwlock_wrlock(0x7762890) for ~column > clearing column 1.e0id152 > column[1.e0id152]::writeLock -- pthread_rwlock_unlock(0x7762890) for ~column > bitvector (0x7762820) clear the content of bitvector with m_vec at 0x7762830 > array_t<j>::freeMemory this=0x7762830 actual=0x7762930 and m_begin=0 > (active references: 0, past references: 1) > fileManager::storage(0x7762930, 0) cleared > column[1.e0id153]::writeLock -- pthread_rwlock_wrlock(0x7763a60) for ~column > clearing column 1.e0id153 > column[1.e0id153]::writeLock -- pthread_rwlock_unlock(0x7763a60) for ~column > bitvector (0x77639f0) clear the content of bitvector with m_vec at 0x7763a00 > array_t<j>::freeMemory this=0x7763a00 actual=0x7763b00 and m_begin=0 > (active references: 0, past references: 1) > fileManager::storage(0x7763b00, 0) cleared > column[1.e0id2]::writeLock -- pthread_rwlock_wrlock(0x7764c30) for ~column > clearing column 1.e0id2 > column[1.e0id2]::writeLock -- pthread_rwlock_unlock(0x7764c30) for ~column > bitvector (0x7764bc0) clear the content of bitvector with m_vec at 0x7764bd0 > array_t<j>::freeMemory this=0x7764bd0 actual=0x7764cd0 and m_begin=0 > (active references: 0, past references: 1) > fileManager::storage(0x7764cd0, 0) cleared > column[1.e0id4]::writeLock -- pthread_rwlock_wrlock(0x7765df0) for ~column > clearing column 1.e0id4 > column[1.e0id4]::writeLock -- pthread_rwlock_unlock(0x7765df0) for ~column > bitvector (0x7765d80) clear the content of bitvector with m_vec at 0x7765d90 > array_t<j>::freeMemory this=0x7765d90 actual=0x7765e90 and m_begin=0 > (active references: 0, past references: 1) > fileManager::storage(0x7765e90, 0) cleared > column[1.e0id5]::writeLock -- pthread_rwlock_wrlock(0x7766fb0) for ~column > clearing column 1.e0id5 > column[1.e0id5]::writeLock -- pthread_rwlock_unlock(0x7766fb0) for ~column > bitvector (0x7766f40) clear the content of bitvector with m_vec at 0x7766f50 > array_t<j>::freeMemory this=0x7766f50 actual=0x7767050 and m_begin=0 > (active references: 0, past references: 1) > fileManager::storage(0x7767050, 0) cleared > column[1.e0id6]::writeLock -- pthread_rwlock_wrlock(0x7768170) for ~column > clearing column 1.e0id6 > column[1.e0id6]::writeLock -- pthread_rwlock_unlock(0x7768170) for ~column > bitvector (0x7768100) clear the content of bitvector with m_vec at 0x7768110 > array_t<j>::freeMemory this=0x7768110 actual=0x7768210 and m_begin=0 > (active references: 0, past references: 1) > fileManager::storage(0x7768210, 0) cleared > column[1.e0id7]::writeLock -- pthread_rwlock_wrlock(0x7769330) for ~column > clearing column 1.e0id7 > column[1.e0id7]::writeLock -- pthread_rwlock_unlock(0x7769330) for ~column > bitvector (0x77692c0) clear the content of bitvector with m_vec at 0x77692d0 > array_t<j>::freeMemory this=0x77692d0 actual=0x77693d0 and m_begin=0 > (active references: 0, past references: 1) > fileManager::storage(0x77693d0, 0) cleared > column[1.e0id8]::writeLock -- pthread_rwlock_wrlock(0x776a4f0) for ~column > clearing column 1.e0id8 > column[1.e0id8]::writeLock -- pthread_rwlock_unlock(0x776a4f0) for ~column > bitvector (0x776a480) clear the content of bitvector with m_vec at 0x776a490 > array_t<j>::freeMemory this=0x776a490 actual=0x776a590 and m_begin=0 > (active references: 0, past references: 1) > fileManager::storage(0x776a590, 0) cleared > part[1]::releaseAccess -- pthread_rwlock_unlock(0x7fefffbb8) for ~part > array_t<N4ibis5rid_tE>::freeMemory this=0x776b940 actual=0x776b9a0 and > m_begin=0 (active references: 0, past references: 1) > fileManager::storage(0x776b9a0, 0) cleared > bitvector (0x7fefffb10) clear the content of bitvector with m_vec at > 0x7fefffb20 > array_t<j>::freeMemory this=0x7fefffb20 actual=0x776d040 and > m_begin=0x776d0b0 (active references: 0, past references: 1) > fileManager::storage(0x776d040, 0x776d0b0) removed 12 bytes to > decrease totalBytes to 0 > fileManager::storage(0x776d040, 0x776d0b0) cleared > fileManager::clear has nothing to do > fileManager decommissioned > ==25287== > ==25287== HEAP SUMMARY: > ==25287== in use at exit: 40 bytes in 1 blocks > ==25287== total heap usage: 466 allocs, 465 frees, 119,596 bytes allocated > ==25287== > ==25287== 40 bytes in 1 blocks are definitely lost in loss record 1 of 1 > ==25287== at 0x4C292C7: operator new(unsigned long) (in > /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) > ==25287== by 0x5B2F832: ibis::array_t<unsigned int>::array_t() > (array_t.cpp:36) > ==25287== by 0x604C5A9: ibis::bitvector::bitvector() (bitvector.cpp:33) > ==25287== by 0x517990B: ibis::part::part(char const*, bool) (part.cpp:262) > ==25287== by 0x40306B: main (test2.cpp:9) > ==25287== > ==25287== LEAK SUMMARY: > ==25287== definitely lost: 40 bytes in 1 blocks > ==25287== indirectly lost: 0 bytes in 0 blocks > ==25287== possibly lost: 0 bytes in 0 blocks > ==25287== still reachable: 0 bytes in 0 blocks > ==25287== suppressed: 0 bytes in 0 blocks > ==25287== > ==25287== For counts of detected and suppressed errors, rerun with: -v > ==25287== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 6 from 6) > > > On 20 September 2012 07:45, K. John Wu <[email protected]> wrote: >> Hi, Petr, >> >> I just updated examples/thula.cpp to exercise the option of reordering >> read-only data partitions. It seems to work in a small test I tried. >> The new code is in SVN as revision 577. >> >> BTW, if yo invoke thula with a data directory and a orderby clause, >> but without a where clause or select clause, then it will perform the >> reordering on the data partitions in the given directory. >> >> At least on the surface, the following command line (run in tests >> directory after 'make check') seems to work fine. >> >> ../example/thula -d tmp/t2 -o c -v >> >> >> Would you mind give it a try? >> >> Thanks. >> >> John >> >> >> On 9/19/12 2:41 AM, Petr Velan wrote: >>> Hi John, >>> >>> I managed to get the test program running, but I cannot seem to be >>> able to reproduce the deadlock. I'll look further into it when I have >>> some spare time. >>> >>> One thing I noticed using the test program, the reorder does not work >>> on read-only partition, but the documentation says that it should, >>> since it does not "change" the data. I'm not running the latest SVN >>> version, so maybe it is somehow resolved and consistent now. I just >>> wanted to let you know. >>> >>> Thanks, >>> Petr >>> >>> On 4 September 2012 07:30, Kesheng Wu <[email protected]> wrote: >>>> Hi, Petr, >>>> >>>> Attached is a modification of the file tests/setqgen.cpp to mimic your >>>> use case. So far, it seems to produce exactly the same output set (as >>>> a whole, not in the individual partitions) as produced by >>>> tests/setqgen.cpp. It works OK on my laptop. Would you mind take a >>>> look and see if you can get it to behave more like what your program >>>> does? >>>> >>>> Thanks. >>>> >>>> John >>>> >>>> >>>> On Thu, Aug 30, 2012 at 11:45 PM, Petr Velan <[email protected]> wrote: >>>>> Hi John, >>>>> >>>>> The memory management in FastBit is good for batch mode of operation, >>>>> however I need to reduce the memory footprint, since there might be >>>>> some other operations that might require large amount of memory for >>>>> the short time, so that the FastBit cannot have it allocated all the >>>>> time. >>>>> >>>>> I know that there is a limit that allows to use maximum of half of >>>>> available memory and that it can be changed. I think it would be a >>>>> good thing to have two limits. One to set maximum that can be used and >>>>> other that would trigger the unload function. The result would be that >>>>> I would set FastBit to have 0.5GB at ready and allow it to expand to >>>>> 10GB. So when needed, the fastbit would use up to 10GB of memory, but >>>>> after that it would would free it and keep only 0.5GB for further use. >>>>> The default could still be to have both limits at half of available >>>>> memory. What do you think? >>>>> >>>>> We are currently trying to manually call >>>>> ibis::fileManager::instance().flushDir() to see if it helps to keep >>>>> the memory down, but I believe that the solution I described earlier >>>>> is much more generic. >>>>> >>>>> Unfortunately, I do not have any simple code to reproduce the >>>>> deadlock. I'll try to look into it, maybe compile FastBit with >>>>> debugging symbols to help us better understand what is going on. >>>>> >>>>> Petr >>>>> >>>>> On 30 August 2012 20:29, K. John Wu <[email protected]> wrote: >>>>>> Hi, Petr, >>>>>> >>>>>> Thanks for clarifying the use case. Looks like you can not wait for >>>>>> everything to be done before releasing the ibis::part objects. >>>>>> Regarding the memory usage, FastBit does lazy deletions - as long as >>>>>> no one needs new memory, the existing content read from files will be >>>>>> kept in memory. The default maximum memory to be used is a half of >>>>>> the physical memory - which explains what you've observed. Once >>>>>> reaching that limit, ibis::fileManager::unload will be called to >>>>>> remove the content of files that are no in active use. In your case, >>>>>> it sounds like there will be a lot of old files to be removed from >>>>>> memory. >>>>>> >>>>>> Since there is no clear indication which thread is holding on to the >>>>>> mutex lock, we might need to create a multithreaded data generator >>>>>> that can mimic your data ingestion process. If you have simple one >>>>>> that I can borrow, I would greatly appreciate it. >>>>>> >>>>>> Most likely, another copy of ibis::fileManager::getFile is holding on >>>>>> to the ibis::fileManager::mutex. However, logically, that is not >>>>>> possible because that thread can only be waiting on a conditional >>>>>> variable in which case it should have yield the mutex lock already. >>>>>> Anyway, something gnarly is going on here.. >>>>>> >>>>>> John >>>>>> >>>>>> >>>>>> On 8/29/12 10:54 PM, Petr Velan wrote: >>>>>>> Hi John, >>>>>>> >>>>>>> I still do not understand why there is a deadlock, or why is the >>>>>>> access to different partitions managed by same mutex lock. >>>>>>> >>>>>>> Our use case is this: >>>>>>> We have a process that collects data from network and stores them to >>>>>>> fastbit partitions. Each partition contains 5 minutes of data, >>>>>>> approximately 300-400MB. After 5 minutes expire, new thread is >>>>>>> launched that creates ibis::part, runs reorder, deletes the part, >>>>>>> creates ibis::table which is used to create indexes and then deletes >>>>>>> the table. After that the thread ends. >>>>>>> >>>>>>> Since there is data from multiple sources, there are multiple threads >>>>>>> that store the data and reorder/index it. >>>>>>> >>>>>>> What is bothering me are two things: >>>>>>> The deadlock, since the mutex should only synchronize, I wonder who >>>>>>> really holds the lock when both threads are waiting for it. >>>>>>> Second is that the memory used by the process constantly grows. After >>>>>>> the parts and tables are deleted, I would expect the memory to be >>>>>>> released as well, since for next 5 minutes, it will not be needed. >>>>>>> Unfortunately, FastBit does not free the memory until it reaches 50% >>>>>>> of total memory, which in our case is 6GB. That is kind of >>>>>>> unfortunate, since what it should really need is about 1GB of memory >>>>>>> for reorder in the worst case and then the memory should be free to >>>>>>> use by other processes. Is there any way to achieve this? The memory >>>>>>> is consumed even without the reordering, only when building indexes. >>>>>>> >>>>>>> Thank you for the warning about strings, we plan to use them in >>>>>>> future, so we will have to without the reorder in that case. >>>>>>> >>>>>>> Petr >>>>>>> >>>>>>> On 29 August 2012 18:28, K. John Wu <[email protected]> wrote: >>>>>>>> Hi, Petr, >>>>>>>> >>>>>>>> From the stack traces, look like one thread is trying to free a data >>>>>>>> partition object while another one is trying to reorder the rows of >>>>>>>> presumably another data partition. The first mutex lock is invoked >>>>>>>> from the constructor of a storage object (ibis::fileManager::storage). >>>>>>>> This is invoked because the amount of data in memory (tracked by the >>>>>>>> file manager) is close to the prescribed maximum (maxBytes). The >>>>>>>> second mutex lock is invoked from a function called >>>>>>>> ibis::fileManager::removeCleaner (which is invoked by the destructor >>>>>>>> of an ibis::part object). >>>>>>>> >>>>>>>> Running out memory seems to be the fundamental problem here. >>>>>>>> Presumably, you only need to do reordering once and your datasets are >>>>>>>> quite large. I would suggest that you use only a single thread to >>>>>>>> reorder your data - this way all the memory will devoted to a single >>>>>>>> reordering operation. >>>>>>>> >>>>>>>> If you really do have a lot of memory (or each data partition is >>>>>>>> relatively small) and want to do the reordering with multiple threads, >>>>>>>> then delay the operation of freeing the ibis::part objects until you >>>>>>>> are done with all reordering operations. The cleaner objects from >>>>>>>> each data partition will make sure each ibis::part object is taking >>>>>>>> only a minimal amount of memory. >>>>>>>> >>>>>>>> A note of warning, the current code only sort the numerical values, >>>>>>>> any strings or blobs will be left untouched. If your datasets have >>>>>>>> strings or blobs, your datasets will not be coherent after calling the >>>>>>>> function reorder! >>>>>>>> >>>>>>>> John >>>>>>>> >>>>>>>> >>>>>>>> On 8/29/12 4:57 AM, Petr Velan wrote: >>>>>>>>> Hi John, >>>>>>>>> >>>>>>>>> thank you for all the work that you put into the FastBit library, it >>>>>>>>> allows us to achieve great results! >>>>>>>>> >>>>>>>>> I've bumped into a little bug which might be very hard to reproduce or >>>>>>>>> identify. I'm using two thread to reorder and index data that are >>>>>>>>> already stored on disk. It was ok for a little while, but then it >>>>>>>>> stuck in deadlock. Here are gdb traces from both threads, >>>>>>>>> unfortunately without debugging symbols, so that the specific files >>>>>>>>> and lines are unknown. >>>>>>>>> >>>>>>>>> We are currently using the SVN version 532. >>>>>>>>> >>>>>>>>> (gdb) bt >>>>>>>>> #0 0x00007f8983463054 in __lll_lock_wait () from >>>>>>>>> /lib64/libpthread.so.0 >>>>>>>>> #1 0x00007f898345e388 in _L_lock_854 () from /lib64/libpthread.so.0 >>>>>>>>> #2 0x00007f898345e257 in pthread_mutex_lock () from >>>>>>>>> /lib64/libpthread.so.0 >>>>>>>>> #3 0x00007f898271e074 in ibis::fileManager::storage::storage(unsigned >>>>>>>>> long) () from /usr/lib64/libfastbit.so.0 >>>>>>>>> #4 0x00007f898271eb16 in ibis::fileManager::storage::enlarge(unsigned >>>>>>>>> long) () from /usr/lib64/libfastbit.so.0 >>>>>>>>> #5 0x00007f898272214f in ibis::fileManager::roFile::doRead(char >>>>>>>>> const*) () from /usr/lib64/libfastbit.so.0 >>>>>>>>> #6 0x00007f8982723b4b in ibis::fileManager::getFile(char const*, >>>>>>>>> ibis::fileManager::storage**, ibis::fileManager::ACCESS_PREFERENCE) () >>>>>>>>> from /usr/lib64/libfastbit.so.0 >>>>>>>>> #7 0x00007f898273406a in int ibis::fileManager::getFile<unsigned >>>>>>>>> short>(char const*, ibis::array_t<unsigned short>&, >>>>>>>>> ibis::fileManager::ACCESS_PREFERENCE) () from >>>>>>>>> /usr/lib64/libfastbit.so.0 >>>>>>>>> #8 0x00007f8981f9f4a5 in ibis::column::actualMinMax(char const*, >>>>>>>>> ibis::bitvector const&, double&, double&) const () from >>>>>>>>> /usr/lib64/libfastbit.so.0 >>>>>>>>> #9 0x00007f8981fa3546 in ibis::column::computeMinMax() () from >>>>>>>>> /usr/lib64/libfastbit.so.0 >>>>>>>>> #10 0x00007f89827beae6 in >>>>>>>>> ibis::part::gatherSortKeys(ibis::array_t<char const*>&) () from >>>>>>>>> /usr/lib64/libfastbit.so.0 >>>>>>>>> #11 0x00007f89827bfc56 in ibis::part::reorder() () from >>>>>>>>> /usr/lib64/libfastbit.so.0 >>>>>>>>> #12 0x00007f8982c7e2af in reorder_index(void*) () from >>>>>>>>> /usr/share/ipfixcol/plugins/ipfixcol-fastbit-output.so >>>>>>>>> #13 0x00007f898345c851 in start_thread () from /lib64/libpthread.so.0 >>>>>>>>> #14 0x00007f89831aa6dd in next_line () from /lib64/libc.so.6 >>>>>>>>> #15 0x0000000000000000 in ?? () >>>>>>>>> (gdb) >>>>>>>>> >>>>>>>>> >>>>>>>>> (gdb) bt >>>>>>>>> #0 0x00007f8983463054 in __lll_lock_wait () from >>>>>>>>> /lib64/libpthread.so.0 >>>>>>>>> #1 0x00007f898345e388 in _L_lock_854 () from /lib64/libpthread.so.0 >>>>>>>>> #2 0x00007f898345e257 in pthread_mutex_lock () from >>>>>>>>> /lib64/libpthread.so.0 >>>>>>>>> #3 0x00007f898175a6aa in >>>>>>>>> ibis::util::mutexLock::mutexLock(pthread_mutex_t*, char const*) () >>>>>>>>> from /usr/lib64/libfastbit.so.0 >>>>>>>>> #4 0x00007f89827177d4 in >>>>>>>>> ibis::fileManager::removeCleaner(ibis::fileManager::cleaner const*) () >>>>>>>>> from /usr/lib64/libfastbit.so.0 >>>>>>>>> #5 0x00007f8981735952 in ibis::part::~part() () from >>>>>>>>> /usr/lib64/libfastbit.so.0 >>>>>>>>> #6 0x00007f8981735c29 in ibis::part::~part() () from >>>>>>>>> /usr/lib64/libfastbit.so.0 >>>>>>>>> #7 0x00007f8982c7e2cd in reorder_index(void*) () from >>>>>>>>> /usr/share/ipfixcol/plugins/ipfixcol-fastbit-output.so >>>>>>>>> #8 0x00007f898345c851 in start_thread () from /lib64/libpthread.so.0 >>>>>>>>> #9 0x00007f89831aa6dd in next_line () from /lib64/libc.so.6 >>>>>>>>> #10 0x0000000000000000 in ?? () >>>>>>>>> (gdb) >>>>>>>>> >>>>>>>>> >>>>>>>>> Do you have any idea what might be going on? >>>>>>>>> >>>>>>>>> With regards, >>>>>>>>> Petr Velan >>>>>>>>> > _______________________________________________ > FastBit-users mailing list > [email protected] > https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users _______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
