Hi John,
I just updated to latest SVN revision and tried the reorder function
on read-only partition. The reordering is now performed exactly as
described in documentation, so the problem no longer exists.
However, I run into a problem with some memory leaks in the new
revision. My previous version was 3.0.10 (SVN 532) and there were no
leaks in my usecase. Now I'm loosing some memory. To simulate the
memory loss, you only need to create a partition in a simple program:
#include <fastbit/ibis.h>
#include <iostream>
int main(int argc, char *argv[]) {
ibis::gVerbose = 10;
ibis::part part(argv[1], false);
return 0;
}
Attached below is the program log with verbose set to 10. You might
notice that the leak is 40 bytes long, and that it is caused by
fileManager::storage class created in array_t constructor not being
deleted. In my case, the first constucted array_t's actual is probably
not removed. I've managed to trace the problem to "ibis::bitvector
amask" of ibis::part, but I cannot seem to find the problem. Most
likely the fileManager is replaced somewhere without proper
deallocation, but I cannot find it.
Yours sincerely,
Petr Velan
velan@wall:~/Documents/devel/tmp/test> valgrind --leak-check=full
./test2 ../../data/000000000001/1/
==25287== Memcheck, a memory error detector
==25287== Copyright (C) 2002-2010, and GNU GPL'd, by Julian Seward et al.
==25287== Using Valgrind-3.6.1 and LibVEX; rerun with -h for copyright info
==25287== Command: ./test2 ../../data/000000000001/1/
==25287==
FastBit ibis1.3.2.6
Log messages started on Thu Sep 20 17:05:57 2012
fileManager::storage(0x775b8e0, 0) initialization completed
array_t<j> constructed at 0x7fefffb20 with actual=0x775b8e0, m_begin=0
and actual->size()=0
bitvector (0x7fefffb10) constructed with m_vec at 0x7fefffb20
fileManager::ctor found the physical memory size to be 4083470336 bytes
fileManager initialization complete -- maxBytes=2041735168, maxOpenFiles=768
part::readMetaData -- opened ../../data/000000000001/1/-part.txt for reading
Name = "1"
Description = "Generated by ipfixcol fastbit plugin"
Number_of_columns = 11
Number_of_rows = 499992
Timestamp = 1329386614
State = 1
END HEADER
fileManager::storage(0x775f3e0, 0) initialization completed
array_t<j> constructed at 0x775f2e0 with actual=0x775f3e0, m_begin=0
and actual->size()=0
bitvector (0x775f2d0) constructed with m_vec at 0x775f2e0
read info about column 1.e0id1 (ULONG)
part::readMetaData -- got column e0id1 from ../../data/000000000001/1/-part.txt
fileManager::storage(0x77605a0, 0) initialization completed
array_t<j> constructed at 0x77604a0 with actual=0x77605a0, m_begin=0
and actual->size()=0
bitvector (0x7760490) constructed with m_vec at 0x77604a0
read info about column 1.e0id11 (USHORT)
part::readMetaData -- got column e0id11 from ../../data/000000000001/1/-part.txt
fileManager::storage(0x7761770, 0) initialization completed
array_t<j> constructed at 0x7761670 with actual=0x7761770, m_begin=0
and actual->size()=0
bitvector (0x7761660) constructed with m_vec at 0x7761670
read info about column 1.e0id12 (UINT)
part::readMetaData -- got column e0id12 from ../../data/000000000001/1/-part.txt
fileManager::storage(0x7762930, 0) initialization completed
array_t<j> constructed at 0x7762830 with actual=0x7762930, m_begin=0
and actual->size()=0
bitvector (0x7762820) constructed with m_vec at 0x7762830
read info about column 1.e0id152 (ULONG)
part::readMetaData -- got column e0id152 from
../../data/000000000001/1/-part.txt
fileManager::storage(0x7763b00, 0) initialization completed
array_t<j> constructed at 0x7763a00 with actual=0x7763b00, m_begin=0
and actual->size()=0
bitvector (0x77639f0) constructed with m_vec at 0x7763a00
read info about column 1.e0id153 (ULONG)
part::readMetaData -- got column e0id153 from
../../data/000000000001/1/-part.txt
fileManager::storage(0x7764cd0, 0) initialization completed
array_t<j> constructed at 0x7764bd0 with actual=0x7764cd0, m_begin=0
and actual->size()=0
bitvector (0x7764bc0) constructed with m_vec at 0x7764bd0
read info about column 1.e0id2 (ULONG)
part::readMetaData -- got column e0id2 from ../../data/000000000001/1/-part.txt
fileManager::storage(0x7765e90, 0) initialization completed
array_t<j> constructed at 0x7765d90 with actual=0x7765e90, m_begin=0
and actual->size()=0
bitvector (0x7765d80) constructed with m_vec at 0x7765d90
read info about column 1.e0id4 (UBYTE)
part::readMetaData -- got column e0id4 from ../../data/000000000001/1/-part.txt
fileManager::storage(0x7767050, 0) initialization completed
array_t<j> constructed at 0x7766f50 with actual=0x7767050, m_begin=0
and actual->size()=0
bitvector (0x7766f40) constructed with m_vec at 0x7766f50
read info about column 1.e0id5 (UBYTE)
part::readMetaData -- got column e0id5 from ../../data/000000000001/1/-part.txt
fileManager::storage(0x7768210, 0) initialization completed
array_t<j> constructed at 0x7768110 with actual=0x7768210, m_begin=0
and actual->size()=0
bitvector (0x7768100) constructed with m_vec at 0x7768110
read info about column 1.e0id6 (UBYTE)
part::readMetaData -- got column e0id6 from ../../data/000000000001/1/-part.txt
fileManager::storage(0x77693d0, 0) initialization completed
array_t<j> constructed at 0x77692d0 with actual=0x77693d0, m_begin=0
and actual->size()=0
bitvector (0x77692c0) constructed with m_vec at 0x77692d0
read info about column 1.e0id7 (USHORT)
part::readMetaData -- got column e0id7 from ../../data/000000000001/1/-part.txt
fileManager::storage(0x776a590, 0) initialization completed
array_t<j> constructed at 0x776a490 with actual=0x776a590, m_begin=0
and actual->size()=0
bitvector (0x776a480) constructed with m_vec at 0x776a490
read info about column 1.e0id8 (UINT)
part::readMetaData -- got column e0id8 from ../../data/000000000001/1/-part.txt
part[1]::gainReadAccess -- pthread_rwlock_rdlock(0x7fefffbb8) for readRIDs
fileManager::storage(0x776b9a0, 0) initialization completed
array_t<N4ibis5rid_tE> constructed at 0x776b940 with actual=0x776b9a0,
m_begin=0 and actual->size()=0
part[1]::readRIDs -- the file manager failed to read file
"../../data/000000000001/1/-rids". There is no RIDs.
part[1]::releaseAccess -- pthread_rwlock_unlock(0x7fefffbb8) for readRIDs
Warning -- failed to read the content of
../../data/000000000001/1/-part.msk, fileManager::getFile returned
-101
fileManager::storage(0x776d040, 0x776d0b0) added 12 bytes to increase
totalBytes to 12
fileManager::storage(0x776d040, 0x776d0b0) initialization completed
with 12 elements
fileManager::flushFile will do nothing because
"../../data/000000000001/1/-part.msk" is not tracked by the file
manager
part::init -- mask for partition 1 has 499992 set bits out of 499992
Constructed a part named 1
activeDir = "../../data/000000000001/1"
part: 1 (Generated by ipfixcol fastbit plugin) with 499992 rows, 11 columns
Column list:
e0id1: (ULONG) [28, 1.44872e+09]
e0id11: (USHORT) [0, 65535]
e0id12: (UINT) [1.0466e+08, 4.02607e+09]
e0id152: (ULONG) [1.26981e+12, 1.26982e+12]
e0id153: (ULONG) [1.26981e+12, 1.26982e+12]
e0id2: (ULONG) [1, 1.3616e+06]
e0id4: (UBYTE) [1, 41]
e0id5: (UBYTE) [0, 0]
e0id6: (UBYTE) [0, 31]
e0id7: (USHORT) [0, 65535]
e0id8: (UINT) [1.0466e+08, 3.75763e+09]
part[1]::gainWriteAccess -- pthread_rwlock_wrlock(0x7fefffbb8) for ~part
clearing data partition 1
column[1.e0id1]::writeLock -- pthread_rwlock_wrlock(0x775f340) for ~column
clearing column 1.e0id1
column[1.e0id1]::writeLock -- pthread_rwlock_unlock(0x775f340) for ~column
bitvector (0x775f2d0) clear the content of bitvector with m_vec at 0x775f2e0
array_t<j>::freeMemory this=0x775f2e0 actual=0x775f3e0 and m_begin=0
(active references: 0, past references: 1)
fileManager::storage(0x775f3e0, 0) cleared
column[1.e0id11]::writeLock -- pthread_rwlock_wrlock(0x7760500) for ~column
clearing column 1.e0id11
column[1.e0id11]::writeLock -- pthread_rwlock_unlock(0x7760500) for ~column
bitvector (0x7760490) clear the content of bitvector with m_vec at 0x77604a0
array_t<j>::freeMemory this=0x77604a0 actual=0x77605a0 and m_begin=0
(active references: 0, past references: 1)
fileManager::storage(0x77605a0, 0) cleared
column[1.e0id12]::writeLock -- pthread_rwlock_wrlock(0x77616d0) for ~column
clearing column 1.e0id12
column[1.e0id12]::writeLock -- pthread_rwlock_unlock(0x77616d0) for ~column
bitvector (0x7761660) clear the content of bitvector with m_vec at 0x7761670
array_t<j>::freeMemory this=0x7761670 actual=0x7761770 and m_begin=0
(active references: 0, past references: 1)
fileManager::storage(0x7761770, 0) cleared
column[1.e0id152]::writeLock -- pthread_rwlock_wrlock(0x7762890) for ~column
clearing column 1.e0id152
column[1.e0id152]::writeLock -- pthread_rwlock_unlock(0x7762890) for ~column
bitvector (0x7762820) clear the content of bitvector with m_vec at 0x7762830
array_t<j>::freeMemory this=0x7762830 actual=0x7762930 and m_begin=0
(active references: 0, past references: 1)
fileManager::storage(0x7762930, 0) cleared
column[1.e0id153]::writeLock -- pthread_rwlock_wrlock(0x7763a60) for ~column
clearing column 1.e0id153
column[1.e0id153]::writeLock -- pthread_rwlock_unlock(0x7763a60) for ~column
bitvector (0x77639f0) clear the content of bitvector with m_vec at 0x7763a00
array_t<j>::freeMemory this=0x7763a00 actual=0x7763b00 and m_begin=0
(active references: 0, past references: 1)
fileManager::storage(0x7763b00, 0) cleared
column[1.e0id2]::writeLock -- pthread_rwlock_wrlock(0x7764c30) for ~column
clearing column 1.e0id2
column[1.e0id2]::writeLock -- pthread_rwlock_unlock(0x7764c30) for ~column
bitvector (0x7764bc0) clear the content of bitvector with m_vec at 0x7764bd0
array_t<j>::freeMemory this=0x7764bd0 actual=0x7764cd0 and m_begin=0
(active references: 0, past references: 1)
fileManager::storage(0x7764cd0, 0) cleared
column[1.e0id4]::writeLock -- pthread_rwlock_wrlock(0x7765df0) for ~column
clearing column 1.e0id4
column[1.e0id4]::writeLock -- pthread_rwlock_unlock(0x7765df0) for ~column
bitvector (0x7765d80) clear the content of bitvector with m_vec at 0x7765d90
array_t<j>::freeMemory this=0x7765d90 actual=0x7765e90 and m_begin=0
(active references: 0, past references: 1)
fileManager::storage(0x7765e90, 0) cleared
column[1.e0id5]::writeLock -- pthread_rwlock_wrlock(0x7766fb0) for ~column
clearing column 1.e0id5
column[1.e0id5]::writeLock -- pthread_rwlock_unlock(0x7766fb0) for ~column
bitvector (0x7766f40) clear the content of bitvector with m_vec at 0x7766f50
array_t<j>::freeMemory this=0x7766f50 actual=0x7767050 and m_begin=0
(active references: 0, past references: 1)
fileManager::storage(0x7767050, 0) cleared
column[1.e0id6]::writeLock -- pthread_rwlock_wrlock(0x7768170) for ~column
clearing column 1.e0id6
column[1.e0id6]::writeLock -- pthread_rwlock_unlock(0x7768170) for ~column
bitvector (0x7768100) clear the content of bitvector with m_vec at 0x7768110
array_t<j>::freeMemory this=0x7768110 actual=0x7768210 and m_begin=0
(active references: 0, past references: 1)
fileManager::storage(0x7768210, 0) cleared
column[1.e0id7]::writeLock -- pthread_rwlock_wrlock(0x7769330) for ~column
clearing column 1.e0id7
column[1.e0id7]::writeLock -- pthread_rwlock_unlock(0x7769330) for ~column
bitvector (0x77692c0) clear the content of bitvector with m_vec at 0x77692d0
array_t<j>::freeMemory this=0x77692d0 actual=0x77693d0 and m_begin=0
(active references: 0, past references: 1)
fileManager::storage(0x77693d0, 0) cleared
column[1.e0id8]::writeLock -- pthread_rwlock_wrlock(0x776a4f0) for ~column
clearing column 1.e0id8
column[1.e0id8]::writeLock -- pthread_rwlock_unlock(0x776a4f0) for ~column
bitvector (0x776a480) clear the content of bitvector with m_vec at 0x776a490
array_t<j>::freeMemory this=0x776a490 actual=0x776a590 and m_begin=0
(active references: 0, past references: 1)
fileManager::storage(0x776a590, 0) cleared
part[1]::releaseAccess -- pthread_rwlock_unlock(0x7fefffbb8) for ~part
array_t<N4ibis5rid_tE>::freeMemory this=0x776b940 actual=0x776b9a0 and
m_begin=0 (active references: 0, past references: 1)
fileManager::storage(0x776b9a0, 0) cleared
bitvector (0x7fefffb10) clear the content of bitvector with m_vec at 0x7fefffb20
array_t<j>::freeMemory this=0x7fefffb20 actual=0x776d040 and
m_begin=0x776d0b0 (active references: 0, past references: 1)
fileManager::storage(0x776d040, 0x776d0b0) removed 12 bytes to
decrease totalBytes to 0
fileManager::storage(0x776d040, 0x776d0b0) cleared
fileManager::clear has nothing to do
fileManager decommissioned
==25287==
==25287== HEAP SUMMARY:
==25287== in use at exit: 40 bytes in 1 blocks
==25287== total heap usage: 466 allocs, 465 frees, 119,596 bytes allocated
==25287==
==25287== 40 bytes in 1 blocks are definitely lost in loss record 1 of 1
==25287== at 0x4C292C7: operator new(unsigned long) (in
/usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==25287== by 0x5B2F832: ibis::array_t<unsigned int>::array_t()
(array_t.cpp:36)
==25287== by 0x604C5A9: ibis::bitvector::bitvector() (bitvector.cpp:33)
==25287== by 0x517990B: ibis::part::part(char const*, bool) (part.cpp:262)
==25287== by 0x40306B: main (test2.cpp:9)
==25287==
==25287== LEAK SUMMARY:
==25287== definitely lost: 40 bytes in 1 blocks
==25287== indirectly lost: 0 bytes in 0 blocks
==25287== possibly lost: 0 bytes in 0 blocks
==25287== still reachable: 0 bytes in 0 blocks
==25287== suppressed: 0 bytes in 0 blocks
==25287==
==25287== For counts of detected and suppressed errors, rerun with: -v
==25287== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 6 from 6)
On 20 September 2012 07:45, K. John Wu <[email protected]> wrote:
> Hi, Petr,
>
> I just updated examples/thula.cpp to exercise the option of reordering
> read-only data partitions. It seems to work in a small test I tried.
> The new code is in SVN as revision 577.
>
> BTW, if yo invoke thula with a data directory and a orderby clause,
> but without a where clause or select clause, then it will perform the
> reordering on the data partitions in the given directory.
>
> At least on the surface, the following command line (run in tests
> directory after 'make check') seems to work fine.
>
> ../example/thula -d tmp/t2 -o c -v
>
>
> Would you mind give it a try?
>
> Thanks.
>
> John
>
>
> On 9/19/12 2:41 AM, Petr Velan wrote:
>> Hi John,
>>
>> I managed to get the test program running, but I cannot seem to be
>> able to reproduce the deadlock. I'll look further into it when I have
>> some spare time.
>>
>> One thing I noticed using the test program, the reorder does not work
>> on read-only partition, but the documentation says that it should,
>> since it does not "change" the data. I'm not running the latest SVN
>> version, so maybe it is somehow resolved and consistent now. I just
>> wanted to let you know.
>>
>> Thanks,
>> Petr
>>
>> On 4 September 2012 07:30, Kesheng Wu <[email protected]> wrote:
>>> Hi, Petr,
>>>
>>> Attached is a modification of the file tests/setqgen.cpp to mimic your
>>> use case. So far, it seems to produce exactly the same output set (as
>>> a whole, not in the individual partitions) as produced by
>>> tests/setqgen.cpp. It works OK on my laptop. Would you mind take a
>>> look and see if you can get it to behave more like what your program
>>> does?
>>>
>>> Thanks.
>>>
>>> John
>>>
>>>
>>> On Thu, Aug 30, 2012 at 11:45 PM, Petr Velan <[email protected]> wrote:
>>>> Hi John,
>>>>
>>>> The memory management in FastBit is good for batch mode of operation,
>>>> however I need to reduce the memory footprint, since there might be
>>>> some other operations that might require large amount of memory for
>>>> the short time, so that the FastBit cannot have it allocated all the
>>>> time.
>>>>
>>>> I know that there is a limit that allows to use maximum of half of
>>>> available memory and that it can be changed. I think it would be a
>>>> good thing to have two limits. One to set maximum that can be used and
>>>> other that would trigger the unload function. The result would be that
>>>> I would set FastBit to have 0.5GB at ready and allow it to expand to
>>>> 10GB. So when needed, the fastbit would use up to 10GB of memory, but
>>>> after that it would would free it and keep only 0.5GB for further use.
>>>> The default could still be to have both limits at half of available
>>>> memory. What do you think?
>>>>
>>>> We are currently trying to manually call
>>>> ibis::fileManager::instance().flushDir() to see if it helps to keep
>>>> the memory down, but I believe that the solution I described earlier
>>>> is much more generic.
>>>>
>>>> Unfortunately, I do not have any simple code to reproduce the
>>>> deadlock. I'll try to look into it, maybe compile FastBit with
>>>> debugging symbols to help us better understand what is going on.
>>>>
>>>> Petr
>>>>
>>>> On 30 August 2012 20:29, K. John Wu <[email protected]> wrote:
>>>>> Hi, Petr,
>>>>>
>>>>> Thanks for clarifying the use case. Looks like you can not wait for
>>>>> everything to be done before releasing the ibis::part objects.
>>>>> Regarding the memory usage, FastBit does lazy deletions - as long as
>>>>> no one needs new memory, the existing content read from files will be
>>>>> kept in memory. The default maximum memory to be used is a half of
>>>>> the physical memory - which explains what you've observed. Once
>>>>> reaching that limit, ibis::fileManager::unload will be called to
>>>>> remove the content of files that are no in active use. In your case,
>>>>> it sounds like there will be a lot of old files to be removed from memory.
>>>>>
>>>>> Since there is no clear indication which thread is holding on to the
>>>>> mutex lock, we might need to create a multithreaded data generator
>>>>> that can mimic your data ingestion process. If you have simple one
>>>>> that I can borrow, I would greatly appreciate it.
>>>>>
>>>>> Most likely, another copy of ibis::fileManager::getFile is holding on
>>>>> to the ibis::fileManager::mutex. However, logically, that is not
>>>>> possible because that thread can only be waiting on a conditional
>>>>> variable in which case it should have yield the mutex lock already.
>>>>> Anyway, something gnarly is going on here..
>>>>>
>>>>> John
>>>>>
>>>>>
>>>>> On 8/29/12 10:54 PM, Petr Velan wrote:
>>>>>> Hi John,
>>>>>>
>>>>>> I still do not understand why there is a deadlock, or why is the
>>>>>> access to different partitions managed by same mutex lock.
>>>>>>
>>>>>> Our use case is this:
>>>>>> We have a process that collects data from network and stores them to
>>>>>> fastbit partitions. Each partition contains 5 minutes of data,
>>>>>> approximately 300-400MB. After 5 minutes expire, new thread is
>>>>>> launched that creates ibis::part, runs reorder, deletes the part,
>>>>>> creates ibis::table which is used to create indexes and then deletes
>>>>>> the table. After that the thread ends.
>>>>>>
>>>>>> Since there is data from multiple sources, there are multiple threads
>>>>>> that store the data and reorder/index it.
>>>>>>
>>>>>> What is bothering me are two things:
>>>>>> The deadlock, since the mutex should only synchronize, I wonder who
>>>>>> really holds the lock when both threads are waiting for it.
>>>>>> Second is that the memory used by the process constantly grows. After
>>>>>> the parts and tables are deleted, I would expect the memory to be
>>>>>> released as well, since for next 5 minutes, it will not be needed.
>>>>>> Unfortunately, FastBit does not free the memory until it reaches 50%
>>>>>> of total memory, which in our case is 6GB. That is kind of
>>>>>> unfortunate, since what it should really need is about 1GB of memory
>>>>>> for reorder in the worst case and then the memory should be free to
>>>>>> use by other processes. Is there any way to achieve this? The memory
>>>>>> is consumed even without the reordering, only when building indexes.
>>>>>>
>>>>>> Thank you for the warning about strings, we plan to use them in
>>>>>> future, so we will have to without the reorder in that case.
>>>>>>
>>>>>> Petr
>>>>>>
>>>>>> On 29 August 2012 18:28, K. John Wu <[email protected]> wrote:
>>>>>>> Hi, Petr,
>>>>>>>
>>>>>>> From the stack traces, look like one thread is trying to free a data
>>>>>>> partition object while another one is trying to reorder the rows of
>>>>>>> presumably another data partition. The first mutex lock is invoked
>>>>>>> from the constructor of a storage object (ibis::fileManager::storage).
>>>>>>> This is invoked because the amount of data in memory (tracked by the
>>>>>>> file manager) is close to the prescribed maximum (maxBytes). The
>>>>>>> second mutex lock is invoked from a function called
>>>>>>> ibis::fileManager::removeCleaner (which is invoked by the destructor
>>>>>>> of an ibis::part object).
>>>>>>>
>>>>>>> Running out memory seems to be the fundamental problem here.
>>>>>>> Presumably, you only need to do reordering once and your datasets are
>>>>>>> quite large. I would suggest that you use only a single thread to
>>>>>>> reorder your data - this way all the memory will devoted to a single
>>>>>>> reordering operation.
>>>>>>>
>>>>>>> If you really do have a lot of memory (or each data partition is
>>>>>>> relatively small) and want to do the reordering with multiple threads,
>>>>>>> then delay the operation of freeing the ibis::part objects until you
>>>>>>> are done with all reordering operations. The cleaner objects from
>>>>>>> each data partition will make sure each ibis::part object is taking
>>>>>>> only a minimal amount of memory.
>>>>>>>
>>>>>>> A note of warning, the current code only sort the numerical values,
>>>>>>> any strings or blobs will be left untouched. If your datasets have
>>>>>>> strings or blobs, your datasets will not be coherent after calling the
>>>>>>> function reorder!
>>>>>>>
>>>>>>> John
>>>>>>>
>>>>>>>
>>>>>>> On 8/29/12 4:57 AM, Petr Velan wrote:
>>>>>>>> Hi John,
>>>>>>>>
>>>>>>>> thank you for all the work that you put into the FastBit library, it
>>>>>>>> allows us to achieve great results!
>>>>>>>>
>>>>>>>> I've bumped into a little bug which might be very hard to reproduce or
>>>>>>>> identify. I'm using two thread to reorder and index data that are
>>>>>>>> already stored on disk. It was ok for a little while, but then it
>>>>>>>> stuck in deadlock. Here are gdb traces from both threads,
>>>>>>>> unfortunately without debugging symbols, so that the specific files
>>>>>>>> and lines are unknown.
>>>>>>>>
>>>>>>>> We are currently using the SVN version 532.
>>>>>>>>
>>>>>>>> (gdb) bt
>>>>>>>> #0 0x00007f8983463054 in __lll_lock_wait () from
>>>>>>>> /lib64/libpthread.so.0
>>>>>>>> #1 0x00007f898345e388 in _L_lock_854 () from /lib64/libpthread.so.0
>>>>>>>> #2 0x00007f898345e257 in pthread_mutex_lock () from
>>>>>>>> /lib64/libpthread.so.0
>>>>>>>> #3 0x00007f898271e074 in ibis::fileManager::storage::storage(unsigned
>>>>>>>> long) () from /usr/lib64/libfastbit.so.0
>>>>>>>> #4 0x00007f898271eb16 in ibis::fileManager::storage::enlarge(unsigned
>>>>>>>> long) () from /usr/lib64/libfastbit.so.0
>>>>>>>> #5 0x00007f898272214f in ibis::fileManager::roFile::doRead(char
>>>>>>>> const*) () from /usr/lib64/libfastbit.so.0
>>>>>>>> #6 0x00007f8982723b4b in ibis::fileManager::getFile(char const*,
>>>>>>>> ibis::fileManager::storage**, ibis::fileManager::ACCESS_PREFERENCE) ()
>>>>>>>> from /usr/lib64/libfastbit.so.0
>>>>>>>> #7 0x00007f898273406a in int ibis::fileManager::getFile<unsigned
>>>>>>>> short>(char const*, ibis::array_t<unsigned short>&,
>>>>>>>> ibis::fileManager::ACCESS_PREFERENCE) () from
>>>>>>>> /usr/lib64/libfastbit.so.0
>>>>>>>> #8 0x00007f8981f9f4a5 in ibis::column::actualMinMax(char const*,
>>>>>>>> ibis::bitvector const&, double&, double&) const () from
>>>>>>>> /usr/lib64/libfastbit.so.0
>>>>>>>> #9 0x00007f8981fa3546 in ibis::column::computeMinMax() () from
>>>>>>>> /usr/lib64/libfastbit.so.0
>>>>>>>> #10 0x00007f89827beae6 in
>>>>>>>> ibis::part::gatherSortKeys(ibis::array_t<char const*>&) () from
>>>>>>>> /usr/lib64/libfastbit.so.0
>>>>>>>> #11 0x00007f89827bfc56 in ibis::part::reorder() () from
>>>>>>>> /usr/lib64/libfastbit.so.0
>>>>>>>> #12 0x00007f8982c7e2af in reorder_index(void*) () from
>>>>>>>> /usr/share/ipfixcol/plugins/ipfixcol-fastbit-output.so
>>>>>>>> #13 0x00007f898345c851 in start_thread () from /lib64/libpthread.so.0
>>>>>>>> #14 0x00007f89831aa6dd in next_line () from /lib64/libc.so.6
>>>>>>>> #15 0x0000000000000000 in ?? ()
>>>>>>>> (gdb)
>>>>>>>>
>>>>>>>>
>>>>>>>> (gdb) bt
>>>>>>>> #0 0x00007f8983463054 in __lll_lock_wait () from
>>>>>>>> /lib64/libpthread.so.0
>>>>>>>> #1 0x00007f898345e388 in _L_lock_854 () from /lib64/libpthread.so.0
>>>>>>>> #2 0x00007f898345e257 in pthread_mutex_lock () from
>>>>>>>> /lib64/libpthread.so.0
>>>>>>>> #3 0x00007f898175a6aa in
>>>>>>>> ibis::util::mutexLock::mutexLock(pthread_mutex_t*, char const*) ()
>>>>>>>> from /usr/lib64/libfastbit.so.0
>>>>>>>> #4 0x00007f89827177d4 in
>>>>>>>> ibis::fileManager::removeCleaner(ibis::fileManager::cleaner const*) ()
>>>>>>>> from /usr/lib64/libfastbit.so.0
>>>>>>>> #5 0x00007f8981735952 in ibis::part::~part() () from
>>>>>>>> /usr/lib64/libfastbit.so.0
>>>>>>>> #6 0x00007f8981735c29 in ibis::part::~part() () from
>>>>>>>> /usr/lib64/libfastbit.so.0
>>>>>>>> #7 0x00007f8982c7e2cd in reorder_index(void*) () from
>>>>>>>> /usr/share/ipfixcol/plugins/ipfixcol-fastbit-output.so
>>>>>>>> #8 0x00007f898345c851 in start_thread () from /lib64/libpthread.so.0
>>>>>>>> #9 0x00007f89831aa6dd in next_line () from /lib64/libc.so.6
>>>>>>>> #10 0x0000000000000000 in ?? ()
>>>>>>>> (gdb)
>>>>>>>>
>>>>>>>>
>>>>>>>> Do you have any idea what might be going on?
>>>>>>>>
>>>>>>>> With regards,
>>>>>>>> Petr Velan
>>>>>>>>
_______________________________________________
FastBit-users mailing list
[email protected]
https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
