Hi, Petr, FastBit is only tracking the memory usage of a few object types such as ibis::array_t and ibis::buffer. Since you are using a compression layer, I presume that you are also allocating considerable amounts of memory using malloc or new directly. This would mean FastBit is only tracking a part of the memory uses. If you turn up the verboseness of the FastBit code, say, ibis::util::setVerboseLevel(10), and ship me the log messages, I can take a look and see if I can spot anything more useful to suggest.
John On 3/20/14, 6:35 AM, Petr Velan wrote: > Hi John, > > I'm currently working on FastBit support for compressed files. To > simplify things a bit for a start, I have disabled the MMAP support > and forced FastBit to use open, read, write and seek functions. Then I > hooked into these functions and provided a compression layer. The > problem is that just disabling the MMAP causes FastBit to use so much > memory that OOM killer terminates the process for some aggregation > queries. > > Is there anything I can do to tell FastBit to limit the amount of > memory it can use? I thought that only half of the available memory > will be used by default. I've tested this issue on latest SVN revision. > > Thank you in advance for any help, > Petr Velan > > > _______________________________________________ > FastBit-users mailing list > [email protected] > https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users > _______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
