Hi John, did you by any chance have a bit of time to look into this matter?
Thanks, Petr On Fri, Mar 21, 2014 at 1:25 PM, Petr Velan <[email protected]> wrote: > Hi John, > > I did the testing only with disabled mmap, no other modifications. I'm now > sending you an email with link to download the complete log. I'm trying it > on an older version from SVN revision 657. Sadly, the newest is not > compatible with the tested system. > > The log file is quite big 19GB, I compressed it to 2.5GB and the link > should be in another email shortly. I'm using > https://filesender.cesnet.cz/ service for the transfer. > > Let me know if there is anything more that I can do. > > Best, > Petr > > > > > On Thu, Mar 20, 2014 at 5:21 PM, K. John Wu <[email protected]> wrote: > >> Hi, Petr, >> >> FastBit is only tracking the memory usage of a few object types such >> as ibis::array_t and ibis::buffer. Since you are using a compression >> layer, I presume that you are also allocating considerable amounts of >> memory using malloc or new directly. This would mean FastBit is only >> tracking a part of the memory uses. If you turn up the verboseness of >> the FastBit code, say, ibis::util::setVerboseLevel(10), and ship me >> the log messages, I can take a look and see if I can spot anything >> more useful to suggest. >> >> John >> >> >> On 3/20/14, 6:35 AM, Petr Velan wrote: >> > Hi John, >> > >> > I'm currently working on FastBit support for compressed files. To >> > simplify things a bit for a start, I have disabled the MMAP support >> > and forced FastBit to use open, read, write and seek functions. Then I >> > hooked into these functions and provided a compression layer. The >> > problem is that just disabling the MMAP causes FastBit to use so much >> > memory that OOM killer terminates the process for some aggregation >> > queries. >> > >> > Is there anything I can do to tell FastBit to limit the amount of >> > memory it can use? I thought that only half of the available memory >> > will be used by default. I've tested this issue on latest SVN revision. >> > >> > Thank you in advance for any help, >> > Petr Velan >> > >> > >> > _______________________________________________ >> > FastBit-users mailing list >> > [email protected] >> > https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users >> > >> _______________________________________________ >> FastBit-users mailing list >> [email protected] >> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users >> > >
_______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
