Hi, Mian, Thanks for the information. The warning messages appear odd to me. For example, the 1st warning message says that the maximum allowed memory (maxBytes) is about 4GB, the amount of memory used currently (totalBytes) is only 33 MB, but it can not find enough space for 628 MB. From what I can see there are plenty of free memory available -- there is probably something computed in 32-bit integer arithmetic that is causing this problem. It would be helpful for us to get some information about your machine and compiler in order to help us locate the actual cause of these erroneous warnings, more specifically, your OS version, memory size, sizes of size_t, long int, and int.
The reason that FastBit can still produce the expected answer because there are fall-back options in many cases. John On 10/25/2009 9:27 AM, Mian Lu wrote: > Actually when I perform an evaluation for about 15 attributes (each > attribute size is totally 1GB, and the selectivity for each attribute is > 40%), I will receive some warnings like this: > > Warning -- fileManager::unload unable to free enough space for 628021720 > bytes (totalBytes=33032688, maxBytes=4182255616) > Warning -- fileManager::storage is unable to find 628021720 bytes of space > to read file descriptor 3 > Sat Oct 24 19:38:29 2009 > Warning -- column[test2.attr12](INT)::estimateRange -- received a > std::exception -- storage::ctor(read file) failed > Warning -- fileManager::unload unable to free enough space for 628059044 > bytes (totalBytes=66064972, maxBytes=4182255616) > Warning -- fileManager::storage is unable to find 628059044 bytes of space > to read file descriptor 4 > Sat Oct 24 19:40:06 2009 > Warning -- column[test2.attr0](INT)::estimateRange -- received a > std::exception -- storage::ctor(read file) failed > ...... > > But the final result is correct. I guess Fastbit does has some memory > limitations for efficiency reasons. Moreover, I also want to know the > exact answers. Thanks. > > Mian > > >> Hi guys, >> >> I am a newbie of fastbit. I do not know whether bitmap index is loaded by >> fastbit into memory. if so, then the size of the bitmap index will be >> limited by the memory size of that machine runs fastbit. Can fastbit >> overcome that problem? If it can't, how many records at most can fastbit >> build bitmap indices on them? >> >> >> Thanks, >> Min >> -- >> My research interests are distributed systems, parallel computing and >> bytecode based virtual machine. >> >> My profile: >> http://www.linkedin.com/in/coderplay >> My blog: >> http://coderplay.javaeye.com >> _______________________________________________ >> FastBit-users mailing list >> [email protected] >> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users >> > > > _______________________________________________ > FastBit-users mailing list > [email protected] > https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users _______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
