Hi, Bailu, You need to remove the index file from memory before building the index - when you build a new index while the index file is memory mapped (which is usually the case), then the function attempting to write the index file will fail. You can use ibis::fileManager::instance().flushDir(name-of-directory) to flush the unused files out of memory cache
Unfortunately, this function is not accessible through JNI. I would suggest that you limit the scope of FastBit object in java and have the query processor initialize a new FastBit object when needed. John On 6/5/13 3:44 PM, Bailu Ding wrote: > Hi John, > > I have experienced two problems that are probably related, when > running multi-threaded program with FastBit. > > Assume I have two threads, one (the loader) loads data from CSV files > and builds indexes with FastBit, the other (the queryer) issues > queries to FastBit. Whenever the loader finishes building the index of > a partition (FastBit.build_index() returns), it signals the queryer. > The queryer will then be able to query the newly available partition > (along with older partitions). > > The two problems I have are: > > = When I build the partition every 30 seconds, the JVM crashes. Here > each CSV contains tuples injected within this 30 seconds. I got > different crash logs. A relevant crash log is like: > > # A fatal error has been detected by the Java Runtime Environment: > # > # SIGSEGV (0xb) at pc=0x00007f45f7b0fb91, pid=10349, tid=139938978543360 > # > # JRE version: 6.0_27-b07 > # Java VM: Java HotSpot(TM) 64-Bit Server VM (20.2-b06 mixed mode > linux-amd64 compressed oops) > # Problematic frame: > # C [libfastbit.so.0+0x1247b91] fastbit_part_list::find(char > const*)+0x411 > > = When I build the partition every 2 seconds, the JVM crashes. Here > each CSV contains tuples injected within 2 seconds. In addition, I got > a lot of warnings like the following: > > Warning -- fileManager::flushDir can not remove in-memory file > (/dir/to/part/filename.idx). It is in use > Warning -- fileManager::flushDir(/dir/to/part) finished with 1 file > still in memory > > I tried to work around the JVM crash. Here are some observations: > > = If I wait a few seconds before starting to query the newly built > partition, the JVM still crashes. > = If I first load all the data, and then start to query the FastBit, > it works nicely. > > Is that because the FastBit.build_index() is not synchronous? How can > I work around this? > > Cheers, > Bailu > > > > _______________________________________________ > FastBit-users mailing list > [email protected] > https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users > _______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
