Hi John, I have experienced two problems that are probably related, when running multi-threaded program with FastBit.
Assume I have two threads, one (the loader) loads data from CSV files and builds indexes with FastBit, the other (the queryer) issues queries to FastBit. Whenever the loader finishes building the index of a partition (FastBit.build_index() returns), it signals the queryer. The queryer will then be able to query the newly available partition (along with older partitions). The two problems I have are: = When I build the partition every 30 seconds, the JVM crashes. Here each CSV contains tuples injected within this 30 seconds. I got different crash logs. A relevant crash log is like: # A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x00007f45f7b0fb91, pid=10349, tid=139938978543360 # # JRE version: 6.0_27-b07 # Java VM: Java HotSpot(TM) 64-Bit Server VM (20.2-b06 mixed mode linux-amd64 compressed oops) # Problematic frame: # C [libfastbit.so.0+0x1247b91] fastbit_part_list::find(char const*)+0x411 = When I build the partition every 2 seconds, the JVM crashes. Here each CSV contains tuples injected within 2 seconds. In addition, I got a lot of warnings like the following: Warning -- fileManager::flushDir can not remove in-memory file (/dir/to/part/filename.idx). It is in use Warning -- fileManager::flushDir(/dir/to/part) finished with 1 file still in memory I tried to work around the JVM crash. Here are some observations: = If I wait a few seconds before starting to query the newly built partition, the JVM still crashes. = If I first load all the data, and then start to query the FastBit, it works nicely. Is that because the FastBit.build_index() is not synchronous? How can I work around this? Cheers, Bailu
_______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
