Hi, Davey, If you are using command line tool, then there is no coordination about their activities. Logically, someone/something has to remember that the data is being modified and to prevent others from accessing the inconsistent data files while the modification is underway. We used to do run an old version of FastBit as a CORBA server (in a client-server mode similar to many of the database systems). So the core of FastBit should be thread safe should you plan to write a server component around the FastBit code.
To minimize the downtime when the modification is in progress, ibis::part data structure is also able to utilize a shadow (backup) directory. If you are going to write your own server, this feature might be useful. An alternative proposal would be implement some sort of file locking mechanism in ibis.cpp, any options that modify the data has to create a lock file, and all other operations can only proceed if the lock file is not present. This option is not as efficient as the server option, but it probably requires less work. We could look into this option, but it might take a few weeks for us to study the issues and implement this option. You are welcome to implement your own option of course. John On 8/8/12 2:38 AM, Lidawei (Davey) wrote: > Hi John, > I encountered a problem when I access fastbit database through > multi-processes. > > Our system provides a web service to filter records from a fastbit database > with the condition request by client. > And we also provides a command line tool to the operator which can update > data in the same fastbit database. > > When the web service running alone, It works fine. > But, If I use the command line tool to update some data in the database, the > query will reply incorrectly. > The problem can be solved by calling cleanup() function before each query. > But If I do so, the query performance drops from 1000 TPS to 100 TPS. > > Do you have any idea on how to solve this problem without cost of performance? > > Best Regards, > Davey > > _______________________________________________ > FastBit-users mailing list > [email protected] > https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users > _______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
