Hi all, speaking about read-only data, what would also be interesting to know would be if there are any potential file creation race conditions when performing select queries. As far as I know, Fastbit will create an .idx file a column the very first time the column data gets selected. My understanding is that these .idx files are created on the fly when the column data is accessed.
Just in case that there a two parallel selects on the same column data in two different processes/threads/ibis invocations, is it possible that race conditions will occur for the .idx file creation? Or are there are sorts of exclusive write locks on the .idx files? Thanks in advance for letting me know. Best regards Jan -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Andreas Streichardt Sent: Wednesday, January 20, 2010 11:03 PM To: FastBit Users Subject: Re: [FastBit-users] Creating a partition Am 20.01.2010 um 22:34 schrieb K. John Wu: Hi, Andreas, Hi John, You have a point about read-only data partition. It would be useful to be able to create a data partition object representing read-only data or existing data. Let me suggest that we add another argument to the constructor to indicate what we want. Guess the argument has to be an optional one. Do you have any opinion what sort of options it should be able to express? Two obvious ones are whether the data directory must exist, and whether the data is read-only. Anything else? generally speaking i would only distinguish between two cases here. One which writes and one which reads. I don't see a case (yet) where you must distinguish between "is read only" and "directory must exists". For me (with my limited knowledge about fastbit) there are only two. One which is read only and one which is able to write. However i am not yet deep enough into fastbits internals so i may of course be wrong here. For me a simple read only case (no mkdir(), no write()) is fine. As said before: I am catching that in userspace currently and this just works fine so maybe postpone a decision until i am a bit more into the internals (not even sure what a partition really means - speaking in SQL it is a table for me currently ;) ) or somebody else proposes the ultimate solution ;) This was just something i noted when working with fastbit. John Kind regards, Andreas Streichardt _______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
