Hi John, thanks, that was exactly the information I was looking for. I haven't observed any issues so far and was just curious because I hadn't looked into the index creation yet. I think holding a write lock until the index creation is completed is absolutely fine as it sounds like an effective and simple solution to the problem. I can still think of other means of locking for the column data (row locks, page locks etc.) but all that would be much more complicated to implement. Furthermore these solutions would introduce new overhead for lock management.
Thanks for the prompt reply and best regards Jan -----Original Message----- From: K. John Wu [mailto:[email protected]] Sent: Wednesday, January 20, 2010 11:26 PM To: FastBit Users Cc: Jan Steemann Subject: Re: [FastBit-users] Creating a partition Hi, Jan, There is no conflict problem -- if you do observe something, let us know. Here is a bit more information. When multiple threads need to create an index at the same time. The first thread will hold a write lock until the index is created. Other threads will have wait for the index creation to complete. Feel free to suggest something that might work better than this option. John On 1/20/2010 2:11 PM, Jan Steemann wrote: > Hi all, > > speaking about read-only data, what would also be interesting to know would > be if there are any potential file creation race conditions when performing > select queries. > As far as I know, Fastbit will create an .idx file a column the very first > time the column data gets selected. > My understanding is that these .idx files are created on the fly when the > column data is accessed. > > Just in case that there a two parallel selects on the same column data in two > different processes/threads/ibis invocations, is it possible that race > conditions will occur for the .idx file creation? Or are there are sorts of > exclusive write locks on the .idx files? > > Thanks in advance for letting me know. > > Best regards > Jan > > > -----Original Message----- > From: [email protected] > [mailto:[email protected]] On Behalf Of Andreas Streichardt > Sent: Wednesday, January 20, 2010 11:03 PM > To: FastBit Users > Subject: Re: [FastBit-users] Creating a partition > > > Am 20.01.2010 um 22:34 schrieb K. John Wu: > > > Hi, Andreas, > > > > Hi John, > > > You have a point about read-only data partition. It would be useful > to be able to create a data partition object representing read-only > data or existing data. Let me suggest that we add another argument to > the constructor to indicate what we want. Guess the argument has to > be an optional one. Do you have any opinion what sort of options it > should be able to express? Two obvious ones are whether the data > directory must exist, and whether the data is read-only. Anything else? > > > > generally speaking i would only distinguish between two cases here. One which > writes and one which reads. > I don't see a case (yet) where you must distinguish between "is read only" > and "directory must exists". For me (with my limited knowledge about fastbit) > there are only two. One which is read only and one which is able to write. > > However i am not yet deep enough into fastbits internals so i may of course > be wrong here. For me a simple read only case (no mkdir(), no write()) is > fine. As said before: I am catching that in userspace currently and this just > works fine so maybe postpone a decision until i am a bit more into the > internals (not even sure what a partition really means - speaking in SQL it > is a table for me currently ;) ) or somebody else proposes the ultimate > solution ;) This was just something i noted when working with fastbit. > > > John > > > > Kind regards, > > Andreas Streichardt > _______________________________________________ > FastBit-users mailing list > [email protected] > https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users _______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
