Hi, Andrew, As you have suggested, I have added code to regenerate the dictionary and the index associated with the metatags when it detects a missing dictionary or a missing index file. The updated code is checked in as SVN Revision 570. Please give it a try when you get the chance.
Thanks. John On 9/14/12 12:55 PM, Olson, Andrew wrote: > Hi John, > I tried 567 but it doesn't update the index because it is looking for the raw > data. You probably need to check if the column is a metaTag and set the > index accordingly. > Andrew > > thula invoking combineCategories with 1 name > Warning -- fileManager::storage::read(fname=./1/FBchr.int, begin=0, > end=35840624) failed to open the named file > Warning -- category[1.FBchr]::setDictionary failed to open data file > ./1/FBchr to create an index > Warning -- mensa[T-1]::combineCategories(FBchr) failed to update index for > 1.FBchr > > On Sep 14, 2012, at 1:37 AM, K. John Wu wrote: > >> Hi, Andrew, >> >> Looks like on the second run of the -M operation, the program >> misbehaves. This problem should be fixed in SVN revision 567. Please >> give it a try when you get the chance. >> >> Thanks. >> >> John >> >> >> >> On 9/13/12 9:05 AM, Olson, Andrew wrote: >>> Actually, -M didn't work in the case of the metaTag-derived columns, I just >>> didn't see the warnings without -v. Looking at the code, it seems there >>> needs to be a special case for metaTag-derived columns so the .idx files >>> can be regenerated without examining the underlying data (implied .int >>> file). >>> Andrew >>> >>> $ thula -d . -v -M FBchr >>> >>> Constructed a part named 1 >>> Constructed a part named 10 >>> Constructed a part named 2 >>> Constructed a part named 3 >>> Constructed a part named 4 >>> Constructed a part named 5 >>> Constructed a part named 6 >>> Constructed a part named 7 >>> Constructed a part named 8 >>> Constructed a part named 9 >>> mensa::addPartition(.) increases the number partitions from 0 to 10, the >>> number of rows from 0 to 59161937, and the number of columns from 0 to 9 >>> thula invoking combineCategories with 1 name >>> Warning -- fileManager::storage::read(fname=./1/FBchr.int, begin=0, >>> end=35840624) failed to open the named file >>> Warning -- fileManager::storage::read(fname=./10/FBchr.int, begin=0, >>> end=17331856) failed to open the named file >>> Warning -- fileManager::storage::read(fname=./2/FBchr.int, begin=0, >>> end=27313592) failed to open the named file >>> Warning -- fileManager::storage::read(fname=./3/FBchr.int, begin=0, >>> end=26749204) failed to open the named file >>> Warning -- fileManager::storage::read(fname=./4/FBchr.int, begin=0, >>> end=28259840) failed to open the named file >>> Warning -- fileManager::storage::read(fname=./5/FBchr.int, begin=0, >>> end=24325084) failed to open the named file >>> Warning -- fileManager::storage::read(fname=./6/FBchr.int, begin=0, >>> end=18849964) failed to open the named file >>> Warning -- fileManager::storage::read(fname=./7/FBchr.int, begin=0, >>> end=20123184) failed to open the named file >>> Warning -- fileManager::storage::read(fname=./8/FBchr.int, begin=0, >>> end=19779448) failed to open the named file >>> Warning -- fileManager::storage::read(fname=./9/FBchr.int, begin=0, >>> end=18074952) failed to open the named file >>> combineCategories returned 10 >>> fileManager::clear -- completed with 24 bytes of storage remain in memory >>> after removing all managed objects >>> >>> On Sep 13, 2012, at 9:37 AM, K. John Wu wrote: >>> >>>> Hi, Andrew, >>>> >>>> Great to know that the -M option works. To make sure the result table >>>> is always outputted, please use the '-x filename' option. It will >>>> write the results to the named file no matter -v is specified or not. >>>> This option should be a little faster than writing a lot of stuff to >>>> screen. >>>> >>>> This option is undocumented because it is different from the output >>>> option in other programs ;-) >>>> >>>> John >>>> >>>> >>>> On 9/13/12 5:22 AM, Olson, Andrew wrote: >>>>> Hi John, I tried thula with -M to merge the dictionaries. This >>>>> speeds up the allele_string,count(*) query to 1.65s, which is as >>>>> great! The FBchr, count(*) query doesn't return any results unless >>>>> you run with -v (but it still takes 20s that way). >>>>> >>> > _______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
