Hi, Jan, Thanks for the report. There are some pretty extensive changes to address the memory leak issue along with the one you mentioned in this message. Please check out the latest code from SVN and give it a try when you get a chance. Thanks.
John On 7/8/2010 6:23 AM, Krivanek, Jan wrote: > Hi John, > > I did have a time to little look on this. I tried my code on revision > 301 and found following: > > When I use following jNatural constructor: > ibis::jNatural::jNatural(const ibis::part* partr, const ibis::part* > parts, const char* colname, const char* condr, const char* conds, const > char* sel) > it doesn't initialize its internal frm_ (ibis::fromClause *) member (is > this the from clause that you was referring to?), but this member is > then required by ibis::jNatural::select method if you specify fully > qualified column name (line 784: match = > frm_->position(tname.c_str());). > > I am currently OK with workaround when not using fully qualified column > names. But please let me know if I am doing something incorrectly, or if > you need more info (repro) for this. > > Thanks > Jan > > -----Original Message----- > From: K. John Wu [mailto:[email protected]] > Sent: Thursday, June 24, 2010 9:02 PM > To: Krivanek, Jan > Cc: FastBit Users > Subject: Re: [FastBit-users] Unable to dump results of (natural) join > operation > > Hi, Jan, > > I am actually expecting the names passed to ibis::quaere::select to be > fully qualified names, i.e., "data1.a", not "a". > > Did you skipped the from clause? There was a little bug in the > implementation of ibis::fromClause used for name look up. This little > problem was fixed yesterday afternoon. You can get the fix from the > latest SVN repository. > > John > > > On 6/24/2010 8:31 AM, Krivanek, Jan wrote: >> Hi John, >> >> Thank you for update to this topic. I checked out newest code and > tried >> my example that wasn't working. Now I seem to be able to join two >> partitions, and then dump resulting data, but only in case if I do not >> fully specify the column name with name of the partition. >> (So if I change the codeline "colnames.push_back("data1.a");" below to >> "colnames.push_back("a");" then I am able to get the results, if I > leave >> it as is, then the final select fail with exception somewhere inside > the >> code). Anyway, this now doesn't seem to be a issue blocking me from >> performing joins. >> >> Thanks >> Jan >> >> -----Original Message----- >> From: K. John Wu [mailto:[email protected]] >> Sent: Wednesday, June 23, 2010 7:07 AM >> To: Krivanek, Jan >> Cc: FastBit Users >> Subject: Re: [FastBit-users] Unable to dump results of (natural) join >> operation >> >> Hi, Jan, >> >> A number of updates to the join code has been checked into the SVN >> repository. Please take a look and let us know if the problem you've >> observed is fixed. >> >> John >> >> On 6/12/2010 2:35 PM, Krivanek, Jan wrote: >>> Hi John, >>> >>> Thank you for your quick reply and information that fix for this >> should >>> be available soon. I must admit that I am a little bit confused now - >>> does it mean that only the naming of column caused the problem? I >>> experienced this also for using the name of column without specifying >>> table name (I tried to debug it a little bit and found some problem > in >>> fillEquiJoinTable which checked for validity of passed parameters >>> indicating count of rows in each table satisfying join condition, but >> I >>> am not at my PC now, so I can provide more info after weekend if >>> needed). >>> Also in code I found that there are probably several ways how to >> perform >>> joins - using classes for ibis::jNatural, or ibis::jRange (which is >> not >>> implemented yet), or there are some join methods in ibis::part or >>> ibis::query classes - is there any recommended way how to perform >> joins >>> (for later grouping and sorting of joined results)? >>> >>> Great thanks for your time >>> Jan >>> >>> -----Original Message----- >>> From: K. John Wu [mailto:[email protected]] >>> Sent: Friday, June 11, 2010 7:33 PM >>> To: FastBit Users >>> Cc: Krivanek, Jan >>> Subject: Re: [FastBit-users] Unable to dump results of (natural) join >>> operation >>> >>> Hi, Jan, >>> >>> I have just came across this problem as well and am in the process of >>> fixing it. The root cause is that many functions in FastBit are > still >>> using the older way of looking up columns and unable to locate > columns >>> named in the format 'data1.a'. The fix for this particular problem > is >>> already in-place, however, I am in the process of changing something >>> else, which might take a day or two to complete. I will let you know >>> as soon as I am comfortable that I have not broken things with my >>> latest changes... >>> >>> Thanks for your patience. >>> >>> John >>> >>> >>> On 6/11/2010 2:10 AM, Krivanek, Jan wrote: >>>> I have following two partitions: >>>> >>>> ibis::part: data1 (ardea -d data/data1 -m a:int, b:int -t data1.csv) >>>> with 18 rows, 2 columns >>>> >>>> Column list: >>>> >>>> a: (INT) [1.79769e+308, -1.79769e+308] >>>> >>>> b: (INT) [1.79769e+308, -1.79769e+308] >>>> >>>> ibis::part: data2 (ardea -d data/data2 -m c:int, d:int -t data2.csv) >>>> with 18 rows, 2 columns >>>> >>>> Column list: >>>> >>>> a: (INT) [1.79769e+308, -1.79769e+308] >>>> >>>> d: (INT) [1.79769e+308, -1.79769e+308] >>>> >>>> And I am trying to find a way how to join them and dump results. I >>> used >>>> the following code: >>>> >>>> //create the partitions >>>> >>>> ibis::part *tbl1 = new ibis::part("data1"); >>>> >>>> ibis::part *tbl2 = new ibis::part("data2"); >>>> >>>> //create the join >>>> >>>> ibis::jNatural *jn = new ibis::jNatural(*tbl1, *tbl2, "a", "b< 8", >> "d >>>> >>>> 4"); >>>> >>>> //process the join >>>> >>>> int numRows = jn->count(); >>>> >>>> //select the result >>>> >>>> std::vector<const char*> colnames = std::vector<const char*>(); >>>> >>>> colnames.push_back("data1.a"); >>>> >>>> ibis::table *resultTable = jn->select(colnames); >>>> >>>> //dump the results to cout >>>> >>>> resultTable->dump(std::cout); >>>> >>>> The join is created and processed correctly (I can see the number of >>>> resulting records, which is correct), but selection of the result >>>> returns NULL pointer (with warning "join::fillEquiJoinTable can not >>>> proceed due to invalid arguments"). Below I copy the log output when >>>> calling select method. >>>> >>>> select data1.a FROM From data1 Join data2 Using(a) Where ... -- > start >>>> timer ... >>>> >>>> select data1.a FROM From data1 Join data2 Using(a) Where ... -- > found >>> 1 >>>> column from data1 and 0 column from data2 >>>> >>>> column[data1.a](INT)::selectValuesT -- selecting 15 out of 18 values >>>> from ./data/data1\a >>>> >>>> part[data1]::accessHint -- nRows=18, selected=15, #pages=0, first >>>> page=0, last page=0, hint=MMAP_LARGE_FILES >>>> >>>> column[data1.a](INT)::selectValuesT -- got 15 values (15 wanted) > from >>> an >>>> in-memory version of file ./data/data1\a as int >>>> >>>> column[data1.a](INT)::selectInts -- retrieving 15 integers took 0 >>>> sec(CPU), 0.000108394 sec(elapsed) >>>> >>>> Warning -- join::fillEquiJoinTable can not proceed due to invalid >>> arguments >>>> >>>> select data1.a FROM From data1 Join data2 Using(a) Where ... -- >>>> duration: 0.001 sec(CPU), 0.000752051 sec(elapsed) >>>> >>>> I will appreciate any hint on what I am doing wrong, or what are the >>>> best practices to perform joins. >>>> >>>> Regards >>>> >>>> Jan Krivanek >>>> >>>> >>>> >>>> _______________________________________________ >>>> FastBit-users mailing list >>>> [email protected] >>>> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users _______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
