Hi John, I did have a time to little look on this. I tried my code on revision 301 and found following:
When I use following jNatural constructor: ibis::jNatural::jNatural(const ibis::part* partr, const ibis::part* parts, const char* colname, const char* condr, const char* conds, const char* sel) it doesn't initialize its internal frm_ (ibis::fromClause *) member (is this the from clause that you was referring to?), but this member is then required by ibis::jNatural::select method if you specify fully qualified column name (line 784: match = frm_->position(tname.c_str());). I am currently OK with workaround when not using fully qualified column names. But please let me know if I am doing something incorrectly, or if you need more info (repro) for this. Thanks Jan -----Original Message----- From: K. John Wu [mailto:[email protected]] Sent: Thursday, June 24, 2010 9:02 PM To: Krivanek, Jan Cc: FastBit Users Subject: Re: [FastBit-users] Unable to dump results of (natural) join operation Hi, Jan, I am actually expecting the names passed to ibis::quaere::select to be fully qualified names, i.e., "data1.a", not "a". Did you skipped the from clause? There was a little bug in the implementation of ibis::fromClause used for name look up. This little problem was fixed yesterday afternoon. You can get the fix from the latest SVN repository. John On 6/24/2010 8:31 AM, Krivanek, Jan wrote: > Hi John, > > Thank you for update to this topic. I checked out newest code and tried > my example that wasn't working. Now I seem to be able to join two > partitions, and then dump resulting data, but only in case if I do not > fully specify the column name with name of the partition. > (So if I change the codeline "colnames.push_back("data1.a");" below to > "colnames.push_back("a");" then I am able to get the results, if I leave > it as is, then the final select fail with exception somewhere inside the > code). Anyway, this now doesn't seem to be a issue blocking me from > performing joins. > > Thanks > Jan > > -----Original Message----- > From: K. John Wu [mailto:[email protected]] > Sent: Wednesday, June 23, 2010 7:07 AM > To: Krivanek, Jan > Cc: FastBit Users > Subject: Re: [FastBit-users] Unable to dump results of (natural) join > operation > > Hi, Jan, > > A number of updates to the join code has been checked into the SVN > repository. Please take a look and let us know if the problem you've > observed is fixed. > > John > > On 6/12/2010 2:35 PM, Krivanek, Jan wrote: >> Hi John, >> >> Thank you for your quick reply and information that fix for this > should >> be available soon. I must admit that I am a little bit confused now - >> does it mean that only the naming of column caused the problem? I >> experienced this also for using the name of column without specifying >> table name (I tried to debug it a little bit and found some problem in >> fillEquiJoinTable which checked for validity of passed parameters >> indicating count of rows in each table satisfying join condition, but > I >> am not at my PC now, so I can provide more info after weekend if >> needed). >> Also in code I found that there are probably several ways how to > perform >> joins - using classes for ibis::jNatural, or ibis::jRange (which is > not >> implemented yet), or there are some join methods in ibis::part or >> ibis::query classes - is there any recommended way how to perform > joins >> (for later grouping and sorting of joined results)? >> >> Great thanks for your time >> Jan >> >> -----Original Message----- >> From: K. John Wu [mailto:[email protected]] >> Sent: Friday, June 11, 2010 7:33 PM >> To: FastBit Users >> Cc: Krivanek, Jan >> Subject: Re: [FastBit-users] Unable to dump results of (natural) join >> operation >> >> Hi, Jan, >> >> I have just came across this problem as well and am in the process of >> fixing it. The root cause is that many functions in FastBit are still >> using the older way of looking up columns and unable to locate columns >> named in the format 'data1.a'. The fix for this particular problem is >> already in-place, however, I am in the process of changing something >> else, which might take a day or two to complete. I will let you know >> as soon as I am comfortable that I have not broken things with my >> latest changes... >> >> Thanks for your patience. >> >> John >> >> >> On 6/11/2010 2:10 AM, Krivanek, Jan wrote: >>> I have following two partitions: >>> >>> ibis::part: data1 (ardea -d data/data1 -m a:int, b:int -t data1.csv) >>> with 18 rows, 2 columns >>> >>> Column list: >>> >>> a: (INT) [1.79769e+308, -1.79769e+308] >>> >>> b: (INT) [1.79769e+308, -1.79769e+308] >>> >>> ibis::part: data2 (ardea -d data/data2 -m c:int, d:int -t data2.csv) >>> with 18 rows, 2 columns >>> >>> Column list: >>> >>> a: (INT) [1.79769e+308, -1.79769e+308] >>> >>> d: (INT) [1.79769e+308, -1.79769e+308] >>> >>> And I am trying to find a way how to join them and dump results. I >> used >>> the following code: >>> >>> //create the partitions >>> >>> ibis::part *tbl1 = new ibis::part("data1"); >>> >>> ibis::part *tbl2 = new ibis::part("data2"); >>> >>> //create the join >>> >>> ibis::jNatural *jn = new ibis::jNatural(*tbl1, *tbl2, "a", "b< 8", > "d >>> >>> 4"); >>> >>> //process the join >>> >>> int numRows = jn->count(); >>> >>> //select the result >>> >>> std::vector<const char*> colnames = std::vector<const char*>(); >>> >>> colnames.push_back("data1.a"); >>> >>> ibis::table *resultTable = jn->select(colnames); >>> >>> //dump the results to cout >>> >>> resultTable->dump(std::cout); >>> >>> The join is created and processed correctly (I can see the number of >>> resulting records, which is correct), but selection of the result >>> returns NULL pointer (with warning "join::fillEquiJoinTable can not >>> proceed due to invalid arguments"). Below I copy the log output when >>> calling select method. >>> >>> select data1.a FROM From data1 Join data2 Using(a) Where ... -- start >>> timer ... >>> >>> select data1.a FROM From data1 Join data2 Using(a) Where ... -- found >> 1 >>> column from data1 and 0 column from data2 >>> >>> column[data1.a](INT)::selectValuesT -- selecting 15 out of 18 values >>> from ./data/data1\a >>> >>> part[data1]::accessHint -- nRows=18, selected=15, #pages=0, first >>> page=0, last page=0, hint=MMAP_LARGE_FILES >>> >>> column[data1.a](INT)::selectValuesT -- got 15 values (15 wanted) from >> an >>> in-memory version of file ./data/data1\a as int >>> >>> column[data1.a](INT)::selectInts -- retrieving 15 integers took 0 >>> sec(CPU), 0.000108394 sec(elapsed) >>> >>> Warning -- join::fillEquiJoinTable can not proceed due to invalid >> arguments >>> >>> select data1.a FROM From data1 Join data2 Using(a) Where ... -- >>> duration: 0.001 sec(CPU), 0.000752051 sec(elapsed) >>> >>> I will appreciate any hint on what I am doing wrong, or what are the >>> best practices to perform joins. >>> >>> Regards >>> >>> Jan Krivanek >>> >>> >>> >>> _______________________________________________ >>> FastBit-users mailing list >>> [email protected] >>> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users _______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
