Hi John,
Thank you for update to this topic. I checked out newest code and tried
my example that wasn't working. Now I seem to be able to join two
partitions, and then dump resulting data, but only in case if I do not
fully specify the column name with name of the partition.
(So if I change the codeline "colnames.push_back("data1.a");" below to
"colnames.push_back("a");" then I am able to get the results, if I leave
it as is, then the final select fail with exception somewhere inside the
code). Anyway, this now doesn't seem to be a issue blocking me from
performing joins.
Thanks
Jan
-----Original Message-----
From: K. John Wu [mailto:[email protected]]
Sent: Wednesday, June 23, 2010 7:07 AM
To: Krivanek, Jan
Cc: FastBit Users
Subject: Re: [FastBit-users] Unable to dump results of (natural) join
operation
Hi, Jan,
A number of updates to the join code has been checked into the SVN
repository. Please take a look and let us know if the problem you've
observed is fixed.
John
On 6/12/2010 2:35 PM, Krivanek, Jan wrote:
> Hi John,
>
> Thank you for your quick reply and information that fix for this
should
> be available soon. I must admit that I am a little bit confused now -
> does it mean that only the naming of column caused the problem? I
> experienced this also for using the name of column without specifying
> table name (I tried to debug it a little bit and found some problem in
> fillEquiJoinTable which checked for validity of passed parameters
> indicating count of rows in each table satisfying join condition, but
I
> am not at my PC now, so I can provide more info after weekend if
> needed).
> Also in code I found that there are probably several ways how to
perform
> joins - using classes for ibis::jNatural, or ibis::jRange (which is
not
> implemented yet), or there are some join methods in ibis::part or
> ibis::query classes - is there any recommended way how to perform
joins
> (for later grouping and sorting of joined results)?
>
> Great thanks for your time
> Jan
>
> -----Original Message-----
> From: K. John Wu [mailto:[email protected]]
> Sent: Friday, June 11, 2010 7:33 PM
> To: FastBit Users
> Cc: Krivanek, Jan
> Subject: Re: [FastBit-users] Unable to dump results of (natural) join
> operation
>
> Hi, Jan,
>
> I have just came across this problem as well and am in the process of
> fixing it. The root cause is that many functions in FastBit are still
> using the older way of looking up columns and unable to locate columns
> named in the format 'data1.a'. The fix for this particular problem is
> already in-place, however, I am in the process of changing something
> else, which might take a day or two to complete. I will let you know
> as soon as I am comfortable that I have not broken things with my
> latest changes...
>
> Thanks for your patience.
>
> John
>
>
> On 6/11/2010 2:10 AM, Krivanek, Jan wrote:
>> I have following two partitions:
>>
>> ibis::part: data1 (ardea -d data/data1 -m a:int, b:int -t data1.csv)
>> with 18 rows, 2 columns
>>
>> Column list:
>>
>> a: (INT) [1.79769e+308, -1.79769e+308]
>>
>> b: (INT) [1.79769e+308, -1.79769e+308]
>>
>> ibis::part: data2 (ardea -d data/data2 -m c:int, d:int -t data2.csv)
>> with 18 rows, 2 columns
>>
>> Column list:
>>
>> a: (INT) [1.79769e+308, -1.79769e+308]
>>
>> d: (INT) [1.79769e+308, -1.79769e+308]
>>
>> And I am trying to find a way how to join them and dump results. I
> used
>> the following code:
>>
>> //create the partitions
>>
>> ibis::part *tbl1 = new ibis::part("data1");
>>
>> ibis::part *tbl2 = new ibis::part("data2");
>>
>> //create the join
>>
>> ibis::jNatural *jn = new ibis::jNatural(*tbl1, *tbl2, "a", "b< 8",
"d
>>
>> 4");
>>
>> //process the join
>>
>> int numRows = jn->count();
>>
>> //select the result
>>
>> std::vector<const char*> colnames = std::vector<const char*>();
>>
>> colnames.push_back("data1.a");
>>
>> ibis::table *resultTable = jn->select(colnames);
>>
>> //dump the results to cout
>>
>> resultTable->dump(std::cout);
>>
>> The join is created and processed correctly (I can see the number of
>> resulting records, which is correct), but selection of the result
>> returns NULL pointer (with warning "join::fillEquiJoinTable can not
>> proceed due to invalid arguments"). Below I copy the log output when
>> calling select method.
>>
>> select data1.a FROM From data1 Join data2 Using(a) Where ... -- start
>> timer ...
>>
>> select data1.a FROM From data1 Join data2 Using(a) Where ... -- found
> 1
>> column from data1 and 0 column from data2
>>
>> column[data1.a](INT)::selectValuesT -- selecting 15 out of 18 values
>> from ./data/data1\a
>>
>> part[data1]::accessHint -- nRows=18, selected=15, #pages=0, first
>> page=0, last page=0, hint=MMAP_LARGE_FILES
>>
>> column[data1.a](INT)::selectValuesT -- got 15 values (15 wanted) from
> an
>> in-memory version of file ./data/data1\a as int
>>
>> column[data1.a](INT)::selectInts -- retrieving 15 integers took 0
>> sec(CPU), 0.000108394 sec(elapsed)
>>
>> Warning -- join::fillEquiJoinTable can not proceed due to invalid
> arguments
>>
>> select data1.a FROM From data1 Join data2 Using(a) Where ... --
>> duration: 0.001 sec(CPU), 0.000752051 sec(elapsed)
>>
>> I will appreciate any hint on what I am doing wrong, or what are the
>> best practices to perform joins.
>>
>> Regards
>>
>> Jan Krivanek
>>
>>
>>
>> _______________________________________________
>> FastBit-users mailing list
>> [email protected]
>> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
_______________________________________________
FastBit-users mailing list
[email protected]
https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
