I am delighted that someone is implementing an Arrow adapter. It would
be a good replacement for the Enumerable adapter someday. Would you
mind logging a JIRA case for it, to track progress?

I think the reason that ArrowFilter is not being chosen is because of
cost. Note this:

> [rel#56:ArrowFilter.NONE.[](input=rel#17:Subset#0.ARROW.[],expr#0..3={inputs},expr#4=?0,expr#5==($t2,
>  
> $t4),N_NATIONKEY=$t0,N_NAME=$t1,N_REGIONKEY=$t2,N_COMMENT=$t3,$condition=$t5)]
> PLANNER = org.apache.calcite.plan.volcano.VolcanoPlanner@499683c4; TICK = 
> 28/26; PHASE = OPTIMIZE; COST = {inf}

Maybe the cost of ArrowFilter followed ArrowToEnumerableConverter is
higher than ArrowToEnumerableConverter followed by EnumerableCalc.

Julian


On Sun, Feb 4, 2018 at 1:46 AM, Masayuki Takahashi
<masayuki...@gmail.com> wrote:
> Hi,
>
> I try to implement Arrow Adapter.
>
> https://github.com/masayuki038/calcite/tree/arrow/arrow/src/main/java/org/apache/calcite/adapter/arrow
>
> But I can not apply my arrow filter instead of EnumerableCalc. Would
> you tell me the reason?
>
> ArrowFilter was created by ArrowFilterTableScanRule.
> (logs are follow:)
>
> ---
> PLANNER = org.apache.calcite.plan.volcano.VolcanoPlanner@499683c4;
> TICK = 27/25; PHASE = OPTIMIZE; COST = {inf}
> Pop match: rule [ArrowFilterTableScanRule] rels
> [rel#31:LogicalFilter.NONE.[](input=rel#17:Subset#0.ARROW.[],condition==($2,
> ?0)), rel#0:ArrowTableScan.ARROW.[](table=[SAMPLES,
> NATIONSSF],fields=[0, 1, 2, 3])]
> call#254: Apply rule [ArrowFilterTableScanRule] to
> [rel#31:LogicalFilter.NONE.[](input=rel#17:Subset#0.ARROW.[],condition==($2,
> ?0)), rel#0:ArrowTableScan.ARROW.[](table=[SAMPLES,
> NATIONSSF],fields=[0, 1, 2, 3])]
> Transform to: rel#56 via ArrowFilterTableScanRule
> call#254 generated 1 successors:
> [rel#56:ArrowFilter.NONE.[](input=rel#17:Subset#0.ARROW.[],expr#0..3={inputs},expr#4=?0,expr#5==($t2,
> $t4),N_NATIONKEY=$t0,N_NAME=$t1,N_REGIONKEY=$t2,N_COMMENT=$t3,$condition=$t5)]
> PLANNER = org.apache.calcite.plan.volcano.VolcanoPlanner@499683c4;
> TICK = 28/26; PHASE = OPTIMIZE; COST = {inf}
> ---
>
> But it was not present in the final plan.
>
> ---
> Plan after physical tweaks: EnumerableCalc(expr#0..2=[{inputs}],
> expr#3=[?0], expr#4=[=($t2, $t3)], proj#0..1=[{exprs}],
> $condition=[$t4]): rowcount = 15.0, cumulative cost = {125.0 rows,
> 911.0 cpu, 0.0 io}, id = 80
> ArrowToEnumerableConverter: rowcount = 100.0, cumulative cost = {110.0
> rows, 111.0 cpu, 0.0 io}, id = 70
> ArrowTableScan(table=[[SAMPLES, NATIONSSF]], fields=[[0, 1, 2]]):
> rowcount = 100.0, cumulative cost = {100.0 rows, 101.0 cpu, 0.0 io},
> id = 54
> ---
>
> I would like to know what rules EnumerableCalc chosen.
>
> thanks.
>
> --
> Masayuki Takahashi

Reply via email to