> But I can not apply my arrow filter instead of EnumerableCalc. Would > you tell me the reason?
I solved my problem by adding a Arrow Convention to ArrowFilter. Thanks, Julian. > Would you mind logging a JIRA case for it, to track progress? Created: https://issues.apache.org/jira/browse/CALCITE-2173 2018-02-06 2:22 GMT+09:00 Masayuki Takahashi <[email protected]>: > Dear Julian, > >> I am delighted that someone is implementing an Arrow adapter. It would >> be a good replacement for the Enumerable adapter someday. Would you >> mind logging a JIRA case for it, to track progress? > > Now I don't know the right way to scan data with columnar-oriented > style in Calcite, but I have a little idea, so I would like you to > see it if you like. I will create a JIRA case and describe it after I > solved this problem. > >> Maybe the cost of ArrowFilter followed ArrowToEnumerableConverter is >> higher than ArrowToEnumerableConverter followed by EnumerableCalc. > > I see. I will look around the cost calculation. Thanks. > > 2018-02-05 14:05 GMT+09:00 Julian Hyde <[email protected]>: >> I am delighted that someone is implementing an Arrow adapter. It would >> be a good replacement for the Enumerable adapter someday. Would you >> mind logging a JIRA case for it, to track progress? >> >> I think the reason that ArrowFilter is not being chosen is because of >> cost. Note this: >> >>> [rel#56:ArrowFilter.NONE.[](input=rel#17:Subset#0.ARROW.[],expr#0..3={inputs},expr#4=?0,expr#5==($t2, >>> >>> $t4),N_NATIONKEY=$t0,N_NAME=$t1,N_REGIONKEY=$t2,N_COMMENT=$t3,$condition=$t5)] >>> PLANNER = org.apache.calcite.plan.volcano.VolcanoPlanner@499683c4; TICK = >>> 28/26; PHASE = OPTIMIZE; COST = {inf} >> >> Maybe the cost of ArrowFilter followed ArrowToEnumerableConverter is >> higher than ArrowToEnumerableConverter followed by EnumerableCalc. >> >> Julian >> >> >> On Sun, Feb 4, 2018 at 1:46 AM, Masayuki Takahashi >> <[email protected]> wrote: >>> Hi, >>> >>> I try to implement Arrow Adapter. >>> >>> https://github.com/masayuki038/calcite/tree/arrow/arrow/src/main/java/org/apache/calcite/adapter/arrow >>> >>> But I can not apply my arrow filter instead of EnumerableCalc. Would >>> you tell me the reason? >>> >>> ArrowFilter was created by ArrowFilterTableScanRule. >>> (logs are follow:) >>> >>> --- >>> PLANNER = org.apache.calcite.plan.volcano.VolcanoPlanner@499683c4; >>> TICK = 27/25; PHASE = OPTIMIZE; COST = {inf} >>> Pop match: rule [ArrowFilterTableScanRule] rels >>> [rel#31:LogicalFilter.NONE.[](input=rel#17:Subset#0.ARROW.[],condition==($2, >>> ?0)), rel#0:ArrowTableScan.ARROW.[](table=[SAMPLES, >>> NATIONSSF],fields=[0, 1, 2, 3])] >>> call#254: Apply rule [ArrowFilterTableScanRule] to >>> [rel#31:LogicalFilter.NONE.[](input=rel#17:Subset#0.ARROW.[],condition==($2, >>> ?0)), rel#0:ArrowTableScan.ARROW.[](table=[SAMPLES, >>> NATIONSSF],fields=[0, 1, 2, 3])] >>> Transform to: rel#56 via ArrowFilterTableScanRule >>> call#254 generated 1 successors: >>> [rel#56:ArrowFilter.NONE.[](input=rel#17:Subset#0.ARROW.[],expr#0..3={inputs},expr#4=?0,expr#5==($t2, >>> $t4),N_NATIONKEY=$t0,N_NAME=$t1,N_REGIONKEY=$t2,N_COMMENT=$t3,$condition=$t5)] >>> PLANNER = org.apache.calcite.plan.volcano.VolcanoPlanner@499683c4; >>> TICK = 28/26; PHASE = OPTIMIZE; COST = {inf} >>> --- >>> >>> But it was not present in the final plan. >>> >>> --- >>> Plan after physical tweaks: EnumerableCalc(expr#0..2=[{inputs}], >>> expr#3=[?0], expr#4=[=($t2, $t3)], proj#0..1=[{exprs}], >>> $condition=[$t4]): rowcount = 15.0, cumulative cost = {125.0 rows, >>> 911.0 cpu, 0.0 io}, id = 80 >>> ArrowToEnumerableConverter: rowcount = 100.0, cumulative cost = {110.0 >>> rows, 111.0 cpu, 0.0 io}, id = 70 >>> ArrowTableScan(table=[[SAMPLES, NATIONSSF]], fields=[[0, 1, 2]]): >>> rowcount = 100.0, cumulative cost = {100.0 rows, 101.0 cpu, 0.0 io}, >>> id = 54 >>> --- >>> >>> I would like to know what rules EnumerableCalc chosen. >>> >>> thanks. >>> >>> -- >>> Masayuki Takahashi > > > > -- > 高橋 真之 -- 高橋 真之
