Using a ThreadLocal might fix some of those concerns.
> On Mar 8, 2017, at 10:38 AM, Remus Rusanu <[email protected]> wrote:
>
> Thanks Amogh, I will consider this, but replacing the planner inside cached
> tree (or cluster) will require adding locking around access to the ‘one’
> cached MV plan, and in the concurrent HiveServer2 it process may lead to more
> problems (my biggest fear being deadlocks).
>
> Thanks,
> ~Remus
>
> On 3/8/17, 9:24 AM, "Amogh Margoor" <[email protected]> wrote:
>
> We had created MaterializedCluster by extending RelOptCluster for solving
> this issue :
>
>
> https://github.com/qubole/quark/blob/master/optimizer/src/main/java/org/apache/calcite/plan/QuarkMaterializeCluster.java
> New cluster was used in RelNodes of Materialised Views and it allowed
> changing it's RelOptPlanner to that of RelNode being optimised. Caveat is
> that it will not work if there are concurrent optimisations of RelNodes
> happening in different threads using same copy of MaterializedViews.
>
> Thanks,
> Amogh
>
>
> On Wed, Mar 8, 2017 at 10:35 PM, Julian Hyde <[email protected]> wrote:
>
>> The argument should be a RelOptCluster, not a RelOptPlanner. The link from
>> RelNode to planner is indirect currently (via cluster) and will be
>> non-existent after CALCITE-1536.
>>
>> I question whether we need a new method. Putting an abstract method on
>> RelNode is a huge burden because every RelNode sub-class needs to be fixed
>> when people upgrade. Even a non-abstract method imposes a conceptual
>> burden: more methods to understand.
>>
>> So, my approach would be to sub-class RelShuttle. It’s sufficient that it
>> only works for LogicalXxx nodes.
>>
>> No need to copy RexNode expressions. They are immutable.
>>
>> Julian
>>
>>
>>> On Mar 8, 2017, at 4:14 AM, Remus Rusanu <[email protected]>
>> wrote:
>>>
>>> I created CALCITE-1681 https://issues.apache.org/
>> jira/browse/CALCITE-1681 and I intent to work on it for finishing
>> HIVE-15708.
>>> My current thinking is to create a RelCopier based on RelShuttle and add
>> a new abstract RelNode.copyTo(RelOptPlanner) that each concrete Rel type
>> must override. The Rex part is already handled by the existing RexCopier.
>>>
>>> Thanks,
>>> ~Remus
>>>
>>> On 3/6/17, 12:30 PM, "Julian Hyde" <[email protected]> wrote:
>>>
>>> Every RelNode belongs to a RelOptCluster, and basically there is one
>> RelOptCluster created each time a query is prepared. When working with
>> materialized views, the view’s query is represented as a tree of RelNodes,
>> that tree is used for optimizing more than one query. When planning a
>> particular query, the nodes of that query will have a different
>> RelOptCluster than the nodes of the materialized view(s) they are matched
>> against.
>>>
>>> How do we deal with this? Do we copy the nodes into the query’s
>> cluster once we have found a match? If so, how? I couldn’t find a sub-class
>> of RelVisitor or RelShuttle that copies trees to a different RelOptCluster.
>>>
>>> By the way, https://issues.apache.org/jira/browse/CALCITE-1536 <
>> https://issues.apache.org/jira/browse/CALCITE-1536> aims to improve the
>> RelNode life-cycle but I don’t think it will solve this problem.
>>>
>>> Julian
>>>
>>>
>>
>>
>
>
