Hello Julian/Calcite community,

I am working on adding subquery support in HIVE using calcite.  From what I 
have read/understood so far Calcite requires HIVE to create RexSubqueryNode 
corresponding to a subquery and then call SubQueryRemoveRule to get rid of 
RexSubqueryNode and change it to join. This seems to be working for 
Un-correlated queries where SubQueryRemoveRule creates Aggregate + Join to get 
rid of RexSubqueryNode. But I am running into following issues with Co-rrelated 
queries: (Note that I am using FILTER rule)

  *   Looking at SubQueryRemoveRule code it should be creating Correlate node 
if it finds any correlation in given filter. To find if given filter has 
correlation getVariablesSet is called on filter, which supposedly should be 
returning set of correlated variables, but it is always returning empty set as 
filter does not implement this method. Shouldn’t Filter implement this method 
to return appropriate correlated variables ?
  *   Comments in SubQueryRemoveRule mentions that “The correlate can be 
removed using RelDecorrelator”. But I don’t see SubqueryRemoveRule using 
RelDecorrelator to de-correlate given query. Should SubQueryRemoveRule call 
this ? If not is doing de-correlation immediately after SubQueryRemoveRule 
appropriate ?

Here is what I have done so far for co-rrelated queries. Could you please 
comment if this is right ?

  *   While creating RexSubqueryNode and RelNode for the subquery I am creating 
RexCorrelVariable. RexCorrelVariable needs a correlation id. CorrelationId 
requires an integer id. Should this id be same as index of co-relatted column 
in outer table ?
  *   Hive has a HiveFilter which is extended from Filter. I implemented 
getVariableSet method to look at the condition and return all correlated 
variables in condition’s RelNode. Does this sound correct ?
  *   I am calling RelDecorrelator’s decorrelateQuery immediately after calling 
SubQueryRemoveRule.  After implementing getVariableSet in HiveFilter 
SubQueryRemoveRule seems to be creating appropriate LogicalCorrelate for 
correlate queries but decorrelateQuery is throwing an exception.

Thanks,
Vineet G

Reply via email to