Hello Julian/Calcite community, I am working on adding subquery support in HIVE using calcite. From what I have read/understood so far Calcite requires HIVE to create RexSubqueryNode corresponding to a subquery and then call SubQueryRemoveRule to get rid of RexSubqueryNode and change it to join. This seems to be working for Un-correlated queries where SubQueryRemoveRule creates Aggregate + Join to get rid of RexSubqueryNode. But I am running into following issues with Co-rrelated queries: (Note that I am using FILTER rule)
* Looking at SubQueryRemoveRule code it should be creating Correlate node if it finds any correlation in given filter. To find if given filter has correlation getVariablesSet is called on filter, which supposedly should be returning set of correlated variables, but it is always returning empty set as filter does not implement this method. Shouldn’t Filter implement this method to return appropriate correlated variables ? * Comments in SubQueryRemoveRule mentions that “The correlate can be removed using RelDecorrelator”. But I don’t see SubqueryRemoveRule using RelDecorrelator to de-correlate given query. Should SubQueryRemoveRule call this ? If not is doing de-correlation immediately after SubQueryRemoveRule appropriate ? Here is what I have done so far for co-rrelated queries. Could you please comment if this is right ? * While creating RexSubqueryNode and RelNode for the subquery I am creating RexCorrelVariable. RexCorrelVariable needs a correlation id. CorrelationId requires an integer id. Should this id be same as index of co-relatted column in outer table ? * Hive has a HiveFilter which is extended from Filter. I implemented getVariableSet method to look at the condition and return all correlated variables in condition’s RelNode. Does this sound correct ? * I am calling RelDecorrelator’s decorrelateQuery immediately after calling SubQueryRemoveRule. After implementing getVariableSet in HiveFilter SubQueryRemoveRule seems to be creating appropriate LogicalCorrelate for correlate queries but decorrelateQuery is throwing an exception. Thanks, Vineet G
