[jira] [Updated] (HIVE-17405) HoS DPP ConstantPropagate should use ConstantPropagateOption.SHORTCUT
[ https://issues.apache.org/jira/browse/HIVE-17405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-17405: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Thanks for the review [~lirui], merged to master. > HoS DPP ConstantPropagate should use ConstantPropagateOption.SHORTCUT > - > > Key: HIVE-17405 > URL: https://issues.apache.org/jira/browse/HIVE-17405 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Sahil Takiar >Assignee: Sahil Takiar > Fix For: 3.0.0 > > Attachments: HIVE-17405.1.patch, HIVE-17405.2.patch, > HIVE-17405.3.patch, HIVE-17405.4.patch, HIVE-17405.5.patch, > HIVE-17405.6.patch, HIVE-17405.7.patch > > > In {{SparkCompiler#runDynamicPartitionPruning}} we should change {{new > ConstantPropagate().transform(parseContext)}} to {{new > ConstantPropagate(ConstantPropagateOption.SHORTCUT).transform(parseContext)}} > Hive-on-Tez does the same thing. > Running the full constant propagation isn't really necessary, we just want to > eliminate any {{and true}} predicates that were introduced by > {{SyntheticJoinPredicate}} and {{DynamicPartitionPruningOptimization}}. The > {{SyntheticJoinPredicate}} will introduce dummy filter predicates into the > operator tree, and {{DynamicPartitionPruningOptimization}} will replace them. > The predicates introduced via {{SyntheticJoinPredicate}} are necessary to > help {{DynamicPartitionPruningOptimization}} determine if DPP can be used or > not. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17405) HoS DPP ConstantPropagate should use ConstantPropagateOption.SHORTCUT
[ https://issues.apache.org/jira/browse/HIVE-17405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-17405: Attachment: HIVE-17405.6.patch Rebased on top of HIVE-17412 > HoS DPP ConstantPropagate should use ConstantPropagateOption.SHORTCUT > - > > Key: HIVE-17405 > URL: https://issues.apache.org/jira/browse/HIVE-17405 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Sahil Takiar >Assignee: Sahil Takiar > Attachments: HIVE-17405.1.patch, HIVE-17405.2.patch, > HIVE-17405.3.patch, HIVE-17405.4.patch, HIVE-17405.5.patch, HIVE-17405.6.patch > > > In {{SparkCompiler#runDynamicPartitionPruning}} we should change {{new > ConstantPropagate().transform(parseContext)}} to {{new > ConstantPropagate(ConstantPropagateOption.SHORTCUT).transform(parseContext)}} > Hive-on-Tez does the same thing. > Running the full constant propagation isn't really necessary, we just want to > eliminate any {{and true}} predicates that were introduced by > {{SyntheticJoinPredicate}} and {{DynamicPartitionPruningOptimization}}. The > {{SyntheticJoinPredicate}} will introduce dummy filter predicates into the > operator tree, and {{DynamicPartitionPruningOptimization}} will replace them. > The predicates introduced via {{SyntheticJoinPredicate}} are necessary to > help {{DynamicPartitionPruningOptimization}} determine if DPP can be used or > not. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17405) HoS DPP ConstantPropagate should use ConstantPropagateOption.SHORTCUT
[ https://issues.apache.org/jira/browse/HIVE-17405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-17405: Attachment: HIVE-17405.5.patch > HoS DPP ConstantPropagate should use ConstantPropagateOption.SHORTCUT > - > > Key: HIVE-17405 > URL: https://issues.apache.org/jira/browse/HIVE-17405 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Sahil Takiar >Assignee: Sahil Takiar > Attachments: HIVE-17405.1.patch, HIVE-17405.2.patch, > HIVE-17405.3.patch, HIVE-17405.4.patch, HIVE-17405.5.patch > > > In {{SparkCompiler#runDynamicPartitionPruning}} we should change {{new > ConstantPropagate().transform(parseContext)}} to {{new > ConstantPropagate(ConstantPropagateOption.SHORTCUT).transform(parseContext)}} > Hive-on-Tez does the same thing. > Running the full constant propagation isn't really necessary, we just want to > eliminate any {{and true}} predicates that were introduced by > {{SyntheticJoinPredicate}} and {{DynamicPartitionPruningOptimization}}. The > {{SyntheticJoinPredicate}} will introduce dummy filter predicates into the > operator tree, and {{DynamicPartitionPruningOptimization}} will replace them. > The predicates introduced via {{SyntheticJoinPredicate}} are necessary to > help {{DynamicPartitionPruningOptimization}} determine if DPP can be used or > not. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17405) HoS DPP ConstantPropagate should use ConstantPropagateOption.SHORTCUT
[ https://issues.apache.org/jira/browse/HIVE-17405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-17405: Attachment: HIVE-17405.4.patch > HoS DPP ConstantPropagate should use ConstantPropagateOption.SHORTCUT > - > > Key: HIVE-17405 > URL: https://issues.apache.org/jira/browse/HIVE-17405 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Sahil Takiar >Assignee: Sahil Takiar > Attachments: HIVE-17405.1.patch, HIVE-17405.2.patch, > HIVE-17405.3.patch, HIVE-17405.4.patch > > > In {{SparkCompiler#runDynamicPartitionPruning}} we should change {{new > ConstantPropagate().transform(parseContext)}} to {{new > ConstantPropagate(ConstantPropagateOption.SHORTCUT).transform(parseContext)}} > Hive-on-Tez does the same thing. > Running the full constant propagation isn't really necessary, we just want to > eliminate any {{and true}} predicates that were introduced by > {{SyntheticJoinPredicate}} and {{DynamicPartitionPruningOptimization}}. The > {{SyntheticJoinPredicate}} will introduce dummy filter predicates into the > operator tree, and {{DynamicPartitionPruningOptimization}} will replace them. > The predicates introduced via {{SyntheticJoinPredicate}} are necessary to > help {{DynamicPartitionPruningOptimization}} determine if DPP can be used or > not. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17405) HoS DPP ConstantPropagate should use ConstantPropagateOption.SHORTCUT
[ https://issues.apache.org/jira/browse/HIVE-17405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-17405: Attachment: HIVE-17405.3.patch > HoS DPP ConstantPropagate should use ConstantPropagateOption.SHORTCUT > - > > Key: HIVE-17405 > URL: https://issues.apache.org/jira/browse/HIVE-17405 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Sahil Takiar >Assignee: Sahil Takiar > Attachments: HIVE-17405.1.patch, HIVE-17405.2.patch, > HIVE-17405.3.patch > > > In {{SparkCompiler#runDynamicPartitionPruning}} we should change {{new > ConstantPropagate().transform(parseContext)}} to {{new > ConstantPropagate(ConstantPropagateOption.SHORTCUT).transform(parseContext)}} > Hive-on-Tez does the same thing. > Running the full constant propagation isn't really necessary, we just want to > eliminate any {{and true}} predicates that were introduced by > {{SyntheticJoinPredicate}} and {{DynamicPartitionPruningOptimization}}. The > {{SyntheticJoinPredicate}} will introduce dummy filter predicates into the > operator tree, and {{DynamicPartitionPruningOptimization}} will replace them. > The predicates introduced via {{SyntheticJoinPredicate}} are necessary to > help {{DynamicPartitionPruningOptimization}} determine if DPP can be used or > not. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17405) HoS DPP ConstantPropagate should use ConstantPropagateOption.SHORTCUT
[ https://issues.apache.org/jira/browse/HIVE-17405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-17405: Attachment: HIVE-17405.2.patch > HoS DPP ConstantPropagate should use ConstantPropagateOption.SHORTCUT > - > > Key: HIVE-17405 > URL: https://issues.apache.org/jira/browse/HIVE-17405 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Sahil Takiar >Assignee: Sahil Takiar > Attachments: HIVE-17405.1.patch, HIVE-17405.2.patch > > > In {{SparkCompiler#runDynamicPartitionPruning}} we should change {{new > ConstantPropagate().transform(parseContext)}} to {{new > ConstantPropagate(ConstantPropagateOption.SHORTCUT).transform(parseContext)}} > Hive-on-Tez does the same thing. > Running the full constant propagation isn't really necessary, we just want to > eliminate any {{and true}} predicates that were introduced by > {{SyntheticJoinPredicate}} and {{DynamicPartitionPruningOptimization}}. The > {{SyntheticJoinPredicate}} will introduce dummy filter predicates into the > operator tree, and {{DynamicPartitionPruningOptimization}} will replace them. > The predicates introduced via {{SyntheticJoinPredicate}} are necessary to > help {{DynamicPartitionPruningOptimization}} determine if DPP can be used or > not. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17405) HoS DPP ConstantPropagate should use ConstantPropagateOption.SHORTCUT
[ https://issues.apache.org/jira/browse/HIVE-17405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-17405: Attachment: HIVE-17405.1.patch > HoS DPP ConstantPropagate should use ConstantPropagateOption.SHORTCUT > - > > Key: HIVE-17405 > URL: https://issues.apache.org/jira/browse/HIVE-17405 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Sahil Takiar >Assignee: Sahil Takiar > Attachments: HIVE-17405.1.patch > > > In {{SparkCompiler#runDynamicPartitionPruning}} we should change {{new > ConstantPropagate().transform(parseContext)}} to {{new > ConstantPropagate(ConstantPropagateOption.SHORTCUT).transform(parseContext)}} > Hive-on-Tez does the same thing. > Running the full constant propagation isn't really necessary, we just want to > eliminate any {{and true}} predicates that were introduced by > {{SyntheticJoinPredicate}} and {{DynamicPartitionPruningOptimization}}. The > {{SyntheticJoinPredicate}} will introduce dummy filter predicates into the > operator tree, and {{DynamicPartitionPruningOptimization}} will replace them. > The predicates introduced via {{SyntheticJoinPredicate}} are necessary to > help {{DynamicPartitionPruningOptimization}} determine if DPP can be used or > not. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17405) HoS DPP ConstantPropagate should use ConstantPropagateOption.SHORTCUT
[ https://issues.apache.org/jira/browse/HIVE-17405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-17405: Status: Patch Available (was: Open) > HoS DPP ConstantPropagate should use ConstantPropagateOption.SHORTCUT > - > > Key: HIVE-17405 > URL: https://issues.apache.org/jira/browse/HIVE-17405 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Sahil Takiar >Assignee: Sahil Takiar > Attachments: HIVE-17405.1.patch > > > In {{SparkCompiler#runDynamicPartitionPruning}} we should change {{new > ConstantPropagate().transform(parseContext)}} to {{new > ConstantPropagate(ConstantPropagateOption.SHORTCUT).transform(parseContext)}} > Hive-on-Tez does the same thing. > Running the full constant propagation isn't really necessary, we just want to > eliminate any {{and true}} predicates that were introduced by > {{SyntheticJoinPredicate}} and {{DynamicPartitionPruningOptimization}}. The > {{SyntheticJoinPredicate}} will introduce dummy filter predicates into the > operator tree, and {{DynamicPartitionPruningOptimization}} will replace them. > The predicates introduced via {{SyntheticJoinPredicate}} are necessary to > help {{DynamicPartitionPruningOptimization}} determine if DPP can be used or > not. -- This message was sent by Atlassian JIRA (v6.4.14#64029)