okumin commented on code in PR #5077:
URL: https://github.com/apache/hive/pull/5077#discussion_r1570549898
##########
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveTableScan.java:
##########
@@ -202,6 +203,12 @@ public HiveTableScan copyIncludingTable(RelDataType
newRowtype) {
// Also include partition list key to trigger cost evaluation even if an
// expression was already generated.
@Override public RelWriter explainTerms(RelWriter pw) {
+ if (pw instanceof RelTreeSignatureWriter) {
Review Comment:
Thanks for the suggestion. I have no objections. Could you please check if
this update follows your suggestion?
https://github.com/apache/hive/pull/5077/commits/bd22d94e3cf1bc4375e7f91c91026ecc036b0ac6
##########
ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java:
##########
@@ -1046,11 +1046,16 @@ public Object process(Node nd, Stack<Node> stack,
NodeProcessorCtx procCtx, Obje
}
}
if (nd instanceof TableScanOperator) {
+ TableScanOperator ts = (TableScanOperator) nd;
// If the tablescan operator is making use of filtering capabilities
of readers then
// we will not see the actual incoming rowcount which was processed -
so we may not use it for relNodes
- TableScanOperator ts = (TableScanOperator) nd;
if (ts.getConf().getPredicateString() != null) {
- planMapper.link(ts, new OperatorStats.MayNotUseForRelNodes());
+ invalidateForRelNodes(ts, false);
+ }
+ // If sampling is configured, the table scan could be canceled in the
middle. We avoid using runtime stats
+ // for HiveTableScan and its descendants as it is not pushed down to
HiveTableScan RelNodes
+ if (ts.getConf().getRowLimit() >= 0) {
+ invalidateForRelNodes(ts, true);
Review Comment:
The answer is yes. This is the only place where linked MayNotUseForRelNodes
are used, and it is interested in if the number of instances is empty or not. I
think the situation also happens when a signature links two TableScanOperators.
https://github.com/apache/hive/blob/rel/release-4.0.0/ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/StatsSources.java#L107-L109
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
