ramitg254 commented on code in PR #6413:
URL: https://github.com/apache/hive/pull/6413#discussion_r3388184363
##########
ql/src/java/org/apache/hadoop/hive/ql/optimizer/DynamicPartitionPruningOptimization.java:
##########
@@ -169,7 +169,12 @@ public Object process(Node nd, Stack<Node> stack,
NodeProcessorCtx procCtx, Obje
Table table = ts.getConf().getTableMetadata();
boolean nonEquiJoin = isNonEquiJoin(ctx.parent);
- if (table != null && table.isPartitionKey(column) && !nonEquiJoin) {
+ if (table != null && table.isNonNative() && !nonEquiJoin
Review Comment:
done
##########
ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java:
##########
@@ -521,16 +521,27 @@ public static void setupNeededColumns(TableScanOperator
scanOp, RowSchema inputR
cols.add(new FieldNode(VirtualColumn.RAWDATASIZE.getName()));
}
+ Table tableMetadata = desc.getTableMetadata();
+ Set<String> nonNativePartitionKeyNames = null;
+ if (tableMetadata != null && tableMetadata.hasNonNativePartitionSupport())
{
+ nonNativePartitionKeyNames = new HashSet<>();
+ for (FieldSchema partCol : tableMetadata.getPartCols()) {
+ nonNativePartitionKeyNames.add(partCol.getName().toLowerCase());
+ }
+ }
+
for (FieldNode fn : cols) {
String column = fn.getFieldName();
ColumnInfo colInfo = inputRS.getColumnInfo(column);
if (colInfo == null) {
continue;
}
referencedColumnNames.add(column);
- if (colInfo.getIsVirtualCol() && !colInfo.isHiddenPartitionCol()) {
+ boolean isNonNativePartitionSourceCol = nonNativePartitionKeyNames !=
null
Review Comment:
I think we should avoid this:
`Set<String> names = new TreeSet<>(String.CASE_INSENSITIVE_ORDER);`
and use HashSet with lowerCase partition names as TreeSet seaching is done
in O(log(n)) while for HashSet it is O(1)
refractored please check
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
