pltbkd commented on code in PR #20612:
URL: https://github.com/apache/flink/pull/20612#discussion_r950067992
##########
flink-connectors/flink-connector-hive/src/main/java/org/apache/flink/connectors/hive/HiveSourceDynamicFileEnumerator.java:
##########
@@ -159,6 +161,9 @@ RowData createRowData(RowType rowType, Map<String, String>
partitionSpec) {
throw new UnsupportedOperationException(
"Unsupported type for dynamic filtering:" +
rowType.getTypeAt(i));
}
+ if (!fieldType.isNullable() && convertedValue == null) {
Review Comment:
The type is acquired from the DynamicFilteringData, which is a subset of the
row type of the dim side. Since the DynamicFilteringData is built with the data
from dim side, no null values should exist in a non-null field in the
DynamicFilteringData.
While a null value presents here probably because the partition name is
DEFAULT_PARTITION_NAME for a non-string field. The null value is impossible to
match the DynamicFilteringData if the field is non-null, so it should be safe
to skip the matching and discard the partition immediately.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
