hehuiyuan commented on code in PR #10234:
URL: https://github.com/apache/hudi/pull/10234#discussion_r1413595424
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/source/prune/PartitionPruners.java:
##########
@@ -94,7 +94,7 @@ private boolean evaluate(String partition) {
Map<String, ColumnStats> partStats = new LinkedHashMap<>();
for (int idx = 0; idx < partitionKeys.length; idx++) {
String partKey = partitionKeys[idx];
- Object partVal = partKey.equals(defaultParName)
+ Object partVal = partStrArray[idx].equals(defaultParName)
Review Comment:
If part filed value is null , use `defaultParName` instead of when writing
data.
When reading:
1. If partVal equals `defaultParName`, the value of part field is null
and the code return null.
```
Object partVal = partStrArray[idx].equals(defaultParName)
? null : DataTypeUtils.resolvePartition(partStrArray[idx],
partitionTypes.get(idx));
```
ColumnStats columnStats = new ColumnStats(null, null, 1);
2. if parVal equals`defaultParName` and parKey is not equal to
`defaultParName`, the code return `defaultParName`
```
Object partVal = partKey.equals(defaultParName)
? null : DataTypeUtils.resolvePartition(partStrArray[idx],
partitionTypes.get(idx));
```
ColumnStats columnStats = new ColumnStats(`defaultParName`,`defaultParName`,
0);
3. For example IsNotNull Evaluator:
```
public static class IsNotNull extends LeafEvaluator {
private static final long serialVersionUID = 1L;
public static IsNotNull getInstance() {
return new IsNotNull();
}
@Override
public boolean eval(Map<String, ColumnStats> columnStatsMap) {
ColumnStats columnStats = getColumnStats(columnStatsMap);
// should consider FLOAT/DOUBLE & NAN
return columnStats.getMinVal() != null || columnStats.getNullCnt() <=
0;
}
}
```
if part filed value is null ,
apply 2 to 3 ， the `eval` method return true
apply 1 to 3 ， the `eval` method return false.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
