chenjunjiedada edited a comment on issue #2405:
URL: https://github.com/apache/iceberg/issues/2405#issuecomment-826281826
Still hit this problem. Now I know how to reproduce this.
1. Write the iceberg table with the 0.10.0 version.
2. Read the iceberg table manifests with the 0.11.1 version.
`ManifestTable.partitionSummarirsToRows` puts `summary.containsNaN()`,
which is null, to the spark row that will be parsed to `Boolean` class and
show out.
The following code could fix the issue but I think we should consider with
#2495 as well:
```
--- a/api/src/main/java/org/apache/iceberg/ManifestFile.java
+++ b/api/src/main/java/org/apache/iceberg/ManifestFile.java
@@ -207,7 +207,7 @@ public interface ManifestFile {
* Default to return null to ensure backward compatibility.
*/
default Boolean containsNaN() {
- return null;
+ return true;
}
/**
--- a/core/src/main/java/org/apache/iceberg/GenericPartitionFieldSummary.java
+++ b/core/src/main/java/org/apache/iceberg/GenericPartitionFieldSummary.java
@@ -121,7 +121,7 @@ public class GenericPartitionFieldSummary
@Override
public Boolean containsNaN() {
- return containsNaN;
+ return containsNaN == null || containsNull;
}
```
@RussellSpitzer @yyanyy
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
