syun64 opened a new pull request, #473:
URL: https://github.com/apache/iceberg-python/pull/473
The existing PyIceberg `cleanup_snapshot_id` validator creates tables that
are not backward compatible.
On table creation, the existing behavior in Java is to create tables with
current_snapshot_id = -1. Older versions of the Java implementations also seem
to require that current_snapshot_id is a None-null long value, and throws an
exception if it is None:
```
Retrying task after failure: Cannot parse missing long: current-snapshot-id
java.lang.IllegalArgumentException: Cannot parse missing long:
current-snapshot-id
at
org.apache.iceberg.relocated.com.google.common.base.Preconditions.checkArgument(Preconditions.java:220)
at org.apache.iceberg.util.JsonUtil.getLong(JsonUtil.java:133)
at
org.apache.iceberg.TableMetadataParser.fromJson(TableMetadataParser.java:436)
at
org.apache.iceberg.TableMetadataParser.fromJson(TableMetadataParser.java:303)
at
org.apache.iceberg.TableMetadataParser.read(TableMetadataParser.java:273)
at
org.apache.iceberg.TableMetadataParser.read(TableMetadataParser.java:266)
```
In order to preserve backward compatibility, we should cast `None`
current_snapshot_id to `-1`, rather than casting `-1` current_snapshot_id value
to `None`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
