Visorgood commented on issue #5644: URL: https://github.com/apache/datafusion-comet/issues/5644#issuecomment-6878081906
@andygrove I'd like to take the manifest metrics parity item from the checklist above, unless you've already started on it. Plan: for each data type, write the same rows through iceberg-java and the native writer, then compare value, null and NaN counts and lower/upper bounds per column in the resulting manifests. Two questions: 1. Spark 3.4 has been dropped, so should this cover only Iceberg 1.8.1, 1.10.0 and 1.11.0, or is 1.5.2 still needed? 2. Should the tests go into Comet's own suites (next to the NaN count checks in `CometIcebergWriteActionSuite`) or into the Iceberg CI jobs? If it helps, I can open a separate sub-issue for this so it can be assigned and tracked. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
