damansingh1313 commented on issue #17805: URL: https://github.com/apache/iceberg/issues/17805#issuecomment-5494023302
Hi @GGraziadei, I'd love to pick this up as I have done my research on this. I've mapped the change surface against #16827: the two new files (SparkHilbertUDF, SparkHilbertFileRewriteRunner) port as‑is since SparkZOrderUDF / SparkZOrderFileRewriteRunner / SparkShufflingFileRewriteRunner are identical across v4.1/v4.0/v3.5, and the three wiring files (Spark3Util, RewriteDataFilesSparkAction, RewriteDataFilesProcedure) are at the same pre‑patch state in v4.0 and v3.5. Plan: port to Spark 4.0 first, then 3.5 (both 2.12/2.13), port the Hilbert cases in TestRewriteDataFilesAction, TestSparkFileRewriteRunners, and TestRewriteDataFilesProcedure, plus a parity check against HilbertByteUtils. Happy to do it as one PR or split per version — preference? @GGraziadei cc @RussellSpitzer -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
