dwsmith1983 commented on PR #7597: URL: https://github.com/apache/datafusion-comet/pull/7597#issuecomment-6801362982
@ErikBPF pushed the Delta serde and the expression suite (dcbbdec8a). Until the two pieces underneath merge, this diff also carries them: - #7600: the native kernel. - The core `CometExpressionContrib` hook, which lets a contrib register an expression serde. I'll open it as its own PR once the design question in #7573 is answered. After those merge and this branch merges `main`, what's left here is the serde, the suite and a "Z-Order Keys" section in the Delta user guide. I removed the probe under `dev/repros/delta` because each of its checks now has a test in `CometDeltaInterleaveBitsSuite`, under both Delta paths; bring it back if you'd rather keep it as a baseline. It's yours from here for cleanup and extending. Open items I left: - The OPTIMIZE test checks `zOrderBy` by substring; an exact match on `["a","b"]` would be tighter. - `delta.md` says the OPTIMIZE path "does not benefit yet"; the docs style guide keeps plans out of the user guide. - The empty-table test writes no batches, so the kernel's zero-length-batch path isn't exercised. - No benchmark against Spark yet. It would also tell us whether a faster bit-spread interleave is worth adding to the kernel. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
