sunchao commented on PR #5409: URL: https://github.com/apache/datafusion-comet/pull/5409#issuecomment-5442788571
@andygrove, a few clarifications against `a96957cc` for [these questions](https://github.com/apache/datafusion-comet/pull/5409#issuecomment-5441723664): - `BatchSplitExec` addresses the [previously demonstrated post-explode replacement overflow](https://github.com/apache/datafusion-comet/pull/5409#discussion_r3837642570), so there is a dependency even though its effect is broader than `replace`. The [current wrapper forwards Unnest metrics](https://github.com/apache/datafusion-comet/blob/a96957cce86116427e6a056e878807c9c0bf7f05/native/core/src/execution/operators/batch_split.rs#L102); its own `batches_split` counter is not declared in the SQL operator metrics. It bounds downstream batch sizes, not peak memory while Unnest constructs the expanded batch. Wrapper overhead remains unmeasured in this follow-up. - The [codegen tests check Spark result parity](https://github.com/apache/datafusion-comet/blob/a96957cce86116427e6a056e878807c9c0bf7f05/spark/src/test/scala/org/apache/comet/CometCodegenSuite.scala#L393), not only routing. The [SQL fixture](https://github.com/apache/datafusion-comet/blob/a96957cce86116427e6a056e878807c9c0bf7f05/spark/src/test/resources/sql-tests/expressions/string/string_replace.sql#L45) includes overlap, multibyte search, searches longer than source rows, and NULL-source cases. I did not find explicit multibyte-replacement or replacement-containing-search cases in the changed fixtures. - The malformed-column and general batch-cap questions are not resolved merely by the literal guard. [#5310](https://github.com/apache/datafusion-comet/pull/5310) remains open and unmerged; its import protection cannot be assumed present. I also did not establish a global cap across every producer/configuration/AQE transition. Those concerns overlap the existing source-safety/overflow discussions, rather than establishing another verified finding here. This was a source/discussion check; I did not rerun runtime tests or benchmarks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
