uros-db commented on PR #46597: URL: https://github.com/apache/spark/pull/46597#issuecomment-2185169957
@GideonPotok Yup, I would suggest to undo the changes in CollationBenchmark, as well as the collation benchmark results. There's been enough exploring and measuring, so from now on let's only focus on the `Mode` expression in this PR - for example, I think we should definitely have some more tests for this expression (depending on how regular non-collated Mode is tested in the Spark codebase, ideally you can consider adding unit tests or just some more sql tests) After that, we can decide on how to proceed with mode. Currently, I see 2 outstanding tasks: - support for complex types - support for PandasMode (but there also may be something that we're missing) Depending on how you feel about these changes, you could do it all in this PR, or make separate PRs - feel free to mix & match depending on your preference after decluttering the PR from benchmark-related stuff -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
