dongjoon-hyun commented on PR #59163: URL: https://github.com/apache/spark/pull/59163#issuecomment-5918459785
Thank you for the review, @viirya. I addressed all comments in f001a772208. - `DataSourceV2ScanExecBase.outputOrdering` also keeps the partition key sort orders past a pruned column when the output partitioning is a `KeyedPartitioning`. So the non-merge coalescing path of `GroupPartitionsExec` keeps `(id)` for a reported `(ts, id)` with `ts` pruned, as before this PR. - The test has a new table reporting `(ts, id)` and checks the non-merge `GroupPartitionsExec(scan).outputOrdering` for all three tables. - `V2ScanPartitioningAndOrdering` has a note on why its `ordering` pass keeps the reported ordering as is. - The PR description is updated accordingly. `GroupPartitionsExecSuite`, `KeyGroupedPartitioningSuite` (with its two runtime filter suites), `DataSourceV2Suite`, and `MergeSubplansSuite` pass locally (388 tests). Could you take another look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
