GitHub user daphnenhuch-at added a comment to the discussion: How does 'sort' interact with record batches?
The provenance doesn't actually matter here. What matters is that the ultimately merged file is globally sorted and the row numbers are correct and accurate. The sorted data in the newly merged file has to have a globally sorted and correct primary key column and a globally sorted and correct row number column GitHub link: https://github.com/apache/datafusion/discussions/15711#discussioncomment-13041183 ---- This is an automatically sent email for github@datafusion.apache.org. To unsubscribe, please send an email to: github-unsubscr...@datafusion.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
