mustafasrepo commented on issue #9488: URL: https://github.com/apache/arrow-datafusion/issues/9488#issuecomment-1984320850
This might be related to built-in parallelism of the Datafusion. Datafusion by default executes in parallel, if query planner thinks it is helpful. You can force DataFusion to work in. single partition by executing command `set datafusion.execution.target_partitions = 1;` In this case, scan order of the file wouldn't change (At least this is what I expect). Another option might be to specify desired order in the query. if you add `ORDER BY index` to the query it should produce expected result even if query works in multi partitions. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
