jychen7 commented on issue #5969: URL: https://github.com/apache/arrow-datafusion/issues/5969#issuecomment-1507903769
when running with `RUST_LOG=debug datafusion-cli`, I find out it is slow during `do_sort`, but not sure which part is slow: `insert_batch` or final `sort`. So I add debug log in https://github.com/apache/arrow-datafusion/compare/main...jychen7:arrow-datafusion:debug-clickbench-q32?expand=1. Surprisingly, it takes 13 mins between `Start do_sort` and `Start insert_batch` ``` [2023-04-14T02:26:51Z DEBUG datafusion::physical_plan::sorts::sort] Start do_sort for partition 11 of context session_id 039882c2-6b74-4d6a-9739-a50c4fbe1ac7 and task_id None [2023-04-14T02:39:01Z DEBUG datafusion::physical_plan::sorts::sort] Start insert_batch for partition 11 with input size 295992 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
