andygrove opened a new pull request #9035: URL: https://github.com/apache/arrow/pull/9035
Implement metrics for `HashJoinExec` to help debug performance issues. Example output: ``` [2020-12-28T20:46:28Z DEBUG datafusion::physical_plan::hash_join] Built build-side of hash join containing 3115341 rows in 9926 ms [2020-12-28T20:46:36Z DEBUG datafusion::physical_plan::hash_join] Processed 144 stream-side input batches containing 18750000 rows and produced 144 output batches containing 388922 rows in 7994 ms [2020-12-28T20:46:36Z DEBUG datafusion::physical_plan::hash_join] Processed 144 stream-side input batches containing 18750000 rows and produced 144 output batches containing 389294 rows in 7986 ms [2020-12-28T20:46:37Z DEBUG datafusion::physical_plan::hash_join] Processed 144 stream-side input batches containing 18750000 rows and produced 144 output batches containing 389558 rows in 8326 ms [2020-12-28T20:46:37Z DEBUG datafusion::physical_plan::hash_join] Processed 144 stream-side input batches containing 18750000 rows and produced 144 output batches containing 388841 rows in 8499 ms [2020-12-28T20:46:37Z DEBUG datafusion::physical_plan::hash_join] Processed 144 stream-side input batches containing 18750000 rows and produced 144 output batches containing 390056 rows in 8643 ms [2020-12-28T20:46:37Z DEBUG datafusion::physical_plan::hash_join] Processed 144 stream-side input batches containing 18749999 rows and produced 144 output batches containing 389143 rows in 8711 ms [2020-12-28T20:46:37Z DEBUG datafusion::physical_plan::hash_join] Processed 144 stream-side input batches containing 18750000 rows and produced 144 output batches containing 389605 rows in 8712 ms [2020-12-28T20:46:37Z DEBUG datafusion::physical_plan::hash_join] Processed 144 stream-side input batches containing 18750000 rows and produced 144 output batches containing 389922 rows in 8877 ms ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
