tustvold commented on pull request #1732: URL: https://github.com/apache/arrow-datafusion/pull/1732#issuecomment-1028240813
@Dandandan I've reverted the changes to those, the main operator I'm interested in excluding is UnionExec as it is showing up in IOx scan plans and yielding 3000 odd partitions as it repartitions every parquet file 28 times. :laughing: For context one of the reasons I'm interested in reducing the concurrency is somewhat stupid, the plans and the resulting metrics and tracing become HUGE, but I also can't help but feel that always pursuing the maximum concurrency is liable to hit a wall somewhere - probably memory :grin: -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
