LiaCastaneda commented on PR #23985: URL: https://github.com/apache/datafusion/pull/23985#issuecomment-5167087918
I missed the notification for this PR, the change makes sense to me! We do something similar internally to track peak memory (from the `MemoryPool`) per query. The query id is stored in an OTel context object, and on each grow we record the peak for the query that called grow. Wondering if something similar could live in the common pools. The tricky part might be that the pool only knows about memory consumers, so it can't do per-query attribution right now :/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
