zheniasigayev commented on issue #14510: URL: https://github.com/apache/datafusion/issues/14510#issuecomment-3181216887
Very exciting progress! One thing that would be incredibly helpful is to extend the configuration flag, `--top-memory-consumers` from the DataFusion CLI into the configs (https://datafusion.apache.org/user-guide/configs.html) to get informative error messages. This would display N number of top memory consumers when the query fails due to memory exhaustion. By default it can be set to 0 to disable this. I think a big leap forward would be to implement continuous profiling of DataFusion, since it's been quite difficult to do so with https://datafusion.apache.org/library-user-guide/profiling.html, unless more capabilities have been added. Thinking even farther out, some nice-to-haves would be equivalent tools such as https://github.com/DBatUTuebingen/duckdb-explain-visualizer. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
