GitHub user joolean closed a discussion: Guidance for integrating a progress bar
Hello! I'm attempting to use Datafusion to implement an embedded query engine for a tool that occasionally needs to perform CPU-intensive, long-running operations, such as sorting a dataset that is too large to fit in memory. The debug logging produced during the execution of the physical plan during these types of operations is useful to me (as a developer) but for my users I would prefer to render a progress bar instead, using something like `indicatif`. Before I jump into the Datafusion internals, I'm wondering if anyone has any suggestions as to the best way to go about this. (Do you think it's even possible in the first place?) Are there any existing hooks for progress events? Or extension points for alternate implementations of, say, an `ExecutionPlan`? Thanks in advance. And thanks for all the work that's gone into Datafusion! I've found it to be an extraordinarily useful library. GitHub link: https://github.com/apache/datafusion/discussions/4668 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
