Dandandan commented on issue #21650: URL: https://github.com/apache/datafusion/issues/21650#issuecomment-4262927382
Nice to hear the concern is shared (both from a idealogical / architecture standpoint as with some concrete technical examples). I will add that I also think the state via `TaskContext` should share as little as possible: * Only keep it local to each returned stream when it is only for each partition * Shared among the streams across partitions for a single operator (so it is dropped when all streams are done) * Only bound to `TaskContext` when it has to be shared during the full query (and will be dropped when TaskContext is dropped <- perhaps this can be avoided in majority of cases -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
