hntd187 commented on issue #1544: URL: https://github.com/apache/arrow-datafusion/issues/1544#issuecomment-1013641369
> > I don't know if outside of a custom DataFrame impl this would be possible in a contrib module. > > If this is not clear from the get go, you can also complete the PoC within datafusion and then we can look at the diff together to see which part should be factored out into extensions. > > > I guess the struggle I am facing here is it seems like the dataframe right now has to get a "wait or end" of data reading notification > > I am also not following the question here. The `SendableRecordBatchStream` returned from `execute_stream` should be able to represent a never ending stream. Do you have a more concrete example of what you are trying to achieve? I think I may have just confused myself thinking while I was writing the original post, execute stream seems good for now. Sorry for the confusion. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
