viirya edited a comment on pull request #31570: URL: https://github.com/apache/spark/pull/31570#issuecomment-779996209
This approach is intuitive, so the change footprint is relatively smaller. It is also easier to maintain. Per performance wise, it in on par with the alternative one. I optimized it more from original implementation. There are some I think we can work. E.g. isolating state store format to an manager class, so we may change state store format if we need. Dynamic session window gap, etc. I would not do that here now before we reach a consensus if we are all agreed to pursue this approach. BTW, I may also take time to play with the alternative approach. cc @dbtsai ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
