viirya edited a comment on pull request #31570:
URL: https://github.com/apache/spark/pull/31570#issuecomment-779996209
This approach is intuitive, so the change footprint is relatively smaller.
It is also easier to maintain. Per performance wise, it in on par with the
alternative one.
I optimized it more from original implementation. There are some I think we
can work. E.g.:
* isolating state store format to an manager class, so we may change state
store format if we need
* optimize state store format
* dynamic session window gap
* ...etc.
I would not do that here now before we reach a consensus if we are all
agreed to pursue this approach.
BTW, I may also take time to play with the alternative approach.
cc @dbtsai
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
