arunmahadevan commented on issue #23576: [SPARK-26655] [SS] Support multiple aggregates in append mode URL: https://github.com/apache/spark/pull/23576#issuecomment-467663947 @jose-torres , thanks for the comment. The current approach allows the users to chain aggregates in append mode as long as they define a watermark per aggregate. Its a reasonable approach I could figure out within the existing framework. We could relax this restriction but need a way to choose the timestamp field per aggregate if the user does not explicitly specify it. The other alternative I explored would require a separate watermark channel and would need more disruptive changes. Could you review the doc and comment on the areas you would like more info and/or alternatives that can be explored so that we can agree on a more solid design to proceed further ?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
