HeartSaVioR commented on issue #26201: [SPARK-29543][SS][UI] Init structured streaming ui URL: https://github.com/apache/spark/pull/26201#issuecomment-570100896 I think this feature would be the most useful addition on end-users perspective for Structured Streaming among all changes in SS for Spark 3.0; it would be ideal if we can deal with major bugs earlier and get this in before feature freeze of Spark 3.0 (seems to be end of Jan), and handle minors later as we will have more weeks to deal with. My major comments are mostly related to use StreamingQueryListener instead of directly referring StreamingQuery, in line with review comment from @zsxwing. All major issues come from the fact that the PR is accessing the instance which are changing from outside and not read-only. If we define an individual data structure and leverage KV store to read/write, it would resolve various issues for thread-safety, isolation, availability on SHS, etc. I'd like to help dealing with this if @uncleGen is busy with other stuffs. It's ideal for @uncleGen to deal with this as well to get full credit (deserves it), but I can work on it if that's the only way we can finish it in time. WDYT?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
