hvanhovell commented on issue #26127: [SPARK-29348][SQL] Add observable Metrics for Streaming queries URL: https://github.com/apache/spark/pull/26127#issuecomment-543192165 @cloud-fan @HeartSaVioR the current approach uses `ProgressReporter` so it currently does not work with `ContinuousExecution` (See https://issues.apache.org/jira/browse/SPARK-23887). I would rather not wait until this is fixed. At some point `ContinuousExecution` should report progress through `ProgressReporter` (the approach taken in https://github.com/apache/spark/pull/24537), when that we can make observable metrics work with it. I will update the description to underline that this only works with (micro) batch.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
