Re: How to access metrics for Structured Streaming 2.1

You can use the monitoring APIs of Structured Streaming to get metrics. See http://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#monitoring-streaming-queries On Tue, Jan 17, 2017 at 5:01 PM, Heji Kim wrote: > Hello. We are trying to

How to access metrics for Structured Streaming 2.1

Hello. We are trying to migrate and performance test the kafka sink for structured streaming in 2.1. Obviously we miss the beautiful Streaming Statistics ui tab and we are trying to figure out the most reasonable way to monitor event processing rates and lag time. 1. Are the SourceStatus and