Sam Davarnia created SPARK-33833: ------------------------------------ Summary: Allow Spark Structured Streaming report Kafka Lag through Burrow Key: SPARK-33833 URL: https://issues.apache.org/jira/browse/SPARK-33833 Project: Spark Issue Type: Improvement Components: Structured Streaming Affects Versions: 3.0.1 Reporter: Sam Davarnia
Because structured streaming tracks Kafka offset consumption by itself, It is not possible to track total Kafka lag using Burrow similar to DStreams We have used Stream hooks as mentioned [here|https://medium.com/@ronbarabash/how-to-measure-consumer-lag-in-spark-structured-streaming-6c3645e45a37] It would be great if Spark supports this feature out of the box. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org