Sam Davarnia created SPARK-33833:
------------------------------------
Summary: Allow Spark Structured Streaming report Kafka Lag through
Burrow
Key: SPARK-33833
URL: https://issues.apache.org/jira/browse/SPARK-33833
Project: Spark
Issue Type: Improvement
Components: Structured Streaming
Affects Versions: 3.0.1
Reporter: Sam Davarnia
Because structured streaming tracks Kafka offset consumption by itself,
It is not possible to track total Kafka lag using Burrow similar to DStreams
We have used Stream hooks as mentioned
[here|https://medium.com/@ronbarabash/how-to-measure-consumer-lag-in-spark-structured-streaming-6c3645e45a37]
It would be great if Spark supports this feature out of the box.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
