Saisai Shao created SPARK-7111:
----------------------------------
Summary: Add a tracker to track the direct (receiver-less) streams
Key: SPARK-7111
URL: https://issues.apache.org/jira/browse/SPARK-7111
Project: Spark
Issue Type: Improvement
Reporter: Saisai Shao
Currently for receiver-based input streams, Spark Streaming offers
ReceiverTracker and ReceivedBlockTracker to track the status of receivers as
well as block information. Also this status and block information can be
retrieved from StreamingListener to expose to the users.
But for direct-based (receiver-less) input streams, Current Spark Streaming
lacks such mechanism to track the registered direct streams, also lacks the way
to track the processed number of data for direct-based input streams.
Here propose a mechanism to track the register direct stream, also expose the
processing statistics to the BatchInfo and StreamingListener.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
