Looks like probably https://issues.apache.org/jira/browse/SPARK-8701
so 1.5.0 On Wed, Dec 9, 2015 at 10:25 AM, Dan Dutrow <dan.dut...@gmail.com> wrote: > I'm on spark version 1.4.1. I couldn't find documentation that said it was > fixed, so I thought maybe it was still an open issue. Any idea what the fix > version is? > > On Wed, Dec 9, 2015 at 11:10 AM Cody Koeninger <c...@koeninger.org> wrote: > >> Which version of spark are you on? I thought that was added to the spark >> UI in recent versions. >> >> DIrect api doesn't have any inherent interaction with zookeeper. If you >> need number of messages per batch and aren't on a recent enough version of >> spark to see them in the ui, you can get them programatically from the >> offset ranges. See the definition of count() in recent versions of >> KafkaRDD for an example. >> >> On Wed, Dec 9, 2015 at 9:39 AM, Dan Dutrow <dan.dut...@gmail.com> wrote: >> >>> Is there documentation for how to update the metrics (#messages per >>> batch) in the Spark Streaming tab when using the Direct API? Does the >>> Streaming tab get its information from Zookeeper or something else >>> internally? >>> -- >>> Dan ✆ >>> >> >> -- > Dan ✆ >
