robert towne <robert.to...@webtrends.com<mailto:robert.to...@webtrends.com>>
Cc: "user@spark.apache.org<mailto:user@spark.apache.org>"
<user@spark.apache.org<mailto:user@spark.apache.org>>
Subject: Re: Problems w/YARN Spark Streaming app reading from Kafka
W
I have a Spark Streaming app that reads using a reciever-less connection (
KafkaUtils.createDirectStream) with an interval of 1 minute.
For about 15 hours it was running fine, ranging in input size of 3,861,758 to
16,836 events.
Then about 3 hours ago, every minute batch brought in the same
What do you mean by "the current documentation states it isn’t used"?
http://spark.apache.org/docs/latest/configuration.html still lists the
value and its meaning.
As far as the issue you're seeing, are you measuring records by looking at
logs, the spark ui, or actual downstream sinks of data?
