Samsung Mobile.
Original message From: Diwakar Dhanuskodi
<diwakar.dhanusk...@gmail.com> Date:07/02/2016 01:39 (GMT+05:30)
To: Cody Koeninger <c...@koeninger.org> Cc:
user@spark.apache.org Subject: Re: Kafka directsream receiving rate
Thanks Cody for trying to
org> Date:06/02/2016 22:32 (GMT+05:30)
To: Diwakar Dhanuskodi <diwakar.dhanusk...@gmail.com> Cc:
user@spark.apache.org Subject: Re: Kafka directsream receiving rate
I am not at all clear on what you are saying.
"Yes , I am printing each messages . It is processing all messages u
er batch .
>
>
>
>
> Sent from Samsung Mobile.
>
>
> Original message
> From: Cody Koeninger <c...@koeninger.org>
> Date:06/02/2016 01:30 (GMT+05:30)
> To: Diwakar Dhanuskodi <diwakar.dhanusk...@gmail.com>
> Cc: user@spark.apache.org
>
Mobile.
Original message From: Cody Koeninger
<c...@koeninger.org> Date:06/02/2016 01:30 (GMT+05:30)
To: Diwakar Dhanuskodi <diwakar.dhanusk...@gmail.com> Cc:
user@spark.apache.org Subject: Re: Kafka directsream receiving rate
Have you tried just printing each message, to see w
If you're using the direct stream, you have 0 receivers. Do you mean you
have 1 executor?
Can you post the relevant call to createDirectStream from your code, as
well as any relevant spark configuration?
On Thu, Feb 4, 2016 at 8:13 PM, Diwakar Dhanuskodi <
diwakar.dhanusk...@gmail.com> wrote:
> /root/dime/jars/spark-streaming-kafka-assembly_2.10-1.5.1.jar,/root/Jars/sparkreceiver.jar
> /root/Jars/sparkreceiver.jar
>
>
>
>
> Sent from Samsung Mobile.
>
>
> Original message
> From: Cody Koeninger <c...@koeninger.org>
> Date:05/02/2016 22:07 (
er.jar
/root/Jars/sparkreceiver.jar
Sent from Samsung Mobile.
Original message From: Cody Koeninger
<c...@koeninger.org> Date:05/02/2016 22:07 (GMT+05:30)
To: Diwakar Dhanuskodi <diwakar.dhanusk...@gmail.com> Cc:
user@spark.apache.org Subject: Re: Kaf
<c...@koeninger.org> Date:06/02/2016 00:33 (GMT+05:30)
To: Diwakar Dhanuskodi <diwakar.dhanusk...@gmail.com> Cc:
user@spark.apache.org Subject: Re: Kafka directsream receiving rate
How are you counting the number of messages?
I'd go ahead and remove the settings for
:33 (GMT+05:30)
> To: Diwakar Dhanuskodi <diwakar.dhanusk...@gmail.com>
> Cc: user@spark.apache.org
> Subject: Re: Kafka directsream receiving rate
>
> How are you counting the number of messages?
>
> I'd go ahead and remove the settings for backpressure and
> maxrateperpartit
Adding more info
Batch interval is 2000ms.
I expect all 100 messages go thru one dstream from directsream but it
receives at rate of 10 messages at time. Am I missing some configurations
here. Any help appreciated.
Regards
Diwakar.
Sent from Samsung Mobile.
Original
10 matches
Mail list logo