Sent from Samsung Mobile.
Original message From: Diwakar Dhanuskodi
Date:07/02/2016 01:39 (GMT+05:30)
To: Cody Koeninger Cc:
user@spark.apache.org Subject: Re: Kafka directsream receiving rate
Thanks Cody for trying to understand the issue .
Sorry if I am not c
2/2016 22:32 (GMT+05:30)
To: Diwakar Dhanuskodi Cc:
user@spark.apache.org Subject: Re: Kafka directsream receiving rate
I am not at all clear on what you are saying.
"Yes , I am printing each messages . It is processing all messages under
each dstream block." If it is pro
sing
> millions per batch .
>
>
>
>
> Sent from Samsung Mobile.
>
>
> Original message
> From: Cody Koeninger
> Date:06/02/2016 01:30 (GMT+05:30)
> To: Diwakar Dhanuskodi
> Cc: user@spark.apache.org
> Subject: Re: Kafka directsrea
Mobile.
Original message From: Cody Koeninger
Date:06/02/2016 01:30 (GMT+05:30)
To: Diwakar Dhanuskodi Cc:
user@spark.apache.org Subject: Re: Kafka directsream receiving rate
Have you tried just printing each message, to see which ones are being
processed?
On Fri, Feb 5, 2016 at
Dhanuskodi
> Cc: user@spark.apache.org
> Subject: Re: Kafka directsream receiving rate
>
> How are you counting the number of messages?
>
> I'd go ahead and remove the settings for backpressure and
> maxrateperpartition, just to eliminate that as a variable.
>
>
Date:06/02/2016 00:33 (GMT+05:30)
To: Diwakar Dhanuskodi Cc:
user@spark.apache.org Subject: Re: Kafka directsream receiving rate
How are you counting the number of messages?
I'd go ahead and remove the settings for backpressure and maxrateperpartition,
just to eliminate that as a var
/root/dime/jars/spark-streaming-kafka-assembly_2.10-1.5.1.jar,/root/Jars/sparkreceiver.jar
> /root/Jars/sparkreceiver.jar
>
>
>
>
> Sent from Samsung Mobile.
>
>
> Original message
> From: Cody Koeninger
> Date:05/02/2016 22:07 (GMT+05:30)
> T
ar
/root/Jars/sparkreceiver.jar
Sent from Samsung Mobile.
Original message ----From: Cody Koeninger
Date:05/02/2016 22:07 (GMT+05:30)
To: Diwakar Dhanuskodi Cc:
user@spark.apache.org Subject: Re: Kafka directsream receiving rate
If you're using the direct stream, you
iwakar.
>
>
> Sent from Samsung Mobile.
>
>
> Original message
> From: Diwakar Dhanuskodi
> Date:05/02/2016 07:33 (GMT+05:30)
> To: user@spark.apache.org
> Cc:
> Subject: Kafka directsream receiving rate
>
> Hi,
> Using spark 1.5.1.
> I have
message From: Diwakar Dhanuskodi
Date:05/02/2016 07:33 (GMT+05:30)
To: user@spark.apache.org Cc: Subject: Kafka
directsream receiving rate
Hi,
Using spark 1.5.1.
I have a topic with 20 partitions. When I publish 100 messages. Spark direct
stream is receiving 10 messages per dstream
Hi,
Using spark 1.5.1.
I have a topic with 20 partitions. When I publish 100 messages. Spark direct
stream is receiving 10 messages per dstream. I have only one receiver .
When I used createStream the receiver received entire 100 messages at once.
Appreciate any help .
Regards
Di
11 matches
Mail list logo
