Re: Kafka exception in Apache Spark

2016-04-26 Thread Cody Koeninger 
That error indicates a message bigger than the buffer's capacity

https://issues.apache.org/jira/browse/KAFKA-1196


On Tue, Apr 26, 2016 at 3:07 AM, Michel Hubert  wrote:
> Hi,
>
>
>
>
>
> I use a Kafka direct stream approach.
>
> My Spark application was running ok.
>
> This morning we upgraded to CDH 5.7.0
>
> And when I re-started my Spark application I get exceptions.
>
>
>
> It seems a problem with the direct stream approach.
>
> Any ideas how to fix this?
>
>
>
>
>
>
>
> User class threw exception: org.apache.spark.SparkException: Job aborted due
> to stage failure: Task 3 in stage 0.0 failed 4 times, most recent failure:
> Lost task 3.3 in stage 0.0 (TID 26, bfravicsvr81439-cld.opentsp.com):
> java.lang.IllegalArgumentException
>
> at java.nio.Buffer.limit(Buffer.java:267)
>
> at kafka.api.FetchResponsePartitionData$.readFrom(FetchResponse.scala:38)
>
> at kafka.api.TopicData$$anonfun$1.apply(FetchResponse.scala:100)
>
> at kafka.api.TopicData$$anonfun$1.apply(FetchResponse.scala:98)
>
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
>
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
>
> at scala.collection.immutable.Range.foreach(Range.scala:141)
>
> at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
>
> at scala.collection.AbstractTraversable.map(Traversable.scala:105)
>
> at kafka.api.TopicData$.readFrom(FetchResponse.scala:98)
>
> at kafka.api.FetchResponse$$anonfun$4.apply(FetchResponse.scala:170)
>
> at kafka.api.FetchResponse$$anonfun$4.apply(FetchResponse.scala:169)
>
> at
> scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:251)
>
> at
> scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:251)
>
> at scala.collection.immutable.Range.foreach(Range.scala:141)
>
> at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:251)
>
> at scala.collection.AbstractTraversable.flatMap(Traversable.scala:105)
>
> at kafka.api.FetchResponse$.readFrom(FetchResponse.scala:169)
>
> at kafka.consumer.SimpleConsumer.fetch(SimpleConsumer.scala:135)
>
> at
> org.apache.spark.streaming.kafka.KafkaRDD$KafkaRDDIterator.fetchBatch(KafkaRDD.scala:192)
>
> at
> org.apache.spark.streaming.kafka.KafkaRDD$KafkaRDDIterator.getNext(KafkaRDD.scala:208)
>
> at org.apache.spark.util.NextIterator.hasNext(NextIterator.scala:73)
>
> at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:327)
>
> at scala.collection.Iterator$$anon$14.hasNext(Iterator.scala:388)
>
> at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:327)
>
> at
> org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:126)
>
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:73)
>
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
>
> at org.apache.spark.scheduler.Task.run(Task.scala:89)
>
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>
> at java.lang.Thread.run(Thread.java:745)
>
>

-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org

RE: Kafka exception in Apache Spark

2016-04-26 Thread Michel Hubert 
This is production.

Van: Mich Talebzadeh [mailto:mich.talebza...@gmail.com]
Verzonden: dinsdag 26 april 2016 12:01
Aan: Michel Hubert <mich...@phact.nl>
CC: user@spark.apache.org
Onderwerp: Re: Kafka exception in Apache Spark

Hi Michael,

Is this production or test?


Dr Mich Talebzadeh



LinkedIn  
https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw



http://talebzadehmich.wordpress.com<http://talebzadehmich.wordpress.com/>



On 26 April 2016 at 09:07, Michel Hubert 
<mich...@phact.nl<mailto:mich...@phact.nl>> wrote:
Hi,


I use a Kafka direct stream approach.
My Spark application was running ok.
This morning we upgraded to CDH 5.7.0
And when I re-started my Spark application I get exceptions.

It seems a problem with the direct stream approach.
Any ideas how to fix this?



User class threw exception: org.apache.spark.SparkException: Job aborted due to 
stage failure: Task 3 in stage 0.0 failed 4 times, most recent failure: Lost 
task 3.3 in stage 0.0 (TID 26, 
bfravicsvr81439-cld.opentsp.com<http://bfravicsvr81439-cld.opentsp.com>): 
java.lang.IllegalArgumentException
at java.nio.Buffer.limit(Buffer.java:267)
at kafka.api.FetchResponsePartitionData$.readFrom(FetchResponse.scala:38)
at kafka.api.TopicData$$anonfun$1.apply(FetchResponse.scala:100)
at kafka.api.TopicData$$anonfun$1.apply(FetchResponse.scala:98)
at 
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at 
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at scala.collection.immutable.Range.foreach(Range.scala:141)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
at scala.collection.AbstractTraversable.map(Traversable.scala:105)
at kafka.api.TopicData$.readFrom(FetchResponse.scala:98)
at kafka.api.FetchResponse$$anonfun$4.apply(FetchResponse.scala:170)
at kafka.api.FetchResponse$$anonfun$4.apply(FetchResponse.scala:169)
at 
scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:251)
at 
scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:251)
at scala.collection.immutable.Range.foreach(Range.scala:141)
at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:251)
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:105)
at kafka.api.FetchResponse$.readFrom(FetchResponse.scala:169)
at kafka.consumer.SimpleConsumer.fetch(SimpleConsumer.scala:135)
at 
org.apache.spark.streaming.kafka.KafkaRDD$KafkaRDDIterator.fetchBatch(KafkaRDD.scala:192)
at 
org.apache.spark.streaming.kafka.KafkaRDD$KafkaRDDIterator.getNext(KafkaRDD.scala:208)
at org.apache.spark.util.NextIterator.hasNext(NextIterator.scala:73)
at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:327)
at scala.collection.Iterator$$anon$14.hasNext(Iterator.scala:388)
at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:327)
at 
org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:126)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:73)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
at org.apache.spark.scheduler.Task.run(Task.scala:89)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)

Re: Kafka exception in Apache Spark

2016-04-26 Thread Mich Talebzadeh 
Hi Michael,

Is this production or test?

Dr Mich Talebzadeh



LinkedIn * 
https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
*



http://talebzadehmich.wordpress.com



On 26 April 2016 at 09:07, Michel Hubert  wrote:

> Hi,
>
>
>
>
>
> I use a Kafka direct stream approach.
>
> My Spark application was running ok.
>
> This morning we upgraded to CDH 5.7.0
>
> And when I re-started my Spark application I get exceptions.
>
>
>
> It seems a problem with the direct stream approach.
>
> Any ideas how to fix this?
>
>
>
>
>
>
>
> User class threw exception: org.apache.spark.SparkException: Job aborted
> due to stage failure: Task 3 in stage 0.0 failed 4 times, most recent
> failure: Lost task 3.3 in stage 0.0 (TID 26,
> bfravicsvr81439-cld.opentsp.com): java.lang.IllegalArgumentException
>
> at java.nio.Buffer.limit(Buffer.java:267)
>
> at kafka.api.FetchResponsePartitionData$.readFrom(FetchResponse.scala:38)
>
> at kafka.api.TopicData$$anonfun$1.apply(FetchResponse.scala:100)
>
> at kafka.api.TopicData$$anonfun$1.apply(FetchResponse.scala:98)
>
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
>
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
>
> at scala.collection.immutable.Range.foreach(Range.scala:141)
>
> at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
>
> at scala.collection.AbstractTraversable.map(Traversable.scala:105)
>
> at kafka.api.TopicData$.readFrom(FetchResponse.scala:98)
>
> at kafka.api.FetchResponse$$anonfun$4.apply(FetchResponse.scala:170)
>
> at kafka.api.FetchResponse$$anonfun$4.apply(FetchResponse.scala:169)
>
> at
> scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:251)
>
> at
> scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:251)
>
> at scala.collection.immutable.Range.foreach(Range.scala:141)
>
> at
> scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:251)
>
> at scala.collection.AbstractTraversable.flatMap(Traversable.scala:105)
>
> at kafka.api.FetchResponse$.readFrom(FetchResponse.scala:169)
>
> at kafka.consumer.SimpleConsumer.fetch(SimpleConsumer.scala:135)
>
> at
> org.apache.spark.streaming.kafka.KafkaRDD$KafkaRDDIterator.fetchBatch(KafkaRDD.scala:192)
>
> at
> org.apache.spark.streaming.kafka.KafkaRDD$KafkaRDDIterator.getNext(KafkaRDD.scala:208)
>
> at org.apache.spark.util.NextIterator.hasNext(NextIterator.scala:73)
>
> at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:327)
>
> at scala.collection.Iterator$$anon$14.hasNext(Iterator.scala:388)
>
> at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:327)
>
> at
> org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:126)
>
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:73)
>
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
>
> at org.apache.spark.scheduler.Task.run(Task.scala:89)
>
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>
> at java.lang.Thread.run(Thread.java:745)
>
>
>