[
https://issues.apache.org/jira/browse/SPARK-20611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen resolved SPARK-20611.
-------------------------------
Resolution: Not A Problem
If a question is specific to CDH, it doesn't belong here, but rather at
Cloudera.
No, it doesn't actually fix anything to duplicate the Logging trait.
We do not use patches. You should read http://spark.apache.org/contributing.html
The problem is version mismatch. I don't think you have built vs the same
version of Spark that you run against.
> Spark kinesis connector doesnt work with cloudera distribution
> ---------------------------------------------------------------
>
> Key: SPARK-20611
> URL: https://issues.apache.org/jira/browse/SPARK-20611
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 1.6.0
> Reporter: sumit
> Labels: cloudera
> Attachments: spark-kcl.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> Facing below exception on CDH5.10
> 17/04/27 05:34:04 WARN scheduler.TaskSetManager: Lost task 0.0 in stage 58.0
> (TID 179, hadoop1.local, executor 5): java.lang.AbstractMethodError
> at org.apache.spark.Logging$class.log(Logging.scala:50)
> at
> org.apache.spark.streaming.kinesis.KinesisCheckpointer.log(KinesisCheckpointer.scala:39)
> at org.apache.spark.Logging$class.logDebug(Logging.scala:62)
> at
> org.apache.spark.streaming.kinesis.KinesisCheckpointer.logDebug(KinesisCheckpointer.scala:39)
> at
> org.apache.spark.streaming.kinesis.KinesisCheckpointer.startCheckpointerThread(KinesisCheckpointer.scala:119)
> at
> org.apache.spark.streaming.kinesis.KinesisCheckpointer.<init>(KinesisCheckpointer.scala:50)
> at
> org.apache.spark.streaming.kinesis.KinesisReceiver.onStart(KinesisReceiver.scala:149)
> at
> org.apache.spark.streaming.receiver.ReceiverSupervisor.startReceiver(ReceiverSupervisor.scala:148)
> at
> org.apache.spark.streaming.receiver.ReceiverSupervisor.start(ReceiverSupervisor.scala:130)
> at
> org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:575)
> at
> org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:565)
> at org.apache.spark.SparkContext$$anonfun$38.apply(SparkContext.scala:2000)
> at org.apache.spark.SparkContext$$anonfun$38.apply(SparkContext.scala:2000)
> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
> at org.apache.spark.scheduler.Task.run(Task.scala:89)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:242)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> below is my POM file
> <dependency>
> <groupId>org.apache.spark</groupId>
> <artifactId>spark-streaming_2.10</artifactId>
> <version>1.6.0</version>
> </dependency>
> <dependency>
> <groupId>org.apache.spark</groupId>
> <artifactId>spark-core_2.10</artifactId>
> <version>1.6.0</version>
> </dependency>
> <dependency>
> <groupId>com.amazonaws</groupId>
> <artifactId>amazon-kinesis-client</artifactId>
> <version>1.6.1</version>
> </dependency>
> <dependency>
> <groupId>org.apache.spark</groupId>
> <artifactId>spark-streaming-kinesis-asl_2.10</artifactId>
> <version>1.6.0</version>
> </dependency>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
