Tsai Li Ming created SPARK-15039: ------------------------------------ Summary: Kinesis reciever does not work in Yarn Key: SPARK-15039 URL: https://issues.apache.org/jira/browse/SPARK-15039 Project: Spark Issue Type: Bug Components: Streaming Affects Versions: 1.6.0 Environment: YARN HDP 2.4.0 Reporter: Tsai Li Ming
Hi, Using the pyspark kinesis example, it does not receive any messages from Kinesis when submitting to a YARN cluster, though it is working when using local mode. ``` spark-submit \ --executor-cores 4 \ --num-executors 4 \ --packages com.databricks:spark-redshift_2.10:0.6.0,com.databricks:spark-csv_2.10:1.4.0,org.apache.spark:spark-streaming-kinesis-asl_2.10:1.5.1 ``` I had to downgrade the package to 1.5.1 before it can work. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org