[jira] [Commented] (SPARK-27554) org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS]
[
https://issues.apache.org/jira/browse/SPARK-27554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16849594#comment-16849594
]
Jepson commented on SPARK-27554:
[~hyukjin.kwon] Thanks for concern. The code issue, i can resolve it.
> org.apache.hadoop.security.AccessControlException: Client cannot authenticate
> via:[TOKEN, KERBEROS]
> ---
>
> Key: SPARK-27554
> URL: https://issues.apache.org/jira/browse/SPARK-27554
> Project: Spark
> Issue Type: Bug
> Components: Scheduler
>Affects Versions: 2.4.0, 2.4.1
> Environment: cdh5.16.1
> spark2.4.0
> spark2.4.1
> spark2.4.2
>Reporter: Jepson
>Priority: Major
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> {code}
> #basic env
> JAVA_HOME=/usr/java/jdk1.8.0_181
> HADOOP_CONF_DIR=/etc/hadoop/conf
> export SPARK_HOME=/opt/software/spark/spark-2.4.2
> {code}
> {code}
> #project env
> KERBEROS_USER=h...@hadoop.com
> KERBEROS_USER_KEYTAB=/etc/kerberos/hdfs.keytab
> PROJECT_MAIN_CLASS=com.jy.dwexercise.OrderDeliveryModel
> PROJECT_JAR=/opt/maintain/scripts/bas-1.0-SNAPSHOT.jar
> {code}
> {code}
> #spark resource
> DRIVER_MEMORY=4G
> EXECUTORS_NUM=20
> EXECUTOR_MEMORY=6G
> EXECUTOR_VCORES=4
> QUEQUE=idss
> {code}
> {code}
> #submit job
> /opt/software/spark/spark-2.4.2/bin/spark-submit \
> --master yarn \
> --deploy-mode cluster \
> --queue ${QUEQUE} \
> --driver-memory ${DRIVER_MEMORY} \
> --num-executors ${EXECUTORS_NUM} \
> --executor-memory ${EXECUTOR_MEMORY} \
> --executor-cores ${EXECUTOR_VCORES} \
> --principal ${KERBEROS_USER} \
> --keytab ${KERBEROS_USER_KEYTAB} \
> --class ${PROJECT_MAIN_CLASS} \
> --conf
> "spark.yarn.archive=hdfs://jybigdata/spark/spark-archive-20190422.zip" \
> --conf "spark.sql.autoBroadcastJoinThreshold=20971520" \
> ${PROJECT_JAR}
> {code}
>
> {code}
> *ERROR log:*
> 19/04/24 13:35:07 INFO ConfiguredRMFailoverProxyProvider: Failing over to rm78
> 19/04/24 13:35:07 WARN UserGroupInformation: PriviledgedActionException
> as:hdfs (auth:SIMPLE)
> cause:org.apache.hadoop.security.AccessControlException: Client cannot
> authenticate via:[TOKEN, KERBEROS]
> 19/04/24 13:35:07 WARN Client: Exception encountered while connecting to the
> server : org.apache.hadoop.security.AccessControlException: Client cannot
> authenticate via:[TOKEN, KERBEROS]
> 19/04/24 13:35:07 WARN UserGroupInformation: PriviledgedActionException
> as:hdfs (auth:SIMPLE) cause:java.io.IOException:
> org.apache.hadoop.security.AccessControlException: Client cannot authenticate
> via:[TOKEN, KERBEROS]
> 19/04/24 13:35:07 INFO RetryInvocationHandler: Exception while invoking
> getClusterMetrics of class ApplicationClientProtocolPBClientImpl over rm78
> after 10 fail over attempts. Trying to fail over immediately.
> java.io.IOException: Failed on local exception: java.io.IOException:
> org.apache.hadoop.security.AccessControlException: Client cannot authenticate
> via:[TOKEN, KERBEROS]; Host Details : local host is:
> "hadoopnode143/192.168.209.143"; destination host is:
> "hadoopmanager136":8032;
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)
> at org.apache.hadoop.ipc.Client.call(Client.java:1508)
> at org.apache.hadoop.ipc.Client.call(Client.java:1441)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:231)
> at com.sun.proxy.$Proxy25.getClusterMetrics(Unknown Source)
> at
> org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getClusterMetrics(ApplicationClientProtocolPBClientImpl.java:202)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:258)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:104)
> at com.sun.proxy.$Proxy26.getClusterMetrics(Unknown Source)
> at
> org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getYarnClusterMetrics(YarnClientImpl.java:483)
> at
> org.apache.spark.deploy.yarn.Client$$anonfun$submitApplication$1.apply(Client.scala:164)
> at
> org.apache.spark.deploy.yarn.Client$$anonfun$submitApplication$1.apply(Client.scala:164)
> at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
> at org.apache.spark.deploy.yarn.Client.logInfo(Client.scala:59)
> at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:163)
> at
> org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedul
[jira] [Commented] (SPARK-27554) org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS]
[
https://issues.apache.org/jira/browse/SPARK-27554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16826699#comment-16826699
]
Jepson commented on SPARK-27554:
[~hyukjin.kwon] Thanks for quick reply.
But, other spark job are normal. I spent a long time for troubleshooting.
> org.apache.hadoop.security.AccessControlException: Client cannot authenticate
> via:[TOKEN, KERBEROS]
> ---
>
> Key: SPARK-27554
> URL: https://issues.apache.org/jira/browse/SPARK-27554
> Project: Spark
> Issue Type: Bug
> Components: Scheduler
>Affects Versions: 2.4.0, 2.4.1
> Environment: cdh5.16.1
> spark2.4.0
> spark2.4.1
> spark2.4.2
>Reporter: Jepson
>Priority: Major
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> {code}
> #basic env
> JAVA_HOME=/usr/java/jdk1.8.0_181
> HADOOP_CONF_DIR=/etc/hadoop/conf
> export SPARK_HOME=/opt/software/spark/spark-2.4.2
> {code}
> {code}
> #project env
> KERBEROS_USER=h...@hadoop.com
> KERBEROS_USER_KEYTAB=/etc/kerberos/hdfs.keytab
> PROJECT_MAIN_CLASS=com.jy.dwexercise.OrderDeliveryModel
> PROJECT_JAR=/opt/maintain/scripts/bas-1.0-SNAPSHOT.jar
> {code}
> {code}
> #spark resource
> DRIVER_MEMORY=4G
> EXECUTORS_NUM=20
> EXECUTOR_MEMORY=6G
> EXECUTOR_VCORES=4
> QUEQUE=idss
> {code}
> {code}
> #submit job
> /opt/software/spark/spark-2.4.2/bin/spark-submit \
> --master yarn \
> --deploy-mode cluster \
> --queue ${QUEQUE} \
> --driver-memory ${DRIVER_MEMORY} \
> --num-executors ${EXECUTORS_NUM} \
> --executor-memory ${EXECUTOR_MEMORY} \
> --executor-cores ${EXECUTOR_VCORES} \
> --principal ${KERBEROS_USER} \
> --keytab ${KERBEROS_USER_KEYTAB} \
> --class ${PROJECT_MAIN_CLASS} \
> --conf
> "spark.yarn.archive=hdfs://jybigdata/spark/spark-archive-20190422.zip" \
> --conf "spark.sql.autoBroadcastJoinThreshold=20971520" \
> ${PROJECT_JAR}
> {code}
>
> {code}
> *ERROR log:*
> 19/04/24 13:35:07 INFO ConfiguredRMFailoverProxyProvider: Failing over to rm78
> 19/04/24 13:35:07 WARN UserGroupInformation: PriviledgedActionException
> as:hdfs (auth:SIMPLE)
> cause:org.apache.hadoop.security.AccessControlException: Client cannot
> authenticate via:[TOKEN, KERBEROS]
> 19/04/24 13:35:07 WARN Client: Exception encountered while connecting to the
> server : org.apache.hadoop.security.AccessControlException: Client cannot
> authenticate via:[TOKEN, KERBEROS]
> 19/04/24 13:35:07 WARN UserGroupInformation: PriviledgedActionException
> as:hdfs (auth:SIMPLE) cause:java.io.IOException:
> org.apache.hadoop.security.AccessControlException: Client cannot authenticate
> via:[TOKEN, KERBEROS]
> 19/04/24 13:35:07 INFO RetryInvocationHandler: Exception while invoking
> getClusterMetrics of class ApplicationClientProtocolPBClientImpl over rm78
> after 10 fail over attempts. Trying to fail over immediately.
> java.io.IOException: Failed on local exception: java.io.IOException:
> org.apache.hadoop.security.AccessControlException: Client cannot authenticate
> via:[TOKEN, KERBEROS]; Host Details : local host is:
> "hadoopnode143/192.168.209.143"; destination host is:
> "hadoopmanager136":8032;
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)
> at org.apache.hadoop.ipc.Client.call(Client.java:1508)
> at org.apache.hadoop.ipc.Client.call(Client.java:1441)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:231)
> at com.sun.proxy.$Proxy25.getClusterMetrics(Unknown Source)
> at
> org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getClusterMetrics(ApplicationClientProtocolPBClientImpl.java:202)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:258)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:104)
> at com.sun.proxy.$Proxy26.getClusterMetrics(Unknown Source)
> at
> org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getYarnClusterMetrics(YarnClientImpl.java:483)
> at
> org.apache.spark.deploy.yarn.Client$$anonfun$submitApplication$1.apply(Client.scala:164)
> at
> org.apache.spark.deploy.yarn.Client$$anonfun$submitApplication$1.apply(Client.scala:164)
> at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
> at org.apache.spark.deploy.yarn.Client.logInfo(Client.scala:59)
> at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:163)
> at
> org.apache.spark.scheduler.cluster.YarnCl
[jira] [Updated] (SPARK-27554) org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS]
[
https://issues.apache.org/jira/browse/SPARK-27554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jepson updated SPARK-27554:
---
Description:
{code:java}
{code}
#basic env
JAVA_HOME=/usr/java/jdk1.8.0_181
HADOOP_CONF_DIR=/etc/hadoop/conf
export SPARK_HOME=/opt/software/spark/spark-2.4.2
#project env
KERBEROS_USER=h...@hadoop.com
KERBEROS_USER_KEYTAB=/etc/kerberos/hdfs.keytab
PROJECT_MAIN_CLASS=com.jy.dwexercise.OrderDeliveryModel
PROJECT_JAR=/opt/maintain/scripts/bas-1.0-SNAPSHOT.jar
#spark resource
DRIVER_MEMORY=4G
EXECUTORS_NUM=20
EXECUTOR_MEMORY=6G
EXECUTOR_VCORES=4
QUEQUE=idss
#submit job
/opt/software/spark/spark-2.4.2/bin/spark-submit \
--master yarn \
--deploy-mode cluster \
--queue ${QUEQUE} \
--driver-memory ${DRIVER_MEMORY} \
--num-executors ${EXECUTORS_NUM} \
--executor-memory ${EXECUTOR_MEMORY} \
--executor-cores ${EXECUTOR_VCORES} \
--principal ${KERBEROS_USER} \
--keytab ${KERBEROS_USER_KEYTAB} \
--class ${PROJECT_MAIN_CLASS} \
--conf "spark.yarn.archive=hdfs://jybigdata/spark/spark-archive-20190422.zip" \
--conf "spark.sql.autoBroadcastJoinThreshold=20971520" \
${PROJECT_JAR}
*ERROR log:*
19/04/24 13:35:07 INFO ConfiguredRMFailoverProxyProvider: Failing over to rm78
19/04/24 13:35:07 WARN UserGroupInformation: PriviledgedActionException
as:hdfs (auth:SIMPLE) cause:org.apache.hadoop.security.AccessControlException:
Client cannot authenticate via:[TOKEN, KERBEROS]
19/04/24 13:35:07 WARN Client: Exception encountered while connecting to the
server : org.apache.hadoop.security.AccessControlException: Client cannot
authenticate via:[TOKEN, KERBEROS]
19/04/24 13:35:07 WARN UserGroupInformation: PriviledgedActionException
as:hdfs (auth:SIMPLE) cause:java.io.IOException:
org.apache.hadoop.security.AccessControlException: Client cannot authenticate
via:[TOKEN, KERBEROS]
19/04/24 13:35:07 INFO RetryInvocationHandler: Exception while invoking
getClusterMetrics of class ApplicationClientProtocolPBClientImpl over rm78
after 10 fail over attempts. Trying to fail over immediately.
java.io.IOException: Failed on local exception: java.io.IOException:
org.apache.hadoop.security.AccessControlException: Client cannot authenticate
via:[TOKEN, KERBEROS]; Host Details : local host is:
"hadoopnode143/192.168.209.143"; destination host is: "hadoopmanager136":8032;
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)
at org.apache.hadoop.ipc.Client.call(Client.java:1508)
at org.apache.hadoop.ipc.Client.call(Client.java:1441)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:231)
at com.sun.proxy.$Proxy25.getClusterMetrics(Unknown Source)
at
org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getClusterMetrics(ApplicationClientProtocolPBClientImpl.java:202)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:258)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:104)
at com.sun.proxy.$Proxy26.getClusterMetrics(Unknown Source)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getYarnClusterMetrics(YarnClientImpl.java:483)
at
org.apache.spark.deploy.yarn.Client$$anonfun$submitApplication$1.apply(Client.scala:164)
at
org.apache.spark.deploy.yarn.Client$$anonfun$submitApplication$1.apply(Client.scala:164)
at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
at org.apache.spark.deploy.yarn.Client.logInfo(Client.scala:59)
at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:163)
at
org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:57)
at
org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:183)
at org.apache.spark.SparkContext.(SparkContext.scala:501)
at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2520)
at
org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:935)
at
org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:926)
at scala.Option.getOrElse(Option.scala:121)
at
org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:926)
at com.jiuye.dwexercise.OrderDeliveryModel$.(OrderDeliveryModel.scala:25)
at com.jiuye.dwexercise.OrderDeliveryModel$.(OrderDeliveryModel.scala)
at
com.jiuye.dwexercise.OrderDeliveryModel$$anonfun$5.apply(OrderDeliveryModel.scala:55)
at
com.jiuye.dwexercise.OrderDeliveryModel$$anonfun$5.apply(OrderDeliveryModel.scala:55)
at
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage49
[jira] [Created] (SPARK-27554) org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS]
Jepson created SPARK-27554:
--
Summary: org.apache.hadoop.security.AccessControlException: Client
cannot authenticate via:[TOKEN, KERBEROS]
Key: SPARK-27554
URL: https://issues.apache.org/jira/browse/SPARK-27554
Project: Spark
Issue Type: Bug
Components: Scheduler
Affects Versions: 2.4.1, 2.4.0
Environment: cdh5.16.1
spark2.4.0
spark2.4.1
spark2.4.2
Reporter: Jepson
{code:java}
{code}
#basic env
JAVA_HOME=/usr/java/jdk1.8.0_181
HADOOP_CONF_DIR=/etc/hadoop/conf
export SPARK_HOME=/opt/software/spark/spark-2.4.2
#project env
KERBEROS_USER=h...@hadoop.com
KERBEROS_USER_KEYTAB=/etc/kerberos/hdfs.keytab
PROJECT_MAIN_CLASS=com.jiuye.dwexercise.OrderDeliveryModel
PROJECT_JAR=/opt/maintain/scripts/bas-1.0-SNAPSHOT.jar
#spark resource
DRIVER_MEMORY=4G
EXECUTORS_NUM=20
EXECUTOR_MEMORY=6G
EXECUTOR_VCORES=4
QUEQUE=idss
#submit job
/opt/software/spark/spark-2.4.2/bin/spark-submit \
--master yarn \
--deploy-mode cluster \
--queue ${QUEQUE} \
--driver-memory ${DRIVER_MEMORY} \
--num-executors ${EXECUTORS_NUM} \
--executor-memory ${EXECUTOR_MEMORY} \
--executor-cores ${EXECUTOR_VCORES} \
--principal ${KERBEROS_USER} \
--keytab ${KERBEROS_USER_KEYTAB} \
--class ${PROJECT_MAIN_CLASS} \
--conf "spark.yarn.archive=hdfs://jybigdata/spark/spark-archive-20190422.zip" \
--conf "spark.sql.autoBroadcastJoinThreshold=20971520" \
${PROJECT_JAR}
*ERROR log:*
19/04/24 13:35:07 INFO ConfiguredRMFailoverProxyProvider: Failing over to rm78
19/04/24 13:35:07 WARN UserGroupInformation: PriviledgedActionException
as:hdfs (auth:SIMPLE) cause:org.apache.hadoop.security.AccessControlException:
Client cannot authenticate via:[TOKEN, KERBEROS]
19/04/24 13:35:07 WARN Client: Exception encountered while connecting to the
server : org.apache.hadoop.security.AccessControlException: Client cannot
authenticate via:[TOKEN, KERBEROS]
19/04/24 13:35:07 WARN UserGroupInformation: PriviledgedActionException
as:hdfs (auth:SIMPLE) cause:java.io.IOException:
org.apache.hadoop.security.AccessControlException: Client cannot authenticate
via:[TOKEN, KERBEROS]
19/04/24 13:35:07 INFO RetryInvocationHandler: Exception while invoking
getClusterMetrics of class ApplicationClientProtocolPBClientImpl over rm78
after 10 fail over attempts. Trying to fail over immediately.
java.io.IOException: Failed on local exception: java.io.IOException:
org.apache.hadoop.security.AccessControlException: Client cannot authenticate
via:[TOKEN, KERBEROS]; Host Details : local host is:
"hadoopnode143/192.168.209.143"; destination host is: "hadoopmanager136":8032;
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)
at org.apache.hadoop.ipc.Client.call(Client.java:1508)
at org.apache.hadoop.ipc.Client.call(Client.java:1441)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:231)
at com.sun.proxy.$Proxy25.getClusterMetrics(Unknown Source)
at
org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getClusterMetrics(ApplicationClientProtocolPBClientImpl.java:202)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:258)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:104)
at com.sun.proxy.$Proxy26.getClusterMetrics(Unknown Source)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getYarnClusterMetrics(YarnClientImpl.java:483)
at
org.apache.spark.deploy.yarn.Client$$anonfun$submitApplication$1.apply(Client.scala:164)
at
org.apache.spark.deploy.yarn.Client$$anonfun$submitApplication$1.apply(Client.scala:164)
at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
at org.apache.spark.deploy.yarn.Client.logInfo(Client.scala:59)
at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:163)
at
org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:57)
at
org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:183)
at org.apache.spark.SparkContext.(SparkContext.scala:501)
at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2520)
at
org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:935)
at
org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:926)
at scala.Option.getOrElse(Option.scala:121)
at
org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:926)
at com.jiuye.dwexercise.OrderDeliveryModel$.(OrderDeliveryModel.scala:25)
at com.jiuye.dwexercise.OrderDeliveryModel$.(OrderDeliveryModel.sca
[jira] [Commented] (SPARK-24579) SPIP: Standardize Optimized Data Exchange between Spark and DL/AI frameworks
[ https://issues.apache.org/jira/browse/SPARK-24579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16519956#comment-16519956 ] Jepson commented on SPARK-24579: Very nice. > SPIP: Standardize Optimized Data Exchange between Spark and DL/AI frameworks > > > Key: SPARK-24579 > URL: https://issues.apache.org/jira/browse/SPARK-24579 > Project: Spark > Issue Type: Epic > Components: ML, PySpark, SQL >Affects Versions: 3.0.0 >Reporter: Xiangrui Meng >Assignee: Xiangrui Meng >Priority: Major > Labels: Hydrogen > Attachments: [SPARK-24579] SPIP_ Standardize Optimized Data Exchange > between Apache Spark and DL%2FAI Frameworks .pdf > > > (see attached SPIP pdf for more details) > At the crossroads of big data and AI, we see both the success of Apache Spark > as a unified > analytics engine and the rise of AI frameworks like TensorFlow and Apache > MXNet (incubating). > Both big data and AI are indispensable components to drive business > innovation and there have > been multiple attempts from both communities to bring them together. > We saw efforts from AI community to implement data solutions for AI > frameworks like tf.data and tf.Transform. However, with 50+ data sources and > built-in SQL, DataFrames, and Streaming features, Spark remains the community > choice for big data. This is why we saw many efforts to integrate DL/AI > frameworks with Spark to leverage its power, for example, TFRecords data > source for Spark, TensorFlowOnSpark, TensorFrames, etc. As part of Project > Hydrogen, this SPIP takes a different angle at Spark + AI unification. > None of the integrations are possible without exchanging data between Spark > and external DL/AI frameworks. And the performance matters. However, there > doesn’t exist a standard way to exchange data and hence implementation and > performance optimization fall into pieces. For example, TensorFlowOnSpark > uses Hadoop InputFormat/OutputFormat for TensorFlow’s TFRecords to load and > save data and pass the RDD records to TensorFlow in Python. And TensorFrames > converts Spark DataFrames Rows to/from TensorFlow Tensors using TensorFlow’s > Java API. How can we reduce the complexity? > The proposal here is to standardize the data exchange interface (or format) > between Spark and DL/AI frameworks and optimize data conversion from/to this > interface. So DL/AI frameworks can leverage Spark to load data virtually > from anywhere without spending extra effort building complex data solutions, > like reading features from a production data warehouse or streaming model > inference. Spark users can use DL/AI frameworks without learning specific > data APIs implemented there. And developers from both sides can work on > performance optimizations independently given the interface itself doesn’t > introduce big overhead. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-22968) java.lang.IllegalStateException: No current assignment for partition kssh-2
[
https://issues.apache.org/jira/browse/SPARK-22968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16433536#comment-16433536
]
Jepson commented on SPARK-22968:
[~apachespark] Thank you very much.
> java.lang.IllegalStateException: No current assignment for partition kssh-2
> ---
>
> Key: SPARK-22968
> URL: https://issues.apache.org/jira/browse/SPARK-22968
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
>Affects Versions: 2.1.1
> Environment: Kafka: 0.10.0 (CDH5.12.0)
> Apache Spark 2.1.1
> Spark streaming+Kafka
>Reporter: Jepson
>Priority: Major
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> *Kafka Broker:*
> {code:java}
>message.max.bytes : 2621440
> {code}
> *Spark Streaming+Kafka Code:*
> {code:java}
> , "max.partition.fetch.bytes" -> (5242880: java.lang.Integer) //default:
> 1048576
> , "request.timeout.ms" -> (9: java.lang.Integer) //default: 6
> , "session.timeout.ms" -> (6: java.lang.Integer) //default: 3
> , "heartbeat.interval.ms" -> (5000: java.lang.Integer)
> , "receive.buffer.bytes" -> (10485760: java.lang.Integer)
> {code}
> *Error message:*
> {code:java}
> 8/01/05 09:48:27 INFO internals.ConsumerCoordinator: Revoking previously
> assigned partitions [kssh-7, kssh-4, kssh-3, kssh-6, kssh-5, kssh-0, kssh-2,
> kssh-1] for group use_a_separate_group_id_for_each_stream
> 18/01/05 09:48:27 INFO internals.AbstractCoordinator: (Re-)joining group
> use_a_separate_group_id_for_each_stream
> 18/01/05 09:48:27 INFO internals.AbstractCoordinator: Successfully joined
> group use_a_separate_group_id_for_each_stream with generation 4
> 18/01/05 09:48:27 INFO internals.ConsumerCoordinator: Setting newly assigned
> partitions [kssh-7, kssh-4, kssh-6, kssh-5] for group
> use_a_separate_group_id_for_each_stream
> 18/01/05 09:48:27 ERROR scheduler.JobScheduler: Error generating jobs for
> time 1515116907000 ms
> java.lang.IllegalStateException: No current assignment for partition kssh-2
> at
> org.apache.kafka.clients.consumer.internals.SubscriptionState.assignedState(SubscriptionState.java:231)
> at
> org.apache.kafka.clients.consumer.internals.SubscriptionState.needOffsetReset(SubscriptionState.java:295)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.seekToEnd(KafkaConsumer.java:1169)
> at
> org.apache.spark.streaming.kafka010.DirectKafkaInputDStream.latestOffsets(DirectKafkaInputDStream.scala:197)
> at
> org.apache.spark.streaming.kafka010.DirectKafkaInputDStream.compute(DirectKafkaInputDStream.scala:214)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
> at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
> at
> org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:335)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:333)
> at scala.Option.orElse(Option.scala:289)
> at
> org.apache.spark.streaming.dstream.DStream.getOrCompute(DStream.scala:330)
> at
> org.apache.spark.streaming.dstream.ForEachDStream.generateJob(ForEachDStream.scala:48)
> at
> org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:117)
> at
> org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:116)
> at
> scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
> at
> scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
> at
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
> at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
> at
> scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
> at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
> at
> org.apache.spark.streaming.DStreamGraph.generateJobs(DStreamGraph.scala:116)
> at
> org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:249)
> at
> org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:247)
> at scala.util.Try$.apply(Try.scala
[jira] [Commented] (SPARK-22968) java.lang.IllegalStateException: No current assignment for partition kssh-2
[
https://issues.apache.org/jira/browse/SPARK-22968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16420215#comment-16420215
]
Jepson commented on SPARK-22968:
Adjust these parameters,keep monitor.
“request.timeout.ms“ -> (21: java.lang.Integer)
“session.timeout.ms“ -> (18: java.lang.Integer)
“heartbeat.interval.ms“ -> (6000: java.lang.Integer)
> java.lang.IllegalStateException: No current assignment for partition kssh-2
> ---
>
> Key: SPARK-22968
> URL: https://issues.apache.org/jira/browse/SPARK-22968
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
>Affects Versions: 2.1.1
> Environment: Kafka: 0.10.0 (CDH5.12.0)
> Apache Spark 2.1.1
> Spark streaming+Kafka
>Reporter: Jepson
>Priority: Major
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> *Kafka Broker:*
> {code:java}
>message.max.bytes : 2621440
> {code}
> *Spark Streaming+Kafka Code:*
> {code:java}
> , "max.partition.fetch.bytes" -> (5242880: java.lang.Integer) //default:
> 1048576
> , "request.timeout.ms" -> (9: java.lang.Integer) //default: 6
> , "session.timeout.ms" -> (6: java.lang.Integer) //default: 3
> , "heartbeat.interval.ms" -> (5000: java.lang.Integer)
> , "receive.buffer.bytes" -> (10485760: java.lang.Integer)
> {code}
> *Error message:*
> {code:java}
> 8/01/05 09:48:27 INFO internals.ConsumerCoordinator: Revoking previously
> assigned partitions [kssh-7, kssh-4, kssh-3, kssh-6, kssh-5, kssh-0, kssh-2,
> kssh-1] for group use_a_separate_group_id_for_each_stream
> 18/01/05 09:48:27 INFO internals.AbstractCoordinator: (Re-)joining group
> use_a_separate_group_id_for_each_stream
> 18/01/05 09:48:27 INFO internals.AbstractCoordinator: Successfully joined
> group use_a_separate_group_id_for_each_stream with generation 4
> 18/01/05 09:48:27 INFO internals.ConsumerCoordinator: Setting newly assigned
> partitions [kssh-7, kssh-4, kssh-6, kssh-5] for group
> use_a_separate_group_id_for_each_stream
> 18/01/05 09:48:27 ERROR scheduler.JobScheduler: Error generating jobs for
> time 1515116907000 ms
> java.lang.IllegalStateException: No current assignment for partition kssh-2
> at
> org.apache.kafka.clients.consumer.internals.SubscriptionState.assignedState(SubscriptionState.java:231)
> at
> org.apache.kafka.clients.consumer.internals.SubscriptionState.needOffsetReset(SubscriptionState.java:295)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.seekToEnd(KafkaConsumer.java:1169)
> at
> org.apache.spark.streaming.kafka010.DirectKafkaInputDStream.latestOffsets(DirectKafkaInputDStream.scala:197)
> at
> org.apache.spark.streaming.kafka010.DirectKafkaInputDStream.compute(DirectKafkaInputDStream.scala:214)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
> at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
> at
> org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:335)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:333)
> at scala.Option.orElse(Option.scala:289)
> at
> org.apache.spark.streaming.dstream.DStream.getOrCompute(DStream.scala:330)
> at
> org.apache.spark.streaming.dstream.ForEachDStream.generateJob(ForEachDStream.scala:48)
> at
> org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:117)
> at
> org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:116)
> at
> scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
> at
> scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
> at
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
> at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
> at
> scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
> at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
> at
> org.apache.spark.streaming.DStreamGraph.generateJobs(DStreamGraph.scala:116)
> at
> org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerato
[jira] [Reopened] (SPARK-22968) java.lang.IllegalStateException: No current assignment for partition kssh-2
[
https://issues.apache.org/jira/browse/SPARK-22968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jepson reopened SPARK-22968:
This issue is reappear.
> java.lang.IllegalStateException: No current assignment for partition kssh-2
> ---
>
> Key: SPARK-22968
> URL: https://issues.apache.org/jira/browse/SPARK-22968
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
>Affects Versions: 2.1.1
> Environment: Kafka: 0.10.0 (CDH5.12.0)
> Apache Spark 2.1.1
> Spark streaming+Kafka
>Reporter: Jepson
>Priority: Major
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> *Kafka Broker:*
> {code:java}
>message.max.bytes : 2621440
> {code}
> *Spark Streaming+Kafka Code:*
> {code:java}
> , "max.partition.fetch.bytes" -> (5242880: java.lang.Integer) //default:
> 1048576
> , "request.timeout.ms" -> (9: java.lang.Integer) //default: 6
> , "session.timeout.ms" -> (6: java.lang.Integer) //default: 3
> , "heartbeat.interval.ms" -> (5000: java.lang.Integer)
> , "receive.buffer.bytes" -> (10485760: java.lang.Integer)
> {code}
> *Error message:*
> {code:java}
> 8/01/05 09:48:27 INFO internals.ConsumerCoordinator: Revoking previously
> assigned partitions [kssh-7, kssh-4, kssh-3, kssh-6, kssh-5, kssh-0, kssh-2,
> kssh-1] for group use_a_separate_group_id_for_each_stream
> 18/01/05 09:48:27 INFO internals.AbstractCoordinator: (Re-)joining group
> use_a_separate_group_id_for_each_stream
> 18/01/05 09:48:27 INFO internals.AbstractCoordinator: Successfully joined
> group use_a_separate_group_id_for_each_stream with generation 4
> 18/01/05 09:48:27 INFO internals.ConsumerCoordinator: Setting newly assigned
> partitions [kssh-7, kssh-4, kssh-6, kssh-5] for group
> use_a_separate_group_id_for_each_stream
> 18/01/05 09:48:27 ERROR scheduler.JobScheduler: Error generating jobs for
> time 1515116907000 ms
> java.lang.IllegalStateException: No current assignment for partition kssh-2
> at
> org.apache.kafka.clients.consumer.internals.SubscriptionState.assignedState(SubscriptionState.java:231)
> at
> org.apache.kafka.clients.consumer.internals.SubscriptionState.needOffsetReset(SubscriptionState.java:295)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.seekToEnd(KafkaConsumer.java:1169)
> at
> org.apache.spark.streaming.kafka010.DirectKafkaInputDStream.latestOffsets(DirectKafkaInputDStream.scala:197)
> at
> org.apache.spark.streaming.kafka010.DirectKafkaInputDStream.compute(DirectKafkaInputDStream.scala:214)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
> at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
> at
> org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:335)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:333)
> at scala.Option.orElse(Option.scala:289)
> at
> org.apache.spark.streaming.dstream.DStream.getOrCompute(DStream.scala:330)
> at
> org.apache.spark.streaming.dstream.ForEachDStream.generateJob(ForEachDStream.scala:48)
> at
> org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:117)
> at
> org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:116)
> at
> scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
> at
> scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
> at
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
> at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
> at
> scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
> at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
> at
> org.apache.spark.streaming.DStreamGraph.generateJobs(DStreamGraph.scala:116)
> at
> org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:249)
> at
> org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:247)
> at scala.util.Try$.apply(Try.scala:192)
> at
> org.apache.spark.streaming.scheduler.JobGenerato
[jira] [Resolved] (SPARK-22968) java.lang.IllegalStateException: No current assignment for partition kssh-2
[
https://issues.apache.org/jira/browse/SPARK-22968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jepson resolved SPARK-22968.
Resolution: Fixed
> java.lang.IllegalStateException: No current assignment for partition kssh-2
> ---
>
> Key: SPARK-22968
> URL: https://issues.apache.org/jira/browse/SPARK-22968
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
>Affects Versions: 2.1.1
> Environment: Kafka: 0.10.0 (CDH5.12.0)
> Apache Spark 2.1.1
> Spark streaming+Kafka
>Reporter: Jepson
>Priority: Major
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> *Kafka Broker:*
> {code:java}
>message.max.bytes : 2621440
> {code}
> *Spark Streaming+Kafka Code:*
> {code:java}
> , "max.partition.fetch.bytes" -> (5242880: java.lang.Integer) //default:
> 1048576
> , "request.timeout.ms" -> (9: java.lang.Integer) //default: 6
> , "session.timeout.ms" -> (6: java.lang.Integer) //default: 3
> , "heartbeat.interval.ms" -> (5000: java.lang.Integer)
> , "receive.buffer.bytes" -> (10485760: java.lang.Integer)
> {code}
> *Error message:*
> {code:java}
> 8/01/05 09:48:27 INFO internals.ConsumerCoordinator: Revoking previously
> assigned partitions [kssh-7, kssh-4, kssh-3, kssh-6, kssh-5, kssh-0, kssh-2,
> kssh-1] for group use_a_separate_group_id_for_each_stream
> 18/01/05 09:48:27 INFO internals.AbstractCoordinator: (Re-)joining group
> use_a_separate_group_id_for_each_stream
> 18/01/05 09:48:27 INFO internals.AbstractCoordinator: Successfully joined
> group use_a_separate_group_id_for_each_stream with generation 4
> 18/01/05 09:48:27 INFO internals.ConsumerCoordinator: Setting newly assigned
> partitions [kssh-7, kssh-4, kssh-6, kssh-5] for group
> use_a_separate_group_id_for_each_stream
> 18/01/05 09:48:27 ERROR scheduler.JobScheduler: Error generating jobs for
> time 1515116907000 ms
> java.lang.IllegalStateException: No current assignment for partition kssh-2
> at
> org.apache.kafka.clients.consumer.internals.SubscriptionState.assignedState(SubscriptionState.java:231)
> at
> org.apache.kafka.clients.consumer.internals.SubscriptionState.needOffsetReset(SubscriptionState.java:295)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.seekToEnd(KafkaConsumer.java:1169)
> at
> org.apache.spark.streaming.kafka010.DirectKafkaInputDStream.latestOffsets(DirectKafkaInputDStream.scala:197)
> at
> org.apache.spark.streaming.kafka010.DirectKafkaInputDStream.compute(DirectKafkaInputDStream.scala:214)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
> at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
> at
> org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:335)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:333)
> at scala.Option.orElse(Option.scala:289)
> at
> org.apache.spark.streaming.dstream.DStream.getOrCompute(DStream.scala:330)
> at
> org.apache.spark.streaming.dstream.ForEachDStream.generateJob(ForEachDStream.scala:48)
> at
> org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:117)
> at
> org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:116)
> at
> scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
> at
> scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
> at
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
> at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
> at
> scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
> at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
> at
> org.apache.spark.streaming.DStreamGraph.generateJobs(DStreamGraph.scala:116)
> at
> org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:249)
> at
> org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:247)
> at scala.util.Try$.apply(Try.scala:192)
> at
> org.apache.spark.streaming.scheduler.JobGenerator.g
[jira] [Commented] (SPARK-22968) java.lang.IllegalStateException: No current assignment for partition kssh-2
[
https://issues.apache.org/jira/browse/SPARK-22968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16379639#comment-16379639
]
Jepson commented on SPARK-22968:
It's been a long time.I close it first.
> java.lang.IllegalStateException: No current assignment for partition kssh-2
> ---
>
> Key: SPARK-22968
> URL: https://issues.apache.org/jira/browse/SPARK-22968
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
>Affects Versions: 2.1.1
> Environment: Kafka: 0.10.0 (CDH5.12.0)
> Apache Spark 2.1.1
> Spark streaming+Kafka
>Reporter: Jepson
>Priority: Major
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> *Kafka Broker:*
> {code:java}
>message.max.bytes : 2621440
> {code}
> *Spark Streaming+Kafka Code:*
> {code:java}
> , "max.partition.fetch.bytes" -> (5242880: java.lang.Integer) //default:
> 1048576
> , "request.timeout.ms" -> (9: java.lang.Integer) //default: 6
> , "session.timeout.ms" -> (6: java.lang.Integer) //default: 3
> , "heartbeat.interval.ms" -> (5000: java.lang.Integer)
> , "receive.buffer.bytes" -> (10485760: java.lang.Integer)
> {code}
> *Error message:*
> {code:java}
> 8/01/05 09:48:27 INFO internals.ConsumerCoordinator: Revoking previously
> assigned partitions [kssh-7, kssh-4, kssh-3, kssh-6, kssh-5, kssh-0, kssh-2,
> kssh-1] for group use_a_separate_group_id_for_each_stream
> 18/01/05 09:48:27 INFO internals.AbstractCoordinator: (Re-)joining group
> use_a_separate_group_id_for_each_stream
> 18/01/05 09:48:27 INFO internals.AbstractCoordinator: Successfully joined
> group use_a_separate_group_id_for_each_stream with generation 4
> 18/01/05 09:48:27 INFO internals.ConsumerCoordinator: Setting newly assigned
> partitions [kssh-7, kssh-4, kssh-6, kssh-5] for group
> use_a_separate_group_id_for_each_stream
> 18/01/05 09:48:27 ERROR scheduler.JobScheduler: Error generating jobs for
> time 1515116907000 ms
> java.lang.IllegalStateException: No current assignment for partition kssh-2
> at
> org.apache.kafka.clients.consumer.internals.SubscriptionState.assignedState(SubscriptionState.java:231)
> at
> org.apache.kafka.clients.consumer.internals.SubscriptionState.needOffsetReset(SubscriptionState.java:295)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.seekToEnd(KafkaConsumer.java:1169)
> at
> org.apache.spark.streaming.kafka010.DirectKafkaInputDStream.latestOffsets(DirectKafkaInputDStream.scala:197)
> at
> org.apache.spark.streaming.kafka010.DirectKafkaInputDStream.compute(DirectKafkaInputDStream.scala:214)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
> at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
> at
> org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:335)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:333)
> at scala.Option.orElse(Option.scala:289)
> at
> org.apache.spark.streaming.dstream.DStream.getOrCompute(DStream.scala:330)
> at
> org.apache.spark.streaming.dstream.ForEachDStream.generateJob(ForEachDStream.scala:48)
> at
> org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:117)
> at
> org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:116)
> at
> scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
> at
> scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
> at
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
> at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
> at
> scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
> at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
> at
> org.apache.spark.streaming.DStreamGraph.generateJobs(DStreamGraph.scala:116)
> at
> org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:249)
> at
> org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:247)
> at scala.util.Try$.apply(Try.sc
[jira] [Updated] (SPARK-22968) java.lang.IllegalStateException: No current assignment for partition kssh-2
[
https://issues.apache.org/jira/browse/SPARK-22968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jepson updated SPARK-22968:
---
Component/s: (was: Structured Streaming)
Spark Core
> java.lang.IllegalStateException: No current assignment for partition kssh-2
> ---
>
> Key: SPARK-22968
> URL: https://issues.apache.org/jira/browse/SPARK-22968
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
>Affects Versions: 2.1.1
> Environment: Kafka: 0.10.0 (CDH5.12.0)
> Apache Spark 2.1.1
> Spark streaming+Kafka
>Reporter: Jepson
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> *Kafka Broker:*
> {code:java}
>message.max.bytes : 2621440
> {code}
> *Spark Streaming+Kafka Code:*
> {code:java}
> , "max.partition.fetch.bytes" -> (5242880: java.lang.Integer) //default:
> 1048576
> , "request.timeout.ms" -> (9: java.lang.Integer) //default: 6
> , "session.timeout.ms" -> (6: java.lang.Integer) //default: 3
> , "heartbeat.interval.ms" -> (5000: java.lang.Integer)
> , "receive.buffer.bytes" -> (10485760: java.lang.Integer)
> {code}
> *Error message:*
> {code:java}
> 8/01/05 09:48:27 INFO internals.ConsumerCoordinator: Revoking previously
> assigned partitions [kssh-7, kssh-4, kssh-3, kssh-6, kssh-5, kssh-0, kssh-2,
> kssh-1] for group use_a_separate_group_id_for_each_stream
> 18/01/05 09:48:27 INFO internals.AbstractCoordinator: (Re-)joining group
> use_a_separate_group_id_for_each_stream
> 18/01/05 09:48:27 INFO internals.AbstractCoordinator: Successfully joined
> group use_a_separate_group_id_for_each_stream with generation 4
> 18/01/05 09:48:27 INFO internals.ConsumerCoordinator: Setting newly assigned
> partitions [kssh-7, kssh-4, kssh-6, kssh-5] for group
> use_a_separate_group_id_for_each_stream
> 18/01/05 09:48:27 ERROR scheduler.JobScheduler: Error generating jobs for
> time 1515116907000 ms
> java.lang.IllegalStateException: No current assignment for partition kssh-2
> at
> org.apache.kafka.clients.consumer.internals.SubscriptionState.assignedState(SubscriptionState.java:231)
> at
> org.apache.kafka.clients.consumer.internals.SubscriptionState.needOffsetReset(SubscriptionState.java:295)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.seekToEnd(KafkaConsumer.java:1169)
> at
> org.apache.spark.streaming.kafka010.DirectKafkaInputDStream.latestOffsets(DirectKafkaInputDStream.scala:197)
> at
> org.apache.spark.streaming.kafka010.DirectKafkaInputDStream.compute(DirectKafkaInputDStream.scala:214)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
> at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
> at
> org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:335)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:333)
> at scala.Option.orElse(Option.scala:289)
> at
> org.apache.spark.streaming.dstream.DStream.getOrCompute(DStream.scala:330)
> at
> org.apache.spark.streaming.dstream.ForEachDStream.generateJob(ForEachDStream.scala:48)
> at
> org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:117)
> at
> org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:116)
> at
> scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
> at
> scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
> at
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
> at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
> at
> scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
> at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
> at
> org.apache.spark.streaming.DStreamGraph.generateJobs(DStreamGraph.scala:116)
> at
> org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:249)
> at
> org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:247)
> at scala.util.Try$.apply(Try.scala:192)
> at
> org.apache.spark.streaming.
[jira] [Commented] (SPARK-22968) java.lang.IllegalStateException: No current assignment for partition kssh-2
[
https://issues.apache.org/jira/browse/SPARK-22968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16315552#comment-16315552
]
Jepson commented on SPARK-22968:
[~srowen] Thanks for quick response.
I turn up the parameter "max.partition.fetch.bytes" from 5242880 to10485760.
In the later days, i'll look at it again.
> java.lang.IllegalStateException: No current assignment for partition kssh-2
> ---
>
> Key: SPARK-22968
> URL: https://issues.apache.org/jira/browse/SPARK-22968
> Project: Spark
> Issue Type: Bug
> Components: Structured Streaming
>Affects Versions: 2.1.1
> Environment: Kafka: 0.10.0 (CDH5.12.0)
> Apache Spark 2.1.1
> Spark streaming+Kafka
>Reporter: Jepson
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> *Kafka Broker:*
> {code:java}
>message.max.bytes : 2621440
> {code}
> *Spark Streaming+Kafka Code:*
> {code:java}
> , "max.partition.fetch.bytes" -> (5242880: java.lang.Integer) //default:
> 1048576
> , "request.timeout.ms" -> (9: java.lang.Integer) //default: 6
> , "session.timeout.ms" -> (6: java.lang.Integer) //default: 3
> , "heartbeat.interval.ms" -> (5000: java.lang.Integer)
> , "receive.buffer.bytes" -> (10485760: java.lang.Integer)
> {code}
> *Error message:*
> {code:java}
> 8/01/05 09:48:27 INFO internals.ConsumerCoordinator: Revoking previously
> assigned partitions [kssh-7, kssh-4, kssh-3, kssh-6, kssh-5, kssh-0, kssh-2,
> kssh-1] for group use_a_separate_group_id_for_each_stream
> 18/01/05 09:48:27 INFO internals.AbstractCoordinator: (Re-)joining group
> use_a_separate_group_id_for_each_stream
> 18/01/05 09:48:27 INFO internals.AbstractCoordinator: Successfully joined
> group use_a_separate_group_id_for_each_stream with generation 4
> 18/01/05 09:48:27 INFO internals.ConsumerCoordinator: Setting newly assigned
> partitions [kssh-7, kssh-4, kssh-6, kssh-5] for group
> use_a_separate_group_id_for_each_stream
> 18/01/05 09:48:27 ERROR scheduler.JobScheduler: Error generating jobs for
> time 1515116907000 ms
> java.lang.IllegalStateException: No current assignment for partition kssh-2
> at
> org.apache.kafka.clients.consumer.internals.SubscriptionState.assignedState(SubscriptionState.java:231)
> at
> org.apache.kafka.clients.consumer.internals.SubscriptionState.needOffsetReset(SubscriptionState.java:295)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.seekToEnd(KafkaConsumer.java:1169)
> at
> org.apache.spark.streaming.kafka010.DirectKafkaInputDStream.latestOffsets(DirectKafkaInputDStream.scala:197)
> at
> org.apache.spark.streaming.kafka010.DirectKafkaInputDStream.compute(DirectKafkaInputDStream.scala:214)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
> at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
> at
> org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:335)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:333)
> at scala.Option.orElse(Option.scala:289)
> at
> org.apache.spark.streaming.dstream.DStream.getOrCompute(DStream.scala:330)
> at
> org.apache.spark.streaming.dstream.ForEachDStream.generateJob(ForEachDStream.scala:48)
> at
> org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:117)
> at
> org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:116)
> at
> scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
> at
> scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
> at
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
> at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
> at
> scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
> at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
> at
> org.apache.spark.streaming.DStreamGraph.generateJobs(DStreamGraph.scala:116)
> at
> org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:249)
> at
> org.apache.spark.streaming.sched
[jira] [Commented] (SPARK-19547) KafkaUtil throw 'No current assignment for partition' Exception
[
https://issues.apache.org/jira/browse/SPARK-19547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16312505#comment-16312505
]
Jepson commented on SPARK-19547:
I have encounter this problem too.
https://issues.apache.org/jira/browse/SPARK-22968
> KafkaUtil throw 'No current assignment for partition' Exception
> ---
>
> Key: SPARK-19547
> URL: https://issues.apache.org/jira/browse/SPARK-19547
> Project: Spark
> Issue Type: Question
> Components: DStreams
>Affects Versions: 1.6.1
>Reporter: wuchang
>
> Below is my scala code to create spark kafka stream:
> val kafkaParams = Map[String, Object](
> "bootstrap.servers" -> "server110:2181,server110:9092",
> "zookeeper" -> "server110:2181",
> "key.deserializer" -> classOf[StringDeserializer],
> "value.deserializer" -> classOf[StringDeserializer],
> "group.id" -> "example",
> "auto.offset.reset" -> "latest",
> "enable.auto.commit" -> (false: java.lang.Boolean)
> )
> val topics = Array("ABTest")
> val stream = KafkaUtils.createDirectStream[String, String](
> ssc,
> PreferConsistent,
> Subscribe[String, String](topics, kafkaParams)
> )
> But after run for 10 hours, it throws exceptions:
> 2017-02-10 10:56:20,000 INFO [JobGenerator] internals.ConsumerCoordinator:
> Revoking previously assigned partitions [ABTest-0, ABTest-1] for group example
> 2017-02-10 10:56:20,000 INFO [JobGenerator] internals.AbstractCoordinator:
> (Re-)joining group example
> 2017-02-10 10:56:20,011 INFO [JobGenerator] internals.AbstractCoordinator:
> (Re-)joining group example
> 2017-02-10 10:56:40,057 INFO [JobGenerator] internals.AbstractCoordinator:
> Successfully joined group example with generation 5
> 2017-02-10 10:56:40,058 INFO [JobGenerator] internals.ConsumerCoordinator:
> Setting newly assigned partitions [ABTest-1] for group example
> 2017-02-10 10:56:40,080 ERROR [JobScheduler] scheduler.JobScheduler: Error
> generating jobs for time 148669538 ms
> java.lang.IllegalStateException: No current assignment for partition ABTest-0
> at
> org.apache.kafka.clients.consumer.internals.SubscriptionState.assignedState(SubscriptionState.java:231)
> at
> org.apache.kafka.clients.consumer.internals.SubscriptionState.needOffsetReset(SubscriptionState.java:295)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.seekToEnd(KafkaConsumer.java:1169)
> at
> org.apache.spark.streaming.kafka010.DirectKafkaInputDStream.latestOffsets(DirectKafkaInputDStream.scala:179)
> at
> org.apache.spark.streaming.kafka010.DirectKafkaInputDStream.compute(DirectKafkaInputDStream.scala:196)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
> at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
> at
> org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:335)
> at
> org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:333)
> at scala.Option.orElse(Option.scala:289)
> at
> org.apache.spark.streaming.dstream.DStream.getOrCompute(DStream.scala:330)
> at
> org.apache.spark.streaming.dstream.ForEachDStream.generateJob(ForEachDStream.scala:48)
> at
> org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:117)
> at
> org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:116)
> at
> scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
> at
> scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
> at
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
> at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
> at
> scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
> at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
> at
> org.apache.spark.streaming.DStreamGraph.generateJobs(DStreamGraph.scala:116)
> at
> org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:248)
> at
> org.
[jira] [Updated] (SPARK-22968) java.lang.IllegalStateException: No current assignment for partition kssh-2
[
https://issues.apache.org/jira/browse/SPARK-22968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jepson updated SPARK-22968:
---
Description:
*Kafka Broker:*
{code:java}
message.max.bytes : 2621440
{code}
*Spark Streaming+Kafka Code:*
{code:java}
, "max.partition.fetch.bytes" -> (5242880: java.lang.Integer) //default: 1048576
, "request.timeout.ms" -> (9: java.lang.Integer) //default: 6
, "session.timeout.ms" -> (6: java.lang.Integer) //default: 3
, "heartbeat.interval.ms" -> (5000: java.lang.Integer)
, "receive.buffer.bytes" -> (10485760: java.lang.Integer)
{code}
*Error message:*
{code:java}
8/01/05 09:48:27 INFO internals.ConsumerCoordinator: Revoking previously
assigned partitions [kssh-7, kssh-4, kssh-3, kssh-6, kssh-5, kssh-0, kssh-2,
kssh-1] for group use_a_separate_group_id_for_each_stream
18/01/05 09:48:27 INFO internals.AbstractCoordinator: (Re-)joining group
use_a_separate_group_id_for_each_stream
18/01/05 09:48:27 INFO internals.AbstractCoordinator: Successfully joined group
use_a_separate_group_id_for_each_stream with generation 4
18/01/05 09:48:27 INFO internals.ConsumerCoordinator: Setting newly assigned
partitions [kssh-7, kssh-4, kssh-6, kssh-5] for group
use_a_separate_group_id_for_each_stream
18/01/05 09:48:27 ERROR scheduler.JobScheduler: Error generating jobs for time
1515116907000 ms
java.lang.IllegalStateException: No current assignment for partition kssh-2
at
org.apache.kafka.clients.consumer.internals.SubscriptionState.assignedState(SubscriptionState.java:231)
at
org.apache.kafka.clients.consumer.internals.SubscriptionState.needOffsetReset(SubscriptionState.java:295)
at
org.apache.kafka.clients.consumer.KafkaConsumer.seekToEnd(KafkaConsumer.java:1169)
at
org.apache.spark.streaming.kafka010.DirectKafkaInputDStream.latestOffsets(DirectKafkaInputDStream.scala:197)
at
org.apache.spark.streaming.kafka010.DirectKafkaInputDStream.compute(DirectKafkaInputDStream.scala:214)
at
org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
at
org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at
org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
at
org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
at
org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at
org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:335)
at
org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:333)
at scala.Option.orElse(Option.scala:289)
at
org.apache.spark.streaming.dstream.DStream.getOrCompute(DStream.scala:330)
at
org.apache.spark.streaming.dstream.ForEachDStream.generateJob(ForEachDStream.scala:48)
at
org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:117)
at
org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:116)
at
scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at
scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at
scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at
scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
at
org.apache.spark.streaming.DStreamGraph.generateJobs(DStreamGraph.scala:116)
at
org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:249)
at
org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:247)
at scala.util.Try$.apply(Try.scala:192)
at
org.apache.spark.streaming.scheduler.JobGenerator.generateJobs(JobGenerator.scala:247)
at
org.apache.spark.streaming.scheduler.JobGenerator.org$apache$spark$streaming$scheduler$JobGenerator$$processEvent(JobGenerator.scala:183)
at
org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:89)
at
org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:88)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
18/01/05 09:48:27 INFO yarn.ApplicationMaster: Final app status: SUCCEEDED,
exitCode: 0
18/01/05 09:48:27 INFO streaming.StreamingContext: Invoking
stop(stopGracefully=false) from shutdown hook
18/01/05 09:48:27 INFO scheduler.ReceiverT
[jira] [Updated] (SPARK-22968) java.lang.IllegalStateException: No current assignment for partition kssh-2
[
https://issues.apache.org/jira/browse/SPARK-22968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jepson updated SPARK-22968:
---
Description:
*Kafka Broker:*
message.max.bytes : 2621440
*Spark Streaming+Kafka Code:*
*, "max.partition.fetch.bytes" -> (5242880: java.lang.Integer) //default:
1048576*
, "request.timeout.ms" -> (9: java.lang.Integer) //default: 6
, "session.timeout.ms" -> (6: java.lang.Integer) //default: 3
, "heartbeat.interval.ms" -> (5000: java.lang.Integer)
*, "receive.buffer.bytes" -> (10485760: java.lang.Integer)*
*Error message:*
{code:java}
8/01/05 09:48:27 INFO internals.ConsumerCoordinator: Revoking previously
assigned partitions [kssh-7, kssh-4, kssh-3, kssh-6, kssh-5, kssh-0, kssh-2,
kssh-1] for group use_a_separate_group_id_for_each_stream
18/01/05 09:48:27 INFO internals.AbstractCoordinator: (Re-)joining group
use_a_separate_group_id_for_each_stream
18/01/05 09:48:27 INFO internals.AbstractCoordinator: Successfully joined group
use_a_separate_group_id_for_each_stream with generation 4
18/01/05 09:48:27 INFO internals.ConsumerCoordinator: Setting newly assigned
partitions [kssh-7, kssh-4, kssh-6, kssh-5] for group
use_a_separate_group_id_for_each_stream
18/01/05 09:48:27 ERROR scheduler.JobScheduler: Error generating jobs for time
1515116907000 ms
java.lang.IllegalStateException: No current assignment for partition kssh-2
at
org.apache.kafka.clients.consumer.internals.SubscriptionState.assignedState(SubscriptionState.java:231)
at
org.apache.kafka.clients.consumer.internals.SubscriptionState.needOffsetReset(SubscriptionState.java:295)
at
org.apache.kafka.clients.consumer.KafkaConsumer.seekToEnd(KafkaConsumer.java:1169)
at
org.apache.spark.streaming.kafka010.DirectKafkaInputDStream.latestOffsets(DirectKafkaInputDStream.scala:197)
at
org.apache.spark.streaming.kafka010.DirectKafkaInputDStream.compute(DirectKafkaInputDStream.scala:214)
at
org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
at
org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at
org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
at
org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
at
org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at
org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:335)
at
org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:333)
at scala.Option.orElse(Option.scala:289)
at
org.apache.spark.streaming.dstream.DStream.getOrCompute(DStream.scala:330)
at
org.apache.spark.streaming.dstream.ForEachDStream.generateJob(ForEachDStream.scala:48)
at
org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:117)
at
org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:116)
at
scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at
scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at
scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at
scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
at
org.apache.spark.streaming.DStreamGraph.generateJobs(DStreamGraph.scala:116)
at
org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:249)
at
org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:247)
at scala.util.Try$.apply(Try.scala:192)
at
org.apache.spark.streaming.scheduler.JobGenerator.generateJobs(JobGenerator.scala:247)
at
org.apache.spark.streaming.scheduler.JobGenerator.org$apache$spark$streaming$scheduler$JobGenerator$$processEvent(JobGenerator.scala:183)
at
org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:89)
at
org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:88)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
18/01/05 09:48:27 INFO yarn.ApplicationMaster: Final app status: SUCCEEDED,
exitCode: 0
18/01/05 09:48:27 INFO streaming.StreamingContext: Invoking
stop(stopGracefully=false) from shutdown hook
18/01/05 09:48:27 INFO scheduler.ReceiverTracker: ReceiverTracker stopped
[jira] [Updated] (SPARK-22968) java.lang.IllegalStateException: No current assignment for partition kssh-2
[
https://issues.apache.org/jira/browse/SPARK-22968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jepson updated SPARK-22968:
---
Description:
CDH-KAFKA:
message.max.bytes : 2621440
, "max.partition.fetch.bytes" -> (5242880: java.lang.Integer) //default: 1048576
, "request.timeout.ms" -> (9: java.lang.Integer) //default: 6
, "session.timeout.ms" -> (6: java.lang.Integer) //default: 3
, "heartbeat.interval.ms" -> (5000: java.lang.Integer)
, "receive.buffer.bytes" -> (10485760: java.lang.Integer)
*Error message:*
{code:java}
8/01/05 09:48:27 INFO internals.ConsumerCoordinator: Revoking previously
assigned partitions [kssh-7, kssh-4, kssh-3, kssh-6, kssh-5, kssh-0, kssh-2,
kssh-1] for group use_a_separate_group_id_for_each_stream
18/01/05 09:48:27 INFO internals.AbstractCoordinator: (Re-)joining group
use_a_separate_group_id_for_each_stream
18/01/05 09:48:27 INFO internals.AbstractCoordinator: Successfully joined group
use_a_separate_group_id_for_each_stream with generation 4
18/01/05 09:48:27 INFO internals.ConsumerCoordinator: Setting newly assigned
partitions [kssh-7, kssh-4, kssh-6, kssh-5] for group
use_a_separate_group_id_for_each_stream
18/01/05 09:48:27 ERROR scheduler.JobScheduler: Error generating jobs for time
1515116907000 ms
java.lang.IllegalStateException: No current assignment for partition kssh-2
at
org.apache.kafka.clients.consumer.internals.SubscriptionState.assignedState(SubscriptionState.java:231)
at
org.apache.kafka.clients.consumer.internals.SubscriptionState.needOffsetReset(SubscriptionState.java:295)
at
org.apache.kafka.clients.consumer.KafkaConsumer.seekToEnd(KafkaConsumer.java:1169)
at
org.apache.spark.streaming.kafka010.DirectKafkaInputDStream.latestOffsets(DirectKafkaInputDStream.scala:197)
at
org.apache.spark.streaming.kafka010.DirectKafkaInputDStream.compute(DirectKafkaInputDStream.scala:214)
at
org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
at
org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at
org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
at
org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
at
org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at
org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:335)
at
org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:333)
at scala.Option.orElse(Option.scala:289)
at
org.apache.spark.streaming.dstream.DStream.getOrCompute(DStream.scala:330)
at
org.apache.spark.streaming.dstream.ForEachDStream.generateJob(ForEachDStream.scala:48)
at
org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:117)
at
org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:116)
at
scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at
scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at
scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at
scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
at
org.apache.spark.streaming.DStreamGraph.generateJobs(DStreamGraph.scala:116)
at
org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:249)
at
org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:247)
at scala.util.Try$.apply(Try.scala:192)
at
org.apache.spark.streaming.scheduler.JobGenerator.generateJobs(JobGenerator.scala:247)
at
org.apache.spark.streaming.scheduler.JobGenerator.org$apache$spark$streaming$scheduler$JobGenerator$$processEvent(JobGenerator.scala:183)
at
org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:89)
at
org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:88)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
18/01/05 09:48:27 INFO yarn.ApplicationMaster: Final app status: SUCCEEDED,
exitCode: 0
18/01/05 09:48:27 INFO streaming.StreamingContext: Invoking
stop(stopGracefully=false) from shutdown hook
18/01/05 09:48:27 INFO scheduler.ReceiverTracker: ReceiverTracker stopped
18/01/05 09:48:27 INFO scheduler.JobGen
[jira] [Created] (SPARK-22968) java.lang.IllegalStateException: No current assignment for partition kssh-2
Jepson created SPARK-22968:
--
Summary: java.lang.IllegalStateException: No current assignment
for partition kssh-2
Key: SPARK-22968
URL: https://issues.apache.org/jira/browse/SPARK-22968
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 2.1.1
Environment: Kafka: 0.10.0 (CDH5.12.0)
Apache Spark 2.1.1
Spark streaming+Kafka
Reporter: Jepson
*Error message:*
{code:java}
8/01/05 09:48:27 INFO internals.ConsumerCoordinator: Revoking previously
assigned partitions [kssh-7, kssh-4, kssh-3, kssh-6, kssh-5, kssh-0, kssh-2,
kssh-1] for group use_a_separate_group_id_for_each_stream
18/01/05 09:48:27 INFO internals.AbstractCoordinator: (Re-)joining group
use_a_separate_group_id_for_each_stream
18/01/05 09:48:27 INFO internals.AbstractCoordinator: Successfully joined group
use_a_separate_group_id_for_each_stream with generation 4
18/01/05 09:48:27 INFO internals.ConsumerCoordinator: Setting newly assigned
partitions [kssh-7, kssh-4, kssh-6, kssh-5] for group
use_a_separate_group_id_for_each_stream
18/01/05 09:48:27 ERROR scheduler.JobScheduler: Error generating jobs for time
1515116907000 ms
java.lang.IllegalStateException: No current assignment for partition kssh-2
at
org.apache.kafka.clients.consumer.internals.SubscriptionState.assignedState(SubscriptionState.java:231)
at
org.apache.kafka.clients.consumer.internals.SubscriptionState.needOffsetReset(SubscriptionState.java:295)
at
org.apache.kafka.clients.consumer.KafkaConsumer.seekToEnd(KafkaConsumer.java:1169)
at
org.apache.spark.streaming.kafka010.DirectKafkaInputDStream.latestOffsets(DirectKafkaInputDStream.scala:197)
at
org.apache.spark.streaming.kafka010.DirectKafkaInputDStream.compute(DirectKafkaInputDStream.scala:214)
at
org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
at
org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at
org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
at
org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
at
org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
at
org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:335)
at
org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:333)
at scala.Option.orElse(Option.scala:289)
at
org.apache.spark.streaming.dstream.DStream.getOrCompute(DStream.scala:330)
at
org.apache.spark.streaming.dstream.ForEachDStream.generateJob(ForEachDStream.scala:48)
at
org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:117)
at
org.apache.spark.streaming.DStreamGraph$$anonfun$1.apply(DStreamGraph.scala:116)
at
scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at
scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at
scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at
scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
at
org.apache.spark.streaming.DStreamGraph.generateJobs(DStreamGraph.scala:116)
at
org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:249)
at
org.apache.spark.streaming.scheduler.JobGenerator$$anonfun$3.apply(JobGenerator.scala:247)
at scala.util.Try$.apply(Try.scala:192)
at
org.apache.spark.streaming.scheduler.JobGenerator.generateJobs(JobGenerator.scala:247)
at
org.apache.spark.streaming.scheduler.JobGenerator.org$apache$spark$streaming$scheduler$JobGenerator$$processEvent(JobGenerator.scala:183)
at
org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:89)
at
org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:88)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
18/01/05 09:48:27 INFO yarn.ApplicationMaster: Final app status: SUCCEEDED,
exitCode: 0
18/01/05 09:48:27 INFO streaming.StreamingContext: Invoking
stop(stopGracefully=false) from shutdown hook
18/01/05 09:48:27 INFO scheduler.ReceiverTracker: ReceiverTracker stopped
18/01/05 09:48:27 INFO scheduler.JobGenerator: Stopping JobGenerator immediately
18/01/05 09:48:27 INFO util.RecurringTim
[jira] [Comment Edited] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
[
https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136668#comment-16136668
]
Jepson edited comment on SPARK-21733 at 8/22/17 11:57 AM:
--
I add this parameter of "spark.network.timeout=42". And I test this, is ok.
*Specific command parameters:*
{code:java}
spark-submit \
--name KSSH-0.3 \
--class com.jiuye.KSSH \
--master yarn \
--deploy-mode cluster \
--driver-memory 2g \
--executor-memory 2g \
--executor-cores 1 \
--num-executors 8 \
--jars $(echo /opt/software/spark2.1.1/spark_on_yarn/libs/*.jar | tr ' ' ',') \
--conf "spark.ui.showConsoleProgress=false" \
--conf "spark.yarn.am.memory=1024m" \
--conf "spark.yarn.am.memoryOverhead=1024m" \
--conf "spark.yarn.driver.memoryOverhead=1024m" \
--conf "spark.yarn.executor.memoryOverhead=1024m" \
--conf "spark.yarn.am.extraJavaOptions=-XX:+UseG1GC -XX:MaxGCPauseMillis=300
-XX:InitiatingHeapOccupancyPercent=50 -XX:G1ReservePercent=20
-XX:+DisableExplicitGC -Dcdh.version=5.12.0" \
--conf "spark.driver.extraJavaOptions=-XX:+UseG1GC -XX:MaxGCPauseMillis=300
-XX:InitiatingHeapOccupancyPercent=50 -XX:G1ReservePercent=20
-XX:+DisableExplicitGC -Dcdh.version=5.12.0" \
--conf "spark.executor.extraJavaOptions=-XX:+UseG1GC -XX:MaxGCPauseMillis=300
-XX:InitiatingHeapOccupancyPercent=50 -XX:G1ReservePercent=20
-XX:+DisableExplicitGC -Dcdh.version=5.12.0" \
--conf "spark.streaming.backpressure.enabled=true" \
--conf "spark.streaming.kafka.maxRatePerPartition=1250" \
--conf "spark.locality.wait=1s" \
--conf "spark.shuffle.consolidateFiles=true" \
--conf "spark.executor.heartbeatInterval=36" \
--conf "spark.network.timeout=42" \
--conf "spark.serializer=org.apache.spark.serializer.KryoSerializer" \
--conf "spark.hadoop.fs.hdfs.impl.disable.cache=true" \
/opt/software/spark2.1.1/spark_on_yarn/KSSH-0.3.jar
{code}
was (Author: 1028344...@qq.com):
I add this parameter of "spark.network.timeout=42". And I test this, is ok.
*Specific command parameters:*
{code:java}
spark-submit \
--name KSSH-0.3 \
--class com.jiuye.KSSH \
--master yarn \
--deploy-mode cluster \
--driver-memory 2g \
--executor-memory 2g \
--executor-cores 1 \
--num-executors 8 \
--jars $(echo /opt/software/spark2.1.1/spark_on_yarn/libs/*.jar | tr ' ' ',') \
--conf "spark.ui.showConsoleProgress=false" \
--conf "spark.yarn.am.memory=1024m" \
--conf "spark.yarn.am.memoryOverhead=1024m" \
--conf "spark.yarn.driver.memoryOverhead=1024m" \
--conf "spark.yarn.executor.memoryOverhead=1024m" \
--conf "spark.yarn.am.extraJavaOptions=-XX:+UseG1GC -XX:MaxGCPauseMillis=300
-XX:InitiatingHeapOccupancyPercent=50 -XX:G1ReservePercent=20
-XX:+DisableExplicitGC -Dcdh.version=5.12.0" \
--conf "spark.driver.extraJavaOptions=-XX:+UseG1GC -XX:MaxGCPauseMillis=300
-XX:InitiatingHeapOccupancyPercent=50 -XX:G1ReservePercent=20
-XX:+DisableExplicitGC -Dcdh.version=5.12.0" \
--conf "spark.executor.extraJavaOptions=-XX:+UseG1GC -XX:MaxGCPauseMillis=300
-XX:InitiatingHeapOccupancyPercent=50 -XX:G1ReservePercent=20
-XX:+DisableExplicitGC -Dcdh.version=5.12.0" \
--conf "spark.streaming.backpressure.enabled=true" \
--conf "spark.streaming.kafka.maxRatePerPartition=1250" \
--conf "spark.locality.wait=1s" \
--conf "spark.shuffle.consolidateFiles=true" \
*--conf "spark.executor.heartbeatInterval=36" \
--conf "spark.network.timeout=42" \*
--conf "spark.serializer=org.apache.spark.serializer.KryoSerializer" \
--conf "spark.hadoop.fs.hdfs.impl.disable.cache=true" \
/opt/software/spark2.1.1/spark_on_yarn/KSSH-0.3.jar
{code}
> ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
> -
>
> Key: SPARK-21733
> URL: https://issues.apache.org/jira/browse/SPARK-21733
> Project: Spark
> Issue Type: Bug
> Components: DStreams
>Affects Versions: 2.1.1
> Environment: Apache Spark2.1.1
> CDH5.12.0 Yarn
>Reporter: Jepson
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> Kafka+Spark streaming ,throw these error:
> {code:java}
> 17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003_piece0 stored
> as bytes in memory (estimated size 1895.0 B, free 1643.2 MB)
> 17/08/15 09:34:14 INFO broadcast.TorrentBroadcast: Reading broadcast variable
> 8003 took 11 ms
> 17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003 stored as
> values in memory (estimated size 2.9 KB, free 1643.2 MB)
> 17/08/15 09:34:14 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the
> same as ending offset skipping kssh 5
> 17/08/15 09:34:14 INFO executor.Executor: Finished task 7.0 in stage 8003.0
> (TID 64178). 1740 bytes result sent to driver
> 17/08/15 09:34:21 INFO storage.BlockManager: Removing RDD 8002
> 17/08/15 09:34:21 INFO executor.CoarseGrainedExe
[jira] [Comment Edited] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
[
https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136668#comment-16136668
]
Jepson edited comment on SPARK-21733 at 8/22/17 11:40 AM:
--
I add this parameter of "spark.network.timeout=42". And I test this, is ok.
*Specific command parameters:*
{code:java}
spark-submit \
--name KSSH-0.3 \
--class com.jiuye.KSSH \
--master yarn \
--deploy-mode cluster \
--driver-memory 2g \
--executor-memory 2g \
--executor-cores 1 \
--num-executors 8 \
--jars $(echo /opt/software/spark2.1.1/spark_on_yarn/libs/*.jar | tr ' ' ',') \
--conf "spark.ui.showConsoleProgress=false" \
--conf "spark.yarn.am.memory=1024m" \
--conf "spark.yarn.am.memoryOverhead=1024m" \
--conf "spark.yarn.driver.memoryOverhead=1024m" \
--conf "spark.yarn.executor.memoryOverhead=1024m" \
--conf "spark.yarn.am.extraJavaOptions=-XX:+UseG1GC -XX:MaxGCPauseMillis=300
-XX:InitiatingHeapOccupancyPercent=50 -XX:G1ReservePercent=20
-XX:+DisableExplicitGC -Dcdh.version=5.12.0" \
--conf "spark.driver.extraJavaOptions=-XX:+UseG1GC -XX:MaxGCPauseMillis=300
-XX:InitiatingHeapOccupancyPercent=50 -XX:G1ReservePercent=20
-XX:+DisableExplicitGC -Dcdh.version=5.12.0" \
--conf "spark.executor.extraJavaOptions=-XX:+UseG1GC -XX:MaxGCPauseMillis=300
-XX:InitiatingHeapOccupancyPercent=50 -XX:G1ReservePercent=20
-XX:+DisableExplicitGC -Dcdh.version=5.12.0" \
--conf "spark.streaming.backpressure.enabled=true" \
--conf "spark.streaming.kafka.maxRatePerPartition=1250" \
--conf "spark.locality.wait=1s" \
--conf "spark.shuffle.consolidateFiles=true" \
*--conf "spark.executor.heartbeatInterval=36" \
--conf "spark.network.timeout=42" \*
--conf "spark.serializer=org.apache.spark.serializer.KryoSerializer" \
--conf "spark.hadoop.fs.hdfs.impl.disable.cache=true" \
/opt/software/spark2.1.1/spark_on_yarn/KSSH-0.3.jar
{code}
was (Author: 1028344...@qq.com):
I add this parameter of "spark.network.timeout=42". And I test this, is ok.
spark-submit \
--name KSSH-0.3 \
--class com.jiuye.KSSH \
--master yarn \
--deploy-mode cluster \
--driver-memory 2g \
--executor-memory 2g \
--executor-cores 1 \
--num-executors 8 \
--jars $(echo /opt/software/spark2.1.1/spark_on_yarn/libs/*.jar | tr ' ' ',') \
--conf "spark.ui.showConsoleProgress=false" \
--conf "spark.yarn.am.memory=1024m" \
--conf "spark.yarn.am.memoryOverhead=1024m" \
--conf "spark.yarn.driver.memoryOverhead=1024m" \
--conf "spark.yarn.executor.memoryOverhead=1024m" \
--conf "spark.yarn.am.extraJavaOptions=-XX:+UseG1GC -XX:MaxGCPauseMillis=300
-XX:InitiatingHeapOccupancyPercent=50 -XX:G1ReservePercent=20
-XX:+DisableExplicitGC -Dcdh.version=5.12.0" \
--conf "spark.driver.extraJavaOptions=-XX:+UseG1GC -XX:MaxGCPauseMillis=300
-XX:InitiatingHeapOccupancyPercent=50 -XX:G1ReservePercent=20
-XX:+DisableExplicitGC -Dcdh.version=5.12.0" \
--conf "spark.executor.extraJavaOptions=-XX:+UseG1GC -XX:MaxGCPauseMillis=300
-XX:InitiatingHeapOccupancyPercent=50 -XX:G1ReservePercent=20
-XX:+DisableExplicitGC -Dcdh.version=5.12.0" \
--conf "spark.streaming.backpressure.enabled=true" \
--conf "spark.streaming.kafka.maxRatePerPartition=1250" \
--conf "spark.locality.wait=1s" \
--conf "spark.shuffle.consolidateFiles=true" \
*--conf "spark.executor.heartbeatInterval=36" \
--conf "spark.network.timeout=42" \*
--conf "spark.serializer=org.apache.spark.serializer.KryoSerializer" \
--conf "spark.hadoop.fs.hdfs.impl.disable.cache=true" \
/opt/software/spark2.1.1/spark_on_yarn/KSSH-0.3.jar
> ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
> -
>
> Key: SPARK-21733
> URL: https://issues.apache.org/jira/browse/SPARK-21733
> Project: Spark
> Issue Type: Bug
> Components: DStreams
>Affects Versions: 2.1.1
> Environment: Apache Spark2.1.1
> CDH5.12.0 Yarn
>Reporter: Jepson
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> Kafka+Spark streaming ,throw these error:
> {code:java}
> 17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003_piece0 stored
> as bytes in memory (estimated size 1895.0 B, free 1643.2 MB)
> 17/08/15 09:34:14 INFO broadcast.TorrentBroadcast: Reading broadcast variable
> 8003 took 11 ms
> 17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003 stored as
> values in memory (estimated size 2.9 KB, free 1643.2 MB)
> 17/08/15 09:34:14 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the
> same as ending offset skipping kssh 5
> 17/08/15 09:34:14 INFO executor.Executor: Finished task 7.0 in stage 8003.0
> (TID 64178). 1740 bytes result sent to driver
> 17/08/15 09:34:21 INFO storage.BlockManager: Removing RDD 8002
> 17/08/15 09:34:21 INFO executor.CoarseGrainedExecutorBackend: Got assigned
> task 64186
> 17/08/15
[jira] [Commented] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
[
https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136668#comment-16136668
]
Jepson commented on SPARK-21733:
I add this parameter of "spark.network.timeout=42". And I test this, is ok.
spark-submit \
--name KSSH-0.3 \
--class com.jiuye.KSSH \
--master yarn \
--deploy-mode cluster \
--driver-memory 2g \
--executor-memory 2g \
--executor-cores 1 \
--num-executors 8 \
--jars $(echo /opt/software/spark2.1.1/spark_on_yarn/libs/*.jar | tr ' ' ',') \
--conf "spark.ui.showConsoleProgress=false" \
--conf "spark.yarn.am.memory=1024m" \
--conf "spark.yarn.am.memoryOverhead=1024m" \
--conf "spark.yarn.driver.memoryOverhead=1024m" \
--conf "spark.yarn.executor.memoryOverhead=1024m" \
--conf "spark.yarn.am.extraJavaOptions=-XX:+UseG1GC -XX:MaxGCPauseMillis=300
-XX:InitiatingHeapOccupancyPercent=50 -XX:G1ReservePercent=20
-XX:+DisableExplicitGC -Dcdh.version=5.12.0" \
--conf "spark.driver.extraJavaOptions=-XX:+UseG1GC -XX:MaxGCPauseMillis=300
-XX:InitiatingHeapOccupancyPercent=50 -XX:G1ReservePercent=20
-XX:+DisableExplicitGC -Dcdh.version=5.12.0" \
--conf "spark.executor.extraJavaOptions=-XX:+UseG1GC -XX:MaxGCPauseMillis=300
-XX:InitiatingHeapOccupancyPercent=50 -XX:G1ReservePercent=20
-XX:+DisableExplicitGC -Dcdh.version=5.12.0" \
--conf "spark.streaming.backpressure.enabled=true" \
--conf "spark.streaming.kafka.maxRatePerPartition=1250" \
--conf "spark.locality.wait=1s" \
--conf "spark.shuffle.consolidateFiles=true" \
*--conf "spark.executor.heartbeatInterval=36" \
--conf "spark.network.timeout=42" \*
--conf "spark.serializer=org.apache.spark.serializer.KryoSerializer" \
--conf "spark.hadoop.fs.hdfs.impl.disable.cache=true" \
/opt/software/spark2.1.1/spark_on_yarn/KSSH-0.3.jar
> ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
> -
>
> Key: SPARK-21733
> URL: https://issues.apache.org/jira/browse/SPARK-21733
> Project: Spark
> Issue Type: Bug
> Components: DStreams
>Affects Versions: 2.1.1
> Environment: Apache Spark2.1.1
> CDH5.12.0 Yarn
>Reporter: Jepson
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> Kafka+Spark streaming ,throw these error:
> {code:java}
> 17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003_piece0 stored
> as bytes in memory (estimated size 1895.0 B, free 1643.2 MB)
> 17/08/15 09:34:14 INFO broadcast.TorrentBroadcast: Reading broadcast variable
> 8003 took 11 ms
> 17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003 stored as
> values in memory (estimated size 2.9 KB, free 1643.2 MB)
> 17/08/15 09:34:14 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the
> same as ending offset skipping kssh 5
> 17/08/15 09:34:14 INFO executor.Executor: Finished task 7.0 in stage 8003.0
> (TID 64178). 1740 bytes result sent to driver
> 17/08/15 09:34:21 INFO storage.BlockManager: Removing RDD 8002
> 17/08/15 09:34:21 INFO executor.CoarseGrainedExecutorBackend: Got assigned
> task 64186
> 17/08/15 09:34:21 INFO executor.Executor: Running task 7.0 in stage 8004.0
> (TID 64186)
> 17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Started reading broadcast
> variable 8004
> 17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004_piece0 stored
> as bytes in memory (estimated size 1895.0 B, free 1643.2 MB)
> 17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Reading broadcast variable
> 8004 took 8 ms
> 17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004 stored as
> values in memory (estimated size 2.9 KB, free 1643.2 MB)
> 17/08/15 09:34:21 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the
> same as ending offset skipping kssh 5
> 17/08/15 09:34:21 INFO executor.Executor: Finished task 7.0 in stage 8004.0
> (TID 64186). 1740 bytes result sent to driver
> h3. 17/08/15 09:34:29 ERROR executor.CoarseGrainedExecutorBackend: RECEIVED
> SIGNAL TERM
> 17/08/15 09:34:29 INFO storage.DiskBlockManager: Shutdown hook called
> 17/08/15 09:34:29 INFO util.ShutdownHookManager: Shutdown hook called
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
[
https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136478#comment-16136478
]
Jepson edited comment on SPARK-21733 at 8/22/17 7:57 AM:
-
[~jerryshao] Sorry, I'll pay attention next time.
I check the log without any error detail. And the parameters with:
yarn.nodemanager.pmem-check-enabled
false
yarn.nodemanager.vmem-check-enabled
false
Then the executor still be kill.
was (Author: 1028344...@qq.com):
[~jerryshao] Sorry, I'll pay attention next time.
I check the log without any error detail. And the parameters with:
yarn.nodemanager.pmem-check-enabled
false
yarn.nodemanager.vmem-check-enabled
false
The the executor still be kill.
> ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
> -
>
> Key: SPARK-21733
> URL: https://issues.apache.org/jira/browse/SPARK-21733
> Project: Spark
> Issue Type: Bug
> Components: DStreams
>Affects Versions: 2.1.1
> Environment: Apache Spark2.1.1
> CDH5.12.0 Yarn
>Reporter: Jepson
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> Kafka+Spark streaming ,throw these error:
> {code:java}
> 17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003_piece0 stored
> as bytes in memory (estimated size 1895.0 B, free 1643.2 MB)
> 17/08/15 09:34:14 INFO broadcast.TorrentBroadcast: Reading broadcast variable
> 8003 took 11 ms
> 17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003 stored as
> values in memory (estimated size 2.9 KB, free 1643.2 MB)
> 17/08/15 09:34:14 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the
> same as ending offset skipping kssh 5
> 17/08/15 09:34:14 INFO executor.Executor: Finished task 7.0 in stage 8003.0
> (TID 64178). 1740 bytes result sent to driver
> 17/08/15 09:34:21 INFO storage.BlockManager: Removing RDD 8002
> 17/08/15 09:34:21 INFO executor.CoarseGrainedExecutorBackend: Got assigned
> task 64186
> 17/08/15 09:34:21 INFO executor.Executor: Running task 7.0 in stage 8004.0
> (TID 64186)
> 17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Started reading broadcast
> variable 8004
> 17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004_piece0 stored
> as bytes in memory (estimated size 1895.0 B, free 1643.2 MB)
> 17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Reading broadcast variable
> 8004 took 8 ms
> 17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004 stored as
> values in memory (estimated size 2.9 KB, free 1643.2 MB)
> 17/08/15 09:34:21 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the
> same as ending offset skipping kssh 5
> 17/08/15 09:34:21 INFO executor.Executor: Finished task 7.0 in stage 8004.0
> (TID 64186). 1740 bytes result sent to driver
> h3. 17/08/15 09:34:29 ERROR executor.CoarseGrainedExecutorBackend: RECEIVED
> SIGNAL TERM
> 17/08/15 09:34:29 INFO storage.DiskBlockManager: Shutdown hook called
> 17/08/15 09:34:29 INFO util.ShutdownHookManager: Shutdown hook called
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
[
https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136478#comment-16136478
]
Jepson commented on SPARK-21733:
[~jerryshao] Sorry, I'll pay attention next time.
I check the log without any error detail. And the parameters with:
yarn.nodemanager.pmem-check-enabled
false
yarn.nodemanager.vmem-check-enabled
false
The the executor still be kill.
> ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
> -
>
> Key: SPARK-21733
> URL: https://issues.apache.org/jira/browse/SPARK-21733
> Project: Spark
> Issue Type: Bug
> Components: DStreams
>Affects Versions: 2.1.1
> Environment: Apache Spark2.1.1
> CDH5.12.0 Yarn
>Reporter: Jepson
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> Kafka+Spark streaming ,throw these error:
> {code:java}
> 17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003_piece0 stored
> as bytes in memory (estimated size 1895.0 B, free 1643.2 MB)
> 17/08/15 09:34:14 INFO broadcast.TorrentBroadcast: Reading broadcast variable
> 8003 took 11 ms
> 17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003 stored as
> values in memory (estimated size 2.9 KB, free 1643.2 MB)
> 17/08/15 09:34:14 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the
> same as ending offset skipping kssh 5
> 17/08/15 09:34:14 INFO executor.Executor: Finished task 7.0 in stage 8003.0
> (TID 64178). 1740 bytes result sent to driver
> 17/08/15 09:34:21 INFO storage.BlockManager: Removing RDD 8002
> 17/08/15 09:34:21 INFO executor.CoarseGrainedExecutorBackend: Got assigned
> task 64186
> 17/08/15 09:34:21 INFO executor.Executor: Running task 7.0 in stage 8004.0
> (TID 64186)
> 17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Started reading broadcast
> variable 8004
> 17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004_piece0 stored
> as bytes in memory (estimated size 1895.0 B, free 1643.2 MB)
> 17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Reading broadcast variable
> 8004 took 8 ms
> 17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004 stored as
> values in memory (estimated size 2.9 KB, free 1643.2 MB)
> 17/08/15 09:34:21 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the
> same as ending offset skipping kssh 5
> 17/08/15 09:34:21 INFO executor.Executor: Finished task 7.0 in stage 8004.0
> (TID 64186). 1740 bytes result sent to driver
> h3. 17/08/15 09:34:29 ERROR executor.CoarseGrainedExecutorBackend: RECEIVED
> SIGNAL TERM
> 17/08/15 09:34:29 INFO storage.DiskBlockManager: Shutdown hook called
> 17/08/15 09:34:29 INFO util.ShutdownHookManager: Shutdown hook called
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
[ https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136400#comment-16136400 ] Jepson edited comment on SPARK-21733 at 8/22/17 7:47 AM: - [~jerryshao] Thanks for you quick reply. The spark streaming with kafka scala code : scc.start() scc.awaitTermination() *1.And I set the parameters:* --driver-memory 4g \ --executor-memory 4g \ --executor-cores 4 \ --num-executors 4 \ --conf "spark.yarn.am.memory=1024m" \ --conf "spark.yarn.am.memoryOverhead=1024m" \ --conf "spark.yarn.driver.memoryOverhead=4096m" \ --conf "spark.yarn.executor.memoryOverhead=4096m" \ *2.The error again.* was (Author: 1028344...@qq.com): [~jerryshao] Thanks for you quick reply. The spark streaming with kafka scala code : scc.start() scc.awaitTermination() *1.And I set the parameters:* --driver-memory 4g \ --executor-memory 4g \ --executor-cores 4 \ --num-executors 4 \ --conf "spark.yarn.am.memory=1024m" \ --conf "spark.yarn.am.memoryOverhead=1024m" \ --conf "spark.yarn.driver.memoryOverhead=4096m" \ --conf "spark.yarn.executor.memoryOverhead=4096m" \ *2.The error again:* 2017-08-22 15:06:32,082 *INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 5382 for container-id container_e65_1503383442059_0002_01_06: 573.9 MB of 8 GB physical memory used; 6.2 GB of 40 GB virtual memory used* 2017-08-22 15:06:33,026 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node's health-status : true, 2017-08-22 15:06:33,026 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_e65_1503383442059_0002_01_06, State: RUNNING, Diagnostics: , ExitStatus: -1000, ]] 2017-08-22 15:06:33,026 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2036704540) connection to hadoop37.jiuye/192.168.17.37:8031 from yarn sending #3069 2017-08-22 15:06:33,027 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2036704540) connection to hadoop37.jiuye/192.168.17.37:8031 from yarn got value #3069 2017-08-22 15:06:33,027 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: nodeHeartbeat took 1ms 2017-08-22 15:06:34,028 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node's health-status : true, 2017-08-22 15:06:34,028 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_e65_1503383442059_0002_01_06, State: RUNNING, Diagnostics: , ExitStatus: -1000, ]] 2017-08-22 15:06:34,028 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2036704540) connection to hadoop37.jiuye/192.168.17.37:8031 from yarn sending #3070 2017-08-22 15:06:34,029 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2036704540) connection to hadoop37.jiuye/192.168.17.37:8031 from yarn got value #3070 2017-08-22 15:06:34,029 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: nodeHeartbeat took 1ms 2017-08-22 15:06:35,030 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node's health-status : true, 2017-08-22 15:06:35,030 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_e65_1503383442059_0002_01_06, State: RUNNING, Diagnostics: , ExitStatus: -1000, ]] 2017-08-22 15:06:35,030 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2036704540) connection to hadoop37.jiuye/192.168.17.37:8031 from yarn sending #3071 2017-08-22 15:06:35,031 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2036704540) connection to hadoop37.jiuye/192.168.17.37:8031 from yarn got value #3071 2017-08-22 15:06:35,031 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: nodeHeartbeat took 1ms 2017-08-22 15:06:35,084 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Current ProcessTree list : [ 5382 ] 2017-08-22 15:06:35,084 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Constructing ProcessTree for : PID = 5382 ContainerId = container_e65_1503383442059_0002_01_06 2017-08-22 15:06:35,092 DEBUG org.apache.hadoop.yarn.util.ProcfsBasedProcessTree: [ 5382 5532 ] 2017-08-22 15:06:35,092 *INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 5382 for container-id container_e65_1503383442059_0002_01_06: 573.9 MB of 8 GB physical memory used; 6.2 GB of 40 GB virtual memory used* 2017-08-22 15:06:36,031 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node's health-status : true, 2017-08-22 15:06:36,032 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 1 cont
[jira] [Comment Edited] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
[ https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136400#comment-16136400 ] Jepson edited comment on SPARK-21733 at 8/22/17 7:18 AM: - [~jerryshao] Thanks for you quick reply. The spark streaming with kafka scala code : scc.start() scc.awaitTermination() *1.And I set the parameters:* --driver-memory 4g \ --executor-memory 4g \ --executor-cores 4 \ --num-executors 4 \ --conf "spark.yarn.am.memory=1024m" \ --conf "spark.yarn.am.memoryOverhead=1024m" \ --conf "spark.yarn.driver.memoryOverhead=4096m" \ --conf "spark.yarn.executor.memoryOverhead=4096m" \ *2.The error again:* 2017-08-22 15:06:32,082 *INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 5382 for container-id container_e65_1503383442059_0002_01_06: 573.9 MB of 8 GB physical memory used; 6.2 GB of 40 GB virtual memory used* 2017-08-22 15:06:33,026 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node's health-status : true, 2017-08-22 15:06:33,026 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_e65_1503383442059_0002_01_06, State: RUNNING, Diagnostics: , ExitStatus: -1000, ]] 2017-08-22 15:06:33,026 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2036704540) connection to hadoop37.jiuye/192.168.17.37:8031 from yarn sending #3069 2017-08-22 15:06:33,027 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2036704540) connection to hadoop37.jiuye/192.168.17.37:8031 from yarn got value #3069 2017-08-22 15:06:33,027 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: nodeHeartbeat took 1ms 2017-08-22 15:06:34,028 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node's health-status : true, 2017-08-22 15:06:34,028 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_e65_1503383442059_0002_01_06, State: RUNNING, Diagnostics: , ExitStatus: -1000, ]] 2017-08-22 15:06:34,028 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2036704540) connection to hadoop37.jiuye/192.168.17.37:8031 from yarn sending #3070 2017-08-22 15:06:34,029 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2036704540) connection to hadoop37.jiuye/192.168.17.37:8031 from yarn got value #3070 2017-08-22 15:06:34,029 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: nodeHeartbeat took 1ms 2017-08-22 15:06:35,030 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node's health-status : true, 2017-08-22 15:06:35,030 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_e65_1503383442059_0002_01_06, State: RUNNING, Diagnostics: , ExitStatus: -1000, ]] 2017-08-22 15:06:35,030 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2036704540) connection to hadoop37.jiuye/192.168.17.37:8031 from yarn sending #3071 2017-08-22 15:06:35,031 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2036704540) connection to hadoop37.jiuye/192.168.17.37:8031 from yarn got value #3071 2017-08-22 15:06:35,031 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: nodeHeartbeat took 1ms 2017-08-22 15:06:35,084 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Current ProcessTree list : [ 5382 ] 2017-08-22 15:06:35,084 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Constructing ProcessTree for : PID = 5382 ContainerId = container_e65_1503383442059_0002_01_06 2017-08-22 15:06:35,092 DEBUG org.apache.hadoop.yarn.util.ProcfsBasedProcessTree: [ 5382 5532 ] 2017-08-22 15:06:35,092 *INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 5382 for container-id container_e65_1503383442059_0002_01_06: 573.9 MB of 8 GB physical memory used; 6.2 GB of 40 GB virtual memory used* 2017-08-22 15:06:36,031 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node's health-status : true, 2017-08-22 15:06:36,032 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_e65_1503383442059_0002_01_06, State: RUNNING, Diagnostics: , ExitStatus: -1000, ]] 2017-08-22 15:06:36,032 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2036704540) connection to hadoop37.jiuye/192.168.17.37:8031 from yarn sending #3072 2017-08-22 15:06:36,032 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2036704540) connection to hadoop37.jiuye/192.168.17.37:8031 from yarn got value #3072 2017-08-22 15:06:36,033 DEBUG org.apache.
[jira] [Comment Edited] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
[
https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136400#comment-16136400
]
Jepson edited comment on SPARK-21733 at 8/22/17 6:44 AM:
-
[~jerryshao] Thanks for you quick reply.
The spark streaming with kafka scala code :
scc.start()
scc.awaitTermination()
{color:#f6c342} scc.stop(){color}
The "scc.stop()" is can't added?
was (Author: 1028344...@qq.com):
[~jerryshao] the spark streaming with kafka scala code :
scc.start()
scc.awaitTermination()
{color:#f6c342} scc.stop(){color}
The "scc.stop()" is can't added?
> ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
> -
>
> Key: SPARK-21733
> URL: https://issues.apache.org/jira/browse/SPARK-21733
> Project: Spark
> Issue Type: Bug
> Components: DStreams
>Affects Versions: 2.1.1
> Environment: Apache Spark2.1.1
> CDH5.12.0 Yarn
>Reporter: Jepson
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> Kafka+Spark streaming ,throw these error:
> {code:java}
> 17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003_piece0 stored
> as bytes in memory (estimated size 1895.0 B, free 1643.2 MB)
> 17/08/15 09:34:14 INFO broadcast.TorrentBroadcast: Reading broadcast variable
> 8003 took 11 ms
> 17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003 stored as
> values in memory (estimated size 2.9 KB, free 1643.2 MB)
> 17/08/15 09:34:14 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the
> same as ending offset skipping kssh 5
> 17/08/15 09:34:14 INFO executor.Executor: Finished task 7.0 in stage 8003.0
> (TID 64178). 1740 bytes result sent to driver
> 17/08/15 09:34:21 INFO storage.BlockManager: Removing RDD 8002
> 17/08/15 09:34:21 INFO executor.CoarseGrainedExecutorBackend: Got assigned
> task 64186
> 17/08/15 09:34:21 INFO executor.Executor: Running task 7.0 in stage 8004.0
> (TID 64186)
> 17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Started reading broadcast
> variable 8004
> 17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004_piece0 stored
> as bytes in memory (estimated size 1895.0 B, free 1643.2 MB)
> 17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Reading broadcast variable
> 8004 took 8 ms
> 17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004 stored as
> values in memory (estimated size 2.9 KB, free 1643.2 MB)
> 17/08/15 09:34:21 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the
> same as ending offset skipping kssh 5
> 17/08/15 09:34:21 INFO executor.Executor: Finished task 7.0 in stage 8004.0
> (TID 64186). 1740 bytes result sent to driver
> h3. 17/08/15 09:34:29 ERROR executor.CoarseGrainedExecutorBackend: RECEIVED
> SIGNAL TERM
> 17/08/15 09:34:29 INFO storage.DiskBlockManager: Shutdown hook called
> 17/08/15 09:34:29 INFO util.ShutdownHookManager: Shutdown hook called
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
[
https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136400#comment-16136400
]
Jepson commented on SPARK-21733:
[~jerryshao] the spark streaming with kafka scala code :
scc.start()
scc.awaitTermination()
{color:#f6c342} scc.stop(){color}
The "scc.stop()" is can't added?
> ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
> -
>
> Key: SPARK-21733
> URL: https://issues.apache.org/jira/browse/SPARK-21733
> Project: Spark
> Issue Type: Bug
> Components: DStreams
>Affects Versions: 2.1.1
> Environment: Apache Spark2.1.1
> CDH5.12.0 Yarn
>Reporter: Jepson
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> Kafka+Spark streaming ,throw these error:
> {code:java}
> 17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003_piece0 stored
> as bytes in memory (estimated size 1895.0 B, free 1643.2 MB)
> 17/08/15 09:34:14 INFO broadcast.TorrentBroadcast: Reading broadcast variable
> 8003 took 11 ms
> 17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003 stored as
> values in memory (estimated size 2.9 KB, free 1643.2 MB)
> 17/08/15 09:34:14 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the
> same as ending offset skipping kssh 5
> 17/08/15 09:34:14 INFO executor.Executor: Finished task 7.0 in stage 8003.0
> (TID 64178). 1740 bytes result sent to driver
> 17/08/15 09:34:21 INFO storage.BlockManager: Removing RDD 8002
> 17/08/15 09:34:21 INFO executor.CoarseGrainedExecutorBackend: Got assigned
> task 64186
> 17/08/15 09:34:21 INFO executor.Executor: Running task 7.0 in stage 8004.0
> (TID 64186)
> 17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Started reading broadcast
> variable 8004
> 17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004_piece0 stored
> as bytes in memory (estimated size 1895.0 B, free 1643.2 MB)
> 17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Reading broadcast variable
> 8004 took 8 ms
> 17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004 stored as
> values in memory (estimated size 2.9 KB, free 1643.2 MB)
> 17/08/15 09:34:21 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the
> same as ending offset skipping kssh 5
> 17/08/15 09:34:21 INFO executor.Executor: Finished task 7.0 in stage 8004.0
> (TID 64186). 1740 bytes result sent to driver
> h3. 17/08/15 09:34:29 ERROR executor.CoarseGrainedExecutorBackend: RECEIVED
> SIGNAL TERM
> 17/08/15 09:34:29 INFO storage.DiskBlockManager: Shutdown hook called
> 17/08/15 09:34:29 INFO util.ShutdownHookManager: Shutdown hook called
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
[
https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136260#comment-16136260
]
Jepson edited comment on SPARK-21733 at 8/22/17 4:32 AM:
-
*The nodemanager log detail:*
2017-08-22 11:20:07,984 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl:
Memory usage of ProcessTree 16747 for container-id
container_e56_1503371613444_0001_01_02: 586.8 MB of 3 GB physical memory
used; 4.5 GB of 6.3 GB virtual memory used
2017-08-22 11:20:07,984 DEBUG
org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl:
Constructing ProcessTree for : PID = 16766 ContainerId =
container_e56_1503371613444_0001_01_03
2017-08-22 11:20:07,992 DEBUG
org.apache.hadoop.yarn.util.ProcfsBasedProcessTree: [ 17066 16766 ]
2017-08-22 11:20:07,992 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl:
Memory usage of ProcessTree 16766 for container-id
container_e56_1503371613444_0001_01_03: 580.4 MB of 3 GB physical memory
used; 4.6 GB of 6.3 GB virtual memory used
2017-08-22 11:20:08,716 DEBUG
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node's
health-status : true,
2017-08-22 11:20:08,717 DEBUG
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 3
container statuses: [ContainerStatus: [ContainerId:
container_e56_1503371613444_0001_01_01, State: RUNNING, Diagnostics: ,
ExitStatus: -1000, ], ContainerStatus: [ContainerId:
container_e56_1503371613444_0001_01_02, State: RUNNING, Diagnostics: ,
ExitStatus: -1000, ], ContainerStatus: [ContainerId:
container_e56_1503371613444_0001_01_03, State: RUNNING, Diagnostics: ,
ExitStatus: -1000, ]]
2017-08-22 11:20:08,717 TRACE org.apache.hadoop.ipc.ProtobufRpcEngine: 102:
Call -> hadoop37.jiuye/192.168.17.37:8031: nodeHeartbeat {node_status { node_id
{ host: "hadoop44.jiuye" port: 8041 } response_id: 389 containersStatuses {
container_id { app_attempt_id { application_id { id: 1 cluster_timestamp:
1503371613444 } attemptId: 1 } id: 61572651155457 } state: C_RUNNING
diagnostics: "" exit_status: -1000 } containersStatuses { container_id {
app_attempt_id { application_id { id: 1 cluster_timestamp: 1503371613444 }
attemptId: 1 } id: 61572651155458 } state: C_RUNNING diagnostics: ""
exit_status: -1000 } containersStatuses { container_id { app_attempt_id {
application_id { id: 1 cluster_timestamp: 1503371613444 } attemptId: 1 } id:
61572651155459 } state: C_RUNNING diagnostics: "" exit_status: -1000 }
nodeHealthStatus { is_node_healthy: true health_report: ""
last_health_report_time: 1503371969299 } }
last_known_container_token_master_key { key_id: -966413074 bytes:
"a\021&\346gs\031n" } last_known_nm_token_master_key { key_id: -1126930838
bytes: "$j@\322\331dr`" }}
2017-08-22 11:20:08,717 DEBUG org.apache.hadoop.ipc.Client: IPC Client
(1778801068) connection to hadoop37.jiuye/192.168.17.37:8031 from yarn sending
#851
2017-08-22 11:20:08,720 DEBUG org.apache.hadoop.ipc.Client: IPC Client
(1778801068) connection to hadoop37.jiuye/192.168.17.37:8031 from yarn got
value #851
2017-08-22 11:20:08,720 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call:
nodeHeartbeat took 3ms
2017-08-22 11:20:08,720 TRACE org.apache.hadoop.ipc.ProtobufRpcEngine: 102:
Response <- hadoop37.jiuye/192.168.17.37:8031: nodeHeartbeat {response_id: 390
nodeAction: NORMAL containers_to_cleanup { app_attempt_id { application_id {
id: 1 cluster_timestamp: 1503371613444 } attemptId: 1 } id: 61572651155458 }
containers_to_cleanup { app_attempt_id { application_id { id: 1
cluster_timestamp: 1503371613444 } attemptId: 1 } id: 61572651155459 }
nextHeartBeatInterval: 1000}
2017-08-22 11:20:08,721 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher:
Dispatching the event
org.apache.hadoop.yarn.server.nodemanager.CMgrCompletedContainersEvent.EventType:
FINISH_CONTAINERS
*2017-08-22 11:20:08,722 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher:
Dispatching the event
org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerKillEvent.EventType:
KILL_CONTAINER*
2017-08-22 11:20:08,722 DEBUG
org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container:
Processing container_e56_1503371613444_0001_01_02 of type KILL_CONTAINER
2017-08-22 11:20:08,722 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container:
Container container_e56_1503371613444_0001_01_02 transitioned from RUNNING
to KILLING
2017-08-22 11:20:08,722 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher:
Dispatching the event
org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerKillEvent.EventType:
KILL_CONTAINER
2017-08-22 11:20:08,722 DEBUG
org.apache.hadoop.yarn.server.nodemanager.containermanager.containe
[jira] [Comment Edited] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
[
https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136260#comment-16136260
]
Jepson edited comment on SPARK-21733 at 8/22/17 4:29 AM:
-
*The nodemanager log detail:*
{code:java}
2017-08-22 11:20:07,984 DEBUG
org.apache.hadoop.yarn.util.ProcfsBasedProcessTree: [ 17040 16747 ]
2017-08-22 11:20:07,984 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl:
Memory usage of ProcessTree 16747 for container-id
container_e56_1503371613444_0001_01_02: 586.8 MB of 3 GB physical memory
used; 4.5 GB of 6.3 GB virtual memory used
2017-08-22 11:20:07,984 DEBUG
org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl:
Constructing ProcessTree for : PID = 16766 ContainerId =
container_e56_1503371613444_0001_01_03
2017-08-22 11:20:07,992 DEBUG
org.apache.hadoop.yarn.util.ProcfsBasedProcessTree: [ 17066 16766 ]
2017-08-22 11:20:07,992 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl:
Memory usage of ProcessTree 16766 for container-id
container_e56_1503371613444_0001_01_03: 580.4 MB of 3 GB physical memory
used; 4.6 GB of 6.3 GB virtual memory used
2017-08-22 11:20:08,716 DEBUG
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node's
health-status : true,
2017-08-22 11:20:08,717 DEBUG
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 3
container statuses: [ContainerStatus: [ContainerId:
container_e56_1503371613444_0001_01_01, State: RUNNING, Diagnostics: ,
ExitStatus: -1000, ], ContainerStatus: [ContainerId:
container_e56_1503371613444_0001_01_02, State: RUNNING, Diagnostics: ,
ExitStatus: -1000, ], ContainerStatus: [ContainerId:
container_e56_1503371613444_0001_01_03, State: RUNNING, Diagnostics: ,
ExitStatus: -1000, ]]
2017-08-22 11:20:08,717 TRACE org.apache.hadoop.ipc.ProtobufRpcEngine: 102:
Call -> hadoop37.jiuye/192.168.17.37:8031: nodeHeartbeat {node_status { node_id
{ host: "hadoop44.jiuye" port: 8041 } response_id: 389 containersStatuses {
container_id { app_attempt_id { application_id { id: 1 cluster_timestamp:
1503371613444 } attemptId: 1 } id: 61572651155457 } state: C_RUNNING
diagnostics: "" exit_status: -1000 } containersStatuses { container_id {
app_attempt_id { application_id { id: 1 cluster_timestamp: 1503371613444 }
attemptId: 1 } id: 61572651155458 } state: C_RUNNING diagnostics: ""
exit_status: -1000 } containersStatuses { container_id { app_attempt_id {
application_id { id: 1 cluster_timestamp: 1503371613444 } attemptId: 1 } id:
61572651155459 } state: C_RUNNING diagnostics: "" exit_status: -1000 }
nodeHealthStatus { is_node_healthy: true health_report: ""
last_health_report_time: 1503371969299 } }
last_known_container_token_master_key { key_id: -966413074 bytes:
"a\021&\346gs\031n" } last_known_nm_token_master_key { key_id: -1126930838
bytes: "$j@\322\331dr`" }}
2017-08-22 11:20:08,717 DEBUG org.apache.hadoop.ipc.Client: IPC Client
(1778801068) connection to hadoop37.jiuye/192.168.17.37:8031 from yarn sending
#851
2017-08-22 11:20:08,720 DEBUG org.apache.hadoop.ipc.Client: IPC Client
(1778801068) connection to hadoop37.jiuye/192.168.17.37:8031 from yarn got
value #851
2017-08-22 11:20:08,720 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call:
nodeHeartbeat took 3ms
2017-08-22 11:20:08,720 TRACE org.apache.hadoop.ipc.ProtobufRpcEngine: 102:
Response <- hadoop37.jiuye/192.168.17.37:8031: nodeHeartbeat {response_id: 390
nodeAction: NORMAL containers_to_cleanup { app_attempt_id { application_id {
id: 1 cluster_timestamp: 1503371613444 } attemptId: 1 } id: 61572651155458 }
containers_to_cleanup { app_attempt_id { application_id { id: 1
cluster_timestamp: 1503371613444 } attemptId: 1 } id: 61572651155459 }
nextHeartBeatInterval: 1000}
2017-08-22 11:20:08,721 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher:
Dispatching the event
org.apache.hadoop.yarn.server.nodemanager.CMgrCompletedContainersEvent.EventType:
FINISH_CONTAINERS
*{color:#f6c342}2017-08-22 11:20:08,722 DEBUG
org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event
org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerKillEvent.EventType:
KILL_CONTAINER{color}*
2017-08-22 11:20:08,722 DEBUG
org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container:
Processing container_e56_1503371613444_0001_01_02 of type KILL_CONTAINER
2017-08-22 11:20:08,722 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container:
Container container_e56_1503371613444_0001_01_02 transitioned from RUNNING
to KILLING
2017-08-22 11:20:08,722 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher:
Dispatching the event
org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerK
[jira] [Commented] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
[
https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136260#comment-16136260
]
Jepson commented on SPARK-21733:
*The nodemanager log detail:*
{code:java}
2017-08-22 11:20:07,984 DEBUG
org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl:
Constructing ProcessTree for : PID = 16766 ContainerId =
container_e56_1503371613444_0001_01_03
2017-08-22 11:20:07,992 DEBUG
org.apache.hadoop.yarn.util.ProcfsBasedProcessTree: [ 17066 16766 ]
2017-08-22 11:20:07,992 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl:
Memory usage of ProcessTree 16766 for container-id
container_e56_1503371613444_0001_01_03: 580.4 MB of 3 GB physical memory
used; 4.6 GB of 6.3 GB virtual memory used
2017-08-22 11:20:08,716 DEBUG
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node's
health-status : true,
2017-08-22 11:20:08,717 DEBUG
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 3
container statuses: [ContainerStatus: [ContainerId:
container_e56_1503371613444_0001_01_01, State: RUNNING, Diagnostics: ,
ExitStatus: -1000, ], ContainerStatus: [ContainerId:
container_e56_1503371613444_0001_01_02, State: RUNNING, Diagnostics: ,
ExitStatus: -1000, ], ContainerStatus: [ContainerId:
container_e56_1503371613444_0001_01_03, State: RUNNING, Diagnostics: ,
ExitStatus: -1000, ]]
2017-08-22 11:20:08,717 TRACE org.apache.hadoop.ipc.ProtobufRpcEngine: 102:
Call -> hadoop37.jiuye/192.168.17.37:8031: nodeHeartbeat {node_status { node_id
{ host: "hadoop44.jiuye" port: 8041 } response_id: 389 containersStatuses {
container_id { app_attempt_id { application_id { id: 1 cluster_timestamp:
1503371613444 } attemptId: 1 } id: 61572651155457 } state: C_RUNNING
diagnostics: "" exit_status: -1000 } containersStatuses { container_id {
app_attempt_id { application_id { id: 1 cluster_timestamp: 1503371613444 }
attemptId: 1 } id: 61572651155458 } state: C_RUNNING diagnostics: ""
exit_status: -1000 } containersStatuses { container_id { app_attempt_id {
application_id { id: 1 cluster_timestamp: 1503371613444 } attemptId: 1 } id:
61572651155459 } state: C_RUNNING diagnostics: "" exit_status: -1000 }
nodeHealthStatus { is_node_healthy: true health_report: ""
last_health_report_time: 1503371969299 } }
last_known_container_token_master_key { key_id: -966413074 bytes:
"a\021&\346gs\031n" } last_known_nm_token_master_key { key_id: -1126930838
bytes: "$j@\322\331dr`" }}
2017-08-22 11:20:08,717 DEBUG org.apache.hadoop.ipc.Client: IPC Client
(1778801068) connection to hadoop37.jiuye/192.168.17.37:8031 from yarn sending
#851
2017-08-22 11:20:08,720 DEBUG org.apache.hadoop.ipc.Client: IPC Client
(1778801068) connection to hadoop37.jiuye/192.168.17.37:8031 from yarn got
value #851
2017-08-22 11:20:08,720 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call:
nodeHeartbeat took 3ms
2017-08-22 11:20:08,720 TRACE org.apache.hadoop.ipc.ProtobufRpcEngine: 102:
Response <- hadoop37.jiuye/192.168.17.37:8031: nodeHeartbeat {response_id: 390
nodeAction: NORMAL containers_to_cleanup { app_attempt_id { application_id {
id: 1 cluster_timestamp: 1503371613444 } attemptId: 1 } id: 61572651155458 }
containers_to_cleanup { app_attempt_id { application_id { id: 1
cluster_timestamp: 1503371613444 } attemptId: 1 } id: 61572651155459 }
nextHeartBeatInterval: 1000}
2017-08-22 11:20:08,721 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher:
Dispatching the event
org.apache.hadoop.yarn.server.nodemanager.CMgrCompletedContainersEvent.EventType:
FINISH_CONTAINERS
2017-08-22 11:20:08,722 {color:#59afe1}DEBUG
org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event
org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerKillEvent.EventType:
KILL_CONTAINER{color}
2017-08-22 11:20:08,722 DEBUG
org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container:
Processing container_e56_1503371613444_0001_01_02 of type KILL_CONTAINER
2017-08-22 11:20:08,722 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container:
Container container_e56_1503371613444_0001_01_02 transitioned from RUNNING
to KILLING
2017-08-22 11:20:08,722 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher:
Dispatching the event
org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerKillEvent.EventType:
KILL_CONTAINER
2017-08-22 11:20:08,722 DEBUG
org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container:
Processing container_e56_1503371613444_0001_01_03 of type KILL_CONTAINER
2017-08-22 11:20:08,722 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container:
Container container_e56_1503371613444_0001_01_03 transitioned from RUNNING
to KILLING
2017-08-22 11:20:
[jira] [Issue Comment Deleted] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
[
https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jepson updated SPARK-21733:
---
Comment: was deleted
(was: I have resolve this issue.Thanks for [~jerryshao] and [~srowen]
spark-submit \
--class com.jiuye.KSSH \
--master yarn \
--deploy-mode cluster \
--driver-memory 3g \
--executor-memory 2g \
--executor-cores 2 \
--num-executors 8 \
--jars $(echo /opt/software/spark/spark_on_yarn/libs/*.jar | tr ' ' ',') \
--conf "spark.ui.showConsoleProgress=false" \
--conf "spark.driver.extraJavaOptions=-XX:MaxPermSize=2G
-XX:+UseConcMarkSweepGC" \
--conf "spark.executor.extraJavaOptions=-XX:+UseConcMarkSweepGC" \
--conf "spark.sql.tungsten.enabled=false" \
--conf "spark.sql.codegen=false" \
--conf "spark.sql.unsafe.enabled=false" \
--conf "spark.streaming.backpressure.enabled=true" \
--conf "spark.streaming.kafka.maxRatePerPartition=1000" \
--conf "spark.locality.wait=1s" \
--conf "spark.streaming.blockInterval=1500ms" \
--conf "spark.shuffle.consolidateFiles=true" \
--conf "spark.executor.heartbeatInterval=36" \
--conf "spark.yarn.am.memoryOverhead=512m" \
--conf "spark.yarn.driver.memoryOverhead=512m" \
--conf "spark.yarn.executor.memoryOverhead=512m" \
/opt/software/spark/spark_on_yarn/KSSH-0.3.jar)
> ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
> -
>
> Key: SPARK-21733
> URL: https://issues.apache.org/jira/browse/SPARK-21733
> Project: Spark
> Issue Type: Bug
> Components: DStreams
>Affects Versions: 2.1.1
> Environment: Apache Spark2.1.1
> CDH5.12.0 Yarn
>Reporter: Jepson
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> Kafka+Spark streaming ,throw these error:
> {code:java}
> 17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003_piece0 stored
> as bytes in memory (estimated size 1895.0 B, free 1643.2 MB)
> 17/08/15 09:34:14 INFO broadcast.TorrentBroadcast: Reading broadcast variable
> 8003 took 11 ms
> 17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003 stored as
> values in memory (estimated size 2.9 KB, free 1643.2 MB)
> 17/08/15 09:34:14 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the
> same as ending offset skipping kssh 5
> 17/08/15 09:34:14 INFO executor.Executor: Finished task 7.0 in stage 8003.0
> (TID 64178). 1740 bytes result sent to driver
> 17/08/15 09:34:21 INFO storage.BlockManager: Removing RDD 8002
> 17/08/15 09:34:21 INFO executor.CoarseGrainedExecutorBackend: Got assigned
> task 64186
> 17/08/15 09:34:21 INFO executor.Executor: Running task 7.0 in stage 8004.0
> (TID 64186)
> 17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Started reading broadcast
> variable 8004
> 17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004_piece0 stored
> as bytes in memory (estimated size 1895.0 B, free 1643.2 MB)
> 17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Reading broadcast variable
> 8004 took 8 ms
> 17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004 stored as
> values in memory (estimated size 2.9 KB, free 1643.2 MB)
> 17/08/15 09:34:21 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the
> same as ending offset skipping kssh 5
> 17/08/15 09:34:21 INFO executor.Executor: Finished task 7.0 in stage 8004.0
> (TID 64186). 1740 bytes result sent to driver
> h3. 17/08/15 09:34:29 ERROR executor.CoarseGrainedExecutorBackend: RECEIVED
> SIGNAL TERM
> 17/08/15 09:34:29 INFO storage.DiskBlockManager: Shutdown hook called
> 17/08/15 09:34:29 INFO util.ShutdownHookManager: Shutdown hook called
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Issue Comment Deleted] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
[
https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jepson updated SPARK-21733:
---
Comment: was deleted
(was: [~sowen], thank you for correcting me, I will notice it at the next time.)
> ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
> -
>
> Key: SPARK-21733
> URL: https://issues.apache.org/jira/browse/SPARK-21733
> Project: Spark
> Issue Type: Bug
> Components: DStreams
>Affects Versions: 2.1.1
> Environment: Apache Spark2.1.1
> CDH5.12.0 Yarn
>Reporter: Jepson
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> Kafka+Spark streaming ,throw these error:
> {code:java}
> 17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003_piece0 stored
> as bytes in memory (estimated size 1895.0 B, free 1643.2 MB)
> 17/08/15 09:34:14 INFO broadcast.TorrentBroadcast: Reading broadcast variable
> 8003 took 11 ms
> 17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003 stored as
> values in memory (estimated size 2.9 KB, free 1643.2 MB)
> 17/08/15 09:34:14 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the
> same as ending offset skipping kssh 5
> 17/08/15 09:34:14 INFO executor.Executor: Finished task 7.0 in stage 8003.0
> (TID 64178). 1740 bytes result sent to driver
> 17/08/15 09:34:21 INFO storage.BlockManager: Removing RDD 8002
> 17/08/15 09:34:21 INFO executor.CoarseGrainedExecutorBackend: Got assigned
> task 64186
> 17/08/15 09:34:21 INFO executor.Executor: Running task 7.0 in stage 8004.0
> (TID 64186)
> 17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Started reading broadcast
> variable 8004
> 17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004_piece0 stored
> as bytes in memory (estimated size 1895.0 B, free 1643.2 MB)
> 17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Reading broadcast variable
> 8004 took 8 ms
> 17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004 stored as
> values in memory (estimated size 2.9 KB, free 1643.2 MB)
> 17/08/15 09:34:21 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the
> same as ending offset skipping kssh 5
> 17/08/15 09:34:21 INFO executor.Executor: Finished task 7.0 in stage 8004.0
> (TID 64186). 1740 bytes result sent to driver
> h3. 17/08/15 09:34:29 ERROR executor.CoarseGrainedExecutorBackend: RECEIVED
> SIGNAL TERM
> 17/08/15 09:34:29 INFO storage.DiskBlockManager: Shutdown hook called
> 17/08/15 09:34:29 INFO util.ShutdownHookManager: Shutdown hook called
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Reopened] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
[
https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jepson reopened SPARK-21733:
*This error is happening again.*
> ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
> -
>
> Key: SPARK-21733
> URL: https://issues.apache.org/jira/browse/SPARK-21733
> Project: Spark
> Issue Type: Bug
> Components: DStreams
>Affects Versions: 2.1.1
> Environment: Apache Spark2.1.1
> CDH5.12.0 Yarn
>Reporter: Jepson
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> Kafka+Spark streaming ,throw these error:
> {code:java}
> 17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003_piece0 stored
> as bytes in memory (estimated size 1895.0 B, free 1643.2 MB)
> 17/08/15 09:34:14 INFO broadcast.TorrentBroadcast: Reading broadcast variable
> 8003 took 11 ms
> 17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003 stored as
> values in memory (estimated size 2.9 KB, free 1643.2 MB)
> 17/08/15 09:34:14 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the
> same as ending offset skipping kssh 5
> 17/08/15 09:34:14 INFO executor.Executor: Finished task 7.0 in stage 8003.0
> (TID 64178). 1740 bytes result sent to driver
> 17/08/15 09:34:21 INFO storage.BlockManager: Removing RDD 8002
> 17/08/15 09:34:21 INFO executor.CoarseGrainedExecutorBackend: Got assigned
> task 64186
> 17/08/15 09:34:21 INFO executor.Executor: Running task 7.0 in stage 8004.0
> (TID 64186)
> 17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Started reading broadcast
> variable 8004
> 17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004_piece0 stored
> as bytes in memory (estimated size 1895.0 B, free 1643.2 MB)
> 17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Reading broadcast variable
> 8004 took 8 ms
> 17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004 stored as
> values in memory (estimated size 2.9 KB, free 1643.2 MB)
> 17/08/15 09:34:21 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the
> same as ending offset skipping kssh 5
> 17/08/15 09:34:21 INFO executor.Executor: Finished task 7.0 in stage 8004.0
> (TID 64186). 1740 bytes result sent to driver
> h3. 17/08/15 09:34:29 ERROR executor.CoarseGrainedExecutorBackend: RECEIVED
> SIGNAL TERM
> 17/08/15 09:34:29 INFO storage.DiskBlockManager: Shutdown hook called
> 17/08/15 09:34:29 INFO util.ShutdownHookManager: Shutdown hook called
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
[
https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16126978#comment-16126978
]
Jepson commented on SPARK-21733:
[~sowen], thank you for correcting me, I will notice it at the next time.
> ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
> -
>
> Key: SPARK-21733
> URL: https://issues.apache.org/jira/browse/SPARK-21733
> Project: Spark
> Issue Type: Bug
> Components: DStreams
>Affects Versions: 2.1.1
> Environment: Apache Spark2.1.1
> CDH5.12.0 Yarn
>Reporter: Jepson
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> Kafka+Spark streaming ,throw these error:
> {code:java}
> 17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003_piece0 stored
> as bytes in memory (estimated size 1895.0 B, free 1643.2 MB)
> 17/08/15 09:34:14 INFO broadcast.TorrentBroadcast: Reading broadcast variable
> 8003 took 11 ms
> 17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003 stored as
> values in memory (estimated size 2.9 KB, free 1643.2 MB)
> 17/08/15 09:34:14 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the
> same as ending offset skipping kssh 5
> 17/08/15 09:34:14 INFO executor.Executor: Finished task 7.0 in stage 8003.0
> (TID 64178). 1740 bytes result sent to driver
> 17/08/15 09:34:21 INFO storage.BlockManager: Removing RDD 8002
> 17/08/15 09:34:21 INFO executor.CoarseGrainedExecutorBackend: Got assigned
> task 64186
> 17/08/15 09:34:21 INFO executor.Executor: Running task 7.0 in stage 8004.0
> (TID 64186)
> 17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Started reading broadcast
> variable 8004
> 17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004_piece0 stored
> as bytes in memory (estimated size 1895.0 B, free 1643.2 MB)
> 17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Reading broadcast variable
> 8004 took 8 ms
> 17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004 stored as
> values in memory (estimated size 2.9 KB, free 1643.2 MB)
> 17/08/15 09:34:21 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the
> same as ending offset skipping kssh 5
> 17/08/15 09:34:21 INFO executor.Executor: Finished task 7.0 in stage 8004.0
> (TID 64186). 1740 bytes result sent to driver
> h3. 17/08/15 09:34:29 ERROR executor.CoarseGrainedExecutorBackend: RECEIVED
> SIGNAL TERM
> 17/08/15 09:34:29 INFO storage.DiskBlockManager: Shutdown hook called
> 17/08/15 09:34:29 INFO util.ShutdownHookManager: Shutdown hook called
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
[
https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jepson resolved SPARK-21733.
Resolution: Fixed
Fix Version/s: 2.1.1
> ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
> -
>
> Key: SPARK-21733
> URL: https://issues.apache.org/jira/browse/SPARK-21733
> Project: Spark
> Issue Type: Bug
> Components: DStreams
>Affects Versions: 2.1.1
> Environment: Apache Spark2.1.1
> CDH5.12.0 Yarn
>Reporter: Jepson
> Fix For: 2.1.1
>
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> Kafka+Spark streaming ,throw these error:
> {code:java}
> 17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003_piece0 stored
> as bytes in memory (estimated size 1895.0 B, free 1643.2 MB)
> 17/08/15 09:34:14 INFO broadcast.TorrentBroadcast: Reading broadcast variable
> 8003 took 11 ms
> 17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003 stored as
> values in memory (estimated size 2.9 KB, free 1643.2 MB)
> 17/08/15 09:34:14 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the
> same as ending offset skipping kssh 5
> 17/08/15 09:34:14 INFO executor.Executor: Finished task 7.0 in stage 8003.0
> (TID 64178). 1740 bytes result sent to driver
> 17/08/15 09:34:21 INFO storage.BlockManager: Removing RDD 8002
> 17/08/15 09:34:21 INFO executor.CoarseGrainedExecutorBackend: Got assigned
> task 64186
> 17/08/15 09:34:21 INFO executor.Executor: Running task 7.0 in stage 8004.0
> (TID 64186)
> 17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Started reading broadcast
> variable 8004
> 17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004_piece0 stored
> as bytes in memory (estimated size 1895.0 B, free 1643.2 MB)
> 17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Reading broadcast variable
> 8004 took 8 ms
> 17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004 stored as
> values in memory (estimated size 2.9 KB, free 1643.2 MB)
> 17/08/15 09:34:21 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the
> same as ending offset skipping kssh 5
> 17/08/15 09:34:21 INFO executor.Executor: Finished task 7.0 in stage 8004.0
> (TID 64186). 1740 bytes result sent to driver
> h3. 17/08/15 09:34:29 ERROR executor.CoarseGrainedExecutorBackend: RECEIVED
> SIGNAL TERM
> 17/08/15 09:34:29 INFO storage.DiskBlockManager: Shutdown hook called
> 17/08/15 09:34:29 INFO util.ShutdownHookManager: Shutdown hook called
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
[
https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16126970#comment-16126970
]
Jepson commented on SPARK-21733:
I have resolve this issue.Thanks for [~jerryshao] and [~srowen]
spark-submit \
--class com.jiuye.KSSH \
--master yarn \
--deploy-mode cluster \
--driver-memory 3g \
--executor-memory 2g \
--executor-cores 2 \
--num-executors 8 \
--jars $(echo /opt/software/spark/spark_on_yarn/libs/*.jar | tr ' ' ',') \
--conf "spark.ui.showConsoleProgress=false" \
--conf "spark.driver.extraJavaOptions=-XX:MaxPermSize=2G
-XX:+UseConcMarkSweepGC" \
--conf "spark.executor.extraJavaOptions=-XX:+UseConcMarkSweepGC" \
--conf "spark.sql.tungsten.enabled=false" \
--conf "spark.sql.codegen=false" \
--conf "spark.sql.unsafe.enabled=false" \
--conf "spark.streaming.backpressure.enabled=true" \
--conf "spark.streaming.kafka.maxRatePerPartition=1000" \
--conf "spark.locality.wait=1s" \
--conf "spark.streaming.blockInterval=1500ms" \
--conf "spark.shuffle.consolidateFiles=true" \
--conf "spark.executor.heartbeatInterval=36" \
--conf "spark.yarn.am.memoryOverhead=512m" \
--conf "spark.yarn.driver.memoryOverhead=512m" \
--conf "spark.yarn.executor.memoryOverhead=512m" \
/opt/software/spark/spark_on_yarn/KSSH-0.3.jar
> ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
> -
>
> Key: SPARK-21733
> URL: https://issues.apache.org/jira/browse/SPARK-21733
> Project: Spark
> Issue Type: Bug
> Components: DStreams
>Affects Versions: 2.1.1
> Environment: Apache Spark2.1.1
> CDH5.12.0 Yarn
>Reporter: Jepson
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> Kafka+Spark streaming ,throw these error:
> {code:java}
> 17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003_piece0 stored
> as bytes in memory (estimated size 1895.0 B, free 1643.2 MB)
> 17/08/15 09:34:14 INFO broadcast.TorrentBroadcast: Reading broadcast variable
> 8003 took 11 ms
> 17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003 stored as
> values in memory (estimated size 2.9 KB, free 1643.2 MB)
> 17/08/15 09:34:14 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the
> same as ending offset skipping kssh 5
> 17/08/15 09:34:14 INFO executor.Executor: Finished task 7.0 in stage 8003.0
> (TID 64178). 1740 bytes result sent to driver
> 17/08/15 09:34:21 INFO storage.BlockManager: Removing RDD 8002
> 17/08/15 09:34:21 INFO executor.CoarseGrainedExecutorBackend: Got assigned
> task 64186
> 17/08/15 09:34:21 INFO executor.Executor: Running task 7.0 in stage 8004.0
> (TID 64186)
> 17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Started reading broadcast
> variable 8004
> 17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004_piece0 stored
> as bytes in memory (estimated size 1895.0 B, free 1643.2 MB)
> 17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Reading broadcast variable
> 8004 took 8 ms
> 17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004 stored as
> values in memory (estimated size 2.9 KB, free 1643.2 MB)
> 17/08/15 09:34:21 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the
> same as ending offset skipping kssh 5
> 17/08/15 09:34:21 INFO executor.Executor: Finished task 7.0 in stage 8004.0
> (TID 64186). 1740 bytes result sent to driver
> h3. 17/08/15 09:34:29 ERROR executor.CoarseGrainedExecutorBackend: RECEIVED
> SIGNAL TERM
> 17/08/15 09:34:29 INFO storage.DiskBlockManager: Shutdown hook called
> 17/08/15 09:34:29 INFO util.ShutdownHookManager: Shutdown hook called
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
[
https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jepson updated SPARK-21733:
---
Description:
Kafka+Spark streaming ,throw these error:
{code:java}
17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003_piece0 stored
as bytes in memory (estimated size 1895.0 B, free 1643.2 MB)
17/08/15 09:34:14 INFO broadcast.TorrentBroadcast: Reading broadcast variable
8003 took 11 ms
17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003 stored as
values in memory (estimated size 2.9 KB, free 1643.2 MB)
17/08/15 09:34:14 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the same
as ending offset skipping kssh 5
17/08/15 09:34:14 INFO executor.Executor: Finished task 7.0 in stage 8003.0
(TID 64178). 1740 bytes result sent to driver
17/08/15 09:34:21 INFO storage.BlockManager: Removing RDD 8002
17/08/15 09:34:21 INFO executor.CoarseGrainedExecutorBackend: Got assigned task
64186
17/08/15 09:34:21 INFO executor.Executor: Running task 7.0 in stage 8004.0 (TID
64186)
17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Started reading broadcast
variable 8004
17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004_piece0 stored
as bytes in memory (estimated size 1895.0 B, free 1643.2 MB)
17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Reading broadcast variable
8004 took 8 ms
17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004 stored as
values in memory (estimated size 2.9 KB, free 1643.2 MB)
17/08/15 09:34:21 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the same
as ending offset skipping kssh 5
17/08/15 09:34:21 INFO executor.Executor: Finished task 7.0 in stage 8004.0
(TID 64186). 1740 bytes result sent to driver
h3. 17/08/15 09:34:29 ERROR executor.CoarseGrainedExecutorBackend: RECEIVED
SIGNAL TERM
17/08/15 09:34:29 INFO storage.DiskBlockManager: Shutdown hook called
17/08/15 09:34:29 INFO util.ShutdownHookManager: Shutdown hook called
{code}
was:
Kafka+Spark streaming ,throw these error:
{code:java}
17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003_piece0 stored
as bytes in memory (estimated size 1895.0 B, free 1643.2 MB)
17/08/15 09:34:14 INFO broadcast.TorrentBroadcast: Reading broadcast variable
8003 took 11 ms
17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003 stored as
values in memory (estimated size 2.9 KB, free 1643.2 MB)
17/08/15 09:34:14 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the same
as ending offset skipping kssh 5
17/08/15 09:34:14 INFO executor.Executor: Finished task 7.0 in stage 8003.0
(TID 64178). 1740 bytes result sent to driver
17/08/15 09:34:21 INFO storage.BlockManager: Removing RDD 8002
17/08/15 09:34:21 INFO executor.CoarseGrainedExecutorBackend: Got assigned task
64186
17/08/15 09:34:21 INFO executor.Executor: Running task 7.0 in stage 8004.0 (TID
64186)
17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Started reading broadcast
variable 8004
17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004_piece0 stored
as bytes in memory (estimated size 1895.0 B, free 1643.2 MB)
17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Reading broadcast variable
8004 took 8 ms
17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004 stored as
values in memory (estimated size 2.9 KB, free 1643.2 MB)
17/08/15 09:34:21 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the same
as ending offset skipping kssh 5
17/08/15 09:34:21 INFO executor.Executor: Finished task 7.0 in stage 8004.0
(TID 64186). 1740 bytes result sent to driver
17/08/15 09:34:29 {color:#59afe1}ERROR executor.CoarseGrainedExecutorBackend:
RECEIVED SIGNAL TERM{color}
17/08/15 09:34:29 INFO storage.DiskBlockManager: Shutdown hook called
17/08/15 09:34:29 INFO util.ShutdownHookManager: Shutdown hook called
{code}
> ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
> -
>
> Key: SPARK-21733
> URL: https://issues.apache.org/jira/browse/SPARK-21733
> Project: Spark
> Issue Type: Bug
> Components: DStreams
>Affects Versions: 2.1.1
> Environment: Apache Spark2.1.1
> CDH5.12.0 Yarn
>Reporter: Jepson
> Labels: patch
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> Kafka+Spark streaming ,throw these error:
> {code:java}
> 17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003_piece0 stored
> as bytes in memory (estimated size 1895.0 B, free 1643.2 MB)
> 17/08/15 09:34:14 INFO broadcast.TorrentBroadcast: Reading broadcast variable
> 8003 took 11 ms
> 17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003 stored as
> values in memory (estimated size 2.9 KB, free 1643.2 MB)
> 17/08/15 09:34:14 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the
> same as ending offset skipping kssh 5
> 17/08/15 09:34:1
[jira] [Updated] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
[
https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jepson updated SPARK-21733:
---
Description:
Kafka+Spark streaming ,throw these error:
{code:java}
17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003_piece0 stored
as bytes in memory (estimated size 1895.0 B, free 1643.2 MB)
17/08/15 09:34:14 INFO broadcast.TorrentBroadcast: Reading broadcast variable
8003 took 11 ms
17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003 stored as
values in memory (estimated size 2.9 KB, free 1643.2 MB)
17/08/15 09:34:14 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the same
as ending offset skipping kssh 5
17/08/15 09:34:14 INFO executor.Executor: Finished task 7.0 in stage 8003.0
(TID 64178). 1740 bytes result sent to driver
17/08/15 09:34:21 INFO storage.BlockManager: Removing RDD 8002
17/08/15 09:34:21 INFO executor.CoarseGrainedExecutorBackend: Got assigned task
64186
17/08/15 09:34:21 INFO executor.Executor: Running task 7.0 in stage 8004.0 (TID
64186)
17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Started reading broadcast
variable 8004
17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004_piece0 stored
as bytes in memory (estimated size 1895.0 B, free 1643.2 MB)
17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Reading broadcast variable
8004 took 8 ms
17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004 stored as
values in memory (estimated size 2.9 KB, free 1643.2 MB)
17/08/15 09:34:21 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the same
as ending offset skipping kssh 5
17/08/15 09:34:21 INFO executor.Executor: Finished task 7.0 in stage 8004.0
(TID 64186). 1740 bytes result sent to driver
17/08/15 09:34:29 {color:#59afe1}ERROR executor.CoarseGrainedExecutorBackend:
RECEIVED SIGNAL TERM{color}
17/08/15 09:34:29 INFO storage.DiskBlockManager: Shutdown hook called
17/08/15 09:34:29 INFO util.ShutdownHookManager: Shutdown hook called
{code}
was:
Kafka+Spark streaming ,throw these error:
{code:java}
17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003_piece0 stored
as bytes in memory (estimated size 1895.0 B, free 1643.2 MB)
17/08/15 09:34:14 INFO broadcast.TorrentBroadcast: Reading broadcast variable
8003 took 11 ms
17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003 stored as
values in memory (estimated size 2.9 KB, free 1643.2 MB)
17/08/15 09:34:14 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the same
as ending offset skipping kssh 5
17/08/15 09:34:14 INFO executor.Executor: Finished task 7.0 in stage 8003.0
(TID 64178). 1740 bytes result sent to driver
17/08/15 09:34:21 INFO storage.BlockManager: Removing RDD 8002
17/08/15 09:34:21 INFO executor.CoarseGrainedExecutorBackend: Got assigned task
64186
17/08/15 09:34:21 INFO executor.Executor: Running task 7.0 in stage 8004.0 (TID
64186)
17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Started reading broadcast
variable 8004
17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004_piece0 stored
as bytes in memory (estimated size 1895.0 B, free 1643.2 MB)
17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Reading broadcast variable
8004 took 8 ms
17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004 stored as
values in memory (estimated size 2.9 KB, free 1643.2 MB)
17/08/15 09:34:21 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the same
as ending offset skipping kssh 5
17/08/15 09:34:21 INFO executor.Executor: Finished task 7.0 in stage 8004.0
(TID 64186). 1740 bytes result sent to driver
17/08/15 09:34:29 ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL
TERM
17/08/15 09:34:29 INFO storage.DiskBlockManager: Shutdown hook called
17/08/15 09:34:29 INFO util.ShutdownHookManager: Shutdown hook called
{code}
> ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
> -
>
> Key: SPARK-21733
> URL: https://issues.apache.org/jira/browse/SPARK-21733
> Project: Spark
> Issue Type: Bug
> Components: DStreams
>Affects Versions: 2.1.1
> Environment: Apache Spark2.1.1
> CDH5.12.0 Yarn
>Reporter: Jepson
> Labels: patch
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> Kafka+Spark streaming ,throw these error:
> {code:java}
> 17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003_piece0 stored
> as bytes in memory (estimated size 1895.0 B, free 1643.2 MB)
> 17/08/15 09:34:14 INFO broadcast.TorrentBroadcast: Reading broadcast variable
> 8003 took 11 ms
> 17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003 stored as
> values in memory (estimated size 2.9 KB, free 1643.2 MB)
> 17/08/15 09:34:14 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the
> same as ending offset skipping kssh 5
> 17/08/15 09:34:14 IN
[jira] [Created] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
Jepson created SPARK-21733:
--
Summary: ERROR executor.CoarseGrainedExecutorBackend: RECEIVED
SIGNAL TERM
Key: SPARK-21733
URL: https://issues.apache.org/jira/browse/SPARK-21733
Project: Spark
Issue Type: Bug
Components: DStreams
Affects Versions: 2.1.1
Environment: Apache Spark2.1.1
CDH5.12.0 Yarn
Reporter: Jepson
Kafka+Spark streaming ,throw these error:
{code:java}
17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003_piece0 stored
as bytes in memory (estimated size 1895.0 B, free 1643.2 MB)
17/08/15 09:34:14 INFO broadcast.TorrentBroadcast: Reading broadcast variable
8003 took 11 ms
17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003 stored as
values in memory (estimated size 2.9 KB, free 1643.2 MB)
17/08/15 09:34:14 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the same
as ending offset skipping kssh 5
17/08/15 09:34:14 INFO executor.Executor: Finished task 7.0 in stage 8003.0
(TID 64178). 1740 bytes result sent to driver
17/08/15 09:34:21 INFO storage.BlockManager: Removing RDD 8002
17/08/15 09:34:21 INFO executor.CoarseGrainedExecutorBackend: Got assigned task
64186
17/08/15 09:34:21 INFO executor.Executor: Running task 7.0 in stage 8004.0 (TID
64186)
17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Started reading broadcast
variable 8004
17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004_piece0 stored
as bytes in memory (estimated size 1895.0 B, free 1643.2 MB)
17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Reading broadcast variable
8004 took 8 ms
17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004 stored as
values in memory (estimated size 2.9 KB, free 1643.2 MB)
17/08/15 09:34:21 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the same
as ending offset skipping kssh 5
17/08/15 09:34:21 INFO executor.Executor: Finished task 7.0 in stage 8004.0
(TID 64186). 1740 bytes result sent to driver
17/08/15 09:34:29 ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL
TERM
17/08/15 09:34:29 INFO storage.DiskBlockManager: Shutdown hook called
17/08/15 09:34:29 INFO util.ShutdownHookManager: Shutdown hook called
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
