Re: spark-submit hangs forever after all tasks finish(spark 2.0.0 stable version on yarn)

2016-07-31 Thread Zhuo Tao 
Yarn client

On Sunday, July 31, 2016, Pradeep  wrote:

> Hi,
>
> Are you running on yarn-client or cluster mode?
>
> Pradeep
>
> > On Jul 30, 2016, at 7:34 PM, taozhuo >
> wrote:
> >
> > below is the error messages that seem run infinitely:
> >
> >
> > 16/07/30 23:25:38 DEBUG ProtobufRpcEngine: Call: getApplicationReport
> took
> > 1ms
> > 16/07/30 23:25:39 DEBUG Client: IPC Client (1735131305) connection to
> > /10.80.1.168:8032 from zhuotao sending #147247
> > 16/07/30 23:25:39 DEBUG Client: IPC Client (1735131305) connection to
> > /10.80.1.168:8032 from zhuotao got value #147247
> > 16/07/30 23:25:39 DEBUG ProtobufRpcEngine: Call: getApplicationReport
> took
> > 1ms
> > 16/07/30 23:25:40 DEBUG Client: IPC Client (1735131305) connection to
> > /10.80.1.168:8032 from zhuotao sending #147248
> > 16/07/30 23:25:40 DEBUG Client: IPC Client (1735131305) connection to
> > /10.80.1.168:8032 from zhuotao got value #147248
> > 16/07/30 23:25:40 DEBUG ProtobufRpcEngine: Call: getApplicationReport
> took
> > 1ms
> > 16/07/30 23:25:41 DEBUG Client: IPC Client (1735131305) connection to
> > /10.80.1.168:8032 from zhuotao sending #147249
> > 16/07/30 23:25:41 DEBUG Client: IPC Client (1735131305) connection to
> > /10.80.1.168:8032 from zhuotao got value #147249
> > 16/07/30 23:25:41 DEBUG ProtobufRpcEngine: Call: getApplicationReport
> took
> > 1ms
> > 16/07/30 23:25:42 DEBUG Client: IPC Client (1735131305) connection to
> > /10.80.1.168:8032 from zhuotao sending #147250
> > 16/07/30 23:25:42 DEBUG Client: IPC Client (1735131305) connection to
> > /10.80.1.168:8032 from zhuotao got value #147250
> > 16/07/30 23:25:42 DEBUG ProtobufRpcEngine: Call: getApplicationReport
> took
> > 1ms
> > 16/07/30 23:25:43 DEBUG Client: IPC Client (1735131305) connection to
> > /10.80.1.168:8032 from zhuotao sending #147251
> > 16/07/30 23:25:43 DEBUG Client: IPC Client (1735131305) connection to
> > /10.80.1.168:8032 from zhuotao got value #147251
> > 16/07/30 23:25:43 DEBUG ProtobufRpcEngine: Call: getApplicationReport
> took
> > 1ms
> > 16/07/30 23:25:44 DEBUG Client: IPC Client (1735131305) connection to
> > /10.80.1.168:8032 from zhuotao sending #147252
> > 16/07/30 23:25:44 DEBUG Client: IPC Client (1735131305) connection to
> > /10.80.1.168:8032 from zhuotao got value #147252
> > 16/07/30 23:25:44 DEBUG ProtobufRpcEngine: Call: getApplicationReport
> took
> > 0ms
> > 16/07/30 23:25:45 DEBUG Client: IPC Client (1735131305) connection to
> > /10.80.1.168:8032 from zhuotao sending #147253
> > 16/07/30 23:25:45 DEBUG Client: IPC Client (1735131305) connection to
> > /10.80.1.168:8032 from zhuotao got value #147253
> > 16/07/30 23:25:45 DEBUG ProtobufRpcEngine: Call: getApplicationReport
> took
> > 0ms
> > 16/07/30 23:25:46 DEBUG Client: IPC Client (1735131305) connection to
> > /10.80.1.168:8032 from zhuotao sending #147254
> > 16/07/30 23:25:46 DEBUG Client: IPC Client (1735131305) connection to
> > /10.80.1.168:8032 from zhuotao got value #147254
> > 16/07/30 23:25:46 DEBUG ProtobufRpcEngine: Call: getApplicationReport
> took
> > 1ms
> > 16/07/30 23:25:47 DEBUG Client: IPC Client (1735131305) connection to
> > /10.80.1.168:8032 from zhuotao sending #147255
> > 16/07/30 23:25:47 DEBUG Client: IPC Client (1735131305) connection to
> > /10.80.1.168:8032 from zhuotao got value #147255
> > 16/07/30 23:25:47 DEBUG ProtobufRpcEngine: Call: getApplicationReport
> took
> > 1ms
> > 16/07/30 23:25:48 DEBUG Client: IPC Client (1735131305) connection to
> > /10.80.1.168:8032 from zhuotao sending #147256
> > 16/07/30 23:25:48 DEBUG Client: IPC Client (1735131305) connection to
> > /10.80.1.168:8032 from zhuotao got value #147256
> > 16/07/30 23:25:48 DEBUG ProtobufRpcEngine: Call: getApplicationReport
> took
> > 1ms
> >
> >
> >
> >
> > --
> > View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/spark-submit-hangs-forever-after-all-tasks-finish-spark-2-0-0-stable-version-on-yarn-tp27436.html
> > Sent from the Apache Spark User List mailing list archive at Nabble.com.
> >
> > -
> > To unsubscribe e-mail: user-unsubscr...@spark.apache.org 
> >
>
>

Re: spark-submit hangs forever after all tasks finish(spark 2.0.0 stable version on yarn)

2016-07-31 Thread Pradeep 
Hi,

Are you running on yarn-client or cluster mode?

Pradeep

> On Jul 30, 2016, at 7:34 PM, taozhuo  wrote:
> 
> below is the error messages that seem run infinitely:
> 
> 
> 16/07/30 23:25:38 DEBUG ProtobufRpcEngine: Call: getApplicationReport took
> 1ms
> 16/07/30 23:25:39 DEBUG Client: IPC Client (1735131305) connection to
> /10.80.1.168:8032 from zhuotao sending #147247
> 16/07/30 23:25:39 DEBUG Client: IPC Client (1735131305) connection to
> /10.80.1.168:8032 from zhuotao got value #147247
> 16/07/30 23:25:39 DEBUG ProtobufRpcEngine: Call: getApplicationReport took
> 1ms
> 16/07/30 23:25:40 DEBUG Client: IPC Client (1735131305) connection to
> /10.80.1.168:8032 from zhuotao sending #147248
> 16/07/30 23:25:40 DEBUG Client: IPC Client (1735131305) connection to
> /10.80.1.168:8032 from zhuotao got value #147248
> 16/07/30 23:25:40 DEBUG ProtobufRpcEngine: Call: getApplicationReport took
> 1ms
> 16/07/30 23:25:41 DEBUG Client: IPC Client (1735131305) connection to
> /10.80.1.168:8032 from zhuotao sending #147249
> 16/07/30 23:25:41 DEBUG Client: IPC Client (1735131305) connection to
> /10.80.1.168:8032 from zhuotao got value #147249
> 16/07/30 23:25:41 DEBUG ProtobufRpcEngine: Call: getApplicationReport took
> 1ms
> 16/07/30 23:25:42 DEBUG Client: IPC Client (1735131305) connection to
> /10.80.1.168:8032 from zhuotao sending #147250
> 16/07/30 23:25:42 DEBUG Client: IPC Client (1735131305) connection to
> /10.80.1.168:8032 from zhuotao got value #147250
> 16/07/30 23:25:42 DEBUG ProtobufRpcEngine: Call: getApplicationReport took
> 1ms
> 16/07/30 23:25:43 DEBUG Client: IPC Client (1735131305) connection to
> /10.80.1.168:8032 from zhuotao sending #147251
> 16/07/30 23:25:43 DEBUG Client: IPC Client (1735131305) connection to
> /10.80.1.168:8032 from zhuotao got value #147251
> 16/07/30 23:25:43 DEBUG ProtobufRpcEngine: Call: getApplicationReport took
> 1ms
> 16/07/30 23:25:44 DEBUG Client: IPC Client (1735131305) connection to
> /10.80.1.168:8032 from zhuotao sending #147252
> 16/07/30 23:25:44 DEBUG Client: IPC Client (1735131305) connection to
> /10.80.1.168:8032 from zhuotao got value #147252
> 16/07/30 23:25:44 DEBUG ProtobufRpcEngine: Call: getApplicationReport took
> 0ms
> 16/07/30 23:25:45 DEBUG Client: IPC Client (1735131305) connection to
> /10.80.1.168:8032 from zhuotao sending #147253
> 16/07/30 23:25:45 DEBUG Client: IPC Client (1735131305) connection to
> /10.80.1.168:8032 from zhuotao got value #147253
> 16/07/30 23:25:45 DEBUG ProtobufRpcEngine: Call: getApplicationReport took
> 0ms
> 16/07/30 23:25:46 DEBUG Client: IPC Client (1735131305) connection to
> /10.80.1.168:8032 from zhuotao sending #147254
> 16/07/30 23:25:46 DEBUG Client: IPC Client (1735131305) connection to
> /10.80.1.168:8032 from zhuotao got value #147254
> 16/07/30 23:25:46 DEBUG ProtobufRpcEngine: Call: getApplicationReport took
> 1ms
> 16/07/30 23:25:47 DEBUG Client: IPC Client (1735131305) connection to
> /10.80.1.168:8032 from zhuotao sending #147255
> 16/07/30 23:25:47 DEBUG Client: IPC Client (1735131305) connection to
> /10.80.1.168:8032 from zhuotao got value #147255
> 16/07/30 23:25:47 DEBUG ProtobufRpcEngine: Call: getApplicationReport took
> 1ms
> 16/07/30 23:25:48 DEBUG Client: IPC Client (1735131305) connection to
> /10.80.1.168:8032 from zhuotao sending #147256
> 16/07/30 23:25:48 DEBUG Client: IPC Client (1735131305) connection to
> /10.80.1.168:8032 from zhuotao got value #147256
> 16/07/30 23:25:48 DEBUG ProtobufRpcEngine: Call: getApplicationReport took
> 1ms
> 
> 
> 
> 
> --
> View this message in context: 
> http://apache-spark-user-list.1001560.n3.nabble.com/spark-submit-hangs-forever-after-all-tasks-finish-spark-2-0-0-stable-version-on-yarn-tp27436.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
> 
> -
> To unsubscribe e-mail: user-unsubscr...@spark.apache.org
> 


-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org

spark-submit hangs forever after all tasks finish(spark 2.0.0 stable version on yarn)

2016-07-30 Thread taozhuo 
below is the error messages that seem run infinitely:


16/07/30 23:25:38 DEBUG ProtobufRpcEngine: Call: getApplicationReport took
1ms
16/07/30 23:25:39 DEBUG Client: IPC Client (1735131305) connection to
/10.80.1.168:8032 from zhuotao sending #147247
16/07/30 23:25:39 DEBUG Client: IPC Client (1735131305) connection to
/10.80.1.168:8032 from zhuotao got value #147247
16/07/30 23:25:39 DEBUG ProtobufRpcEngine: Call: getApplicationReport took
1ms
16/07/30 23:25:40 DEBUG Client: IPC Client (1735131305) connection to
/10.80.1.168:8032 from zhuotao sending #147248
16/07/30 23:25:40 DEBUG Client: IPC Client (1735131305) connection to
/10.80.1.168:8032 from zhuotao got value #147248
16/07/30 23:25:40 DEBUG ProtobufRpcEngine: Call: getApplicationReport took
1ms
16/07/30 23:25:41 DEBUG Client: IPC Client (1735131305) connection to
/10.80.1.168:8032 from zhuotao sending #147249
16/07/30 23:25:41 DEBUG Client: IPC Client (1735131305) connection to
/10.80.1.168:8032 from zhuotao got value #147249
16/07/30 23:25:41 DEBUG ProtobufRpcEngine: Call: getApplicationReport took
1ms
16/07/30 23:25:42 DEBUG Client: IPC Client (1735131305) connection to
/10.80.1.168:8032 from zhuotao sending #147250
16/07/30 23:25:42 DEBUG Client: IPC Client (1735131305) connection to
/10.80.1.168:8032 from zhuotao got value #147250
16/07/30 23:25:42 DEBUG ProtobufRpcEngine: Call: getApplicationReport took
1ms
16/07/30 23:25:43 DEBUG Client: IPC Client (1735131305) connection to
/10.80.1.168:8032 from zhuotao sending #147251
16/07/30 23:25:43 DEBUG Client: IPC Client (1735131305) connection to
/10.80.1.168:8032 from zhuotao got value #147251
16/07/30 23:25:43 DEBUG ProtobufRpcEngine: Call: getApplicationReport took
1ms
16/07/30 23:25:44 DEBUG Client: IPC Client (1735131305) connection to
/10.80.1.168:8032 from zhuotao sending #147252
16/07/30 23:25:44 DEBUG Client: IPC Client (1735131305) connection to
/10.80.1.168:8032 from zhuotao got value #147252
16/07/30 23:25:44 DEBUG ProtobufRpcEngine: Call: getApplicationReport took
0ms
16/07/30 23:25:45 DEBUG Client: IPC Client (1735131305) connection to
/10.80.1.168:8032 from zhuotao sending #147253
16/07/30 23:25:45 DEBUG Client: IPC Client (1735131305) connection to
/10.80.1.168:8032 from zhuotao got value #147253
16/07/30 23:25:45 DEBUG ProtobufRpcEngine: Call: getApplicationReport took
0ms
16/07/30 23:25:46 DEBUG Client: IPC Client (1735131305) connection to
/10.80.1.168:8032 from zhuotao sending #147254
16/07/30 23:25:46 DEBUG Client: IPC Client (1735131305) connection to
/10.80.1.168:8032 from zhuotao got value #147254
16/07/30 23:25:46 DEBUG ProtobufRpcEngine: Call: getApplicationReport took
1ms
16/07/30 23:25:47 DEBUG Client: IPC Client (1735131305) connection to
/10.80.1.168:8032 from zhuotao sending #147255
16/07/30 23:25:47 DEBUG Client: IPC Client (1735131305) connection to
/10.80.1.168:8032 from zhuotao got value #147255
16/07/30 23:25:47 DEBUG ProtobufRpcEngine: Call: getApplicationReport took
1ms
16/07/30 23:25:48 DEBUG Client: IPC Client (1735131305) connection to
/10.80.1.168:8032 from zhuotao sending #147256
16/07/30 23:25:48 DEBUG Client: IPC Client (1735131305) connection to
/10.80.1.168:8032 from zhuotao got value #147256
16/07/30 23:25:48 DEBUG ProtobufRpcEngine: Call: getApplicationReport took
1ms




--
View this message in context: 
http://apache-spark-user-list.1001560.n3.nabble.com/spark-submit-hangs-forever-after-all-tasks-finish-spark-2-0-0-stable-version-on-yarn-tp27436.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org