Re: Logging in RDD mapToPair of Java Spark application

2017-07-30 Thread ayan guha
# pwd >> /yarn/container-logs/application_1501197841826_0013 >> >> I believe Spark moves these logs to a different place. But where are >> they? >> >> Thanks >> >> John >> >> >> >> >> -- >>

Re: Logging in RDD mapToPair of Java Spark application

2017-07-30 Thread John Zeng
ccardo Ferrari Cc: User Subject: Re: Logging in RDD mapToPair of Java Spark application Hi As you are using yarn log aggregation, yarn moves all the logs to hdfs after the application completes. You can use following command to get the logs: yarn logs -applicationId On Mon, 31 Jul 2017 at 3

Re: Logging in RDD mapToPair of Java Spark application

2017-07-30 Thread ayan guha
13]# pwd > /yarn/container-logs/application_1501197841826_0013 > > I believe Spark moves these logs to a different place. But where are they? > > Thanks > > John > > > > > -- > *From:* Riccardo Ferrari <ferra...@gmail.com> > *S

Re: Logging in RDD mapToPair of Java Spark application

2017-07-30 Thread John Zeng
. But where are they? Thanks John From: Riccardo Ferrari <ferra...@gmail.com> Sent: Saturday, July 29, 2017 8:18 PM To: johnzengspark Cc: User Subject: Re: Logging in RDD mapToPair of Java Spark application Hi John, The reason you don't see the second

Re: Logging in RDD mapToPair of Java Spark application

2017-07-29 Thread Riccardo Ferrari
lled and the statements after the second > logging line were executed. The only issue for me is why the second > logging > is not in JobTracker UI. > > Appreciate your help. > > Thanks > > John > > > > -- > View this message in context: http://apache-sp

Logging in RDD mapToPair of Java Spark application

2017-07-29 Thread johnzengspark
' is called and the statements after the second logging line were executed. The only issue for me is why the second logging is not in JobTracker UI. Appreciate your help. Thanks John -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Logging-in-RDD-