On 27 Apr 2015, at 07:51, ÐΞ€ρ@Ҝ (๏̯͡๏)
mailto:deepuj...@gmail.com>> wrote:
Spark 1.3
1. View stderr/stdout from executor from Web UI: when the job is running i
figured out the executor that am suppose to see, and those two links show 4
special characters on browser.
2. Tail on Yarn logs:
1) Application container logs from Web RM UI never load on browser. I
eventually have to kill the browser.
2) /apache/hadoop/bin/yarn logs -applicationId
application_1429087638744_151059
| less emits logs only after the application has completed.
Are there no better ways to see the logs as they a
You can check container logs from RM web UI or when log-aggregation is
enabled with the yarn command. There are other, but less convenient options.
On Mon, Apr 27, 2015 at 8:53 AM ÐΞ€ρ@Ҝ (๏̯͡๏) wrote:
> Spark 1.3
>
> 1. View stderr/stdout from executor from Web UI: when the job is running i
> fi
Spark 1.3
1. View stderr/stdout from executor from Web UI: when the job is running i
figured out the executor that am suppose to see, and those two links show 4
special characters on browser.
2. Tail on Yarn logs:
/apache/hadoop/bin/yarn logs -applicationId
application_1429087638744_151059 | les
On Fri, Apr 24, 2015 at 11:31 AM, Marcelo Vanzin
wrote:
>
> Spark 1.3 should have links to the executor logs in the UI while the
> application is running. Not yet in the history server, though.
You're absolutely correct -- didn't notice it until now. This is a great
addition!
--
www.skrasser.
On top of what's been said...
On Wed, Apr 22, 2015 at 10:48 PM, ÐΞ€ρ@Ҝ (๏̯͡๏) wrote:
> 1) I can go to Spark UI and see the status of the APP but cannot see the
> logs as the job progresses. How can i see logs of executors as they progress
> ?
Spark 1.3 should have links to the executor logs in t
For #1, click on a worker node on the YARN dashboard. From there,
Tools->Local logs->Userlogs has the logs for each application, and you can
view them by executor even while an application is running. (This is for
Hadoop 2.4, things may have changed in 2.6.)
-Sven
On Thu, Apr 23, 2015 at 6:27 AM,
For step 2, you can pipe application log to a file instead of copy-pasting.
Cheers
> On Apr 22, 2015, at 10:48 PM, ÐΞ€ρ@Ҝ (๏̯͡๏) wrote:
>
> I submit a spark app to YARN and i get these messages
>
>
>
> 15/04/22 22:45:04 INFO yarn.Client: Application report for
> application_1429087638744
I submit a spark app to YARN and i get these messages
15/04/22 22:45:04 INFO yarn.Client: Application report for
application_1429087638744_101363 (state: RUNNING)
15/04/22 22:45:04 INFO yarn.Client: Application report for
application_1429087638744_101363 (state: RUNNING).
...
1) I can go to S