I see, I guess I should have set the historyServer.
Strangely enough peeking in the yarn seems like nothing is "happening", it list
a single application running with 0% progress but each node has 0 running
containers which confuses me to wether anything is actually happening
Should I restart the job with the spark.yarn.historyServer.address ?
[hadoop@sslabnode02 ~]$ yarn node -list
16/01/04 14:43:40 INFO client.RMProxy: Connecting to ResourceManager at
sslabnode01/10.15.235.239:8032
16/01/04 14:43:40 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
Total Nodes:3
Node-Id Node-State Node-Http-Address
Number-of-Running-Containers
sslabnode02:54142 RUNNING sslabnode02:8042
0
sslabnode01:60780 RUNNING sslabnode01:8042
0
sslabnode03:60569 RUNNING sslabnode03:8042
0
[hadoop@sslabnode02 ~]$ yarn application -list
16/01/04 14:43:55 INFO client.RMProxy: Connecting to ResourceManager at
sslabnode01/10.15.235.239:8032
16/01/04 14:43:55 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
Total number of applications (application-types: [] and states: [SUBMITTED,
ACCEPTED, RUNNING]):1
Application-Id Application-Name Application-Type
User Queue State Final-State
Progress Tracking-URL
application_1451947397662_0001 ClusterIncidents SPARK
hadoop default ACCEPTED UNDEFINED
0% N/A
> On Jan 4, 2016, at 2:55 PM, Ted Yu <[email protected]> wrote:
>
> Please look at history server related content under:
> https://spark.apache.org/docs/latest/running-on-yarn.html
> <https://spark.apache.org/docs/latest/running-on-yarn.html>
>
> Note spark.yarn.historyServer.address
> FYI
>
> On Mon, Jan 4, 2016 at 2:49 PM, Daniel Valdivia <[email protected]
> <mailto:[email protected]>> wrote:
> Hello everyone, happy new year,
>
> I submitted an app to yarn, however I'm unable to monitor it's progress on
> the driver node, not in <drive_address>:8080 or <drive_address>:4040 as
> documented, when submitting to the standalone mode I could monitor however
> seems liek its not the case right now.
>
> I submitted my app this way:
>
> spark-submit --class my.class --master yarn --deploy-mode cluster myjar.jar
>
> and so far the job is on it's way it seems, the console is vivid with
> Application report messages, however I can't access the status of the app,
> should I have submitted the app in a different fashion to access the status
> of it?
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> <mailto:[email protected]>
> For additional commands, e-mail: [email protected]
> <mailto:[email protected]>
>
>
