[jira] [Updated] (YARN-2377) Localization exception stack traces are not passed as diagnostic info

2014-10-08 Thread Gera Shegalov (JIRA) 

 [ 
https://issues.apache.org/jira/browse/YARN-2377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gera Shegalov updated YARN-2377:

Attachment: YARN-2377.v02.patch

Thanks for review, [~jlowe]! Your points are valid, uploading v02 to 
accommodate them.

> Localization exception stack traces are not passed as diagnostic info
> -
>
> Key: YARN-2377
> URL: https://issues.apache.org/jira/browse/YARN-2377
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: nodemanager
>Affects Versions: 2.4.0
>Reporter: Gera Shegalov
>Assignee: Gera Shegalov
> Attachments: YARN-2377.v01.patch, YARN-2377.v02.patch
>
>
> In the Localizer log one can only see this kind of message
> {code}
> 14/07/31 10:29:00 INFO localizer.ResourceLocalizationService: DEBUG: FAILED { 
> hdfs://ha-nn-uri-0:8020/tmp/hadoop-yarn/staging/gshegalov/.staging/job_1406825443306_0004/job.jar,
>  1406827248944, PATTERN, (?:classes/|lib/).* }, java.net.UnknownHos 
> tException: ha-nn-uri-0
> {code}
> And then only {{ java.net.UnknownHostException: ha-nn-uri-0}} message is 
> propagated as diagnostics.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-2377) Localization exception stack traces are not passed as diagnostic info

2014-07-31 Thread Gera Shegalov (JIRA) 

 [ 
https://issues.apache.org/jira/browse/YARN-2377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gera Shegalov updated YARN-2377:


Attachment: YARN-2377.v01.patch

v01 for review. With this you get a more actionable stack trace:

{code}
14/07/31 17:46:39 INFO mapreduce.Job: Job job_1406853387336_0001 failed with 
state FAILED due to: Application application_1406853387336_0001 failed 2 times 
due to AM Container for appattempt_1406853387336_0001_02 exited with  
exitCode: -1000
For more detailed output, check application tracking 
page:http://tw-mbp-gshegalov:8088/proxy/application_1406853387336_0001/Then, 
click on links to logs of each attempt.
Diagnostics: java.net.UnknownHostException: ha-nn-uri-0
java.lang.IllegalArgumentException: java.net.UnknownHostException: ha-nn-uri-0
at 
org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:373)
at 
org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:260)
at 
org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:153)
at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:607)
at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:552)
at 
org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:139)
at 
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2590)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:89)
at 
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2624)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2606)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:368)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296)
at org.apache.hadoop.yarn.util.FSDownload.copy(FSDownload.java:248)
at org.apache.hadoop.yarn.util.FSDownload.access$000(FSDownload.java:60)
at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:356)
at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:354)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:394)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1626)
at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:353)
at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:59)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:695)
Caused by: java.net.UnknownHostException: ha-nn-uri-0
... 29 more
Caused by: ha-nn-uri-0
java.lang.IllegalArgumentException: java.net.UnknownHostException: ha-nn-uri-0
at 
org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:373)
at 
org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:260)
at 
org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:153)
at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:607)
at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:552)
at 
org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:139)
at 
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2590)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:89)
at 
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2624)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2606)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:368)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296)
at org.apache.hadoop.yarn.util.FSDownload.copy(FSDownload.java:248)
at org.apache.hadoop.yarn.util.FSDownload.access$000(FSDownload.java:60)
at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:356)
at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:354)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:394)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1626)
at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:353)
at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:59)
at java.util.concurrent.FutureTask$Sync.innerRun

[jira] [Updated] (YARN-2377) Localization exception stack traces are not passed as diagnostic info

2014-07-31 Thread Gera Shegalov (JIRA) 

 [ 
https://issues.apache.org/jira/browse/YARN-2377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gera Shegalov updated YARN-2377:


Description: 
In the Localizer log one can only see this kind of message
{code}
14/07/31 10:29:00 INFO localizer.ResourceLocalizationService: DEBUG: FAILED { 
hdfs://ha-nn-uri-0:8020/tmp/hadoop-yarn/staging/gshegalov/.staging/job_1406825443306_0004/job.jar,
 1406827248944, PATTERN, (?:classes/|lib/).* }, java.net.UnknownHos tException: 
ha-nn-uri-0
{code}

And then only {{ java.net.UnknownHostException: ha-nn-uri-0}} message is 
propagated as diagnostics.

  was:
In the Localizer log one can only see this kind of message
{code}
14/07/31 10:29:00 INFO localizer.ResourceLocalizationService: DEBUG: FAILED { 
hdfs://ha-nn-uri-0:8020/tmp/hadoop-yarn/staging/gshegalov/.staging/job_1406825443306_0004/job.jar,
 1406827248944, PATTERN, (?:classes/|lib/).* }, java.net.UnknownHos tException: 
ha-nn-uri-0
{code}

And then only {{ java.net.UnknownHos tException: ha-nn-uri-0}} message is 
propagated as diagnostics.


> Localization exception stack traces are not passed as diagnostic info
> -
>
> Key: YARN-2377
> URL: https://issues.apache.org/jira/browse/YARN-2377
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: nodemanager
>Affects Versions: 2.4.0
>Reporter: Gera Shegalov
>Assignee: Gera Shegalov
>
> In the Localizer log one can only see this kind of message
> {code}
> 14/07/31 10:29:00 INFO localizer.ResourceLocalizationService: DEBUG: FAILED { 
> hdfs://ha-nn-uri-0:8020/tmp/hadoop-yarn/staging/gshegalov/.staging/job_1406825443306_0004/job.jar,
>  1406827248944, PATTERN, (?:classes/|lib/).* }, java.net.UnknownHos 
> tException: ha-nn-uri-0
> {code}
> And then only {{ java.net.UnknownHostException: ha-nn-uri-0}} message is 
> propagated as diagnostics.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (YARN-2377) Localization exception stack traces are not passed as diagnostic info

2014-07-31 Thread Gera Shegalov (JIRA) 

 [ 
https://issues.apache.org/jira/browse/YARN-2377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gera Shegalov updated YARN-2377:


Description: 
In the Localizer log one can only see this kind of message
{code}
14/07/31 10:29:00 INFO localizer.ResourceLocalizationService: DEBUG: FAILED { 
hdfs://ha-nn-uri-0:8020/tmp/hadoop-yarn/staging/gshegalov/.staging/job_1406825443306_0004/job.jar,
 1406827248944, PATTERN, (?:classes/|lib/).* }, java.net.UnknownHos tException: 
ha-nn-uri-0
{code}

And then only {{ java.net.UnknownHos tException: ha-nn-uri-0}} message is 
propagated as diagnostics.

  was:
In the Localizer log one can only see this kind of message
{code}
14/07/31 10:29:00 INFO localizer.ResourceLocalizationService: DEBUG: FAILED { 
hdfs://ha-nn-uri-0:8020/tmp/hadoop-yarn/staging/gshegalov/.staging/job_1406825443306_0004/job.jar,
 1406827248944, PATTERN, (?:classes/|lib/).* }, java.net.UnknownHos tException: 
ha-nn-uri-0
{code}

And then onlt {{ java.net.UnknownHos tException: ha-nn-uri-0}} message is 
propagated as diagnostics.


> Localization exception stack traces are not passed as diagnostic info
> -
>
> Key: YARN-2377
> URL: https://issues.apache.org/jira/browse/YARN-2377
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: nodemanager
>Affects Versions: 2.4.0
>Reporter: Gera Shegalov
>Assignee: Gera Shegalov
>
> In the Localizer log one can only see this kind of message
> {code}
> 14/07/31 10:29:00 INFO localizer.ResourceLocalizationService: DEBUG: FAILED { 
> hdfs://ha-nn-uri-0:8020/tmp/hadoop-yarn/staging/gshegalov/.staging/job_1406825443306_0004/job.jar,
>  1406827248944, PATTERN, (?:classes/|lib/).* }, java.net.UnknownHos 
> tException: ha-nn-uri-0
> {code}
> And then only {{ java.net.UnknownHos tException: ha-nn-uri-0}} message is 
> propagated as diagnostics.



--
This message was sent by Atlassian JIRA
(v6.2#6252)