[jira] [Assigned] (YARN-6922) Findbugs warning in YARN NodeManager
[
https://issues.apache.org/jira/browse/YARN-6922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Chen reassigned YARN-6922:
---
Assignee: Weiwei Yang (was: Xiao Chen)
> Findbugs warning in YARN NodeManager
>
>
> Key: YARN-6922
> URL: https://issues.apache.org/jira/browse/YARN-6922
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.0.0-beta1
>Reporter: Xuan Gong
>Assignee: Weiwei Yang
>
> Several findbugs warning in YARN NodeManager package.
> {code}
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics
> is a mutable collection which should be package protected
> Bug type MS_MUTABLE_COLLECTION_PKGPROTECT (click for details)
> In class
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics
> Field
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics
> At ContainerMetrics.java:[line 134]
> {code}
> {code}
>
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus()
> makes inefficient use of keySet iterator instead of entrySet iterator
> Bug type WMI_WRONG_MAP_ITERATOR (click for details)
> In class
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer
> In method
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus()
> Field
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.pendingResources
> At ContainerLocalizer.java:[line 357]
> {code}
> {code}
>
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache()
> makes inefficient use of keySet iterator instead of entrySet iterator
> Bug type WMI_WRONG_MAP_ITERATOR (click for details)
> In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl
> In method
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache()
> Field
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.recentlyStoppedContainers
> At NodeStatusUpdaterImpl.java:[line 719]
> {code}
> {code}
> Hard coded reference to an absolute pathname in
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext)
> Bug type DMI_HARDCODED_ABSOLUTE_FILENAME (click for details)
> In class
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime
> In method
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext)
> File name /sys/fs/cgroup
> At DockerLinuxContainerRuntime.java:[line 490]
> {code}
> {code}
> Useless object stored in variable removedNullContainers of method
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List)
> Bug type UC_USELESS_OBJECT (click for details)
> In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl
> In method
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List)
> Value removedNullContainers
> Type java.util.HashSet
> At NodeStatusUpdaterImpl.java:[line 642]
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-6922) Findbugs warning in YARN NodeManager
[
https://issues.apache.org/jira/browse/YARN-6922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Chen reassigned YARN-6922:
---
Assignee: Xiao Chen (was: Weiwei Yang)
> Findbugs warning in YARN NodeManager
>
>
> Key: YARN-6922
> URL: https://issues.apache.org/jira/browse/YARN-6922
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.0.0-beta1
>Reporter: Xuan Gong
>Assignee: Xiao Chen
>
> Several findbugs warning in YARN NodeManager package.
> {code}
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics
> is a mutable collection which should be package protected
> Bug type MS_MUTABLE_COLLECTION_PKGPROTECT (click for details)
> In class
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics
> Field
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics
> At ContainerMetrics.java:[line 134]
> {code}
> {code}
>
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus()
> makes inefficient use of keySet iterator instead of entrySet iterator
> Bug type WMI_WRONG_MAP_ITERATOR (click for details)
> In class
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer
> In method
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus()
> Field
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.pendingResources
> At ContainerLocalizer.java:[line 357]
> {code}
> {code}
>
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache()
> makes inefficient use of keySet iterator instead of entrySet iterator
> Bug type WMI_WRONG_MAP_ITERATOR (click for details)
> In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl
> In method
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache()
> Field
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.recentlyStoppedContainers
> At NodeStatusUpdaterImpl.java:[line 719]
> {code}
> {code}
> Hard coded reference to an absolute pathname in
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext)
> Bug type DMI_HARDCODED_ABSOLUTE_FILENAME (click for details)
> In class
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime
> In method
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext)
> File name /sys/fs/cgroup
> At DockerLinuxContainerRuntime.java:[line 490]
> {code}
> {code}
> Useless object stored in variable removedNullContainers of method
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List)
> Bug type UC_USELESS_OBJECT (click for details)
> In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl
> In method
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List)
> Value removedNullContainers
> Type java.util.HashSet
> At NodeStatusUpdaterImpl.java:[line 642]
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7261) Add debug message in class FSDownload for better download latency monitoring
[ https://issues.apache.org/jira/browse/YARN-7261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16212874#comment-16212874 ] Xiao Chen commented on YARN-7261: - +1 on patch 3, thanks Yufei! > Add debug message in class FSDownload for better download latency monitoring > > > Key: YARN-7261 > URL: https://issues.apache.org/jira/browse/YARN-7261 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager >Reporter: Yufei Gu >Assignee: Yufei Gu > Attachments: YARN-7261.001.patch, YARN-7261.002.patch, > YARN-7261.003.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7261) Add debug message in class FSDownload for better download latency monitoring
[
https://issues.apache.org/jira/browse/YARN-7261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16212189#comment-16212189
]
Xiao Chen commented on YARN-7261:
-
Thanks for revving Yufei!
1 minor comment:
{code}
if (LOG.isDebugEnabled()) {
LOG.debug("Skip downloading resource: " + key + " since it isn't"
+ " ready or it has been downloaded.");
}
{code}
Let's also add {{rsrc.getState()}} to this message. We can say {{"since it's in
state: " + rsrc.getState()}}.
+1 pending that and pre-commit.
> Add debug message in class FSDownload for better download latency monitoring
>
>
> Key: YARN-7261
> URL: https://issues.apache.org/jira/browse/YARN-7261
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager
>Reporter: Yufei Gu
>Assignee: Yufei Gu
> Attachments: YARN-7261.001.patch, YARN-7261.002.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7261) Add debug message in class FSDownload for better download latency monitoring
[
https://issues.apache.org/jira/browse/YARN-7261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16212095#comment-16212095
]
Xiao Chen commented on YARN-7261:
-
Thanks [~yufeigu] for creating the jira and providing a patch.
For context, Yufei and myself have seen an intermittent issue where
localization took very long. It is suspected that the copying from hdfs took
long, but HDFS metrics/logs doesn't show any smoking guns. We'd like to use
this jira to add more debugging information.
The log we collected currently looks like:
{noformat}
2017-09-15 10:55:50,738 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService:
Created localizer for container_e70_1505214525894_75227_01_14
2017-09-15 10:55:50,738 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService:
Downloading public rsrc:{
hdfs://nameservice1/cached/pub/deviceDetailsQuery_1505472717000.xml,
1505472808731, FILE, null }
...
2017-09-15 10:58:38,760 DEBUG org.apache.hadoop.yarn.util.FSDownload: Changing
permissions for path
file:/var/hdfs/5/yarn/nm/filecache/7363_tmp/deviceDetailsQuery_1505472717000.xml
to perm r-xr-xr-x
2017-09-15 10:58:38,775 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container:
Container container_e70_1505214525894_75227_01_14 transitioned from
LOCALIZING to LOCALIZED
{noformat}
But no details on what happened in the 3 minutes.
The patch LGTM. 1 question:
Do you think adding a debug message to
{{ResourceLocalizationService#addResource}}, to indicate the when the following
1 & 2 conditions are false would be helpful?
{code}
/*
* Here multiple containers may request the same resource. So we need
* to start downloading only when
* 1) ResourceState == DOWNLOADING
* 2) We are able to acquire non blocking semaphore lock.
* If not we will skip this resource as either it is getting downloaded
* or it FAILED / LOCALIZED.
*/
{code}
> Add debug message in class FSDownload for better download latency monitoring
>
>
> Key: YARN-7261
> URL: https://issues.apache.org/jira/browse/YARN-7261
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager
>Reporter: Yufei Gu
>Assignee: Yufei Gu
> Attachments: YARN-7261.001.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5641) Localizer leaves behind tarballs after container is complete
[ https://issues.apache.org/jira/browse/YARN-5641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15840153#comment-15840153 ] Xiao Chen commented on YARN-5641: - Thanks for the prompt response. I think the failure is understandable since the backport was clean. Thanks for working on the branch-2 patch. :) > Localizer leaves behind tarballs after container is complete > > > Key: YARN-5641 > URL: https://issues.apache.org/jira/browse/YARN-5641 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Eric Badger >Assignee: Eric Badger > Fix For: 3.0.0-alpha3 > > Attachments: YARN-5641.001.patch, YARN-5641.002.patch, > YARN-5641.003.patch, YARN-5641.004.patch, YARN-5641.005.patch, > YARN-5641.006.patch, YARN-5641.007.patch, YARN-5641.008.patch, > YARN-5641.009.patch, YARN-5641.009.patch, YARN-5641.010-b2.patch, > YARN-5641.010.patch > > > The localizer sometimes fails to clean up extracted tarballs leaving large > footprints that persist on the nodes indefinitely. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5641) Localizer leaves behind tarballs after container is complete
[ https://issues.apache.org/jira/browse/YARN-5641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15839241#comment-15839241 ] Xiao Chen commented on YARN-5641: - It seems this commit broke branch-2 compilation. Could you take a look? Thanks. https://builds.apache.org/job/PreCommit-HADOOP-Build/11511/artifact/patchprocess/branch-compile-root-jdk1.7.0_121.txt > Localizer leaves behind tarballs after container is complete > > > Key: YARN-5641 > URL: https://issues.apache.org/jira/browse/YARN-5641 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Eric Badger >Assignee: Eric Badger > Fix For: 2.9.0, 3.0.0-alpha3 > > Attachments: YARN-5641.001.patch, YARN-5641.002.patch, > YARN-5641.003.patch, YARN-5641.004.patch, YARN-5641.005.patch, > YARN-5641.006.patch, YARN-5641.007.patch, YARN-5641.008.patch, > YARN-5641.009.patch, YARN-5641.009.patch, YARN-5641.010.patch > > > The localizer sometimes fails to clean up extracted tarballs leaving large > footprints that persist on the nodes indefinitely. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5875) TestTokenClientRMService#testTokenRenewalWrongUser fails
[
https://issues.apache.org/jira/browse/YARN-5875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Chen updated YARN-5875:
Summary: TestTokenClientRMService#testTokenRenewalWrongUser fails (was:
TestTokenClientRMService#testTokenRenewalWrongUser fails on trunk)
> TestTokenClientRMService#testTokenRenewalWrongUser fails
>
>
> Key: YARN-5875
> URL: https://issues.apache.org/jira/browse/YARN-5875
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Varun Saxena
>Assignee: Gergely Novák
> Attachments: YARN-5875.001.patch
>
>
> {noformat}
> Running org.apache.hadoop.yarn.server.resourcemanager.TestTokenClientRMService
> Tests run: 6, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.983 sec <<<
> FAILURE! - in
> org.apache.hadoop.yarn.server.resourcemanager.TestTokenClientRMService
> testTokenRenewalWrongUser(org.apache.hadoop.yarn.server.resourcemanager.TestTokenClientRMService)
> Time elapsed: 0.015 sec <<< FAILURE!
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.yarn.server.resourcemanager.TestTokenClientRMService$3.run(TestTokenClientRMService.java:125)
> at
> org.apache.hadoop.yarn.server.resourcemanager.TestTokenClientRMService$3.run(TestTokenClientRMService.java:118)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1857)
> at
> org.apache.hadoop.yarn.server.resourcemanager.TestTokenClientRMService.testTokenRenewalWrongUser(TestTokenClientRMService.java:118)
> Results :
> Failed tests:
> TestTokenClientRMService.testTokenRenewalWrongUser:118 null
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5875) TestTokenClientRMService#testTokenRenewalWrongUser fails on trunk
[
https://issues.apache.org/jira/browse/YARN-5875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15668500#comment-15668500
]
Xiao Chen commented on YARN-5875:
-
+1 committing this. Thanks [~varun_saxena] for reporting the issue and
[~GergelyNovak] for the fix!
> TestTokenClientRMService#testTokenRenewalWrongUser fails on trunk
> -
>
> Key: YARN-5875
> URL: https://issues.apache.org/jira/browse/YARN-5875
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Varun Saxena
>Assignee: Gergely Novák
> Attachments: YARN-5875.001.patch
>
>
> {noformat}
> Running org.apache.hadoop.yarn.server.resourcemanager.TestTokenClientRMService
> Tests run: 6, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.983 sec <<<
> FAILURE! - in
> org.apache.hadoop.yarn.server.resourcemanager.TestTokenClientRMService
> testTokenRenewalWrongUser(org.apache.hadoop.yarn.server.resourcemanager.TestTokenClientRMService)
> Time elapsed: 0.015 sec <<< FAILURE!
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.yarn.server.resourcemanager.TestTokenClientRMService$3.run(TestTokenClientRMService.java:125)
> at
> org.apache.hadoop.yarn.server.resourcemanager.TestTokenClientRMService$3.run(TestTokenClientRMService.java:118)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1857)
> at
> org.apache.hadoop.yarn.server.resourcemanager.TestTokenClientRMService.testTokenRenewalWrongUser(TestTokenClientRMService.java:118)
> Results :
> Failed tests:
> TestTokenClientRMService.testTokenRenewalWrongUser:118 null
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5433) Audit dependencies for Category-X
[
https://issues.apache.org/jira/browse/YARN-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15607567#comment-15607567
]
Xiao Chen commented on YARN-5433:
-
Thanks for the great work here, [~sjlee0].
I see the spreadsheet is done, and patch looks good. +1 pending the following
nit:
IIRC the 'EXHIBIT' section can be omitted, only the license is required.
Somehow I couldn't find the source of this information anymore...
Feel free to wait for potential comments from others, and merge this nit into
the final patch.
(For the record, MPL is category B.
https://www.apache.org/legal/resolved#category-b)
> Audit dependencies for Category-X
> -
>
> Key: YARN-5433
> URL: https://issues.apache.org/jira/browse/YARN-5433
> Project: Hadoop YARN
> Issue Type: Bug
> Components: timelineserver
>Affects Versions: 3.0.0-alpha1
>Reporter: Sean Busbey
>Assignee: Sangjin Lee
>Priority: Blocker
> Attachments: YARN-5433.01.patch, YARN-5433.02.patch,
> YARN-5433.03.patch
>
>
> Recently phoenix has found some category-x dependencies in their build
> (PHOENIX-3084, PHOENIX-3091), which also showed some problems in HBase
> (HBASE-16260).
> Since the Timeline Server work brought in both of these as dependencies, we
> should make sure we don't have any cat-x dependencies either. From what I've
> seen in those projects, our choice of HBase version shouldn't be impacted but
> our Phoenix one is.
> Greping our current dependency list for the timeline server component shows
> some LGPL:
> {code}
> ...
> [INFO]net.sourceforge.findbugs:annotations:jar:1.3.2:compile
> ...
> {code}
> I haven't checked the rest of the dependencies that have changed since
> HADOOP-12893 went in, so ATM I've filed this against YARN since that's where
> this one example came in.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5433) Audit dependencies for Category-X
[
https://issues.apache.org/jira/browse/YARN-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15607384#comment-15607384
]
Xiao Chen commented on YARN-5433:
-
Thanks [~sjlee0] for the new rev!
For MPL, since it's new, we need to put the whole MPL license text in there.
The script can't handle it because we need to find the official license, and
80-char wrap it. I used http://appincredible.com/online/word-wrap/ to do the
wrap in HADOOP-12893.
Also, it seems the spreadsheet here has some {{already analyzed? == Y?}}
entries (namely the glassfish ones). What's the plan on those? I think we
should have the spreadsheet all {{Done==Y}} before resolving the jira. :)
> Audit dependencies for Category-X
> -
>
> Key: YARN-5433
> URL: https://issues.apache.org/jira/browse/YARN-5433
> Project: Hadoop YARN
> Issue Type: Bug
> Components: timelineserver
>Affects Versions: 3.0.0-alpha1
>Reporter: Sean Busbey
>Assignee: Sangjin Lee
>Priority: Blocker
> Attachments: YARN-5433.01.patch, YARN-5433.02.patch
>
>
> Recently phoenix has found some category-x dependencies in their build
> (PHOENIX-3084, PHOENIX-3091), which also showed some problems in HBase
> (HBASE-16260).
> Since the Timeline Server work brought in both of these as dependencies, we
> should make sure we don't have any cat-x dependencies either. From what I've
> seen in those projects, our choice of HBase version shouldn't be impacted but
> our Phoenix one is.
> Greping our current dependency list for the timeline server component shows
> some LGPL:
> {code}
> ...
> [INFO]net.sourceforge.findbugs:annotations:jar:1.3.2:compile
> ...
> {code}
> I haven't checked the rest of the dependencies that have changed since
> HADOOP-12893 went in, so ATM I've filed this against YARN since that's where
> this one example came in.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5433) Audit dependencies for Category-X
[
https://issues.apache.org/jira/browse/YARN-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15606711#comment-15606711
]
Xiao Chen commented on YARN-5433:
-
Thanks [~sjlee0] for working on this (and the ping).
For HADOOP-12893, the
[spreadsheet|https://docs.google.com/spreadsheets/d/1HL2b4PSdQMZDVJmum1GIKrteFr2oainApTLiJTPnfd4/edit#gid=1060066002]
has a 'generate.py' tab, containing the code used to generate the L (which
license to skip L etc.). Probably an overkill here, but that's more accurate
than my memory. :)
So:
- No need to add NOTICE for BSD here. See
https://issues.apache.org/jira/browse/HADOOP-12893?focusedCommentId=15284739=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15284739
for details.
- MPL and CDDL are ok. We should add it to LICENSE.txt under corresponding
license. There's already {{servlet-api 2.5}} in there under CPPL1.0, so we can
skip that.
- Didn't check all, but if there're any dependency with multiple licenses, we
can choose the most friendly one.
> Audit dependencies for Category-X
> -
>
> Key: YARN-5433
> URL: https://issues.apache.org/jira/browse/YARN-5433
> Project: Hadoop YARN
> Issue Type: Bug
> Components: timelineserver
>Affects Versions: 3.0.0-alpha1
>Reporter: Sean Busbey
>Assignee: Sangjin Lee
>Priority: Blocker
> Attachments: YARN-5433.01.patch
>
>
> Recently phoenix has found some category-x dependencies in their build
> (PHOENIX-3084, PHOENIX-3091), which also showed some problems in HBase
> (HBASE-16260).
> Since the Timeline Server work brought in both of these as dependencies, we
> should make sure we don't have any cat-x dependencies either. From what I've
> seen in those projects, our choice of HBase version shouldn't be impacted but
> our Phoenix one is.
> Greping our current dependency list for the timeline server component shows
> some LGPL:
> {code}
> ...
> [INFO]net.sourceforge.findbugs:annotations:jar:1.3.2:compile
> ...
> {code}
> I haven't checked the rest of the dependencies that have changed since
> HADOOP-12893 went in, so ATM I've filed this against YARN since that's where
> this one example came in.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5777) TestLogsCLI#testFetchApplictionLogsAsAnotherUser fails
[
https://issues.apache.org/jira/browse/YARN-5777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15606203#comment-15606203
]
Xiao Chen commented on YARN-5777:
-
Thanks [~ajisakaa] for the catch. The analysis make sense, +1. Committing this.
> TestLogsCLI#testFetchApplictionLogsAsAnotherUser fails
> --
>
> Key: YARN-5777
> URL: https://issues.apache.org/jira/browse/YARN-5777
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
> Attachments: YARN-5777.01.patch
>
>
> {noformat}
> Running org.apache.hadoop.yarn.client.cli.TestLogsCLI
> Tests run: 14, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 5.876 sec
> <<< FAILURE! - in org.apache.hadoop.yarn.client.cli.TestLogsCLI
> testFetchApplictionLogsAsAnotherUser(org.apache.hadoop.yarn.client.cli.TestLogsCLI)
> Time elapsed: 0.199 sec <<< ERROR!
> java.io.IOException: Invalid directory or I/O error occurred for dir:
> /Users/aajisaka/git/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/target/logs/priority/logs/application_1477371285256_1000
> at org.apache.hadoop.fs.FileUtil.list(FileUtil.java:1148)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:469)
> at
> org.apache.hadoop.fs.DelegateToFileSystem.listStatus(DelegateToFileSystem.java:169)
> at org.apache.hadoop.fs.ChecksumFs.listStatus(ChecksumFs.java:519)
> at
> org.apache.hadoop.fs.AbstractFileSystem$1.(AbstractFileSystem.java:890)
> at
> org.apache.hadoop.fs.AbstractFileSystem.listStatusIterator(AbstractFileSystem.java:888)
> at org.apache.hadoop.fs.FileContext$22.next(FileContext.java:1492)
> at org.apache.hadoop.fs.FileContext$22.next(FileContext.java:1487)
> at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
> at org.apache.hadoop.fs.FileContext.listStatus(FileContext.java:1494)
> at
> org.apache.hadoop.yarn.logaggregation.LogCLIHelpers.getRemoteNodeFileDir(LogCLIHelpers.java:592)
> at
> org.apache.hadoop.yarn.logaggregation.LogCLIHelpers.dumpAllContainersLogs(LogCLIHelpers.java:348)
> at
> org.apache.hadoop.yarn.client.cli.LogsCLI.fetchApplicationLogs(LogsCLI.java:971)
> at
> org.apache.hadoop.yarn.client.cli.LogsCLI.runCommand(LogsCLI.java:299)
> at org.apache.hadoop.yarn.client.cli.LogsCLI.run(LogsCLI.java:106)
> at
> org.apache.hadoop.yarn.client.cli.TestLogsCLI.testFetchApplictionLogsAsAnotherUser(TestLogsCLI.java:868)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5664) Fix Yarn documentation to link to correct versions.
[
https://issues.apache.org/jira/browse/YARN-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15518466#comment-15518466
]
Xiao Chen commented on YARN-5664:
-
Thanks [~yufeigu] and [~Naganarasimha] for reviews and (coming) commit!
> Fix Yarn documentation to link to correct versions.
> ---
>
> Key: YARN-5664
> URL: https://issues.apache.org/jira/browse/YARN-5664
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Xiao Chen
>Assignee: Xiao Chen
>Priority: Minor
> Attachments: YARN-5664.01.patch
>
>
> Found out that some links in Yarn's doc is pointing to {{current}}. They
> should point to the version they're on.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5664) Fix Yarn documentation to link to correct versions.
[
https://issues.apache.org/jira/browse/YARN-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Chen updated YARN-5664:
Priority: Minor (was: Major)
> Fix Yarn documentation to link to correct versions.
> ---
>
> Key: YARN-5664
> URL: https://issues.apache.org/jira/browse/YARN-5664
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Xiao Chen
>Assignee: Xiao Chen
>Priority: Minor
> Attachments: YARN-5664.01.patch
>
>
> Found out that some links in Yarn's doc is pointing to {{current}}. They
> should point to the version they're on.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5664) Fix Yarn documentation to link to correct versions.
[
https://issues.apache.org/jira/browse/YARN-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Chen updated YARN-5664:
Attachment: YARN-5664.01.patch
Also some minor typos:
CapacityScheduler.md: near peak -> near-peak
YARN.md: based the resource requirements -> based on the resource requirements
Others are link fixes. Compiled locally and verified the links to work.
> Fix Yarn documentation to link to correct versions.
> ---
>
> Key: YARN-5664
> URL: https://issues.apache.org/jira/browse/YARN-5664
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Xiao Chen
>Assignee: Xiao Chen
> Attachments: YARN-5664.01.patch
>
>
> Found out that some links in Yarn's doc is pointing to {{current}}. They
> should point to the version they're on.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-5664) Fix Yarn documentation to link to correct versions.
Xiao Chen created YARN-5664:
---
Summary: Fix Yarn documentation to link to correct versions.
Key: YARN-5664
URL: https://issues.apache.org/jira/browse/YARN-5664
Project: Hadoop YARN
Issue Type: Bug
Reporter: Xiao Chen
Assignee: Xiao Chen
Found out that some links in Yarn's doc is pointing to {{current}}. They should
point to the version they're on.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
