[jira] [Commented] (YARN-4630) Remove useless boxing/unboxing code (Hadoop YARN)
[ https://issues.apache.org/jira/browse/YARN-4630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15234519#comment-15234519 ] Akira AJISAKA commented on YARN-4630: - +1, the test failures seem to be unrelated. I ran all the failed tests and they passed except TestNodeLabelContainerAllocation, which is tracked by YARN-4890. Committing this. > Remove useless boxing/unboxing code (Hadoop YARN) > - > > Key: YARN-4630 > URL: https://issues.apache.org/jira/browse/YARN-4630 > Project: Hadoop YARN > Issue Type: Improvement > Components: yarn >Affects Versions: 3.0.0 >Reporter: Kousuke Saruta >Assignee: Kousuke Saruta >Priority: Minor > Attachments: YARN-4630.0.patch, YARN-4630.1.patch, YARN-4630.2.patch > > > There are lots of places where useless boxing/unboxing occur. > To avoid performance issue, let's remove them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-4630) Remove useless boxing/unboxing code
[ https://issues.apache.org/jira/browse/YARN-4630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA updated YARN-4630: Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Summary: Remove useless boxing/unboxing code (was: Remove useless boxing/unboxing code (Hadoop YARN)) > Remove useless boxing/unboxing code > --- > > Key: YARN-4630 > URL: https://issues.apache.org/jira/browse/YARN-4630 > Project: Hadoop YARN > Issue Type: Improvement > Components: yarn >Affects Versions: 3.0.0 >Reporter: Kousuke Saruta >Assignee: Kousuke Saruta >Priority: Minor > Fix For: 2.8.0 > > Attachments: YARN-4630.0.patch, YARN-4630.1.patch, YARN-4630.2.patch > > > There are lots of places where useless boxing/unboxing occur. > To avoid performance issue, let's remove them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3344) procfs stat file is not in the expected format warning
[
https://issues.apache.org/jira/browse/YARN-3344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15213852#comment-15213852
]
Akira AJISAKA commented on YARN-3344:
-
Hi [~vinodkv], would you please review the latest patch?
> procfs stat file is not in the expected format warning
> --
>
> Key: YARN-3344
> URL: https://issues.apache.org/jira/browse/YARN-3344
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 2.6.0
>Reporter: Jon Bringhurst
>Assignee: Akira AJISAKA
> Attachments: YARN-3344-trunk.005.patch, YARN-3344.06.patch
>
>
> Although this doesn't appear to be causing any functional issues, it is
> spamming our log files quite a bit. :)
> It appears that the regex in ProcfsBasedProcessTree doesn't work for all
> /proc//stat files.
> Here's the error I'm seeing:
> {noformat}
> "source_host": "asdf",
> "method": "constructProcessInfo",
> "level": "WARN",
> "message": "Unexpected: procfs stat file is not in the expected format
> for process with pid 6953"
> "file": "ProcfsBasedProcessTree.java",
> "line_number": "514",
> "class": "org.apache.hadoop.yarn.util.ProcfsBasedProcessTree",
> {noformat}
> And here's the basic info on process with pid 6953:
> {noformat}
> [asdf ~]$ cat /proc/6953/stat
> 6953 (python2.6 /expo) S 1871 1871 1871 0 -1 4202496 9364 1080 0 0 25 3 0 0
> 20 0 1 0 144918696 205295616 5856 18446744073709551615 1 1 0 0 0 0 0 16781312
> 2 18446744073709551615 0 0 17 13 0 0 0 0 0
> [asdf ~]$ ps aux|grep 6953
> root 6953 0.0 0.0 200484 23424 ?S21:44 0:00 python2.6
> /export/apps/salt/minion-scripts/module-sync.py
> jbringhu 13481 0.0 0.0 105312 872 pts/0S+ 22:13 0:00 grep -i 6953
> [asdf ~]$
> {noformat}
> This is using 2.6.32-431.11.2.el6.x86_64 in RHEL 6.5.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4686) MiniYARNCluster.start() returns before cluster is completely started
[
https://issues.apache.org/jira/browse/YARN-4686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15208103#comment-15208103
]
Akira AJISAKA commented on YARN-4686:
-
Hi [~ebadger] and [~eepayne], TestMRJobs#testJobWithChangePriority is failing
after this issue. Would you fix the test failure?
I've filed MAPREDUCE-6658 for fixing the failure.
> MiniYARNCluster.start() returns before cluster is completely started
>
>
> Key: YARN-4686
> URL: https://issues.apache.org/jira/browse/YARN-4686
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Rohith Sharma K S
>Assignee: Eric Badger
> Fix For: 2.7.3
>
> Attachments: MAPREDUCE-6507.001.patch,
> YARN-4686-branch-2.7.006.patch, YARN-4686.001.patch, YARN-4686.002.patch,
> YARN-4686.003.patch, YARN-4686.004.patch, YARN-4686.005.patch,
> YARN-4686.006.patch
>
>
> TestRMNMInfo fails intermittently. Below is trace for the failure
> {noformat}
> testRMNMInfo(org.apache.hadoop.mapreduce.v2.TestRMNMInfo) Time elapsed: 0.28
> sec <<< FAILURE!
> java.lang.AssertionError: Unexpected number of live nodes: expected:<4> but
> was:<3>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at
> org.apache.hadoop.mapreduce.v2.TestRMNMInfo.testRMNMInfo(TestRMNMInfo.java:111)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-5107) TestContainerMetrics fails
[
https://issues.apache.org/jira/browse/YARN-5107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15290908#comment-15290908
]
Akira AJISAKA commented on YARN-5107:
-
bq. What I want to know is how this test has been failing consistently for 16
months without anyone noticing...
I guess the test was broken by upgrading the version of java compiler.
(HADOOP-11858) The test passed at May 16 01:54, but the test failed at May 16
23:05.
https://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201605.mbox/%3C1735640484.5717.1463363683007.JavaMail.jenkins@crius%3E
https://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201605.mbox/%3C298927843.6177.1463439935479.JavaMail.jenkins@crius%3E
> TestContainerMetrics fails
> --
>
> Key: YARN-5107
> URL: https://issues.apache.org/jira/browse/YARN-5107
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Akira AJISAKA
>Assignee: Akira AJISAKA
> Fix For: 3.0.0-alpha1
>
> Attachments: YARN-5107.01.patch
>
>
> {noformat}
> testContainerMetricsFlow(org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainerMetrics)
> Time elapsed: 0.022 sec <<< ERROR!
> org.mockito.exceptions.misusing.WrongTypeOfReturnValue:
> TestContainerMetrics cannot be returned by register()
> register() should return MetricsSink
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainerMetrics.testContainerMetricsFlow(TestContainerMetrics.java:53)
> testContainerMetricsLimit(org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainerMetrics)
> Time elapsed: 0.001 sec <<< ERROR!
> org.mockito.exceptions.misusing.WrongTypeOfReturnValue:
> TestContainerMetrics cannot be returned by register()
> register() should return MetricsSink
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainerMetrics.testContainerMetricsLimit(TestContainerMetrics.java:93)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5126) Remove CHANGES.txt from branch-2
[ https://issues.apache.org/jira/browse/YARN-5126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA updated YARN-5126: Attachment: YARN-5126.branch-2.01.patch > Remove CHANGES.txt from branch-2 > > > Key: YARN-5126 > URL: https://issues.apache.org/jira/browse/YARN-5126 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Akira AJISAKA > Labels: newbie > Attachments: YARN-5126.branch-2.01.patch > > > CHANGES.txt was added accidentally when [cherry-picking > YARN-2882|https://github.com/apache/hadoop/commit/2c6c701affb99f250ec25abae7f76c44bc5b701a]. > We should remove the changelog. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-2877) Extend YARN to support distributed scheduling
[ https://issues.apache.org/jira/browse/YARN-2877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15295735#comment-15295735 ] Akira AJISAKA commented on YARN-2877: - Hi [~asuresh], would you update the fix version field of the cherry-picked jiras? In addition, CHANGES.txt was added when cherry-picking YARN-2882. I filed YARN-5126 to remove it. > Extend YARN to support distributed scheduling > - > > Key: YARN-2877 > URL: https://issues.apache.org/jira/browse/YARN-2877 > Project: Hadoop YARN > Issue Type: New Feature > Components: nodemanager, resourcemanager >Reporter: Sriram Rao >Assignee: Konstantinos Karanasos > Attachments: distributed-scheduling-design-doc_v1.pdf > > > This is an umbrella JIRA that proposes to extend YARN to support distributed > scheduling. Briefly, some of the motivations for distributed scheduling are > the following: > 1. Improve cluster utilization by opportunistically executing tasks otherwise > idle resources on individual machines. > 2. Reduce allocation latency. Tasks where the scheduling time dominates > (i.e., task execution time is much less compared to the time required for > obtaining a container from the RM). > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-5126) Remove CHANGES.txt from branch-2
Akira AJISAKA created YARN-5126: --- Summary: Remove CHANGES.txt from branch-2 Key: YARN-5126 URL: https://issues.apache.org/jira/browse/YARN-5126 Project: Hadoop YARN Issue Type: Bug Reporter: Akira AJISAKA CHANGES.txt was added accidentally when [cherry-picking YARN-2882|https://github.com/apache/hadoop/commit/2c6c701affb99f250ec25abae7f76c44bc5b701a]. We should remove the changelog. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-3344) procfs stat file is not in the expected format warning
[
https://issues.apache.org/jira/browse/YARN-3344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira AJISAKA updated YARN-3344:
Attachment: YARN-3344.07.patch
Nice catch, [~templedf]. Updated the patch to address the comment.
> procfs stat file is not in the expected format warning
> --
>
> Key: YARN-3344
> URL: https://issues.apache.org/jira/browse/YARN-3344
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 2.6.0
>Reporter: Jon Bringhurst
>Assignee: Akira AJISAKA
> Attachments: YARN-3344-trunk.005.patch, YARN-3344.06.patch,
> YARN-3344.07.patch
>
>
> Although this doesn't appear to be causing any functional issues, it is
> spamming our log files quite a bit. :)
> It appears that the regex in ProcfsBasedProcessTree doesn't work for all
> /proc//stat files.
> Here's the error I'm seeing:
> {noformat}
> "source_host": "asdf",
> "method": "constructProcessInfo",
> "level": "WARN",
> "message": "Unexpected: procfs stat file is not in the expected format
> for process with pid 6953"
> "file": "ProcfsBasedProcessTree.java",
> "line_number": "514",
> "class": "org.apache.hadoop.yarn.util.ProcfsBasedProcessTree",
> {noformat}
> And here's the basic info on process with pid 6953:
> {noformat}
> [asdf ~]$ cat /proc/6953/stat
> 6953 (python2.6 /expo) S 1871 1871 1871 0 -1 4202496 9364 1080 0 0 25 3 0 0
> 20 0 1 0 144918696 205295616 5856 18446744073709551615 1 1 0 0 0 0 0 16781312
> 2 18446744073709551615 0 0 17 13 0 0 0 0 0
> [asdf ~]$ ps aux|grep 6953
> root 6953 0.0 0.0 200484 23424 ?S21:44 0:00 python2.6
> /export/apps/salt/minion-scripts/module-sync.py
> jbringhu 13481 0.0 0.0 105312 872 pts/0S+ 22:13 0:00 grep -i 6953
> [asdf ~]$
> {noformat}
> This is using 2.6.32-431.11.2.el6.x86_64 in RHEL 6.5.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-5075) Fix findbugs warning in hadoop-yarn-common module
Akira AJISAKA created YARN-5075: --- Summary: Fix findbugs warning in hadoop-yarn-common module Key: YARN-5075 URL: https://issues.apache.org/jira/browse/YARN-5075 Project: Hadoop YARN Issue Type: Bug Reporter: Akira AJISAKA -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5075) Fix findbugs warning in hadoop-yarn-common module
[ https://issues.apache.org/jira/browse/YARN-5075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA updated YARN-5075: Attachment: findbugs.html Attaching a html file for the detail. > Fix findbugs warning in hadoop-yarn-common module > - > > Key: YARN-5075 > URL: https://issues.apache.org/jira/browse/YARN-5075 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Akira AJISAKA > Attachments: findbugs.html > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-3344) procfs stat file is not in the expected format warning
[
https://issues.apache.org/jira/browse/YARN-3344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287779#comment-15287779
]
Akira AJISAKA commented on YARN-3344:
-
bq. Clean up your whitespace issues
Is there any trailing whitespaces in my patch? Jenkins says the patch has no
whitespace issues.
> procfs stat file is not in the expected format warning
> --
>
> Key: YARN-3344
> URL: https://issues.apache.org/jira/browse/YARN-3344
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 2.6.0
>Reporter: Jon Bringhurst
>Assignee: Akira AJISAKA
> Attachments: YARN-3344-trunk.005.patch, YARN-3344.06.patch,
> YARN-3344.07.patch, YARN-3344.08.patch
>
>
> Although this doesn't appear to be causing any functional issues, it is
> spamming our log files quite a bit. :)
> It appears that the regex in ProcfsBasedProcessTree doesn't work for all
> /proc//stat files.
> Here's the error I'm seeing:
> {noformat}
> "source_host": "asdf",
> "method": "constructProcessInfo",
> "level": "WARN",
> "message": "Unexpected: procfs stat file is not in the expected format
> for process with pid 6953"
> "file": "ProcfsBasedProcessTree.java",
> "line_number": "514",
> "class": "org.apache.hadoop.yarn.util.ProcfsBasedProcessTree",
> {noformat}
> And here's the basic info on process with pid 6953:
> {noformat}
> [asdf ~]$ cat /proc/6953/stat
> 6953 (python2.6 /expo) S 1871 1871 1871 0 -1 4202496 9364 1080 0 0 25 3 0 0
> 20 0 1 0 144918696 205295616 5856 18446744073709551615 1 1 0 0 0 0 0 16781312
> 2 18446744073709551615 0 0 17 13 0 0 0 0 0
> [asdf ~]$ ps aux|grep 6953
> root 6953 0.0 0.0 200484 23424 ?S21:44 0:00 python2.6
> /export/apps/salt/minion-scripts/module-sync.py
> jbringhu 13481 0.0 0.0 105312 872 pts/0S+ 22:13 0:00 grep -i 6953
> [asdf ~]$
> {noformat}
> This is using 2.6.32-431.11.2.el6.x86_64 in RHEL 6.5.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-3344) procfs stat file is not in the expected format warning
[
https://issues.apache.org/jira/browse/YARN-3344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira AJISAKA updated YARN-3344:
Attachment: YARN-3344.09.patch
Fixed checkstyle issues. Thanks!
> procfs stat file is not in the expected format warning
> --
>
> Key: YARN-3344
> URL: https://issues.apache.org/jira/browse/YARN-3344
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 2.6.0
>Reporter: Jon Bringhurst
>Assignee: Akira AJISAKA
> Attachments: YARN-3344-trunk.005.patch, YARN-3344.06.patch,
> YARN-3344.07.patch, YARN-3344.08.patch, YARN-3344.09.patch
>
>
> Although this doesn't appear to be causing any functional issues, it is
> spamming our log files quite a bit. :)
> It appears that the regex in ProcfsBasedProcessTree doesn't work for all
> /proc//stat files.
> Here's the error I'm seeing:
> {noformat}
> "source_host": "asdf",
> "method": "constructProcessInfo",
> "level": "WARN",
> "message": "Unexpected: procfs stat file is not in the expected format
> for process with pid 6953"
> "file": "ProcfsBasedProcessTree.java",
> "line_number": "514",
> "class": "org.apache.hadoop.yarn.util.ProcfsBasedProcessTree",
> {noformat}
> And here's the basic info on process with pid 6953:
> {noformat}
> [asdf ~]$ cat /proc/6953/stat
> 6953 (python2.6 /expo) S 1871 1871 1871 0 -1 4202496 9364 1080 0 0 25 3 0 0
> 20 0 1 0 144918696 205295616 5856 18446744073709551615 1 1 0 0 0 0 0 16781312
> 2 18446744073709551615 0 0 17 13 0 0 0 0 0
> [asdf ~]$ ps aux|grep 6953
> root 6953 0.0 0.0 200484 23424 ?S21:44 0:00 python2.6
> /export/apps/salt/minion-scripts/module-sync.py
> jbringhu 13481 0.0 0.0 105312 872 pts/0S+ 22:13 0:00 grep -i 6953
> [asdf ~]$
> {noformat}
> This is using 2.6.32-431.11.2.el6.x86_64 in RHEL 6.5.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-5107) TestContainerMetrics fails
Akira AJISAKA created YARN-5107: --- Summary: TestContainerMetrics fails Key: YARN-5107 URL: https://issues.apache.org/jira/browse/YARN-5107 Project: Hadoop YARN Issue Type: Bug Components: test Reporter: Akira AJISAKA -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5107) TestContainerMetrics fails
[
https://issues.apache.org/jira/browse/YARN-5107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira AJISAKA updated YARN-5107:
Description:
{noformat}
testContainerMetricsFlow(org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainerMetrics)
Time elapsed: 0.022 sec <<< ERROR!
org.mockito.exceptions.misusing.WrongTypeOfReturnValue:
TestContainerMetrics cannot be returned by register()
register() should return MetricsSink
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainerMetrics.testContainerMetricsFlow(TestContainerMetrics.java:53)
testContainerMetricsLimit(org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainerMetrics)
Time elapsed: 0.001 sec <<< ERROR!
org.mockito.exceptions.misusing.WrongTypeOfReturnValue:
TestContainerMetrics cannot be returned by register()
register() should return MetricsSink
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainerMetrics.testContainerMetricsLimit(TestContainerMetrics.java:93)
{noformat}
> TestContainerMetrics fails
> --
>
> Key: YARN-5107
> URL: https://issues.apache.org/jira/browse/YARN-5107
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Akira AJISAKA
>
> {noformat}
> testContainerMetricsFlow(org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainerMetrics)
> Time elapsed: 0.022 sec <<< ERROR!
> org.mockito.exceptions.misusing.WrongTypeOfReturnValue:
> TestContainerMetrics cannot be returned by register()
> register() should return MetricsSink
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainerMetrics.testContainerMetricsFlow(TestContainerMetrics.java:53)
> testContainerMetricsLimit(org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainerMetrics)
> Time elapsed: 0.001 sec <<< ERROR!
> org.mockito.exceptions.misusing.WrongTypeOfReturnValue:
> TestContainerMetrics cannot be returned by register()
> register() should return MetricsSink
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainerMetrics.testContainerMetricsLimit(TestContainerMetrics.java:93)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-5107) TestContainerMetrics fails
[
https://issues.apache.org/jira/browse/YARN-5107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira AJISAKA reassigned YARN-5107:
---
Assignee: Akira AJISAKA
> TestContainerMetrics fails
> --
>
> Key: YARN-5107
> URL: https://issues.apache.org/jira/browse/YARN-5107
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Akira AJISAKA
>Assignee: Akira AJISAKA
> Attachments: YARN-5107.01.patch
>
>
> {noformat}
> testContainerMetricsFlow(org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainerMetrics)
> Time elapsed: 0.022 sec <<< ERROR!
> org.mockito.exceptions.misusing.WrongTypeOfReturnValue:
> TestContainerMetrics cannot be returned by register()
> register() should return MetricsSink
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainerMetrics.testContainerMetricsFlow(TestContainerMetrics.java:53)
> testContainerMetricsLimit(org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainerMetrics)
> Time elapsed: 0.001 sec <<< ERROR!
> org.mockito.exceptions.misusing.WrongTypeOfReturnValue:
> TestContainerMetrics cannot be returned by register()
> register() should return MetricsSink
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainerMetrics.testContainerMetricsLimit(TestContainerMetrics.java:93)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5107) TestContainerMetrics fails
[
https://issues.apache.org/jira/browse/YARN-5107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira AJISAKA updated YARN-5107:
Attachment: YARN-5107.01.patch
> TestContainerMetrics fails
> --
>
> Key: YARN-5107
> URL: https://issues.apache.org/jira/browse/YARN-5107
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Akira AJISAKA
> Attachments: YARN-5107.01.patch
>
>
> {noformat}
> testContainerMetricsFlow(org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainerMetrics)
> Time elapsed: 0.022 sec <<< ERROR!
> org.mockito.exceptions.misusing.WrongTypeOfReturnValue:
> TestContainerMetrics cannot be returned by register()
> register() should return MetricsSink
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainerMetrics.testContainerMetricsFlow(TestContainerMetrics.java:53)
> testContainerMetricsLimit(org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainerMetrics)
> Time elapsed: 0.001 sec <<< ERROR!
> org.mockito.exceptions.misusing.WrongTypeOfReturnValue:
> TestContainerMetrics cannot be returned by register()
> register() should return MetricsSink
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainerMetrics.testContainerMetricsLimit(TestContainerMetrics.java:93)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5107) TestContainerMetrics fails
[
https://issues.apache.org/jira/browse/YARN-5107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287879#comment-15287879
]
Akira AJISAKA commented on YARN-5107:
-
This issue seems to be related to JDK8 because the error does not occur in
branch-2.
> TestContainerMetrics fails
> --
>
> Key: YARN-5107
> URL: https://issues.apache.org/jira/browse/YARN-5107
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Akira AJISAKA
>Assignee: Akira AJISAKA
> Attachments: YARN-5107.01.patch
>
>
> {noformat}
> testContainerMetricsFlow(org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainerMetrics)
> Time elapsed: 0.022 sec <<< ERROR!
> org.mockito.exceptions.misusing.WrongTypeOfReturnValue:
> TestContainerMetrics cannot be returned by register()
> register() should return MetricsSink
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainerMetrics.testContainerMetricsFlow(TestContainerMetrics.java:53)
> testContainerMetricsLimit(org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainerMetrics)
> Time elapsed: 0.001 sec <<< ERROR!
> org.mockito.exceptions.misusing.WrongTypeOfReturnValue:
> TestContainerMetrics cannot be returned by register()
> register() should return MetricsSink
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainerMetrics.testContainerMetricsLimit(TestContainerMetrics.java:93)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-3344) procfs stat file is not in the expected format warning
[
https://issues.apache.org/jira/browse/YARN-3344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira AJISAKA updated YARN-3344:
Attachment: YARN-3344.08.patch
Thanks Daniel. Updated the patch to address your comment.
> procfs stat file is not in the expected format warning
> --
>
> Key: YARN-3344
> URL: https://issues.apache.org/jira/browse/YARN-3344
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 2.6.0
>Reporter: Jon Bringhurst
>Assignee: Akira AJISAKA
> Attachments: YARN-3344-trunk.005.patch, YARN-3344.06.patch,
> YARN-3344.07.patch, YARN-3344.08.patch
>
>
> Although this doesn't appear to be causing any functional issues, it is
> spamming our log files quite a bit. :)
> It appears that the regex in ProcfsBasedProcessTree doesn't work for all
> /proc//stat files.
> Here's the error I'm seeing:
> {noformat}
> "source_host": "asdf",
> "method": "constructProcessInfo",
> "level": "WARN",
> "message": "Unexpected: procfs stat file is not in the expected format
> for process with pid 6953"
> "file": "ProcfsBasedProcessTree.java",
> "line_number": "514",
> "class": "org.apache.hadoop.yarn.util.ProcfsBasedProcessTree",
> {noformat}
> And here's the basic info on process with pid 6953:
> {noformat}
> [asdf ~]$ cat /proc/6953/stat
> 6953 (python2.6 /expo) S 1871 1871 1871 0 -1 4202496 9364 1080 0 0 25 3 0 0
> 20 0 1 0 144918696 205295616 5856 18446744073709551615 1 1 0 0 0 0 0 16781312
> 2 18446744073709551615 0 0 17 13 0 0 0 0 0
> [asdf ~]$ ps aux|grep 6953
> root 6953 0.0 0.0 200484 23424 ?S21:44 0:00 python2.6
> /export/apps/salt/minion-scripts/module-sync.py
> jbringhu 13481 0.0 0.0 105312 872 pts/0S+ 22:13 0:00 grep -i 6953
> [asdf ~]$
> {noformat}
> This is using 2.6.32-431.11.2.el6.x86_64 in RHEL 6.5.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5075) Fix findbugs warning in hadoop-yarn-common module
[ https://issues.apache.org/jira/browse/YARN-5075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286103#comment-15286103 ] Akira AJISAKA commented on YARN-5075: - LGTM, +1. Thanks Arun. > Fix findbugs warning in hadoop-yarn-common module > - > > Key: YARN-5075 > URL: https://issues.apache.org/jira/browse/YARN-5075 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Akira AJISAKA >Assignee: Arun Suresh > Attachments: YARN-5075.001.patch, YARN-5075.002.patch, findbugs.html > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5159) Wrong Javadoc tag in MiniYarnCluster
[
https://issues.apache.org/jira/browse/YARN-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15377351#comment-15377351
]
Akira Ajisaka commented on YARN-5159:
-
Thank you for the detailed information! You are right. I checked whether the
mvn command passed, but I didn't actually see the generated html file. +1,
checking this in.
> Wrong Javadoc tag in MiniYarnCluster
>
>
> Key: YARN-5159
> URL: https://issues.apache.org/jira/browse/YARN-5159
> Project: Hadoop YARN
> Issue Type: Test
> Components: documentation
>Affects Versions: 2.6.0
>Reporter: Andras Bokor
>Assignee: Andras Bokor
> Fix For: 2.8.0
>
> Attachments: YARN-5159.01.patch, YARN-5159.02.patch,
> YARN-5159.03.patch
>
>
> {@YarnConfiguration.RM_SCHEDULER_INCLUDE_PORT_IN_NODE_NAME} is wrong. Should
> be changed to
> {@value YarnConfiguration#RM_SCHEDULER_INCLUDE_PORT_IN_NODE_NAME}
> Edit:
> I noted that due to java 8 javadoc restrictions the javadoc:test-javadoc goal
> fails on hadoop-yarn-server-tests project.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5425) TestDirectoryCollection.testCreateDirectories failed
[
https://issues.apache.org/jira/browse/YARN-5425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15397430#comment-15397430
]
Akira Ajisaka commented on YARN-5425:
-
After HADOOP-13073,
{code:title=TestDirectoryCollection#testCreateDirectories}
conf.set(CommonConfigurationKeys.FS_PERMISSIONS_UMASK_KEY, "077");
{code}
has effect on RawLocalFileSystem, but it has no effect on
{{FileContext.umask}}. Therefore {{localFs.getUMask()}} returns 022 and
{{localFs.setPermission(dir, perm)}} is not called in the following code.
{code:title=DirectoryCollection#createDir}
if (!perm.equals(perm.applyUMask(localFs.getUMask( {
localFs.setPermission(dir, perm);
}
{code}
> TestDirectoryCollection.testCreateDirectories failed
>
>
> Key: YARN-5425
> URL: https://issues.apache.org/jira/browse/YARN-5425
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 2.9.0
>Reporter: Yufei Gu
>Assignee: Yufei Gu
>
> {code}
> java.lang.AssertionError: local dir parent not created with proper
> permissions expected: but was:
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestDirectoryCollection.testCreateDirectories(TestDirectoryCollection.java:113)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5425) TestDirectoryCollection.testCreateDirectories failed
[
https://issues.apache.org/jira/browse/YARN-5425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated YARN-5425:
Attachment: YARN-5425.00.patch
Attaching a patch to remove local variable umask from FileContext.
> TestDirectoryCollection.testCreateDirectories failed
>
>
> Key: YARN-5425
> URL: https://issues.apache.org/jira/browse/YARN-5425
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 2.9.0
>Reporter: Yufei Gu
>Assignee: Yufei Gu
> Attachments: YARN-5425.00.patch
>
>
> {code}
> java.lang.AssertionError: local dir parent not created with proper
> permissions expected: but was:
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestDirectoryCollection.testCreateDirectories(TestDirectoryCollection.java:113)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5425) TestDirectoryCollection.testCreateDirectories failed
[
https://issues.apache.org/jira/browse/YARN-5425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15397437#comment-15397437
]
Akira Ajisaka commented on YARN-5425:
-
I noticed [~yufeigu] has already assigned. Sorry for taking this.
> TestDirectoryCollection.testCreateDirectories failed
>
>
> Key: YARN-5425
> URL: https://issues.apache.org/jira/browse/YARN-5425
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 2.9.0
>Reporter: Yufei Gu
>Assignee: Yufei Gu
> Attachments: YARN-5425.00.patch
>
>
> {code}
> java.lang.AssertionError: local dir parent not created with proper
> permissions expected: but was:
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestDirectoryCollection.testCreateDirectories(TestDirectoryCollection.java:113)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5425) TestDirectoryCollection.testCreateDirectories failed
[
https://issues.apache.org/jira/browse/YARN-5425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15398531#comment-15398531
]
Akira Ajisaka commented on YARN-5425:
-
Thanks [~boky01] and [~yufeigu] for the comments. Submitting patch.
> TestDirectoryCollection.testCreateDirectories failed
>
>
> Key: YARN-5425
> URL: https://issues.apache.org/jira/browse/YARN-5425
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 2.9.0
>Reporter: Yufei Gu
>Assignee: Yufei Gu
> Attachments: YARN-5425.00.patch
>
>
> {code}
> java.lang.AssertionError: local dir parent not created with proper
> permissions expected: but was:
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestDirectoryCollection.testCreateDirectories(TestDirectoryCollection.java:113)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-5425) TestDirectoryCollection.testCreateDirectories failed
[
https://issues.apache.org/jira/browse/YARN-5425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka reassigned YARN-5425:
---
Assignee: Akira Ajisaka (was: Yufei Gu)
> TestDirectoryCollection.testCreateDirectories failed
>
>
> Key: YARN-5425
> URL: https://issues.apache.org/jira/browse/YARN-5425
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Yufei Gu
>Assignee: Akira Ajisaka
> Attachments: YARN-5425.00.patch
>
>
> {code}
> java.lang.AssertionError: local dir parent not created with proper
> permissions expected: but was:
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestDirectoryCollection.testCreateDirectories(TestDirectoryCollection.java:113)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5159) Wrong Javadoc tag in MiniYarnCluster
[
https://issues.apache.org/jira/browse/YARN-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15376089#comment-15376089
]
Akira Ajisaka commented on YARN-5159:
-
bq. I tried that locally before but if I remove the package name the javadoc
engine will skip it.
Really? o.a.h.yarn.conf.YarnConfiguration is imported in MiniYarnCluster.java,
so I'm thinking that works. I tried that and succeed the following commands.
{noformat}
$ cd hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
$ mvn javadoc:test-javadoc
{noformat}
> Wrong Javadoc tag in MiniYarnCluster
>
>
> Key: YARN-5159
> URL: https://issues.apache.org/jira/browse/YARN-5159
> Project: Hadoop YARN
> Issue Type: Test
> Components: documentation
>Affects Versions: 2.6.0
>Reporter: Andras Bokor
>Assignee: Andras Bokor
> Fix For: 2.8.0
>
> Attachments: YARN-5159.01.patch, YARN-5159.02.patch,
> YARN-5159.03.patch
>
>
> {@YarnConfiguration.RM_SCHEDULER_INCLUDE_PORT_IN_NODE_NAME} is wrong. Should
> be changed to
> {@value YarnConfiguration#RM_SCHEDULER_INCLUDE_PORT_IN_NODE_NAME}
> Edit:
> I noted that due to java 8 javadoc restrictions the javadoc:test-javadoc goal
> fails on hadoop-yarn-server-tests project.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-4883) Make consistent operation name in AdminService
[
https://issues.apache.org/jira/browse/YARN-4883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated YARN-4883:
Priority: Minor (was: Trivial)
> Make consistent operation name in AdminService
> --
>
> Key: YARN-4883
> URL: https://issues.apache.org/jira/browse/YARN-4883
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
>Reporter: Kai Sasaki
>Assignee: Kai Sasaki
>Priority: Minor
> Attachments: YARN-4883.01.patch, YARN-4883.02.patch
>
>
> YARN-2740 changes {{argName}} to {{operation}} make them final.
> We can make other operation names consistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4883) Make consistent operation name in AdminService
[
https://issues.apache.org/jira/browse/YARN-4883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386824#comment-15386824
]
Akira Ajisaka commented on YARN-4883:
-
LGTM, +1.
> Make consistent operation name in AdminService
> --
>
> Key: YARN-4883
> URL: https://issues.apache.org/jira/browse/YARN-4883
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
>Reporter: Kai Sasaki
>Assignee: Kai Sasaki
>Priority: Trivial
> Attachments: YARN-4883.01.patch, YARN-4883.02.patch
>
>
> YARN-2740 changes {{argName}} to {{operation}} make them final.
> We can make other operation names consistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5383) Fix findbugs for nodemanager & checkstyle warnings in nodemanager.ContainerExecutor
[
https://issues.apache.org/jira/browse/YARN-5383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15378892#comment-15378892
]
Akira Ajisaka commented on YARN-5383:
-
LGTM, +1.
> Fix findbugs for nodemanager & checkstyle warnings in
> nodemanager.ContainerExecutor
> ---
>
> Key: YARN-5383
> URL: https://issues.apache.org/jira/browse/YARN-5383
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 3.0.0-alpha1
>Reporter: Vrushali C
>Assignee: Vrushali C
> Attachments: YARN-5383.01.patch
>
>
> Nodemanager build shows a findbugs warning
> {code}
> Performance Warnings
> Code Warning
> WMI
> org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor.writeLaunchEnv(OutputStream,
> Map, Map, List, Path, String) makes inefficient use of keySet iterator
> instead of entrySet iterator
> Bug type WMI_WRONG_MAP_ITERATOR (click for details)
> In class org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor
> In method
> org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor.writeLaunchEnv(OutputStream,
> Map, Map, List, Path, String)
> At ContainerExecutor.java:[line 330]
> Details
> WMI_WRONG_MAP_ITERATOR: Inefficient use of keySet iterator instead of
> entrySet iterator
> This method accesses the value of a Map entry, using a key that was retrieved
> from a keySet iterator. It is more efficient to use an iterator on the
> entrySet of the map, to avoid the Map.get(key) lookup.
> {code}
> There are also several checkstyle errors in the same class
> org.apache.hadoop.yarn.server.nodemanager.ContainerExecutorContainerExecutor
> {code}
> [ERROR]
> src/main/java/org/apache/hadoop/yarn/server/nodemanager/ContainerExecutor.java[308]
> (indentation) Indentation: 'ContainerLaunch' have incorrect indentation
> level 6, expected level should be 8.
> [ERROR]
> src/main/java/org/apache/hadoop/yarn/server/nodemanager/ContainerExecutor.java[319:29]
> (whitespace) WhitespaceAfter: ',' is not followed by whitespace.
> [ERROR]
> src/main/java/org/apache/hadoop/yarn/server/nodemanager/ContainerExecutor.java[474:52]
> (coding) HiddenField: 'conf' hides a field.
> [ERROR]
> src/main/java/org/apache/hadoop/yarn/server/nodemanager/ContainerExecutor.java[497:52]
> (coding) HiddenField: 'conf' hides a field.
> [ERROR]
> src/main/java/org/apache/hadoop/yarn/server/nodemanager/ContainerExecutor.java[522:52]
> (coding) HiddenField: 'conf' hides a field.
> [ERROR]
> src/main/java/org/apache/hadoop/yarn/server/nodemanager/ContainerExecutor.java[529]
> (sizes) LineLength: Line is longer than 80 characters (found 81).
> [ERROR]
> src/main/java/org/apache/hadoop/yarn/server/nodemanager/ContainerExecutor.java[571:21]
> (coding) HiddenField: 'conf' hides a field.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5047) Refactor nodeUpdate across schedulers
[
https://issues.apache.org/jira/browse/YARN-5047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15380288#comment-15380288
]
Akira Ajisaka commented on YARN-5047:
-
mvn javadoc:javadoc fails by error.
{noformat}
[ERROR]
/testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java:802:
error: unknown tag: returns
[ERROR] * @returns true if the scheduler is cleared to call assignContainer().
[ERROR] ^
{noformat}
Would you fix the error by unknown tag {{@returns}}?
{code}
/**
* Method determine whether assignContainers can be called.
* @returns true if the scheduler is cleared to call assignContainer().
*/
public boolean isReadyToAssignContainers() {
// Determine if work-preserving restart recovery time has not yet passed
return (!rmContext.isWorkPreservingRecoveryEnabled()
|| rmContext.isSchedulerReadyForAllocatingContainers());
}
{code}
> Refactor nodeUpdate across schedulers
> -
>
> Key: YARN-5047
> URL: https://issues.apache.org/jira/browse/YARN-5047
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacityscheduler, fairscheduler, scheduler
>Affects Versions: 3.0.0-alpha1
>Reporter: Ray Chiang
>Assignee: Ray Chiang
> Attachments: YARN-5047.001.patch, YARN-5047.002.patch,
> YARN-5047.003.patch, YARN-5047.004.patch, YARN-5047.005.patch,
> YARN-5047.006.patch
>
>
> FairScheduler#nodeUpdate() and CapacityScheduler#nodeUpdate() have a lot of
> commonality in their code. See about refactoring the common parts into
> AbstractYARNScheduler.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5309) SSLFactory truststore reloader thread leak in TimelineClientImpl
[ https://issues.apache.org/jira/browse/YARN-5309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15380326#comment-15380326 ] Akira Ajisaka commented on YARN-5309: - Hi [~tfriedr] and [~cheersyang], do you think this is a blocker for 2.7.3? If you think so, please set the target version to 2.7.3 so that we don't miss this issue in the release process. Thanks in advance. > SSLFactory truststore reloader thread leak in TimelineClientImpl > > > Key: YARN-5309 > URL: https://issues.apache.org/jira/browse/YARN-5309 > Project: Hadoop YARN > Issue Type: Bug > Components: timelineserver, yarn >Affects Versions: 2.7.1 >Reporter: Thomas Friedrich >Assignee: Weiwei Yang >Priority: Blocker > Attachments: YARN-5309.001.patch, YARN-5309.002.patch, > YARN-5309.003.patch, YARN-5309.004.patch, YARN-5309.005.patch > > > We found a similar issue as HADOOP-11368 in TimelineClientImpl. The class > creates an instance of SSLFactory in newSslConnConfigurator and subsequently > creates the ReloadingX509TrustManager instance which in turn starts a trust > store reloader thread. > However, the SSLFactory is never destroyed and hence the trust store reloader > threads are not killed. > This problem was observed by a customer who had SSL enabled in Hadoop and > submitted many queries against the HiveServer2. After a few days, the HS2 > instance crashed and from the Java dump we could see many (over 13000) > threads like this: > "Truststore reloader thread" #126 daemon prio=5 os_prio=0 > tid=0x7f680d2e3000 nid=0x98fd waiting on > condition [0x7f67e482c000] >java.lang.Thread.State: TIMED_WAITING (sleeping) > at java.lang.Thread.sleep(Native Method) > at org.apache.hadoop.security.ssl.ReloadingX509TrustManager.run > (ReloadingX509TrustManager.java:225) > at java.lang.Thread.run(Thread.java:745) > HiveServer2 uses the JobClient to submit a job: > Thread [HiveServer2-Background-Pool: Thread-188] (Suspended (breakpoint at > line 89 in > ReloadingX509TrustManager)) > owns: Object (id=464) > owns: Object (id=465) > owns: Object (id=466) > owns: ServiceLoader (id=210) > ReloadingX509TrustManager.(String, String, String, long) line: 89 > FileBasedKeyStoresFactory.init(SSLFactory$Mode) line: 209 > SSLFactory.init() line: 131 > TimelineClientImpl.newSslConnConfigurator(int, Configuration) line: 532 > TimelineClientImpl.newConnConfigurator(Configuration) line: 507 > TimelineClientImpl.serviceInit(Configuration) line: 269 > TimelineClientImpl(AbstractService).init(Configuration) line: 163 > YarnClientImpl.serviceInit(Configuration) line: 169 > YarnClientImpl(AbstractService).init(Configuration) line: 163 > ResourceMgrDelegate.serviceInit(Configuration) line: 102 > ResourceMgrDelegate(AbstractService).init(Configuration) line: 163 > ResourceMgrDelegate.(YarnConfiguration) line: 96 > YARNRunner.(Configuration) line: 112 > YarnClientProtocolProvider.create(Configuration) line: 34 > Cluster.initialize(InetSocketAddress, Configuration) line: 95 > Cluster.(InetSocketAddress, Configuration) line: 82 > Cluster.(Configuration) line: 75 > JobClient.init(JobConf) line: 475 > JobClient.(JobConf) line: 454 > MapRedTask(ExecDriver).execute(DriverContext) line: 401 > MapRedTask.execute(DriverContext) line: 137 > MapRedTask(Task).executeTask() line: 160 > TaskRunner.runSequential() line: 88 > Driver.launchTask(Task, String, boolean, String, int, > DriverContext) line: 1653 > Driver.execute() line: 1412 > For every job, a new instance of JobClient/YarnClientImpl/TimelineClientImpl > is created. But because the HS2 process stays up for days, the previous trust > store reloader threads are still hanging around in the HS2 process and > eventually use all the resources available. > It seems like a similar fix as HADOOP-11368 is needed in TimelineClientImpl > but it doesn't have a destroy method to begin with. > One option to avoid this problem is to disable the yarn timeline service > (yarn.timeline-service.enabled=false). -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5159) Wrong Javadoc tag in MiniYarnCluster
[
https://issues.apache.org/jira/browse/YARN-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366758#comment-15366758
]
Akira Ajisaka commented on YARN-5159:
-
Thank you for updating the patch!
{code}
* {@value
org.apache.hadoop.yarn.conf.YarnConfiguration#RM_SCHEDULER_INCLUDE_PORT_IN_NODE_NAME}
* should be set true in the configuration used to initialize the
minicluster.
{code}
I'm thinking we can make the lines shorter by replacing
{{org.apache.hadoop.yarn.conf.YarnConfiguration}} with {{YarnConfiguration}}.
That way the checkstyle warning can be fixed.
> Wrong Javadoc tag in MiniYarnCluster
>
>
> Key: YARN-5159
> URL: https://issues.apache.org/jira/browse/YARN-5159
> Project: Hadoop YARN
> Issue Type: Test
> Components: documentation
>Affects Versions: 2.6.0
>Reporter: Andras Bokor
>Assignee: Andras Bokor
> Fix For: 2.8.0
>
> Attachments: YARN-5159.01.patch, YARN-5159.02.patch
>
>
> {@YarnConfiguration.RM_SCHEDULER_INCLUDE_PORT_IN_NODE_NAME} is wrong. Should
> be changed to
> {@value YarnConfiguration#RM_SCHEDULER_INCLUDE_PORT_IN_NODE_NAME}
> Edit:
> I noted that due to java 8 javadoc restrictions the javadoc:test-javadoc goal
> fails on hadoop-yarn-server-tests project.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5159) Wrong Javadoc tag in MiniYarnCluster
[
https://issues.apache.org/jira/browse/YARN-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15355661#comment-15355661
]
Akira Ajisaka commented on YARN-5159:
-
Mostly looks good to me. Would you fix the checkstyle warning?
{noformat}
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/MiniYARNCluster.java:740:
* @throws InterruptedException if any thread has interrupted the current
thread.: Line is longer than 80 characters (found 83).
{noformat}
I'm +1 if that is addressed.
> Wrong Javadoc tag in MiniYarnCluster
>
>
> Key: YARN-5159
> URL: https://issues.apache.org/jira/browse/YARN-5159
> Project: Hadoop YARN
> Issue Type: Test
> Components: timelineserver
>Affects Versions: 2.6.0
>Reporter: Andras Bokor
>Assignee: Andras Bokor
>Priority: Minor
> Fix For: 2.8.0
>
> Attachments: YARN-5159.01.patch
>
>
> {@YarnConfiguration.RM_SCHEDULER_INCLUDE_PORT_IN_NODE_NAME} is wrong. Should
> be changed to
> {@value YarnConfiguration#RM_SCHEDULER_INCLUDE_PORT_IN_NODE_NAME}
> Edit:
> I noted that due to java 8 javadoc restrictions the javadoc:test-javadoc goal
> fails on hadoop-yarn-server-tests project.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5159) Wrong Javadoc tag in MiniYarnCluster
[
https://issues.apache.org/jira/browse/YARN-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated YARN-5159:
Priority: Major (was: Minor)
Component/s: (was: timelineserver)
documentation
> Wrong Javadoc tag in MiniYarnCluster
>
>
> Key: YARN-5159
> URL: https://issues.apache.org/jira/browse/YARN-5159
> Project: Hadoop YARN
> Issue Type: Test
> Components: documentation
>Affects Versions: 2.6.0
>Reporter: Andras Bokor
>Assignee: Andras Bokor
> Fix For: 2.8.0
>
> Attachments: YARN-5159.01.patch
>
>
> {@YarnConfiguration.RM_SCHEDULER_INCLUDE_PORT_IN_NODE_NAME} is wrong. Should
> be changed to
> {@value YarnConfiguration#RM_SCHEDULER_INCLUDE_PORT_IN_NODE_NAME}
> Edit:
> I noted that due to java 8 javadoc restrictions the javadoc:test-javadoc goal
> fails on hadoop-yarn-server-tests project.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4423) Cleanup lint warnings in resource manager
[
https://issues.apache.org/jira/browse/YARN-4423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851214#comment-15851214
]
Akira Ajisaka commented on YARN-4423:
-
Thanks [~templedf] for the cleanup! Mostly looks good to me.
I'm thinking we can remove the following two {{@SuppressWarnings}} annotations.
{code:title=SchedulingPolicy.java}
@SuppressWarnings("unchecked")
public static SchedulingPolicy parse(String policy)
throws AllocationConfigurationException {
@SuppressWarnings("rawtypes")
Class clazz;
{code}
> Cleanup lint warnings in resource manager
> -
>
> Key: YARN-4423
> URL: https://issues.apache.org/jira/browse/YARN-4423
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
>Affects Versions: 2.7.1
>Reporter: Daniel Templeton
>Assignee: Daniel Templeton
>Priority: Minor
> Labels: oct16-easy
> Attachments: YARN-4423.001.patch, YARN-4423.002.patch,
> YARN-4423.003.patch, YARN-4423.004.patch
>
>
> There are multiple lint warnings about unchecked usage. This JIRA is to
> clean them up, and maybe a few other quibbles as well.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4753) Use doxia macro to generate in-page TOC of YARN site documentation
[ https://issues.apache.org/jira/browse/YARN-4753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852489#comment-15852489 ] Akira Ajisaka commented on YARN-4753: - Hi [~iwasakims], would you rebase the patch? > Use doxia macro to generate in-page TOC of YARN site documentation > -- > > Key: YARN-4753 > URL: https://issues.apache.org/jira/browse/YARN-4753 > Project: Hadoop YARN > Issue Type: Improvement > Components: documentation >Affects Versions: 2.7.0 >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki > Labels: oct16-easy > Attachments: YARN-4753.001.patch > > > Since maven-site-plugin 3.5 was released, we can use toc macro in Markdown. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4520) FinishAppEvent is leaked in leveldb if no app's container running on this node
[ https://issues.apache.org/jira/browse/YARN-4520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871306#comment-15871306 ] Akira Ajisaka commented on YARN-4520: - Thanks [~sandflee] and [~jianhe] for the contribution. Filed YARN-6208 for improving the log. > FinishAppEvent is leaked in leveldb if no app's container running on this node > -- > > Key: YARN-4520 > URL: https://issues.apache.org/jira/browse/YARN-4520 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: sandflee >Assignee: sandflee > Fix For: 2.8.0, 3.0.0-alpha1 > > Attachments: YARN-4520.01.patch, YARN-4520.02.patch > > > once we restart nodemanager we see many logs like : > 2015-12-28 11:59:18,725 WARN > org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: > Event EventType: FINISH_APPLICATION sent to absent application > application_1446103803043_9892 > we find that the app containers are never started on NM and is released by AM > after allocated. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-6208) Improve the log when FinishAppEvent sent to the NodeManager which didn't run the application
Akira Ajisaka created YARN-6208:
---
Summary: Improve the log when FinishAppEvent sent to the
NodeManager which didn't run the application
Key: YARN-6208
URL: https://issues.apache.org/jira/browse/YARN-6208
Project: Hadoop YARN
Issue Type: Improvement
Reporter: Akira Ajisaka
Priority: Minor
When FinishAppEvent of an application is sent to a NodeManager and there are no
applications of the application ran on the NodeManager, we can see the
following log:
{code}
2015-12-28 11:59:18,725 WARN
org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl:
Event EventType: FINISH_APPLICATION sent to absent application
application_1446103803043_9892
{code}
YARN-4520 made the log as follows:
{code}
LOG.warn("couldn't find application " + appID + " while processing"
+ " FINISH_APPS event");
{code}
and I'm thinking it can be improved.
* Make the log WARN from INFO
* Add why the NodeManager couldn't find the application. For example, "because
there were no containers of the application ran on the NodeManager."
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4753) Use doxia macro to generate in-page TOC of YARN site documentation
[ https://issues.apache.org/jira/browse/YARN-4753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15864853#comment-15864853 ] Akira Ajisaka commented on YARN-4753: - +1, thanks Masatake. > Use doxia macro to generate in-page TOC of YARN site documentation > -- > > Key: YARN-4753 > URL: https://issues.apache.org/jira/browse/YARN-4753 > Project: Hadoop YARN > Issue Type: Improvement > Components: documentation >Affects Versions: 2.7.0 >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki > Labels: oct16-easy > Attachments: YARN-4753.001.patch, YARN-4753.002.patch > > > Since maven-site-plugin 3.5 was released, we can use toc macro in Markdown. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4036) Findbugs warnings in hadoop-yarn-server-common
[ https://issues.apache.org/jira/browse/YARN-4036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867096#comment-15867096 ] Akira Ajisaka commented on YARN-4036: - Now there are no findbugs warnings in hadoop-yarn-server-common module. Closing this. > Findbugs warnings in hadoop-yarn-server-common > -- > > Key: YARN-4036 > URL: https://issues.apache.org/jira/browse/YARN-4036 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn >Affects Versions: 2.7.0 >Reporter: Varun Saxena > Attachments: findbugsHtml.html > > > Refer to > https://issues.apache.org/jira/browse/YARN-3232?focusedCommentId=14679146=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14679146 -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-4036) Findbugs warnings in hadoop-yarn-server-common
[ https://issues.apache.org/jira/browse/YARN-4036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka resolved YARN-4036. - Resolution: Not A Problem > Findbugs warnings in hadoop-yarn-server-common > -- > > Key: YARN-4036 > URL: https://issues.apache.org/jira/browse/YARN-4036 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn >Affects Versions: 2.7.0 >Reporter: Varun Saxena > Attachments: findbugsHtml.html > > > Refer to > https://issues.apache.org/jira/browse/YARN-3232?focusedCommentId=14679146=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14679146 -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6110) Fix opportunistic containers document
[ https://issues.apache.org/jira/browse/YARN-6110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated YARN-6110: Attachment: YARN-6110.01.patch > Fix opportunistic containers document > - > > Key: YARN-6110 > URL: https://issues.apache.org/jira/browse/YARN-6110 > Project: Hadoop YARN > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.0-alpha2 >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka >Priority: Minor > Attachments: YARN-6110.01.patch > > > I noticed the following issues: > * There are some dead links > * Bullet point itemization is broken -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6110) Fix opportunistic containers document
[ https://issues.apache.org/jira/browse/YARN-6110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated YARN-6110: Summary: Fix opportunistic containers document (was: Fix opportunistic container document) > Fix opportunistic containers document > - > > Key: YARN-6110 > URL: https://issues.apache.org/jira/browse/YARN-6110 > Project: Hadoop YARN > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.0-alpha2 >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka >Priority: Minor > > I noticed the following issues: > * There are some dead links > * Bullet point itemization is broken -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-6110) Fix opportunistic container document
Akira Ajisaka created YARN-6110: --- Summary: Fix opportunistic container document Key: YARN-6110 URL: https://issues.apache.org/jira/browse/YARN-6110 Project: Hadoop YARN Issue Type: Bug Components: documentation Affects Versions: 3.0.0-alpha2 Reporter: Akira Ajisaka Assignee: Akira Ajisaka Priority: Minor I noticed the following issues: * There are some dead links * Bullet point itemization is broken -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6110) Fix opportunistic containers document
[ https://issues.apache.org/jira/browse/YARN-6110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated YARN-6110: Target Version/s: 2.9.0, 3.0.0-alpha2 (was: 3.0.0-alpha2) > Fix opportunistic containers document > - > > Key: YARN-6110 > URL: https://issues.apache.org/jira/browse/YARN-6110 > Project: Hadoop YARN > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.0-alpha2 >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka >Priority: Minor > Attachments: YARN-6110.01.patch > > > I noticed the following issues: > * There are some dead links > * Bullet point itemization is broken -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6110) Fix opportunistic containers document
[ https://issues.apache.org/jira/browse/YARN-6110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15829508#comment-15829508 ] Akira Ajisaka commented on YARN-6110: - Hi [~asuresh], would you review my patch? > Fix opportunistic containers document > - > > Key: YARN-6110 > URL: https://issues.apache.org/jira/browse/YARN-6110 > Project: Hadoop YARN > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.0-alpha2 >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka >Priority: Minor > Attachments: YARN-6110.01.patch > > > I noticed the following issues: > * There are some dead links > * Bullet point itemization is broken -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6110) Fix opportunistic containers documentation
[ https://issues.apache.org/jira/browse/YARN-6110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated YARN-6110: Release Note: (was: Committed to trunk and branch-2) Thanks [~asuresh] for the review & commit! > Fix opportunistic containers documentation > -- > > Key: YARN-6110 > URL: https://issues.apache.org/jira/browse/YARN-6110 > Project: Hadoop YARN > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.0-alpha2 >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka >Priority: Minor > Fix For: 2.9.0, 3.0.0-alpha2 > > Attachments: YARN-6110.01.patch > > > I noticed the following issues: > * There are some dead links > * Bullet point itemization is broken -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-5679) TestAHSWebServices is failing
Akira Ajisaka created YARN-5679:
---
Summary: TestAHSWebServices is failing
Key: YARN-5679
URL: https://issues.apache.org/jira/browse/YARN-5679
Project: Hadoop YARN
Issue Type: Bug
Components: timelineserver
Reporter: Akira Ajisaka
TestAHSWebServices.testContainerLogsForFinishedApps is failing.
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/176/testReport/
{noformat}
java.lang.AssertionError: null
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at
org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebServices.createContainerLogInLocalDir(TestAHSWebServices.java:675)
at
org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebServices.testContainerLogsForFinishedApps(TestAHSWebServices.java:581)
{noformat}
{noformat}
java.lang.AssertionError: null
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at
org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebServices.testContainerLogsForFinishedApps(TestAHSWebServices.java:519)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Moved] (YARN-5713) Update jackson from 1.9.13 to 2.x in hadoop-yarn
[ https://issues.apache.org/jira/browse/YARN-5713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka moved HADOOP-13677 to YARN-5713: -- Target Version/s: 3.0.0-alpha2 (was: 3.0.0-alpha2) Component/s: (was: build) build Key: YARN-5713 (was: HADOOP-13677) Project: Hadoop YARN (was: Hadoop Common) > Update jackson from 1.9.13 to 2.x in hadoop-yarn > > > Key: YARN-5713 > URL: https://issues.apache.org/jira/browse/YARN-5713 > Project: Hadoop YARN > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka > Attachments: HADOOP-13677.01.patch, HADOOP-13677.02.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5657) TestDefaultContainerExecutor fails
[
https://issues.apache.org/jira/browse/YARN-5657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated YARN-5657:
Assignee: Arun Suresh (was: Akira Ajisaka)
> TestDefaultContainerExecutor fails
> --
>
> Key: YARN-5657
> URL: https://issues.apache.org/jira/browse/YARN-5657
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Akira Ajisaka
>Assignee: Arun Suresh
> Attachments: YARN-5657.001.patch
>
>
> https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/166/testReport/junit/org.apache.hadoop.yarn.server.nodemanager/TestDefaultContainerExecutor/testStartLocalizer/
> {noformat}
> java.lang.AssertionError: StartLocalizer failed to copy token file:
> java.io.FileNotFoundException: File
> file:/testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/TestDefaultContainerExecutor/tokenDir/test.tokens
> does not exist
> at
> org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:645)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:871)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:635)
> at
> org.apache.hadoop.fs.DelegateToFileSystem.getFileStatus(DelegateToFileSystem.java:126)
> at org.apache.hadoop.fs.FilterFs.getFileStatus(FilterFs.java:121)
> at org.apache.hadoop.fs.FileContext$15.next(FileContext.java:1167)
> at org.apache.hadoop.fs.FileContext$15.next(FileContext.java:1163)
> at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
> at org.apache.hadoop.fs.FileContext.getFileStatus(FileContext.java:1169)
> at org.apache.hadoop.fs.FileContext$Util.copy(FileContext.java:2076)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestDefaultContainerExecutor.testStartLocalizer(TestDefaultContainerExecutor.java:452)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
> at org.junit.Assert.fail(Assert.java:88)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestDefaultContainerExecutor.testStartLocalizer(TestDefaultContainerExecutor.java:462)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5657) TestDefaultContainerExecutor fails
[
https://issues.apache.org/jira/browse/YARN-5657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15497094#comment-15497094
]
Akira Ajisaka commented on YARN-5657:
-
Thank you [~asuresh]. LGTM, +1 pending Jenkins.
> TestDefaultContainerExecutor fails
> --
>
> Key: YARN-5657
> URL: https://issues.apache.org/jira/browse/YARN-5657
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Akira Ajisaka
>Assignee: Arun Suresh
> Attachments: YARN-5657.001.patch
>
>
> https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/166/testReport/junit/org.apache.hadoop.yarn.server.nodemanager/TestDefaultContainerExecutor/testStartLocalizer/
> {noformat}
> java.lang.AssertionError: StartLocalizer failed to copy token file:
> java.io.FileNotFoundException: File
> file:/testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/TestDefaultContainerExecutor/tokenDir/test.tokens
> does not exist
> at
> org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:645)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:871)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:635)
> at
> org.apache.hadoop.fs.DelegateToFileSystem.getFileStatus(DelegateToFileSystem.java:126)
> at org.apache.hadoop.fs.FilterFs.getFileStatus(FilterFs.java:121)
> at org.apache.hadoop.fs.FileContext$15.next(FileContext.java:1167)
> at org.apache.hadoop.fs.FileContext$15.next(FileContext.java:1163)
> at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
> at org.apache.hadoop.fs.FileContext.getFileStatus(FileContext.java:1169)
> at org.apache.hadoop.fs.FileContext$Util.copy(FileContext.java:2076)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestDefaultContainerExecutor.testStartLocalizer(TestDefaultContainerExecutor.java:452)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
> at org.junit.Assert.fail(Assert.java:88)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestDefaultContainerExecutor.testStartLocalizer(TestDefaultContainerExecutor.java:462)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-5657) TestDefaultContainerExecutor fails
[
https://issues.apache.org/jira/browse/YARN-5657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka reassigned YARN-5657:
---
Assignee: Akira Ajisaka
> TestDefaultContainerExecutor fails
> --
>
> Key: YARN-5657
> URL: https://issues.apache.org/jira/browse/YARN-5657
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>
> https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/166/testReport/junit/org.apache.hadoop.yarn.server.nodemanager/TestDefaultContainerExecutor/testStartLocalizer/
> {noformat}
> java.lang.AssertionError: StartLocalizer failed to copy token file:
> java.io.FileNotFoundException: File
> file:/testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/TestDefaultContainerExecutor/tokenDir/test.tokens
> does not exist
> at
> org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:645)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:871)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:635)
> at
> org.apache.hadoop.fs.DelegateToFileSystem.getFileStatus(DelegateToFileSystem.java:126)
> at org.apache.hadoop.fs.FilterFs.getFileStatus(FilterFs.java:121)
> at org.apache.hadoop.fs.FileContext$15.next(FileContext.java:1167)
> at org.apache.hadoop.fs.FileContext$15.next(FileContext.java:1163)
> at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
> at org.apache.hadoop.fs.FileContext.getFileStatus(FileContext.java:1169)
> at org.apache.hadoop.fs.FileContext$Util.copy(FileContext.java:2076)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestDefaultContainerExecutor.testStartLocalizer(TestDefaultContainerExecutor.java:452)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
> at org.junit.Assert.fail(Assert.java:88)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestDefaultContainerExecutor.testStartLocalizer(TestDefaultContainerExecutor.java:462)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5622) TestYarnCLI.testGetContainers fails due to mismatched date formats
[
https://issues.apache.org/jira/browse/YARN-5622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15516022#comment-15516022
]
Akira Ajisaka commented on YARN-5622:
-
I couldn't reproduce the failure. Does this issue still occur on the latest
trunk?
> TestYarnCLI.testGetContainers fails due to mismatched date formats
> --
>
> Key: YARN-5622
> URL: https://issues.apache.org/jira/browse/YARN-5622
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Eric Badger
>Assignee: Eric Badger
> Attachments: YARN-5622.001.patch
>
>
> ApplicationCLI.listContainers uses Times.format to print timestamps, while
> TestYarnCLI.testGetContainers formats them using dateFormat.format with its
> own defined format. The test should be consistent and use Times.format.
> {noformat}
> org.junit.ComparisonFailure: expected:<...1234_0005_01_01 [Thu Jan 01
> 00:00:01 + 1970 Thu Jan 01 00:00:05 + 1970 COMPLETE
> host:1234http://host:2345
> logURL
> container_1234_0005_01_02Thu Jan 01 00:00:01 + 1970 Thu Jan
> 01 00:00:05 + 1970 COMPLETE host:1234
> http://host:2345 logURL
> container_1234_0005_01_03Thu Jan 01 00:00:01 + 1970]
> N/...> but was:<...1234_0005_01_01 [ 1-Jan-1970 00:00:01
> 1-Jan-1970 00:00:05COMPLETE host:1234
> http://host:2345 logURL
> container_1234_0005_01_02 1-Jan-1970 00:00:01 1-Jan-1970
> 00:00:05COMPLETE host:1234
> http://host:2345 logURL
> container_1234_0005_01_03 1-Jan-1970 00:00:01]
> N/...>
> at org.junit.Assert.assertEquals(Assert.java:115)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.hadoop.yarn.client.cli.TestYarnCLI.testGetContainers(TestYarnCLI.java:330)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5400) Light cleanup in ZKRMStateStore
[
https://issues.apache.org/jira/browse/YARN-5400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15515889#comment-15515889
]
Akira Ajisaka commented on YARN-5400:
-
LGTM, +1.
> Light cleanup in ZKRMStateStore
> ---
>
> Key: YARN-5400
> URL: https://issues.apache.org/jira/browse/YARN-5400
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
>Affects Versions: 2.9.0
>Reporter: Daniel Templeton
>Assignee: Daniel Templeton
>Priority: Trivial
> Attachments: YARN-5400.001.patch, YARN-5400.002.patch
>
>
> of {{ZKRMStateStore}} contains a plethora whitespace issues as well as some
> icky bits, like unused variables. This JIRA is to clean that up. It should
> have no functional impact.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5622) TestYarnCLI.testGetContainers fails due to mismatched date formats
[
https://issues.apache.org/jira/browse/YARN-5622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15516824#comment-15516824
]
Akira Ajisaka commented on YARN-5622:
-
Thank you for the detailed information. +1 for the improvement.
> TestYarnCLI.testGetContainers fails due to mismatched date formats
> --
>
> Key: YARN-5622
> URL: https://issues.apache.org/jira/browse/YARN-5622
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Eric Badger
>Assignee: Eric Badger
> Attachments: YARN-5622.001.patch
>
>
> ApplicationCLI.listContainers uses Times.format to print timestamps, while
> TestYarnCLI.testGetContainers formats them using dateFormat.format with its
> own defined format. The test should be consistent and use Times.format.
> {noformat}
> org.junit.ComparisonFailure: expected:<...1234_0005_01_01 [Thu Jan 01
> 00:00:01 + 1970 Thu Jan 01 00:00:05 + 1970 COMPLETE
> host:1234http://host:2345
> logURL
> container_1234_0005_01_02Thu Jan 01 00:00:01 + 1970 Thu Jan
> 01 00:00:05 + 1970 COMPLETE host:1234
> http://host:2345 logURL
> container_1234_0005_01_03Thu Jan 01 00:00:01 + 1970]
> N/...> but was:<...1234_0005_01_01 [ 1-Jan-1970 00:00:01
> 1-Jan-1970 00:00:05COMPLETE host:1234
> http://host:2345 logURL
> container_1234_0005_01_02 1-Jan-1970 00:00:01 1-Jan-1970
> 00:00:05COMPLETE host:1234
> http://host:2345 logURL
> container_1234_0005_01_03 1-Jan-1970 00:00:01]
> N/...>
> at org.junit.Assert.assertEquals(Assert.java:115)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.hadoop.yarn.client.cli.TestYarnCLI.testGetContainers(TestYarnCLI.java:330)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5622) TestYarnCLI.testGetContainers fails due to mismatched date formats
[
https://issues.apache.org/jira/browse/YARN-5622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated YARN-5622:
Priority: Minor (was: Major)
Hadoop Flags: Reviewed
Issue Type: Improvement (was: Bug)
> TestYarnCLI.testGetContainers fails due to mismatched date formats
> --
>
> Key: YARN-5622
> URL: https://issues.apache.org/jira/browse/YARN-5622
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Eric Badger
>Assignee: Eric Badger
>Priority: Minor
> Attachments: YARN-5622.001.patch
>
>
> ApplicationCLI.listContainers uses Times.format to print timestamps, while
> TestYarnCLI.testGetContainers formats them using dateFormat.format with its
> own defined format. The test should be consistent and use Times.format.
> {noformat}
> org.junit.ComparisonFailure: expected:<...1234_0005_01_01 [Thu Jan 01
> 00:00:01 + 1970 Thu Jan 01 00:00:05 + 1970 COMPLETE
> host:1234http://host:2345
> logURL
> container_1234_0005_01_02Thu Jan 01 00:00:01 + 1970 Thu Jan
> 01 00:00:05 + 1970 COMPLETE host:1234
> http://host:2345 logURL
> container_1234_0005_01_03Thu Jan 01 00:00:01 + 1970]
> N/...> but was:<...1234_0005_01_01 [ 1-Jan-1970 00:00:01
> 1-Jan-1970 00:00:05COMPLETE host:1234
> http://host:2345 logURL
> container_1234_0005_01_02 1-Jan-1970 00:00:01 1-Jan-1970
> 00:00:05COMPLETE host:1234
> http://host:2345 logURL
> container_1234_0005_01_03 1-Jan-1970 00:00:01]
> N/...>
> at org.junit.Assert.assertEquals(Assert.java:115)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.hadoop.yarn.client.cli.TestYarnCLI.testGetContainers(TestYarnCLI.java:330)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5657) TestDefaultContainerExecutor fails
[
https://issues.apache.org/jira/browse/YARN-5657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15498660#comment-15498660
]
Akira Ajisaka commented on YARN-5657:
-
This seems to be false-positive. Anyway, the following code fixes the warning.
{code}
}}).when(mockUtil).copy(any(Path.class), any(Path.class),
anyBoolean(), anyBoolean());
{code}
> TestDefaultContainerExecutor fails
> --
>
> Key: YARN-5657
> URL: https://issues.apache.org/jira/browse/YARN-5657
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Akira Ajisaka
>Assignee: Arun Suresh
> Attachments: YARN-5657.001.patch
>
>
> https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/166/testReport/junit/org.apache.hadoop.yarn.server.nodemanager/TestDefaultContainerExecutor/testStartLocalizer/
> {noformat}
> java.lang.AssertionError: StartLocalizer failed to copy token file:
> java.io.FileNotFoundException: File
> file:/testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/TestDefaultContainerExecutor/tokenDir/test.tokens
> does not exist
> at
> org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:645)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:871)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:635)
> at
> org.apache.hadoop.fs.DelegateToFileSystem.getFileStatus(DelegateToFileSystem.java:126)
> at org.apache.hadoop.fs.FilterFs.getFileStatus(FilterFs.java:121)
> at org.apache.hadoop.fs.FileContext$15.next(FileContext.java:1167)
> at org.apache.hadoop.fs.FileContext$15.next(FileContext.java:1163)
> at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
> at org.apache.hadoop.fs.FileContext.getFileStatus(FileContext.java:1169)
> at org.apache.hadoop.fs.FileContext$Util.copy(FileContext.java:2076)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestDefaultContainerExecutor.testStartLocalizer(TestDefaultContainerExecutor.java:452)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
> at org.junit.Assert.fail(Assert.java:88)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestDefaultContainerExecutor.testStartLocalizer(TestDefaultContainerExecutor.java:462)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5657) TestDefaultContainerExecutor fails
[
https://issues.apache.org/jira/browse/YARN-5657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15497047#comment-15497047
]
Akira Ajisaka commented on YARN-5657:
-
YARN-5620 made this change
{code:title=DefaultContainerExecutor.java}
protected void copyFile(Path src, Path dst, String owner) throws IOException
{
-lfs.util().copy(src, dst);
+lfs.util().copy(src, dst, false, true);
}
{code}
but the following code was not updated accordingly.
{code:title=TestDefaultContainerExecutor.java}
}).when(mockUtil).copy(any(Path.class), any(Path.class));
{code}
> TestDefaultContainerExecutor fails
> --
>
> Key: YARN-5657
> URL: https://issues.apache.org/jira/browse/YARN-5657
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Akira Ajisaka
>
> https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/166/testReport/junit/org.apache.hadoop.yarn.server.nodemanager/TestDefaultContainerExecutor/testStartLocalizer/
> {noformat}
> java.lang.AssertionError: StartLocalizer failed to copy token file:
> java.io.FileNotFoundException: File
> file:/testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/TestDefaultContainerExecutor/tokenDir/test.tokens
> does not exist
> at
> org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:645)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:871)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:635)
> at
> org.apache.hadoop.fs.DelegateToFileSystem.getFileStatus(DelegateToFileSystem.java:126)
> at org.apache.hadoop.fs.FilterFs.getFileStatus(FilterFs.java:121)
> at org.apache.hadoop.fs.FileContext$15.next(FileContext.java:1167)
> at org.apache.hadoop.fs.FileContext$15.next(FileContext.java:1163)
> at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
> at org.apache.hadoop.fs.FileContext.getFileStatus(FileContext.java:1169)
> at org.apache.hadoop.fs.FileContext$Util.copy(FileContext.java:2076)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestDefaultContainerExecutor.testStartLocalizer(TestDefaultContainerExecutor.java:452)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
> at org.junit.Assert.fail(Assert.java:88)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestDefaultContainerExecutor.testStartLocalizer(TestDefaultContainerExecutor.java:462)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-5657) TestDefaultContainerExecutor fails
Akira Ajisaka created YARN-5657:
---
Summary: TestDefaultContainerExecutor fails
Key: YARN-5657
URL: https://issues.apache.org/jira/browse/YARN-5657
Project: Hadoop YARN
Issue Type: Bug
Components: test
Reporter: Akira Ajisaka
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/166/testReport/junit/org.apache.hadoop.yarn.server.nodemanager/TestDefaultContainerExecutor/testStartLocalizer/
{noformat}
java.lang.AssertionError: StartLocalizer failed to copy token file:
java.io.FileNotFoundException: File
file:/testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/TestDefaultContainerExecutor/tokenDir/test.tokens
does not exist
at
org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:645)
at
org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:871)
at
org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:635)
at
org.apache.hadoop.fs.DelegateToFileSystem.getFileStatus(DelegateToFileSystem.java:126)
at org.apache.hadoop.fs.FilterFs.getFileStatus(FilterFs.java:121)
at org.apache.hadoop.fs.FileContext$15.next(FileContext.java:1167)
at org.apache.hadoop.fs.FileContext$15.next(FileContext.java:1163)
at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
at org.apache.hadoop.fs.FileContext.getFileStatus(FileContext.java:1169)
at org.apache.hadoop.fs.FileContext$Util.copy(FileContext.java:2076)
at
org.apache.hadoop.yarn.server.nodemanager.TestDefaultContainerExecutor.testStartLocalizer(TestDefaultContainerExecutor.java:452)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
at org.junit.Assert.fail(Assert.java:88)
at
org.apache.hadoop.yarn.server.nodemanager.TestDefaultContainerExecutor.testStartLocalizer(TestDefaultContainerExecutor.java:462)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5657) TestDefaultContainerExecutor fails
[
https://issues.apache.org/jira/browse/YARN-5657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15499293#comment-15499293
]
Akira Ajisaka commented on YARN-5657:
-
+1, thanks [~asuresh] for updating the patch.
> TestDefaultContainerExecutor fails
> --
>
> Key: YARN-5657
> URL: https://issues.apache.org/jira/browse/YARN-5657
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Akira Ajisaka
>Assignee: Arun Suresh
> Attachments: YARN-5657.001.patch, YARN-5657.002.patch
>
>
> https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/166/testReport/junit/org.apache.hadoop.yarn.server.nodemanager/TestDefaultContainerExecutor/testStartLocalizer/
> {noformat}
> java.lang.AssertionError: StartLocalizer failed to copy token file:
> java.io.FileNotFoundException: File
> file:/testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/TestDefaultContainerExecutor/tokenDir/test.tokens
> does not exist
> at
> org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:645)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:871)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:635)
> at
> org.apache.hadoop.fs.DelegateToFileSystem.getFileStatus(DelegateToFileSystem.java:126)
> at org.apache.hadoop.fs.FilterFs.getFileStatus(FilterFs.java:121)
> at org.apache.hadoop.fs.FileContext$15.next(FileContext.java:1167)
> at org.apache.hadoop.fs.FileContext$15.next(FileContext.java:1163)
> at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
> at org.apache.hadoop.fs.FileContext.getFileStatus(FileContext.java:1169)
> at org.apache.hadoop.fs.FileContext$Util.copy(FileContext.java:2076)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestDefaultContainerExecutor.testStartLocalizer(TestDefaultContainerExecutor.java:452)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
> at org.junit.Assert.fail(Assert.java:88)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestDefaultContainerExecutor.testStartLocalizer(TestDefaultContainerExecutor.java:462)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5657) TestDefaultContainerExecutor fails
[
https://issues.apache.org/jira/browse/YARN-5657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15496941#comment-15496941
]
Akira Ajisaka commented on YARN-5657:
-
Ran git bisect and found YARN-5620 broke this test.
> TestDefaultContainerExecutor fails
> --
>
> Key: YARN-5657
> URL: https://issues.apache.org/jira/browse/YARN-5657
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Akira Ajisaka
>
> https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/166/testReport/junit/org.apache.hadoop.yarn.server.nodemanager/TestDefaultContainerExecutor/testStartLocalizer/
> {noformat}
> java.lang.AssertionError: StartLocalizer failed to copy token file:
> java.io.FileNotFoundException: File
> file:/testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/TestDefaultContainerExecutor/tokenDir/test.tokens
> does not exist
> at
> org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:645)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:871)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:635)
> at
> org.apache.hadoop.fs.DelegateToFileSystem.getFileStatus(DelegateToFileSystem.java:126)
> at org.apache.hadoop.fs.FilterFs.getFileStatus(FilterFs.java:121)
> at org.apache.hadoop.fs.FileContext$15.next(FileContext.java:1167)
> at org.apache.hadoop.fs.FileContext$15.next(FileContext.java:1163)
> at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
> at org.apache.hadoop.fs.FileContext.getFileStatus(FileContext.java:1169)
> at org.apache.hadoop.fs.FileContext$Util.copy(FileContext.java:2076)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestDefaultContainerExecutor.testStartLocalizer(TestDefaultContainerExecutor.java:452)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
> at org.junit.Assert.fail(Assert.java:88)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestDefaultContainerExecutor.testStartLocalizer(TestDefaultContainerExecutor.java:462)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5620) Core changes in NodeManager to support re-initialization of Containers with new launchContext
[
https://issues.apache.org/jira/browse/YARN-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15496974#comment-15496974
]
Akira Ajisaka commented on YARN-5620:
-
Hi [~asuresh], [~jianhe], and [~vvasudev],
This commit broke YARN-5657. Would you look into this issue?
> Core changes in NodeManager to support re-initialization of Containers with
> new launchContext
> -
>
> Key: YARN-5620
> URL: https://issues.apache.org/jira/browse/YARN-5620
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: Arun Suresh
> Fix For: 2.9.0, 3.0.0-alpha2
>
> Attachments: YARN-5620.001.patch, YARN-5620.002.patch,
> YARN-5620.003.patch, YARN-5620.004.patch, YARN-5620.005.patch,
> YARN-5620.006.patch, YARN-5620.007.patch, YARN-5620.008.patch,
> YARN-5620.009.patch, YARN-5620.010.patch, YARN-5620.011.patch,
> YARN-5620.012.patch, YARN-5620.013.patch, YARN-5620.014.patch,
> YARN-5620.015.patch, YARN-5620.016.patch
>
>
> JIRA proposes to modify the ContainerManager (and other core classes) to
> support upgrade of a running container with a new {{ContainerLaunchContext}}
> as well as the ability to rollback the upgrade if the container is not able
> to restart using the new launch Context.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5575) Many classes use bare yarn. properties instead of the defined constants
[
https://issues.apache.org/jira/browse/YARN-5575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15607601#comment-15607601
]
Akira Ajisaka commented on YARN-5575:
-
+1 make sense to me. Committing this.
> Many classes use bare yarn. properties instead of the defined constants
> ---
>
> Key: YARN-5575
> URL: https://issues.apache.org/jira/browse/YARN-5575
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Daniel Templeton
>Assignee: Daniel Templeton
> Attachments: YARN-5575.001.patch, YARN-5575.002.patch,
> YARN-5575.003.patch
>
>
> MAPREDUCE-5870 introduced the following line:
> {code}
> conf.setInt("yarn.cluster.max-application-priority", 10);
> {code}
> It should instead be:
> {code}
> conf.setInt(YarnConfiguration.MAX_CLUSTER_LEVEL_APPLICATION_PRIORITY,
> 10);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5713) Update jackson from 1.9.13 to 2.x in hadoop-yarn
[ https://issues.apache.org/jira/browse/YARN-5713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated YARN-5713: Attachment: YARN-5713.04.patch Thanks [~ste...@apache.org] for reviewing. Attaching v4 patch: * Set jackson-module-jaxb-annotations and jackson-jaxrs-json-provider versions in hadoop-project pom. * Fixed checkstyle warning. > Update jackson from 1.9.13 to 2.x in hadoop-yarn > > > Key: YARN-5713 > URL: https://issues.apache.org/jira/browse/YARN-5713 > Project: Hadoop YARN > Issue Type: Improvement > Components: build, timelineserver >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka > Labels: oct16-medium > Attachments: HADOOP-13677.01.patch, HADOOP-13677.02.patch, > YARN-5713.03.patch, YARN-5713.04.patch > > > Sub-task of HADOOP-13332. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5007) Remove deprecated constructors of MiniYARNCluster and MiniMRYarnCluster
[ https://issues.apache.org/jira/browse/YARN-5007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated YARN-5007: Priority: Major (was: Minor) Hadoop Flags: Reviewed Component/s: (was: timelineserver) Summary: Remove deprecated constructors of MiniYARNCluster and MiniMRYarnCluster (was: MiniYarnCluster contains deprecated constructor which is called by the other constructors) > Remove deprecated constructors of MiniYARNCluster and MiniMRYarnCluster > --- > > Key: YARN-5007 > URL: https://issues.apache.org/jira/browse/YARN-5007 > Project: Hadoop YARN > Issue Type: Test >Reporter: Andras Bokor >Assignee: Andras Bokor > Attachments: YARN-5007.01.patch, YARN-5007.02.patch > > > MiniYarnCluster has a deprecated constructor which is called by the other > constructors and it causes javac warnings during the build. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5007) MiniYarnCluster contains deprecated constructor which is called by the other constructors
[ https://issues.apache.org/jira/browse/YARN-5007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587780#comment-15587780 ] Akira Ajisaka commented on YARN-5007: - LGTM, +1. > MiniYarnCluster contains deprecated constructor which is called by the other > constructors > - > > Key: YARN-5007 > URL: https://issues.apache.org/jira/browse/YARN-5007 > Project: Hadoop YARN > Issue Type: Test > Components: timelineserver >Reporter: Andras Bokor >Assignee: Andras Bokor >Priority: Minor > Attachments: YARN-5007.01.patch, YARN-5007.02.patch > > > MiniYarnCluster has a deprecated constructor which is called by the other > constructors and it causes javac warnings during the build. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5007) Remove deprecated constructors of MiniYARNCluster and MiniMRYarnCluster
[ https://issues.apache.org/jira/browse/YARN-5007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated YARN-5007: Target Version/s: 3.0.0-alpha2 (was: 2.8.0) > Remove deprecated constructors of MiniYARNCluster and MiniMRYarnCluster > --- > > Key: YARN-5007 > URL: https://issues.apache.org/jira/browse/YARN-5007 > Project: Hadoop YARN > Issue Type: Test >Reporter: Andras Bokor >Assignee: Andras Bokor > Attachments: YARN-5007.01.patch, YARN-5007.02.patch > > > MiniYarnCluster has a deprecated constructor which is called by the other > constructors and it causes javac warnings during the build. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-5007) Remove deprecated constructors of MiniYARNCluster and MiniMRYarnCluster
[ https://issues.apache.org/jira/browse/YARN-5007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587900#comment-15587900 ] Akira Ajisaka edited comment on YARN-5007 at 10/19/16 7:17 AM: --- +1 for this patch but not committing for now. YARN-3573 deprecated the constructors but the fix version (2.8.0) is not released. We need to wait for 2.8.0 release before committing this to trunk. This is the same as the reason why HADOOP-13705 reverted HADOOP-13534. was (Author: ajisakaa): +1 for this patch but not committing for now. YARN-3573 deprecated the constructors but the fix version (2.8.0) is not released. We need to wait for 2.8.0 release before committing this. This is the same as the reason why HADOOP-13705 reverted HADOOP-13534. > Remove deprecated constructors of MiniYARNCluster and MiniMRYarnCluster > --- > > Key: YARN-5007 > URL: https://issues.apache.org/jira/browse/YARN-5007 > Project: Hadoop YARN > Issue Type: Test >Reporter: Andras Bokor >Assignee: Andras Bokor > Attachments: YARN-5007.01.patch, YARN-5007.02.patch > > > MiniYarnCluster has a deprecated constructor which is called by the other > constructors and it causes javac warnings during the build. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5007) Remove deprecated constructors of MiniYARNCluster and MiniMRYarnCluster
[ https://issues.apache.org/jira/browse/YARN-5007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587900#comment-15587900 ] Akira Ajisaka commented on YARN-5007: - +1 for this patch but not committing for now. YARN-3573 deprecated the constructors but the fix version (2.8.0) is not released. We need to wait for 2.8.0 release before committing this. This is the same as the reason why HADOOP-13705 reverted HADOOP-13534. > Remove deprecated constructors of MiniYARNCluster and MiniMRYarnCluster > --- > > Key: YARN-5007 > URL: https://issues.apache.org/jira/browse/YARN-5007 > Project: Hadoop YARN > Issue Type: Test >Reporter: Andras Bokor >Assignee: Andras Bokor > Attachments: YARN-5007.01.patch, YARN-5007.02.patch > > > MiniYarnCluster has a deprecated constructor which is called by the other > constructors and it causes javac warnings during the build. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5679) TestAHSWebServices is failing
[
https://issues.apache.org/jira/browse/YARN-5679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated YARN-5679:
Attachment: YARN-5679.02.patch
> TestAHSWebServices is failing
> -
>
> Key: YARN-5679
> URL: https://issues.apache.org/jira/browse/YARN-5679
> Project: Hadoop YARN
> Issue Type: Bug
> Components: timelineserver
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
> Attachments: YARN-5679.01.patch, YARN-5679.02.patch
>
>
> TestAHSWebServices.testContainerLogsForFinishedApps is failing.
> https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/176/testReport/
> {noformat}
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebServices.createContainerLogInLocalDir(TestAHSWebServices.java:675)
> at
> org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebServices.testContainerLogsForFinishedApps(TestAHSWebServices.java:581)
> {noformat}
> {noformat}
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebServices.testContainerLogsForFinishedApps(TestAHSWebServices.java:519)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5679) TestAHSWebServices is failing
[
https://issues.apache.org/jira/browse/YARN-5679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15577413#comment-15577413
]
Akira Ajisaka commented on YARN-5679:
-
Thanks [~miklos.szeg...@cloudera.com] for the comment!
bq. setUMask will change the umask in the Configuration object, which may be
picked up by any other thread in the process using the same configuration
object before it is reverted.
Agreed.
bq. One option is to create a private configuration object to change the umask,
the other is to apply the both umasks to a permission object and create the
file with this permission object set.
I chose the latter option because creating a private configuration object for
each log costs too much.
> TestAHSWebServices is failing
> -
>
> Key: YARN-5679
> URL: https://issues.apache.org/jira/browse/YARN-5679
> Project: Hadoop YARN
> Issue Type: Bug
> Components: timelineserver
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
> Attachments: YARN-5679.01.patch, YARN-5679.02.patch
>
>
> TestAHSWebServices.testContainerLogsForFinishedApps is failing.
> https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/176/testReport/
> {noformat}
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebServices.createContainerLogInLocalDir(TestAHSWebServices.java:675)
> at
> org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebServices.testContainerLogsForFinishedApps(TestAHSWebServices.java:581)
> {noformat}
> {noformat}
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebServices.testContainerLogsForFinishedApps(TestAHSWebServices.java:519)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5679) TestAHSWebServices is failing
[
https://issues.apache.org/jira/browse/YARN-5679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15598397#comment-15598397
]
Akira Ajisaka commented on YARN-5679:
-
Thanks [~miklos.szeg...@cloudera.com] and [~rkanter]!
> TestAHSWebServices is failing
> -
>
> Key: YARN-5679
> URL: https://issues.apache.org/jira/browse/YARN-5679
> Project: Hadoop YARN
> Issue Type: Bug
> Components: timelineserver
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
> Fix For: 3.0.0-alpha2
>
> Attachments: YARN-5679.01.patch, YARN-5679.02.patch
>
>
> TestAHSWebServices.testContainerLogsForFinishedApps is failing.
> https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/176/testReport/
> {noformat}
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebServices.createContainerLogInLocalDir(TestAHSWebServices.java:675)
> at
> org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebServices.testContainerLogsForFinishedApps(TestAHSWebServices.java:581)
> {noformat}
> {noformat}
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebServices.testContainerLogsForFinishedApps(TestAHSWebServices.java:519)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-5772) Replace old Hadoop logo with new one
Akira Ajisaka created YARN-5772: --- Summary: Replace old Hadoop logo with new one Key: YARN-5772 URL: https://issues.apache.org/jira/browse/YARN-5772 Project: Hadoop YARN Issue Type: Sub-task Components: yarn-ui-v2 Affects Versions: YARN-3368 Reporter: Akira Ajisaka -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5772) Replace old Hadoop logo with new one
[ https://issues.apache.org/jira/browse/YARN-5772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated YARN-5772: Description: YARN-5161 added Apache Hadoop logo in the UI but the logo is old. > Replace old Hadoop logo with new one > > > Key: YARN-5772 > URL: https://issues.apache.org/jira/browse/YARN-5772 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn-ui-v2 >Affects Versions: YARN-3368 >Reporter: Akira Ajisaka > > YARN-5161 added Apache Hadoop logo in the UI but the logo is old. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5313) TestLogsCLI.testFetchApplictionLogsAsAnotherUser fails in trunk
[ https://issues.apache.org/jira/browse/YARN-5313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15604224#comment-15604224 ] Akira Ajisaka commented on YARN-5313: - Now the unit test is failing by another reason, so I filed another jira: YARN-5777 > TestLogsCLI.testFetchApplictionLogsAsAnotherUser fails in trunk > --- > > Key: YARN-5313 > URL: https://issues.apache.org/jira/browse/YARN-5313 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 2.9.0 >Reporter: Xuan Gong >Priority: Blocker > > We have reverted HADOOP-12718 recently which caused this failure. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5777) TestLogsCLI#testFetchApplictionLogsAsAnotherUser fails
[
https://issues.apache.org/jira/browse/YARN-5777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated YARN-5777:
Description:
{noformat}
Running org.apache.hadoop.yarn.client.cli.TestLogsCLI
Tests run: 14, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 5.876 sec <<<
FAILURE! - in org.apache.hadoop.yarn.client.cli.TestLogsCLI
testFetchApplictionLogsAsAnotherUser(org.apache.hadoop.yarn.client.cli.TestLogsCLI)
Time elapsed: 0.199 sec <<< ERROR!
java.io.IOException: Invalid directory or I/O error occurred for dir:
/Users/aajisaka/git/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/target/logs/priority/logs/application_1477371285256_1000
at org.apache.hadoop.fs.FileUtil.list(FileUtil.java:1148)
at
org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:469)
at
org.apache.hadoop.fs.DelegateToFileSystem.listStatus(DelegateToFileSystem.java:169)
at org.apache.hadoop.fs.ChecksumFs.listStatus(ChecksumFs.java:519)
at
org.apache.hadoop.fs.AbstractFileSystem$1.(AbstractFileSystem.java:890)
at
org.apache.hadoop.fs.AbstractFileSystem.listStatusIterator(AbstractFileSystem.java:888)
at org.apache.hadoop.fs.FileContext$22.next(FileContext.java:1492)
at org.apache.hadoop.fs.FileContext$22.next(FileContext.java:1487)
at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
at org.apache.hadoop.fs.FileContext.listStatus(FileContext.java:1494)
at
org.apache.hadoop.yarn.logaggregation.LogCLIHelpers.getRemoteNodeFileDir(LogCLIHelpers.java:592)
at
org.apache.hadoop.yarn.logaggregation.LogCLIHelpers.dumpAllContainersLogs(LogCLIHelpers.java:348)
at
org.apache.hadoop.yarn.client.cli.LogsCLI.fetchApplicationLogs(LogsCLI.java:971)
at
org.apache.hadoop.yarn.client.cli.LogsCLI.runCommand(LogsCLI.java:299)
at org.apache.hadoop.yarn.client.cli.LogsCLI.run(LogsCLI.java:106)
at
org.apache.hadoop.yarn.client.cli.TestLogsCLI.testFetchApplictionLogsAsAnotherUser(TestLogsCLI.java:868)
{noformat}
> TestLogsCLI#testFetchApplictionLogsAsAnotherUser fails
> --
>
> Key: YARN-5777
> URL: https://issues.apache.org/jira/browse/YARN-5777
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Akira Ajisaka
>
> {noformat}
> Running org.apache.hadoop.yarn.client.cli.TestLogsCLI
> Tests run: 14, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 5.876 sec
> <<< FAILURE! - in org.apache.hadoop.yarn.client.cli.TestLogsCLI
> testFetchApplictionLogsAsAnotherUser(org.apache.hadoop.yarn.client.cli.TestLogsCLI)
> Time elapsed: 0.199 sec <<< ERROR!
> java.io.IOException: Invalid directory or I/O error occurred for dir:
> /Users/aajisaka/git/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/target/logs/priority/logs/application_1477371285256_1000
> at org.apache.hadoop.fs.FileUtil.list(FileUtil.java:1148)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:469)
> at
> org.apache.hadoop.fs.DelegateToFileSystem.listStatus(DelegateToFileSystem.java:169)
> at org.apache.hadoop.fs.ChecksumFs.listStatus(ChecksumFs.java:519)
> at
> org.apache.hadoop.fs.AbstractFileSystem$1.(AbstractFileSystem.java:890)
> at
> org.apache.hadoop.fs.AbstractFileSystem.listStatusIterator(AbstractFileSystem.java:888)
> at org.apache.hadoop.fs.FileContext$22.next(FileContext.java:1492)
> at org.apache.hadoop.fs.FileContext$22.next(FileContext.java:1487)
> at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
> at org.apache.hadoop.fs.FileContext.listStatus(FileContext.java:1494)
> at
> org.apache.hadoop.yarn.logaggregation.LogCLIHelpers.getRemoteNodeFileDir(LogCLIHelpers.java:592)
> at
> org.apache.hadoop.yarn.logaggregation.LogCLIHelpers.dumpAllContainersLogs(LogCLIHelpers.java:348)
> at
> org.apache.hadoop.yarn.client.cli.LogsCLI.fetchApplicationLogs(LogsCLI.java:971)
> at
> org.apache.hadoop.yarn.client.cli.LogsCLI.runCommand(LogsCLI.java:299)
> at org.apache.hadoop.yarn.client.cli.LogsCLI.run(LogsCLI.java:106)
> at
> org.apache.hadoop.yarn.client.cli.TestLogsCLI.testFetchApplictionLogsAsAnotherUser(TestLogsCLI.java:868)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-5777) TestLogsCLI#testFetchApplictionLogsAsAnotherUser fails
[
https://issues.apache.org/jira/browse/YARN-5777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka reassigned YARN-5777:
---
Assignee: Akira Ajisaka
> TestLogsCLI#testFetchApplictionLogsAsAnotherUser fails
> --
>
> Key: YARN-5777
> URL: https://issues.apache.org/jira/browse/YARN-5777
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>
> {noformat}
> Running org.apache.hadoop.yarn.client.cli.TestLogsCLI
> Tests run: 14, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 5.876 sec
> <<< FAILURE! - in org.apache.hadoop.yarn.client.cli.TestLogsCLI
> testFetchApplictionLogsAsAnotherUser(org.apache.hadoop.yarn.client.cli.TestLogsCLI)
> Time elapsed: 0.199 sec <<< ERROR!
> java.io.IOException: Invalid directory or I/O error occurred for dir:
> /Users/aajisaka/git/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/target/logs/priority/logs/application_1477371285256_1000
> at org.apache.hadoop.fs.FileUtil.list(FileUtil.java:1148)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:469)
> at
> org.apache.hadoop.fs.DelegateToFileSystem.listStatus(DelegateToFileSystem.java:169)
> at org.apache.hadoop.fs.ChecksumFs.listStatus(ChecksumFs.java:519)
> at
> org.apache.hadoop.fs.AbstractFileSystem$1.(AbstractFileSystem.java:890)
> at
> org.apache.hadoop.fs.AbstractFileSystem.listStatusIterator(AbstractFileSystem.java:888)
> at org.apache.hadoop.fs.FileContext$22.next(FileContext.java:1492)
> at org.apache.hadoop.fs.FileContext$22.next(FileContext.java:1487)
> at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
> at org.apache.hadoop.fs.FileContext.listStatus(FileContext.java:1494)
> at
> org.apache.hadoop.yarn.logaggregation.LogCLIHelpers.getRemoteNodeFileDir(LogCLIHelpers.java:592)
> at
> org.apache.hadoop.yarn.logaggregation.LogCLIHelpers.dumpAllContainersLogs(LogCLIHelpers.java:348)
> at
> org.apache.hadoop.yarn.client.cli.LogsCLI.fetchApplicationLogs(LogsCLI.java:971)
> at
> org.apache.hadoop.yarn.client.cli.LogsCLI.runCommand(LogsCLI.java:299)
> at org.apache.hadoop.yarn.client.cli.LogsCLI.run(LogsCLI.java:106)
> at
> org.apache.hadoop.yarn.client.cli.TestLogsCLI.testFetchApplictionLogsAsAnotherUser(TestLogsCLI.java:868)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5777) TestLogsCLI#testFetchApplictionLogsAsAnotherUser fails
[
https://issues.apache.org/jira/browse/YARN-5777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated YARN-5777:
Attachment: YARN-5777.01.patch
The test fails because RawLocalFileSystem.listStatus throws IOException after
HADOOP-7302.
I wrote a patch to catch IOE and log error message in
LogCLIHelpers#getRemoteNodeFileDir. Before this patch, the method catches
FileNotFoundException, AccessControlException, and AccessDeniedException, so
I'm thinking it's okay to catch IOE as well.
> TestLogsCLI#testFetchApplictionLogsAsAnotherUser fails
> --
>
> Key: YARN-5777
> URL: https://issues.apache.org/jira/browse/YARN-5777
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
> Attachments: YARN-5777.01.patch
>
>
> {noformat}
> Running org.apache.hadoop.yarn.client.cli.TestLogsCLI
> Tests run: 14, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 5.876 sec
> <<< FAILURE! - in org.apache.hadoop.yarn.client.cli.TestLogsCLI
> testFetchApplictionLogsAsAnotherUser(org.apache.hadoop.yarn.client.cli.TestLogsCLI)
> Time elapsed: 0.199 sec <<< ERROR!
> java.io.IOException: Invalid directory or I/O error occurred for dir:
> /Users/aajisaka/git/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/target/logs/priority/logs/application_1477371285256_1000
> at org.apache.hadoop.fs.FileUtil.list(FileUtil.java:1148)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:469)
> at
> org.apache.hadoop.fs.DelegateToFileSystem.listStatus(DelegateToFileSystem.java:169)
> at org.apache.hadoop.fs.ChecksumFs.listStatus(ChecksumFs.java:519)
> at
> org.apache.hadoop.fs.AbstractFileSystem$1.(AbstractFileSystem.java:890)
> at
> org.apache.hadoop.fs.AbstractFileSystem.listStatusIterator(AbstractFileSystem.java:888)
> at org.apache.hadoop.fs.FileContext$22.next(FileContext.java:1492)
> at org.apache.hadoop.fs.FileContext$22.next(FileContext.java:1487)
> at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
> at org.apache.hadoop.fs.FileContext.listStatus(FileContext.java:1494)
> at
> org.apache.hadoop.yarn.logaggregation.LogCLIHelpers.getRemoteNodeFileDir(LogCLIHelpers.java:592)
> at
> org.apache.hadoop.yarn.logaggregation.LogCLIHelpers.dumpAllContainersLogs(LogCLIHelpers.java:348)
> at
> org.apache.hadoop.yarn.client.cli.LogsCLI.fetchApplicationLogs(LogsCLI.java:971)
> at
> org.apache.hadoop.yarn.client.cli.LogsCLI.runCommand(LogsCLI.java:299)
> at org.apache.hadoop.yarn.client.cli.LogsCLI.run(LogsCLI.java:106)
> at
> org.apache.hadoop.yarn.client.cli.TestLogsCLI.testFetchApplictionLogsAsAnotherUser(TestLogsCLI.java:868)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5777) TestLogsCLI#testFetchApplictionLogsAsAnotherUser fails
[
https://issues.apache.org/jira/browse/YARN-5777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15604291#comment-15604291
]
Akira Ajisaka commented on YARN-5777:
-
I ran git bisect and found HADOOP-7352 broke this.
> TestLogsCLI#testFetchApplictionLogsAsAnotherUser fails
> --
>
> Key: YARN-5777
> URL: https://issues.apache.org/jira/browse/YARN-5777
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Akira Ajisaka
>
> {noformat}
> Running org.apache.hadoop.yarn.client.cli.TestLogsCLI
> Tests run: 14, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 5.876 sec
> <<< FAILURE! - in org.apache.hadoop.yarn.client.cli.TestLogsCLI
> testFetchApplictionLogsAsAnotherUser(org.apache.hadoop.yarn.client.cli.TestLogsCLI)
> Time elapsed: 0.199 sec <<< ERROR!
> java.io.IOException: Invalid directory or I/O error occurred for dir:
> /Users/aajisaka/git/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/target/logs/priority/logs/application_1477371285256_1000
> at org.apache.hadoop.fs.FileUtil.list(FileUtil.java:1148)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:469)
> at
> org.apache.hadoop.fs.DelegateToFileSystem.listStatus(DelegateToFileSystem.java:169)
> at org.apache.hadoop.fs.ChecksumFs.listStatus(ChecksumFs.java:519)
> at
> org.apache.hadoop.fs.AbstractFileSystem$1.(AbstractFileSystem.java:890)
> at
> org.apache.hadoop.fs.AbstractFileSystem.listStatusIterator(AbstractFileSystem.java:888)
> at org.apache.hadoop.fs.FileContext$22.next(FileContext.java:1492)
> at org.apache.hadoop.fs.FileContext$22.next(FileContext.java:1487)
> at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
> at org.apache.hadoop.fs.FileContext.listStatus(FileContext.java:1494)
> at
> org.apache.hadoop.yarn.logaggregation.LogCLIHelpers.getRemoteNodeFileDir(LogCLIHelpers.java:592)
> at
> org.apache.hadoop.yarn.logaggregation.LogCLIHelpers.dumpAllContainersLogs(LogCLIHelpers.java:348)
> at
> org.apache.hadoop.yarn.client.cli.LogsCLI.fetchApplicationLogs(LogsCLI.java:971)
> at
> org.apache.hadoop.yarn.client.cli.LogsCLI.runCommand(LogsCLI.java:299)
> at org.apache.hadoop.yarn.client.cli.LogsCLI.run(LogsCLI.java:106)
> at
> org.apache.hadoop.yarn.client.cli.TestLogsCLI.testFetchApplictionLogsAsAnotherUser(TestLogsCLI.java:868)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5575) Many classes use bare yarn. properties instead of the defined constants
[
https://issues.apache.org/jira/browse/YARN-5575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15604194#comment-15604194
]
Akira Ajisaka commented on YARN-5575:
-
Hi [~templedf], would you fix checkstyle warnings? I'm +1 if that is addressed.
Thanks.
> Many classes use bare yarn. properties instead of the defined constants
> ---
>
> Key: YARN-5575
> URL: https://issues.apache.org/jira/browse/YARN-5575
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Daniel Templeton
>Assignee: Daniel Templeton
> Attachments: YARN-5575.001.patch, YARN-5575.002.patch,
> YARN-5575.003.patch
>
>
> MAPREDUCE-5870 introduced the following line:
> {code}
> conf.setInt("yarn.cluster.max-application-priority", 10);
> {code}
> It should instead be:
> {code}
> conf.setInt(YarnConfiguration.MAX_CLUSTER_LEVEL_APPLICATION_PRIORITY,
> 10);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-5777) TestLogsCLI#testFetchApplictionLogsAsAnotherUser fails
Akira Ajisaka created YARN-5777: --- Summary: TestLogsCLI#testFetchApplictionLogsAsAnotherUser fails Key: YARN-5777 URL: https://issues.apache.org/jira/browse/YARN-5777 Project: Hadoop YARN Issue Type: Bug Components: test Reporter: Akira Ajisaka -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-5313) TestLogsCLI.testFetchApplictionLogsAsAnotherUser fails in trunk
[ https://issues.apache.org/jira/browse/YARN-5313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka resolved YARN-5313. - Resolution: Not A Problem Closing this issue because HADOOP-12718 has been reverted. > TestLogsCLI.testFetchApplictionLogsAsAnotherUser fails in trunk > --- > > Key: YARN-5313 > URL: https://issues.apache.org/jira/browse/YARN-5313 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 2.9.0 >Reporter: Xuan Gong >Priority: Blocker > > We have reverted HADOOP-12718 recently which caused this failure. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5559) Analyse 2.8.0/3.0.0 jdiff reports and fix any issues
[ https://issues.apache.org/jira/browse/YARN-5559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15604561#comment-15604561 ] Akira Ajisaka commented on YARN-5559: - Additional comment: Would you add javadoc to document the replacements when deprecating APIs? > Analyse 2.8.0/3.0.0 jdiff reports and fix any issues > > > Key: YARN-5559 > URL: https://issues.apache.org/jira/browse/YARN-5559 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Wangda Tan >Assignee: Wangda Tan >Priority: Blocker > Attachments: YARN-5559.1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5559) Analyse 2.8.0/3.0.0 jdiff reports and fix any issues
[ https://issues.apache.org/jira/browse/YARN-5559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15604551#comment-15604551 ] Akira Ajisaka commented on YARN-5559: - Mostly looks good to me. I checked what issue broke the compatibility. bq. 1) Incompatible changes in GetClusterNodeLabelResponse.java (signature changed) YARN-3413 (fixed in 2.8.0) bq. 3) One of ContainerTokenIdentifier newInstance method removed YARN-2581 (fixed in 2.6.0) I'm thinking we should separate the patch into (3) and the rest. (3) should be fixed in branch-2.6 and above. > Analyse 2.8.0/3.0.0 jdiff reports and fix any issues > > > Key: YARN-5559 > URL: https://issues.apache.org/jira/browse/YARN-5559 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Wangda Tan >Assignee: Wangda Tan >Priority: Blocker > Attachments: YARN-5559.1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5772) Replace old Hadoop logo with new one
[ https://issues.apache.org/jira/browse/YARN-5772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15602622#comment-15602622 ] Akira Ajisaka commented on YARN-5772: - LGTM, +1. > Replace old Hadoop logo with new one > > > Key: YARN-5772 > URL: https://issues.apache.org/jira/browse/YARN-5772 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn-ui-v2 >Affects Versions: YARN-3368 >Reporter: Akira Ajisaka >Assignee: Akhil PB > Attachments: YARN-5772-YARN-3368.0001.patch, ui2-with-newlogo.png > > > YARN-5161 added Apache Hadoop logo in the UI but the logo is old. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5713) Update jackson from 1.9.13 to 2.x in hadoop-yarn
[ https://issues.apache.org/jira/browse/YARN-5713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1516#comment-1516 ] Akira Ajisaka commented on YARN-5713: - Hi [~ste...@apache.org], would you review the latest patch? > Update jackson from 1.9.13 to 2.x in hadoop-yarn > > > Key: YARN-5713 > URL: https://issues.apache.org/jira/browse/YARN-5713 > Project: Hadoop YARN > Issue Type: Improvement > Components: build, timelineserver >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka > Labels: oct16-medium > Attachments: HADOOP-13677.01.patch, HADOOP-13677.02.patch, > YARN-5713.03.patch, YARN-5713.04.patch > > > Sub-task of HADOOP-13332. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5559) Analyse 2.8.0/3.0.0 jdiff reports and fix any issues
[ https://issues.apache.org/jira/browse/YARN-5559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15692436#comment-15692436 ] Akira Ajisaka commented on YARN-5559: - Ping [~leftnoteasy]. How is this issue going on? > Analyse 2.8.0/3.0.0 jdiff reports and fix any issues > > > Key: YARN-5559 > URL: https://issues.apache.org/jira/browse/YARN-5559 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Reporter: Wangda Tan >Assignee: Wangda Tan >Priority: Blocker > Labels: oct16-easy > Attachments: YARN-5559.1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5572) HBaseTimelineWriterImpl appears to reference a bad property name
[
https://issues.apache.org/jira/browse/YARN-5572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15692363#comment-15692363
]
Akira Ajisaka commented on YARN-5572:
-
LGTM, +1.
> HBaseTimelineWriterImpl appears to reference a bad property name
>
>
> Key: YARN-5572
> URL: https://issues.apache.org/jira/browse/YARN-5572
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Daniel Templeton
>Assignee: Varun Saxena
> Attachments: YARN-5572.01.patch
>
>
> This constructor:
> {code}
> public HBaseTimelineWriterImpl(Configuration conf) throws IOException {
> super(conf.get("yarn.application.id",
> HBaseTimelineWriterImpl.class.getName()));
> }
> {code}
> references "yarn.application.id", which is the only reference to this
> property I could find.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-5934) Fix TestTimelineWebServices.testPrimaryFilterNumericString
Akira Ajisaka created YARN-5934:
---
Summary: Fix TestTimelineWebServices.testPrimaryFilterNumericString
Key: YARN-5934
URL: https://issues.apache.org/jira/browse/YARN-5934
Project: Hadoop YARN
Issue Type: Bug
Components: test
Reporter: Akira Ajisaka
{noformat}
Running org.apache.hadoop.yarn.server.timeline.webapp.TestTimelineWebServices
Tests run: 28, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 43.297 sec <<<
FAILURE! - in
org.apache.hadoop.yarn.server.timeline.webapp.TestTimelineWebServices
testPrimaryFilterNumericString(org.apache.hadoop.yarn.server.timeline.webapp.TestTimelineWebServices)
Time elapsed: 1.209 sec <<< FAILURE!
java.lang.AssertionError: expected:<0> but was:<3>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at org.junit.Assert.assertEquals(Assert.java:542)
at
org.apache.hadoop.yarn.server.timeline.webapp.TestTimelineWebServices.testPrimaryFilterNumericString(TestTimelineWebServices.java:348)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5926) clean up registry code for java 7/8
[ https://issues.apache.org/jira/browse/YARN-5926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15689775#comment-15689775 ] Akira Ajisaka commented on YARN-5926: - Sure. I'm okay with changing the title. > clean up registry code for java 7/8 > --- > > Key: YARN-5926 > URL: https://issues.apache.org/jira/browse/YARN-5926 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Affects Versions: 2.7.3 >Reporter: Steve Loughran >Assignee: Steve Loughran > Attachments: YARN-5926-001.patch > > > Clean up the registry code to stop the java 7/8 warnings -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5713) Update jackson from 1.9.13 to 2.x in hadoop-yarn
[ https://issues.apache.org/jira/browse/YARN-5713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15685336#comment-15685336 ] Akira Ajisaka commented on YARN-5713: - bq. I'll sync it up with this code and then, if you can review it, get it in. Okay. Would you ping me after you sync it up? > Update jackson from 1.9.13 to 2.x in hadoop-yarn > > > Key: YARN-5713 > URL: https://issues.apache.org/jira/browse/YARN-5713 > Project: Hadoop YARN > Issue Type: Improvement > Components: build, timelineserver >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka > Labels: oct16-medium > Fix For: 3.0.0-alpha2 > > Attachments: HADOOP-13677.01.patch, HADOOP-13677.02.patch, > YARN-5713.03.patch, YARN-5713.04.patch > > > Sub-task of HADOOP-13332. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5934) Fix TestTimelineWebServices.testPrimaryFilterNumericString
[
https://issues.apache.org/jira/browse/YARN-5934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated YARN-5934:
Attachment: YARN-5934.01.patch
Attaching a patch to remove the tests.
> Fix TestTimelineWebServices.testPrimaryFilterNumericString
> --
>
> Key: YARN-5934
> URL: https://issues.apache.org/jira/browse/YARN-5934
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Akira Ajisaka
> Attachments: YARN-5934.01.patch
>
>
> {noformat}
> Running org.apache.hadoop.yarn.server.timeline.webapp.TestTimelineWebServices
> Tests run: 28, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 43.297 sec
> <<< FAILURE! - in
> org.apache.hadoop.yarn.server.timeline.webapp.TestTimelineWebServices
> testPrimaryFilterNumericString(org.apache.hadoop.yarn.server.timeline.webapp.TestTimelineWebServices)
> Time elapsed: 1.209 sec <<< FAILURE!
> java.lang.AssertionError: expected:<0> but was:<3>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at org.junit.Assert.assertEquals(Assert.java:542)
> at
> org.apache.hadoop.yarn.server.timeline.webapp.TestTimelineWebServices.testPrimaryFilterNumericString(TestTimelineWebServices.java:348)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5934) Fix TestTimelineWebServices.testPrimaryFilterNumericString
[
https://issues.apache.org/jira/browse/YARN-5934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15692576#comment-15692576
]
Akira Ajisaka commented on YARN-5934:
-
It looks like Jackson1 parses 123abc to 123, on the other hand, Jackson2 parses
123abc to "123abc". YARN-3723 documented that Jackson library decides how the
value is casted, so I'm think we can remove the tests.
bq. Note that the value of the key/value pair for primaryFilter and
secondaryFilters parameters can be of different data types, and matching is
data type sensitive. Users need to format the value properly. For example, 123
and "123" means an integer and a string respectively. If the entity has a
string "123" for primaryFilter, but the parameter is set to the integer 123,
the entity will not be matched. Similarly, true means a boolean while "true"
means a string. In general, the value will be casted as a certain Java type in
consistent with jackson library parsing a JSON clip.
What do you think, [~ste...@apache.org], [~zjshen], and [~xgong]?
> Fix TestTimelineWebServices.testPrimaryFilterNumericString
> --
>
> Key: YARN-5934
> URL: https://issues.apache.org/jira/browse/YARN-5934
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Akira Ajisaka
>
> {noformat}
> Running org.apache.hadoop.yarn.server.timeline.webapp.TestTimelineWebServices
> Tests run: 28, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 43.297 sec
> <<< FAILURE! - in
> org.apache.hadoop.yarn.server.timeline.webapp.TestTimelineWebServices
> testPrimaryFilterNumericString(org.apache.hadoop.yarn.server.timeline.webapp.TestTimelineWebServices)
> Time elapsed: 1.209 sec <<< FAILURE!
> java.lang.AssertionError: expected:<0> but was:<3>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at org.junit.Assert.assertEquals(Assert.java:542)
> at
> org.apache.hadoop.yarn.server.timeline.webapp.TestTimelineWebServices.testPrimaryFilterNumericString(TestTimelineWebServices.java:348)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5559) Analyse 2.8.0/3.0.0 jdiff reports and fix any issues
[ https://issues.apache.org/jira/browse/YARN-5559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704159#comment-15704159 ] Akira Ajisaka commented on YARN-5559: - +1, thanks [~jianhe] and [~djp]. > Analyse 2.8.0/3.0.0 jdiff reports and fix any issues > > > Key: YARN-5559 > URL: https://issues.apache.org/jira/browse/YARN-5559 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Reporter: Wangda Tan >Assignee: Wangda Tan >Priority: Blocker > Labels: oct16-easy > Attachments: YARN-5559.1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5713) Update jackson from 1.9.13 to 2.x in hadoop-yarn
[ https://issues.apache.org/jira/browse/YARN-5713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15668520#comment-15668520 ] Akira Ajisaka commented on YARN-5713: - Sure! > Update jackson from 1.9.13 to 2.x in hadoop-yarn > > > Key: YARN-5713 > URL: https://issues.apache.org/jira/browse/YARN-5713 > Project: Hadoop YARN > Issue Type: Improvement > Components: build, timelineserver >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka > Labels: oct16-medium > Attachments: HADOOP-13677.01.patch, HADOOP-13677.02.patch, > YARN-5713.03.patch, YARN-5713.04.patch > > > Sub-task of HADOOP-13332. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5713) Update jackson from 1.9.13 to 2.x in hadoop-yarn
[ https://issues.apache.org/jira/browse/YARN-5713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682292#comment-15682292 ] Akira Ajisaka commented on YARN-5713: - Ping [~ste...@apache.org] again. Thanks. > Update jackson from 1.9.13 to 2.x in hadoop-yarn > > > Key: YARN-5713 > URL: https://issues.apache.org/jira/browse/YARN-5713 > Project: Hadoop YARN > Issue Type: Improvement > Components: build, timelineserver >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka > Labels: oct16-medium > Attachments: HADOOP-13677.01.patch, HADOOP-13677.02.patch, > YARN-5713.03.patch, YARN-5713.04.patch > > > Sub-task of HADOOP-13332. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5713) Update jackson from 1.9.13 to 2.x in hadoop-yarn
[ https://issues.apache.org/jira/browse/YARN-5713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated YARN-5713: Attachment: YARN-5713.03.patch > Update jackson from 1.9.13 to 2.x in hadoop-yarn > > > Key: YARN-5713 > URL: https://issues.apache.org/jira/browse/YARN-5713 > Project: Hadoop YARN > Issue Type: Improvement > Components: build, timelineserver >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka > Labels: oct16-medium > Attachments: HADOOP-13677.01.patch, HADOOP-13677.02.patch, > YARN-5713.03.patch > > > Sub-task of HADOOP-13332. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5713) Update jackson from 1.9.13 to 2.x in hadoop-yarn
[ https://issues.apache.org/jira/browse/YARN-5713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15621723#comment-15621723 ] Akira Ajisaka commented on YARN-5713: - Thanks [~miklos.szeg...@cloudera.com] for the comment. bq. FileSystemTimelineReaderImpl.java has a new newline at the end that is not needed. I couldn't find the unneeded newline. Would you specify this? bq. JsonSerDeser.java: toJson may throw IOException but does not document it. Actually toJson does not throw IOException, so I changed the signature. > Update jackson from 1.9.13 to 2.x in hadoop-yarn > > > Key: YARN-5713 > URL: https://issues.apache.org/jira/browse/YARN-5713 > Project: Hadoop YARN > Issue Type: Improvement > Components: build, timelineserver >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka > Labels: oct16-medium > Attachments: HADOOP-13677.01.patch, HADOOP-13677.02.patch > > > Sub-task of HADOOP-13332. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5713) Update jackson from 1.9.13 to 2.x in hadoop-yarn
[ https://issues.apache.org/jira/browse/YARN-5713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15621726#comment-15621726 ] Akira Ajisaka commented on YARN-5713: - bq. If it also impacts v.1.x, it sounds like we should test this on the trunk? Agreed. Thanks [~gtCarrera9] and [~sjlee0]. > Update jackson from 1.9.13 to 2.x in hadoop-yarn > > > Key: YARN-5713 > URL: https://issues.apache.org/jira/browse/YARN-5713 > Project: Hadoop YARN > Issue Type: Improvement > Components: build, timelineserver >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka > Labels: oct16-medium > Attachments: HADOOP-13677.01.patch, HADOOP-13677.02.patch, > YARN-5713.03.patch > > > Sub-task of HADOOP-13332. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
