[jira] [Commented] (YARN-8873) [YARN-8811] Add CSI java-based client library
[ https://issues.apache.org/jira/browse/YARN-8873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16663148#comment-16663148 ] Arpit Agarwal commented on YARN-8873: - Thanks for the look [~cheersyang]. I asked around and no one else seems to be hitting it. I will investigate further. > [YARN-8811] Add CSI java-based client library > - > > Key: YARN-8873 > URL: https://issues.apache.org/jira/browse/YARN-8873 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Weiwei Yang >Assignee: Weiwei Yang >Priority: Major > Fix For: 3.3.0 > > Attachments: YARN-8873.001.patch, YARN-8873.002.patch, > YARN-8873.003.patch, YARN-8873.004.patch, YARN-8873.005.patch, > YARN-8873.006.patch > > > Build a java-based client to talk to CSI drivers, through CSI gRPC services. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8873) [YARN-8811] Add CSI java-based client library
[
https://issues.apache.org/jira/browse/YARN-8873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662700#comment-16662700
]
Arpit Agarwal commented on YARN-8873:
-
This patch breaks trunk compilation for me:
{code}
[ERROR] PROTOC FAILED: google/protobuf/wrappers.proto: File not found.
csi.proto: Import "google/protobuf/wrappers.proto" was not found or had errors.
csi.proto:192:5: ".google.protobuf.BoolValue" is not defined.
[ERROR]
/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/main/proto/csi.proto
[0:0]: google/protobuf/wrappers.proto: File not found.
csi.proto: Import "google/protobuf/wrappers.proto" was not found or had errors.
csi.proto:192:5: ".google.protobuf.BoolValue" is not defined.
{code}
> [YARN-8811] Add CSI java-based client library
> -
>
> Key: YARN-8873
> URL: https://issues.apache.org/jira/browse/YARN-8873
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
>Priority: Major
> Fix For: 3.3.0
>
> Attachments: YARN-8873.001.patch, YARN-8873.002.patch,
> YARN-8873.003.patch, YARN-8873.004.patch, YARN-8873.005.patch,
> YARN-8873.006.patch
>
>
> Build a java-based client to talk to CSI drivers, through CSI gRPC services.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-8873) [YARN-8811] Add CSI java-based client library
[
https://issues.apache.org/jira/browse/YARN-8873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662700#comment-16662700
]
Arpit Agarwal edited comment on YARN-8873 at 10/24/18 7:09 PM:
---
This patch breaks trunk compilation for me:
{code}
[ERROR] PROTOC FAILED: google/protobuf/wrappers.proto: File not found.
csi.proto: Import "google/protobuf/wrappers.proto" was not found or had errors.
csi.proto:192:5: ".google.protobuf.BoolValue" is not defined.
[ERROR]
/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/main/proto/csi.proto
[0:0]: google/protobuf/wrappers.proto: File not found.
csi.proto: Import "google/protobuf/wrappers.proto" was not found or had errors.
csi.proto:192:5: ".google.protobuf.BoolValue" is not defined.
{code}
Anyone else seeing the same?
was (Author: arpitagarwal):
This patch breaks trunk compilation for me:
{code}
[ERROR] PROTOC FAILED: google/protobuf/wrappers.proto: File not found.
csi.proto: Import "google/protobuf/wrappers.proto" was not found or had errors.
csi.proto:192:5: ".google.protobuf.BoolValue" is not defined.
[ERROR]
/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/main/proto/csi.proto
[0:0]: google/protobuf/wrappers.proto: File not found.
csi.proto: Import "google/protobuf/wrappers.proto" was not found or had errors.
csi.proto:192:5: ".google.protobuf.BoolValue" is not defined.
{code}
> [YARN-8811] Add CSI java-based client library
> -
>
> Key: YARN-8873
> URL: https://issues.apache.org/jira/browse/YARN-8873
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
>Priority: Major
> Fix For: 3.3.0
>
> Attachments: YARN-8873.001.patch, YARN-8873.002.patch,
> YARN-8873.003.patch, YARN-8873.004.patch, YARN-8873.005.patch,
> YARN-8873.006.patch
>
>
> Build a java-based client to talk to CSI drivers, through CSI gRPC services.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8091) Revisit checkUserAccessToQueue RM REST API
[ https://issues.apache.org/jira/browse/YARN-8091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated YARN-8091: Fix Version/s: 3.1.1 Cherry-picked to branch-3.1. > Revisit checkUserAccessToQueue RM REST API > -- > > Key: YARN-8091 > URL: https://issues.apache.org/jira/browse/YARN-8091 > Project: Hadoop YARN > Issue Type: Task >Reporter: Wangda Tan >Assignee: Wangda Tan >Priority: Critical > Fix For: 3.2.0, 3.1.1 > > Attachments: YARN-8091.001.patch > > > As offline suggested by [~sershe]. Currently design of the > checkUserAccessToQueue mixed config-related issues (like user doesn't access > to the URL) and user-facing output (like requested user is not permitted to > access the queue) in the same code. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8028) Support authorizeUserAccessToQueue in RMWebServices
[
https://issues.apache.org/jira/browse/YARN-8028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arpit Agarwal updated YARN-8028:
Fix Version/s: 3.1.1
Cherry-picked to branch-3.1.
> Support authorizeUserAccessToQueue in RMWebServices
> ---
>
> Key: YARN-8028
> URL: https://issues.apache.org/jira/browse/YARN-8028
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Wangda Tan
>Assignee: Wangda Tan
>Priority: Major
> Fix For: 3.2.0, 3.1.1
>
> Attachments: YARN-8028.001.patch, YARN-8028.002.patch
>
>
> Currently we have {{QueueUserACLInfo}} in ApplicationClient, we should
> support similar API in REST API.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8048) Support auto-spawning of admin configured services during bootstrap of rm/apiserver
[ https://issues.apache.org/jira/browse/YARN-8048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated YARN-8048: Fix Version/s: 3.1.1 Cherry-picked to branch-3.1. > Support auto-spawning of admin configured services during bootstrap of > rm/apiserver > --- > > Key: YARN-8048 > URL: https://issues.apache.org/jira/browse/YARN-8048 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > Fix For: 3.2.0, 3.1.1 > > Attachments: YARN-8048.001.patch, YARN-8048.002.patch, > YARN-8048.003.patch, YARN-8048.004.patch, YARN-8048.005.patch, > YARN-8048.006.patch > > > Goal is to support auto-spawning of admin configured services during > bootstrap of resourcemanager/apiserver. > *Requirement:* Some of the services might required to be consumed by yarn > itself ex: Hbase for atsv2. Instead of depending on user installed HBase or > sometimes user may not required to install HBase at all, in such conditions > running HBase app on YARN will help for ATSv2. > Before YARN cluster is started, admin configure these services spec and place > it in common location in HDFS. At the time of RM/apiServer bootstrap, these > services will be submitted. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8040) [UI2] New YARN UI webapp does not respect current pathname for REST api
[ https://issues.apache.org/jira/browse/YARN-8040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated YARN-8040: Fix Version/s: 3.1.1 Cherry-picked to branch-3.1. > [UI2] New YARN UI webapp does not respect current pathname for REST api > --- > > Key: YARN-8040 > URL: https://issues.apache.org/jira/browse/YARN-8040 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Reporter: Sunil G >Assignee: Sunil G >Priority: Major > Fix For: 3.2.0, 3.1.1 > > Attachments: YARN-8040.001.patch > > > When ui2 is accessed behind proxy like knox/nginx, trailing path name should > not be skipped. However trim of "ui2" if its there. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-1151) Ability to configure auxiliary services from HDFS-based JAR files
[ https://issues.apache.org/jira/browse/YARN-1151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated YARN-1151: Fix Version/s: 3.1.1 Cherry-picked to branch-3.1. > Ability to configure auxiliary services from HDFS-based JAR files > - > > Key: YARN-1151 > URL: https://issues.apache.org/jira/browse/YARN-1151 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager >Affects Versions: 2.1.0-beta, 2.9.0 >Reporter: john lilley >Assignee: Xuan Gong >Priority: Major > Labels: auxiliary-service, yarn > Fix For: 3.2.0, 3.1.1 > > Attachments: YARN-1151.1.patch, YARN-1151.2.patch, YARN-1151.3.patch, > YARN-1151.4.patch, YARN-1151.5.patch, YARN-1151.6.patch, > YARN-1151.branch-2.poc.2.patch, YARN-1151.branch-2.poc.3.patch, > YARN-1151.branch-2.poc.patch, [YARN-1151] [Design] Configure auxiliary > services from HDFS-based JAR files.pdf > > > I would like to install an auxiliary service in Hadoop YARN without actually > installing files/services on every node in the system. Discussions on the > user@ list indicate that this is not easily done. The reason we want an > auxiliary service is that our application has some persistent-data components > that are not appropriate for HDFS. In fact, they are somewhat analogous to > the mapper output of MapReduce's shuffle, which is what led me to > auxiliary-services in the first place. It would be much easier if we could > just place our service's JARs in HDFS. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5400) Light cleanup in ZKRMStateStore
[
https://issues.apache.org/jira/browse/YARN-5400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15531223#comment-15531223
]
Arpit Agarwal commented on YARN-5400:
-
No problem and thank you for the quick fix Robert. I verified that your latest
commit fixed branch-2.
> Light cleanup in ZKRMStateStore
> ---
>
> Key: YARN-5400
> URL: https://issues.apache.org/jira/browse/YARN-5400
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
>Affects Versions: 2.9.0
>Reporter: Daniel Templeton
>Assignee: Daniel Templeton
>Priority: Trivial
> Fix For: 2.9.0, 3.0.0-alpha2
>
> Attachments: YARN-5400.001.patch, YARN-5400.002.patch,
> YARN-5400.003.patch, YARN-5400.addendum.patch
>
>
> of {{ZKRMStateStore}} contains a plethora whitespace issues as well as some
> icky bits, like unused variables. This JIRA is to clean that up. It should
> have no functional impact.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5400) Light cleanup in ZKRMStateStore
[
https://issues.apache.org/jira/browse/YARN-5400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15531122#comment-15531122
]
Arpit Agarwal commented on YARN-5400:
-
Looks like this commit broke the build for branch-2. [~rkanter] can you please
take a look?
{code}
[ERROR] COMPILATION ERROR :
[ERROR]
/Users/aagarwal/src/commit/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/ZKRMStateStore.java:[415,40]
method put in interface java.util.Map cannot be applied to given types;
required:
java.lang.String,java.util.Map
found: java.lang.String,java.util.HashMap
reason: actual argument java.util.HashMap
cannot be converted to
java.util.Map
by method invocation conversion
{code}
> Light cleanup in ZKRMStateStore
> ---
>
> Key: YARN-5400
> URL: https://issues.apache.org/jira/browse/YARN-5400
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
>Affects Versions: 2.9.0
>Reporter: Daniel Templeton
>Assignee: Daniel Templeton
>Priority: Trivial
> Fix For: 2.9.0, 3.0.0-alpha2
>
> Attachments: YARN-5400.001.patch, YARN-5400.002.patch,
> YARN-5400.003.patch
>
>
> of {{ZKRMStateStore}} contains a plethora whitespace issues as well as some
> icky bits, like unused variables. This JIRA is to clean that up. It should
> have no functional impact.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5045) hbase unit tests fail due to dependency issues
[
https://issues.apache.org/jira/browse/YARN-5045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15390319#comment-15390319
]
Arpit Agarwal commented on YARN-5045:
-
Thanks [~sjlee0], yes wiping the maven cache worked.
> hbase unit tests fail due to dependency issues
> --
>
> Key: YARN-5045
> URL: https://issues.apache.org/jira/browse/YARN-5045
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Sangjin Lee
>Assignee: Sangjin Lee
>Priority: Blocker
> Fix For: 3.0.0-alpha1
>
> Attachments: YARN-5045-YARN-2928.01.patch,
> YARN-5045-YARN-2928.02.patch, YARN-5045-YARN-2928.03.patch,
> YARN-5045-YARN-2928.poc.patch
>
>
> After the 5/4 rebase, the hbase unit tests in the timeline service project
> are failing:
> {noformat}
> org.apache.hadoop.yarn.server.timelineservice.reader.TestTimelineReaderWebServicesHBaseStorage
> Time elapsed: 5.103 sec <<< ERROR!
> java.io.IOException: Shutting down
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
> at
> org.apache.hadoop.hbase.http.HttpServer.addDefaultServlets(HttpServer.java:677)
> at
> org.apache.hadoop.hbase.http.HttpServer.initializeWebServer(HttpServer.java:546)
> at org.apache.hadoop.hbase.http.HttpServer.(HttpServer.java:500)
> at org.apache.hadoop.hbase.http.HttpServer.(HttpServer.java:104)
> at
> org.apache.hadoop.hbase.http.HttpServer$Builder.build(HttpServer.java:345)
> at org.apache.hadoop.hbase.http.InfoServer.(InfoServer.java:77)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.putUpWebUI(HRegionServer.java:1697)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:550)
> at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:333)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
> at
> org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:139)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:217)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:153)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:213)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:93)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:978)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:938)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:812)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:806)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:750)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TestTimelineReaderWebServicesHBaseStorage.setup(TestTimelineReaderWebServicesHBaseStorage.java:87)
> {noformat}
> The root cause is that the hbase mini server depends on hadoop common's
> {{MetricsServlet}} which has been removed in the trunk (HADOOP-12504):
> {noformat}
> Caused by: java.lang.NoClassDefFoundError:
> org/apache/hadoop/metrics/MetricsServlet
> at
> org.apache.hadoop.hbase.http.HttpServer.addDefaultServlets(HttpServer.java:677)
> at
> org.apache.hadoop.hbase.http.HttpServer.initializeWebServer(HttpServer.java:546)
> at org.apache.hadoop.hbase.http.HttpServer.(HttpServer.java:500)
> at org.apache.hadoop.hbase.http.HttpServer.(HttpServer.java:104)
> at
> org.apache.hadoop.hbase.http.HttpServer$Builder.build(HttpServer.java:345)
> at org.apache.hadoop.hbase.http.InfoServer.(InfoServer.java:77)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.putUpWebUI(HRegionServer.java:1697)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:550)
> at org.apache.hadoop.hbase.master.HMaster.(HMast
[jira] [Commented] (YARN-5045) hbase unit tests fail due to dependency issues
[
https://issues.apache.org/jira/browse/YARN-5045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15390015#comment-15390015
]
Arpit Agarwal commented on YARN-5045:
-
Hi, trunk no longer builds for me and 'git bisect' seems to think this commit
is to blame. I get the following build error:
{code}
[INFO] --- maven-enforcer-plugin:1.4.1:enforce (depcheck) @
hadoop-yarn-server-timelineservice ---
[WARNING]
Dependency convergence error for org.apache.hbase:hbase-annotations:1.1.3 paths
to dependency are:
+-org.apache.hadoop:hadoop-yarn-server-timelineservice:3.0.0-alpha1-SNAPSHOT
+-org.apache.hbase:hbase-common:1.1.3
+-org.apache.hbase:hbase-protocol:1.1.3
+-org.apache.hbase:hbase-annotations:1.1.3
and
+-org.apache.hadoop:hadoop-yarn-server-timelineservice:3.0.0-alpha1-SNAPSHOT
+-org.apache.hbase:hbase-common:1.1.3
+-org.apache.hbase:hbase-annotations:1.1.3
{code}
I'll keep looking but posting in case any of you have ideas.
> hbase unit tests fail due to dependency issues
> --
>
> Key: YARN-5045
> URL: https://issues.apache.org/jira/browse/YARN-5045
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Sangjin Lee
>Assignee: Sangjin Lee
>Priority: Blocker
> Fix For: 3.0.0-alpha1
>
> Attachments: YARN-5045-YARN-2928.01.patch,
> YARN-5045-YARN-2928.02.patch, YARN-5045-YARN-2928.03.patch,
> YARN-5045-YARN-2928.poc.patch
>
>
> After the 5/4 rebase, the hbase unit tests in the timeline service project
> are failing:
> {noformat}
> org.apache.hadoop.yarn.server.timelineservice.reader.TestTimelineReaderWebServicesHBaseStorage
> Time elapsed: 5.103 sec <<< ERROR!
> java.io.IOException: Shutting down
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
> at
> org.apache.hadoop.hbase.http.HttpServer.addDefaultServlets(HttpServer.java:677)
> at
> org.apache.hadoop.hbase.http.HttpServer.initializeWebServer(HttpServer.java:546)
> at org.apache.hadoop.hbase.http.HttpServer.(HttpServer.java:500)
> at org.apache.hadoop.hbase.http.HttpServer.(HttpServer.java:104)
> at
> org.apache.hadoop.hbase.http.HttpServer$Builder.build(HttpServer.java:345)
> at org.apache.hadoop.hbase.http.InfoServer.(InfoServer.java:77)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.putUpWebUI(HRegionServer.java:1697)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:550)
> at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:333)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
> at
> org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:139)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:217)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:153)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:213)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:93)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:978)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:938)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:812)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:806)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:750)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TestTimelineReaderWebServicesHBaseStorage.setup(TestTimelineReaderWebServicesHBaseStorage.java:87)
> {noformat}
> The root cause is that the hbase mini server depends on hadoop common's
> {{MetricsServlet}} which has been removed in the trunk (HADOOP-12504):
> {noformat}
> Caused by: java.lang.NoClassDefFoundError:
> org/apache/hadoop/metrics/MetricsServlet
> at
> org.apach
[jira] [Commented] (YARN-1994) Expose YARN/MR endpoints on multiple interfaces
[ https://issues.apache.org/jira/browse/YARN-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15378062#comment-15378062 ] Arpit Agarwal commented on YARN-1994: - [~kasha], I also think that 0.0.0.0 is a better default. e.g. we made wild-card bind the default (HDFS-10363) in Ozone. Changing the behavior in 2.x feels risky as it may introduce a security risk for existing deployments. We can certainly change it in 3.0 and document it. > Expose YARN/MR endpoints on multiple interfaces > --- > > Key: YARN-1994 > URL: https://issues.apache.org/jira/browse/YARN-1994 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager, resourcemanager, webapp >Affects Versions: 2.4.0 >Reporter: Arpit Agarwal >Assignee: Craig Welch > Fix For: 2.6.0 > > Attachments: YARN-1994.0.patch, YARN-1994.1.patch, > YARN-1994.11.patch, YARN-1994.11.patch, YARN-1994.12.patch, > YARN-1994.13.patch, YARN-1994.14.patch, YARN-1994.15-branch2.patch, > YARN-1994.15.patch, YARN-1994.2.patch, YARN-1994.3.patch, YARN-1994.4.patch, > YARN-1994.5.patch, YARN-1994.6.patch, YARN-1994.7.patch > > > YARN and MapReduce daemons currently do not support specifying a wildcard > address for the server endpoints. This prevents the endpoints from being > accessible from all interfaces on a multihomed machine. > Note that if we do specify INADDR_ANY for any of the options, it will break > clients as they will attempt to connect to 0.0.0.0. We need a solution that > allows specifying a hostname or IP-address for clients while requesting > wildcard bind for the servers. > (List of endpoints is in a comment below) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4928) Some yarn.server.timeline.* tests fail on Windows attempting to use a test root path containing a colon
[
https://issues.apache.org/jira/browse/YARN-4928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15228424#comment-15228424
]
Arpit Agarwal commented on YARN-4928:
-
>From looking at the test cases I think these paths will be only used as HDFS
>paths.
> Some yarn.server.timeline.* tests fail on Windows attempting to use a test
> root path containing a colon
> ---
>
> Key: YARN-4928
> URL: https://issues.apache.org/jira/browse/YARN-4928
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Affects Versions: 2.8.0
> Environment: OS: Windows Server 2012
> JDK: 1.7.0_79
>Reporter: Gergely Novák
>Priority: Minor
> Attachments: YARN-4928.001.patch
>
>
> yarn.server.timeline.TestEntityGroupFSTimelineStore.* and
> yarn.server.timeline.TestLogInfo.* fail on Windows, because they are
> attempting to use a test root paths like
> "/C:/hdp/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage/target/test-dir/TestLogInfo",
> which contains a ":" (after the Windows drive letter) and
> DFSUtil.isValidName() does not accept paths containing ":".
> This problem is identical to HDFS-6189, so I suggest to use the same
> approach: using "/tmp/..." as test root dir instead of
> System.getProperty("test.build.data", System.getProperty("java.io.tmpdir")).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4928) Some yarn.server.timeline.* tests fail on Windows attempting to use a test root path containing a colon
[
https://issues.apache.org/jira/browse/YARN-4928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15228409#comment-15228409
]
Arpit Agarwal commented on YARN-4928:
-
+1 from me.
[~djp], do you have any comments?
> Some yarn.server.timeline.* tests fail on Windows attempting to use a test
> root path containing a colon
> ---
>
> Key: YARN-4928
> URL: https://issues.apache.org/jira/browse/YARN-4928
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Affects Versions: 2.8.0
> Environment: OS: Windows Server 2012
> JDK: 1.7.0_79
>Reporter: Gergely Novák
>Priority: Minor
> Attachments: YARN-4928.001.patch
>
>
> yarn.server.timeline.TestEntityGroupFSTimelineStore.* and
> yarn.server.timeline.TestLogInfo.* fail on Windows, because they are
> attempting to use a test root paths like
> "/C:/hdp/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage/target/test-dir/TestLogInfo",
> which contains a ":" (after the Windows drive letter) and
> DFSUtil.isValidName() does not accept paths containing ":".
> This problem is identical to HDFS-6189, so I suggest to use the same
> approach: using "/tmp/..." as test root dir instead of
> System.getProperty("test.build.data", System.getProperty("java.io.tmpdir")).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-1994) Expose YARN/MR endpoints on multiple interfaces
[ https://issues.apache.org/jira/browse/YARN-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14903614#comment-14903614 ] Arpit Agarwal commented on YARN-1994: - bq. Is it assumed that NM_BIND_HOST is configured to specific IP then NM_ADDRESS is also configured to the same IP ? Hi [~Naganarasimha], if NM_BIND_HOST is an IP address other than 0.0.0.0, then NM_ADDRESS should be set to a host that resolves to that address. Think of NM_BIND_HOST as the server side setting and NM_ADDRESS as a client side setting. If they are different the client cannot connect. I don't think we have tested setting NM_BIND_HOST to anything other than 0.0.0.0. In hindsight it may have been simpler to expose a boolean setting like NM_BIND_WILDCARD. bq. May be this a layman question why is it required to bind to all/multiple interfaces ? Depending on the routing and DNS configs, the client may connect on a different interface than the one bound by the server. Listening on all interfaces ensures connectivity. > Expose YARN/MR endpoints on multiple interfaces > --- > > Key: YARN-1994 > URL: https://issues.apache.org/jira/browse/YARN-1994 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager, resourcemanager, webapp >Affects Versions: 2.4.0 >Reporter: Arpit Agarwal >Assignee: Craig Welch > Fix For: 2.6.0 > > Attachments: YARN-1994.0.patch, YARN-1994.1.patch, > YARN-1994.11.patch, YARN-1994.11.patch, YARN-1994.12.patch, > YARN-1994.13.patch, YARN-1994.14.patch, YARN-1994.15-branch2.patch, > YARN-1994.15.patch, YARN-1994.2.patch, YARN-1994.3.patch, YARN-1994.4.patch, > YARN-1994.5.patch, YARN-1994.6.patch, YARN-1994.7.patch > > > YARN and MapReduce daemons currently do not support specifying a wildcard > address for the server endpoints. This prevents the endpoints from being > accessible from all interfaces on a multihomed machine. > Note that if we do specify INADDR_ANY for any of the options, it will break > clients as they will attempt to connect to 0.0.0.0. We need a solution that > allows specifying a hostname or IP-address for clients while requesting > wildcard bind for the servers. > (List of endpoints is in a comment below) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2549) TestContainerLaunch fails due to classpath problem with hamcrest classes.
[
https://issues.apache.org/jira/browse/YARN-2549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14134303#comment-14134303
]
Arpit Agarwal commented on YARN-2549:
-
+1 for the patch.
> TestContainerLaunch fails due to classpath problem with hamcrest classes.
> -
>
> Key: YARN-2549
> URL: https://issues.apache.org/jira/browse/YARN-2549
> Project: Hadoop YARN
> Issue Type: Test
> Components: nodemanager, test
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
>Priority: Minor
> Attachments: YARN-2549.1.patch
>
>
> The mockito jar bundles its own copy of the hamcrest classes, and it's ahead
> of our hamcrest dependency jar on the test classpath for
> hadoop-yarn-server-nodemanager. Unfortunately, the version bundled in
> mockito doesn't match the version we need, so it's missing the
> {{CoreMatchers#containsString}} method. This causes the tests to fail with
> {{NoSuchMethodError}} on Windows.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2384) Document YARN multihoming settings
[ https://issues.apache.org/jira/browse/YARN-2384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102807#comment-14102807 ] Arpit Agarwal commented on YARN-2384: - Thanks for the contribution [~kj-ki]. Perhaps instead of three different docs and duplicated content we can have a single doc with HDFS, YARN and MR settings and have it under common. We can remove the content from the current HDFS doc and leave a forwarding link to the common doc. What do you think? [~cwelch], would you be able to verify the yarn/mr content for accuracy? > Document YARN multihoming settings > -- > > Key: YARN-2384 > URL: https://issues.apache.org/jira/browse/YARN-2384 > Project: Hadoop YARN > Issue Type: Bug > Components: documentation >Affects Versions: 2.6.0 >Reporter: Arpit Agarwal > Attachments: YARN-2384.patch > > > YARN-1994 introduced new settings to improve multihoming support in YARN/MR. > This Jira is to get the settings documented. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2399) FairScheduler: Merge AppSchedulable and FSSchedulerApp into FSAppAttempt
[ https://issues.apache.org/jira/browse/YARN-2399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14094864#comment-14094864 ] Arpit Agarwal commented on YARN-2399: - Thanks Karthik! Verified the build break is fixed. > FairScheduler: Merge AppSchedulable and FSSchedulerApp into FSAppAttempt > > > Key: YARN-2399 > URL: https://issues.apache.org/jira/browse/YARN-2399 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Affects Versions: 2.5.0 >Reporter: Karthik Kambatla >Assignee: Karthik Kambatla > Fix For: 2.6.0 > > Attachments: yarn-2399-1.patch, yarn-2399-2.patch, yarn-2399-3.patch > > > FairScheduler has two data structures for an application, making the code > hard to track. We should merge these for better maintainability in the > long-term. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2399) FairScheduler: Merge AppSchedulable and FSSchedulerApp into FSAppAttempt
[ https://issues.apache.org/jira/browse/YARN-2399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14094788#comment-14094788 ] Arpit Agarwal commented on YARN-2399: - Hi, looks like this was committed and it broke trunk compilation. Can someone familiar with the patch please take a look? > FairScheduler: Merge AppSchedulable and FSSchedulerApp into FSAppAttempt > > > Key: YARN-2399 > URL: https://issues.apache.org/jira/browse/YARN-2399 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Affects Versions: 2.5.0 >Reporter: Karthik Kambatla >Assignee: Karthik Kambatla > Attachments: yarn-2399-1.patch, yarn-2399-2.patch, yarn-2399-3.patch > > > FairScheduler has two data structures for an application, making the code > hard to track. We should merge these for better maintainability in the > long-term. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (YARN-2384) Document YARN multihoming settings
Arpit Agarwal created YARN-2384: --- Summary: Document YARN multihoming settings Key: YARN-2384 URL: https://issues.apache.org/jira/browse/YARN-2384 Project: Hadoop YARN Issue Type: Bug Components: documentation Affects Versions: 2.6.0 Reporter: Arpit Agarwal YARN-1994 introduced new settings to improve multihoming support in YARN/MR. This Jira is to get the settings documented. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1994) Expose YARN/MR endpoints on multiple interfaces
[ https://issues.apache.org/jira/browse/YARN-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14078195#comment-14078195 ] Arpit Agarwal commented on YARN-1994: - +1 for the latest patch, thanks for all the patch iterations Craig! :-) > Expose YARN/MR endpoints on multiple interfaces > --- > > Key: YARN-1994 > URL: https://issues.apache.org/jira/browse/YARN-1994 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager, resourcemanager, webapp >Affects Versions: 2.4.0 >Reporter: Arpit Agarwal >Assignee: Craig Welch > Attachments: YARN-1994.0.patch, YARN-1994.1.patch, > YARN-1994.11.patch, YARN-1994.11.patch, YARN-1994.12.patch, > YARN-1994.2.patch, YARN-1994.3.patch, YARN-1994.4.patch, YARN-1994.5.patch, > YARN-1994.6.patch, YARN-1994.7.patch > > > YARN and MapReduce daemons currently do not support specifying a wildcard > address for the server endpoints. This prevents the endpoints from being > accessible from all interfaces on a multihomed machine. > Note that if we do specify INADDR_ANY for any of the options, it will break > clients as they will attempt to connect to 0.0.0.0. We need a solution that > allows specifying a hostname or IP-address for clients while requesting > wildcard bind for the servers. > (List of endpoints is in a comment below) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1994) Expose YARN/MR endpoints on multiple interfaces
[
https://issues.apache.org/jira/browse/YARN-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14077401#comment-14077401
]
Arpit Agarwal commented on YARN-1994:
-
+1 from me module one question. Why is the following logic only needed for
ContainerManagerImpl.java? I probably knew this but can't recall now.
{code}
InetSocketAddress connectAddress;
String connectHost = conf.getTrimmed(YarnConfiguration.NM_ADDRESS);
if (connectHost == null || connectHost.isEmpty()) {
// Get hostname and port from the listening endpoint.
connectAddress = NetUtils.getConnectAddress(server);
} else {
// Combine the configured hostname with the port from the listening
// endpoint. This gets the correct port number if the configuration
// specifies an ephemeral port (port number 0).
connectAddress = NetUtils.getConnectAddress(
new InetSocketAddress(connectHost.split(":")[0],
server.getListenerAddress().getPort()));
}
{code}
Thanks.
> Expose YARN/MR endpoints on multiple interfaces
> ---
>
> Key: YARN-1994
> URL: https://issues.apache.org/jira/browse/YARN-1994
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager, resourcemanager, webapp
>Affects Versions: 2.4.0
>Reporter: Arpit Agarwal
>Assignee: Craig Welch
> Attachments: YARN-1994.0.patch, YARN-1994.1.patch,
> YARN-1994.11.patch, YARN-1994.11.patch, YARN-1994.2.patch, YARN-1994.3.patch,
> YARN-1994.4.patch, YARN-1994.5.patch, YARN-1994.6.patch, YARN-1994.7.patch
>
>
> YARN and MapReduce daemons currently do not support specifying a wildcard
> address for the server endpoints. This prevents the endpoints from being
> accessible from all interfaces on a multihomed machine.
> Note that if we do specify INADDR_ANY for any of the options, it will break
> clients as they will attempt to connect to 0.0.0.0. We need a solution that
> allows specifying a hostname or IP-address for clients while requesting
> wildcard bind for the servers.
> (List of endpoints is in a comment below)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-1994) Expose YARN/MR endpoints on multiple interfaces
[ https://issues.apache.org/jira/browse/YARN-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14072751#comment-14072751 ] Arpit Agarwal commented on YARN-1994: - +1 for the v6 patch. I will hold off on committing until Vinod or another YARN committer can sanity check the changes. Thanks [~cwelch] and [~mipoto]! I think there is a Windows line-endings issue with the patch hence Jenkins failed to pick it up. I was able to apply it with _git apply -p0 --whitespace=fix_ > Expose YARN/MR endpoints on multiple interfaces > --- > > Key: YARN-1994 > URL: https://issues.apache.org/jira/browse/YARN-1994 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager, resourcemanager, webapp >Affects Versions: 2.4.0 >Reporter: Arpit Agarwal >Assignee: Craig Welch > Attachments: YARN-1994.0.patch, YARN-1994.1.patch, YARN-1994.2.patch, > YARN-1994.3.patch, YARN-1994.4.patch, YARN-1994.5.patch, YARN-1994.6.patch > > > YARN and MapReduce daemons currently do not support specifying a wildcard > address for the server endpoints. This prevents the endpoints from being > accessible from all interfaces on a multihomed machine. > Note that if we do specify INADDR_ANY for any of the options, it will break > clients as they will attempt to connect to 0.0.0.0. We need a solution that > allows specifying a hostname or IP-address for clients while requesting > wildcard bind for the servers. > (List of endpoints is in a comment below) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1994) Expose YARN/MR endpoints on multiple interfaces
[ https://issues.apache.org/jira/browse/YARN-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14063065#comment-14063065 ] Arpit Agarwal commented on YARN-1994: - [~mipoto], also feel free to incorporate the above feedback if it is applicable to your patch. Thanks, Arpit. > Expose YARN/MR endpoints on multiple interfaces > --- > > Key: YARN-1994 > URL: https://issues.apache.org/jira/browse/YARN-1994 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager, resourcemanager, webapp >Affects Versions: 2.4.0 >Reporter: Arpit Agarwal >Assignee: Craig Welch > Attachments: YARN-1994.0.patch, YARN-1994.1.patch, YARN-1994.2.patch, > YARN-1994.3.patch, YARN-1994.4.patch > > > YARN and MapReduce daemons currently do not support specifying a wildcard > address for the server endpoints. This prevents the endpoints from being > accessible from all interfaces on a multihomed machine. > Note that if we do specify INADDR_ANY for any of the options, it will break > clients as they will attempt to connect to 0.0.0.0. We need a solution that > allows specifying a hostname or IP-address for clients while requesting > wildcard bind for the servers. > (List of endpoints is in a comment below) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1994) Expose YARN/MR endpoints on multiple interfaces
[
https://issues.apache.org/jira/browse/YARN-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14063064#comment-14063064
]
Arpit Agarwal commented on YARN-1994:
-
[~mipoto], I was unable to apply your v4 patch to trunk. Could you please
rebase it to trunk and fix the whitespace errors?. I’ll review it once it is
fixed.
[~cwelch], Thank you for picking up this Jira. I reviewed your v3 patch and it
looks great, just a few comments below.
*AdminService.java:*
- Now that you have the code to construct the correct connectAddress (lines
180-192), {{RPCUtil.updateConnectAddr}} looks a little inaccurate. It undoes
the work of constructing the connectAddress since it always gets the hostname
from the property. I think instead of the call to
{{RPCUtil.updateConnectAddr}}, we can just have
_conf.updateConnectAddr(YarnConfiguration.RM_ADMIN_ADDRESS, connectAddress)_.
It is probably not a big deal since any real-world configuration will define
this property.
*ApplicationMasterService.java, ClientRMService.java,
HistoryClientService.java, ResourceLocalizationService.java,
ResourceTrackerService.java:*
- Do we need a similar logic to compute connectAddress in {{serviceStart}}?
*WebAppUtils.java:*
- {{getNMWebAppBindURLWithoutScheme}} can be simplified by using
{{RPCUtil.getAddressAsString}}.
*YarnConfiguration.java:*
- {{TIMELINE_SERVICE_BIND_HOST}} is unused. Fix the timeline service or just
drop this definition?
I don't understand YARN well enough to be sure that RM webapp endpoints are
being correctly handled, [~vinodkv] could you please review that?
Thanks, Arpit.
> Expose YARN/MR endpoints on multiple interfaces
> ---
>
> Key: YARN-1994
> URL: https://issues.apache.org/jira/browse/YARN-1994
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager, resourcemanager, webapp
>Affects Versions: 2.4.0
>Reporter: Arpit Agarwal
>Assignee: Craig Welch
> Attachments: YARN-1994.0.patch, YARN-1994.1.patch, YARN-1994.2.patch,
> YARN-1994.3.patch, YARN-1994.4.patch
>
>
> YARN and MapReduce daemons currently do not support specifying a wildcard
> address for the server endpoints. This prevents the endpoints from being
> accessible from all interfaces on a multihomed machine.
> Note that if we do specify INADDR_ANY for any of the options, it will break
> clients as they will attempt to connect to 0.0.0.0. We need a solution that
> allows specifying a hostname or IP-address for clients while requesting
> wildcard bind for the servers.
> (List of endpoints is in a comment below)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2023) TestClientRMService fails on trunk
[ https://issues.apache.org/jira/browse/YARN-2023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990777#comment-13990777 ] Arpit Agarwal commented on YARN-2023: - Thanks for reporting this [~zhiguohong]. I committed the fix for YARN-2018. > TestClientRMService fails on trunk > -- > > Key: YARN-2023 > URL: https://issues.apache.org/jira/browse/YARN-2023 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Reporter: Hong Zhiguo >Assignee: Hong Zhiguo > Attachments: YARN-2023.patch > > > testTokenRenewalWrongUser(org.apache.hadoop.yarn.server.resourcemanager.TestClientRMService) > Time elapsed: 0.015 sec <<< FAILURE! > java.lang.AssertionError: null > at org.junit.Assert.fail(Assert.java:86) > at org.junit.Assert.assertTrue(Assert.java:41) > at org.junit.Assert.assertTrue(Assert.java:52) > at > org.apache.hadoop.yarn.server.resourcemanager.TestClientRMService$4.run(TestClientRMService.java:481) > at > org.apache.hadoop.yarn.server.resourcemanager.TestClientRMService$4.run(TestClientRMService.java:474) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1606) > at > org.apache.hadoop.yarn.server.resourcemanager.TestClientRMService.testTokenRenewalWrongUser(TestClientRMService.java:474) > Results : > Failed tests: > TestClientRMService.testTokenRenewalWrongUser:474 null -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (YARN-2018) TestClientRMService.testTokenRenewalWrongUser fails after HADOOP-10562
[ https://issues.apache.org/jira/browse/YARN-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated YARN-2018: Target Version/s: 2.5.0 Hadoop Flags: Reviewed +1 for the patch, I committed this to trunk and branch-2. It's odd that the Jenkins run for HADOOP-10562 did not catch this. Thanks for reporting and fixing this folks. I don't have permissions to resolve issues in YARN project so can someone with the appropriate permissions please take care of that. > TestClientRMService.testTokenRenewalWrongUser fails after HADOOP-10562 > > > Key: YARN-2018 > URL: https://issues.apache.org/jira/browse/YARN-2018 > Project: Hadoop YARN > Issue Type: Test >Affects Versions: 2.5.0 >Reporter: Tsuyoshi OZAWA >Assignee: Ming Ma > Attachments: YARN-2018.patch > > > The test failure is observed on YARN-1945 and YARN-1861. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (YARN-2018) TestClientRMService.testTokenRenewalWrongUser fails after HADOOP-10562
[ https://issues.apache.org/jira/browse/YARN-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated YARN-2018: Affects Version/s: 2.5.0 > TestClientRMService.testTokenRenewalWrongUser fails after HADOOP-10562 > > > Key: YARN-2018 > URL: https://issues.apache.org/jira/browse/YARN-2018 > Project: Hadoop YARN > Issue Type: Test >Affects Versions: 2.5.0 >Reporter: Tsuyoshi OZAWA >Assignee: Ming Ma > Attachments: YARN-2018.patch > > > The test failure is observed on YARN-1945 and YARN-1861. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (YARN-1994) Expose YARN/MR endpoints on multiple interfaces
[ https://issues.apache.org/jira/browse/YARN-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated YARN-1994: Description: YARN and MapReduce daemons currently do not support specifying a wildcard address for the server endpoints. This prevents the endpoints from being accessible from all interfaces on a multihomed machine. Note that if we do specify INADDR_ANY for any of the options, it will break clients as they will attempt to connect to 0.0.0.0. We need a solution that allows specifying a hostname or IP-address for clients while requesting wildcard bind for the servers. (List of endpoints is in a comment below) was: YARN and MapReduce daemons currently do not support specifying a wildcard address for the server endpoints. This prevents the endpoints from being accessible from all interfaces on a multihomed machine. A preliminary shows the following candidates: # yarn.nodemanager.address # yarn.nodemanager.webapp.address # yarn.nodemanager.webapp.https.address # yarn.resourcemanager.address # yarn.resourcemanager.webapp.address # yarn.resourcemanager.webapp.https.address # yarn.resourcemanager.resource-tracker.address # yarn.resourcemanager.scheduler.address # yarn.resourcemanager.admin.address # mapreduce.jobhistory.address # mapreduce.jobhistory.admin.address # mapreduce.jobhistory.webapp.address # mapreduce.jobhistory.webapp.https.address # mapreduce.history.server.http.address (Deprecated) # yarn.timeline-service.webapp.address # yarn.timeline-service.webapp.https.address Note that if we do specify INADDR_ANY for any of the options, it will break clients as they will attempt to connect to 0.0.0.0. We need a solution that allows specifying a hostname or IP-address for clients while requesting wildcard bind for the servers. > Expose YARN/MR endpoints on multiple interfaces > --- > > Key: YARN-1994 > URL: https://issues.apache.org/jira/browse/YARN-1994 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager, resourcemanager, webapp >Affects Versions: 2.4.0 >Reporter: Arpit Agarwal > > YARN and MapReduce daemons currently do not support specifying a wildcard > address for the server endpoints. This prevents the endpoints from being > accessible from all interfaces on a multihomed machine. > Note that if we do specify INADDR_ANY for any of the options, it will break > clients as they will attempt to connect to 0.0.0.0. We need a solution that > allows specifying a hostname or IP-address for clients while requesting > wildcard bind for the servers. > (List of endpoints is in a comment below) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1994) Expose YARN/MR endpoints on multiple interfaces
[ https://issues.apache.org/jira/browse/YARN-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13983828#comment-13983828 ] Arpit Agarwal commented on YARN-1994: - A preliminary shows the following candidates: # yarn.nodemanager.address # yarn.nodemanager.webapp.address # yarn.nodemanager.webapp.https.address # yarn.resourcemanager.address # yarn.resourcemanager.webapp.address # yarn.resourcemanager.webapp.https.address # yarn.resourcemanager.resource-tracker.address # yarn.resourcemanager.scheduler.address # yarn.resourcemanager.admin.address # mapreduce.jobhistory.address # mapreduce.jobhistory.admin.address # mapreduce.jobhistory.webapp.address # mapreduce.jobhistory.webapp.https.address # mapreduce.history.server.http.address (Deprecated) # yarn.timeline-service.webapp.address # yarn.timeline-service.webapp.https.address > Expose YARN/MR endpoints on multiple interfaces > --- > > Key: YARN-1994 > URL: https://issues.apache.org/jira/browse/YARN-1994 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager, resourcemanager, webapp >Affects Versions: 2.4.0 >Reporter: Arpit Agarwal > > YARN and MapReduce daemons currently do not support specifying a wildcard > address for the server endpoints. This prevents the endpoints from being > accessible from all interfaces on a multihomed machine. > Note that if we do specify INADDR_ANY for any of the options, it will break > clients as they will attempt to connect to 0.0.0.0. We need a solution that > allows specifying a hostname or IP-address for clients while requesting > wildcard bind for the servers. > (List of endpoints is in a comment below) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (YARN-1994) Expose YARN/MR endpoints on multiple interfaces
[ https://issues.apache.org/jira/browse/YARN-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated YARN-1994: Description: YARN and MapReduce daemons currently do not support specifying a wildcard address for the server endpoints. This prevents the endpoints from being accessible from all interfaces on a multihomed machine. A preliminary shows the following candidates: # yarn.nodemanager.address # yarn.nodemanager.webapp.address # yarn.nodemanager.webapp.https.address # yarn.resourcemanager.address # yarn.resourcemanager.webapp.address # yarn.resourcemanager.webapp.https.address # yarn.resourcemanager.resource-tracker.address # yarn.resourcemanager.scheduler.address # yarn.resourcemanager.admin.address # mapreduce.jobhistory.address # mapreduce.jobhistory.admin.address # mapreduce.jobhistory.webapp.address # mapreduce.jobhistory.webapp.https.address # mapreduce.history.server.http.address (Deprecated) # yarn.timeline-service.webapp.address # yarn.timeline-service.webapp.https.address Note that if we do specify INADDR_ANY for any of the options, it will break clients as they will attempt to connect to 0.0.0.0. We need a solution that allows specifying a hostname or IP-address for clients while requesting wildcard bind for the servers. was: YARN and MapReduce daemons currently do not support specifying a wildcard address for the server endpoints. This prevents the endpoints from being accessible from all interfaces on a multihomed machine. A preliminary shows the following candidates: # yarn.nodemanager.address # yarn.nodemanager.webapp.address # yarn.nodemanager.webapp.https.address # yarn.resourcemanager.address # yarn.resourcemanager.webapp.address # yarn.resourcemanager.webapp.https.address # yarn.resourcemanager.resource-tracker.address # yarn.resourcemanager.scheduler.address # yarn.resourcemanager.admin.address # mapreduce.jobhistory.address Note that if we do specify INADDR_ANY for any of the options, it will break clients as they will attempt to connect to 0.0.0.0. We need a solution that allows specifying a hostname or IP-address for clients while requesting wildcard bind for the servers. # mapreduce.jobhistory.admin.address # mapreduce.jobhistory.webapp.address # mapreduce.jobhistory.webapp.https.address # mapreduce.history.server.http.address (Deprecated) # yarn.timeline-service.webapp.address # yarn.timeline-service.webapp.https.address > Expose YARN/MR endpoints on multiple interfaces > --- > > Key: YARN-1994 > URL: https://issues.apache.org/jira/browse/YARN-1994 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager, resourcemanager, webapp >Affects Versions: 2.4.0 >Reporter: Arpit Agarwal > > YARN and MapReduce daemons currently do not support specifying a wildcard > address for the server endpoints. This prevents the endpoints from being > accessible from all interfaces on a multihomed machine. > A preliminary shows the following candidates: > # yarn.nodemanager.address > # yarn.nodemanager.webapp.address > # yarn.nodemanager.webapp.https.address > # yarn.resourcemanager.address > # yarn.resourcemanager.webapp.address > # yarn.resourcemanager.webapp.https.address > # yarn.resourcemanager.resource-tracker.address > # yarn.resourcemanager.scheduler.address > # yarn.resourcemanager.admin.address > # mapreduce.jobhistory.address > # mapreduce.jobhistory.admin.address > # mapreduce.jobhistory.webapp.address > # mapreduce.jobhistory.webapp.https.address > # mapreduce.history.server.http.address (Deprecated) > # yarn.timeline-service.webapp.address > # yarn.timeline-service.webapp.https.address > Note that if we do specify INADDR_ANY for any of the options, it will break > clients as they will attempt to connect to 0.0.0.0. We need a solution that > allows specifying a hostname or IP-address for clients while requesting > wildcard bind for the servers. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1994) Expose YARN/MR endpoints on multiple interfaces
[
https://issues.apache.org/jira/browse/YARN-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13983774#comment-13983774
]
Arpit Agarwal commented on YARN-1994:
-
Linked to HDFS-6273 and HDFS-5128 for one approach to solving this problem.
We introduced {{*.bind-host}} options which override the hostname portion of
the corresponding {{*.address}} setting, when present.
> Expose YARN/MR endpoints on multiple interfaces
> ---
>
> Key: YARN-1994
> URL: https://issues.apache.org/jira/browse/YARN-1994
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager, resourcemanager, webapp
>Affects Versions: 2.4.0
>Reporter: Arpit Agarwal
>
> YARN and MapReduce daemons currently do not support specifying a wildcard
> address for the server endpoints. This prevents the endpoints from being
> accessible from all interfaces on a multihomed machine.
> A preliminary shows the following candidates:
> # yarn.nodemanager.address
> # yarn.nodemanager.webapp.address
> # yarn.nodemanager.webapp.https.address
> # yarn.resourcemanager.address
> # yarn.resourcemanager.webapp.address
> # yarn.resourcemanager.webapp.https.address
> # yarn.resourcemanager.resource-tracker.address
> # yarn.resourcemanager.scheduler.address
> # yarn.resourcemanager.admin.address
> # mapreduce.jobhistory.address
> Note that if we do specify INADDR_ANY for any of the options, it will break
> clients as they will attempt to connect to 0.0.0.0. We need a solution that
> allows specifying a hostname or IP-address for clients while requesting
> wildcard bind for the servers.
> # mapreduce.jobhistory.admin.address
> # mapreduce.jobhistory.webapp.address
> # mapreduce.jobhistory.webapp.https.address
> # mapreduce.history.server.http.address (Deprecated)
> # yarn.timeline-service.webapp.address
> # yarn.timeline-service.webapp.https.address
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (YARN-1994) Expose YARN/MR endpoints on multiple interfaces
Arpit Agarwal created YARN-1994: --- Summary: Expose YARN/MR endpoints on multiple interfaces Key: YARN-1994 URL: https://issues.apache.org/jira/browse/YARN-1994 Project: Hadoop YARN Issue Type: Improvement Components: nodemanager, resourcemanager, webapp Affects Versions: 2.4.0 Reporter: Arpit Agarwal YARN and MapReduce daemons currently do not support specifying a wildcard address for the server endpoints. This prevents the endpoints from being accessible from all interfaces on a multihomed machine. A preliminary shows the following candidates: # yarn.nodemanager.address # yarn.nodemanager.webapp.address # yarn.nodemanager.webapp.https.address # yarn.resourcemanager.address # yarn.resourcemanager.webapp.address # yarn.resourcemanager.webapp.https.address # yarn.resourcemanager.resource-tracker.address # yarn.resourcemanager.scheduler.address # yarn.resourcemanager.admin.address # mapreduce.jobhistory.address Note that if we do specify INADDR_ANY for any of the options, it will break clients as they will attempt to connect to 0.0.0.0. We need a solution that allows specifying a hostname or IP-address for clients while requesting wildcard bind for the servers. # mapreduce.jobhistory.admin.address # mapreduce.jobhistory.webapp.address # mapreduce.jobhistory.webapp.https.address # mapreduce.history.server.http.address (Deprecated) # yarn.timeline-service.webapp.address # yarn.timeline-service.webapp.https.address -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1970) Prepare YARN codebase for JUnit 4.11.
[ https://issues.apache.org/jira/browse/YARN-1970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976155#comment-13976155 ] Arpit Agarwal commented on YARN-1970: - +1 for the patch. > Prepare YARN codebase for JUnit 4.11. > - > > Key: YARN-1970 > URL: https://issues.apache.org/jira/browse/YARN-1970 > Project: Hadoop YARN > Issue Type: Test >Affects Versions: 3.0.0, 2.4.0 >Reporter: Chris Nauroth >Assignee: Chris Nauroth >Priority: Minor > Attachments: YARN-1970.1.patch > > > HADOOP-10503 upgrades the entire Hadoop repo to use JUnit 4.11. Some of the > YARN code needs some minor updates to fix deprecation warnings and test > isolation problems before the upgrade. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-713) ResourceManager can exit unexpectedly if DNS is unavailable
[ https://issues.apache.org/jira/browse/YARN-713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13906468#comment-13906468 ] Arpit Agarwal commented on YARN-713: Thanks Vinod! > ResourceManager can exit unexpectedly if DNS is unavailable > --- > > Key: YARN-713 > URL: https://issues.apache.org/jira/browse/YARN-713 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 2.1.0-beta >Reporter: Jason Lowe >Assignee: Jian He >Priority: Critical > Fix For: 2.4.0 > > Attachments: YARN-713.09052013.1.patch, YARN-713.09062013.1.patch, > YARN-713.1.patch, YARN-713.2.patch, YARN-713.20130910.1.patch, > YARN-713.3.patch, YARN-713.4.patch, YARN-713.5.patch, YARN-713.6.patch, > YARN-713.patch, YARN-713.patch, YARN-713.patch, YARN-713.patch > > > As discussed in MAPREDUCE-5261, there's a possibility that a DNS outage could > lead to an unhandled exception in the ResourceManager's AsyncDispatcher, and > that ultimately would cause the RM to exit. The RM should not exit during > DNS hiccups. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-713) ResourceManager can exit unexpectedly if DNS is unavailable
[
https://issues.apache.org/jira/browse/YARN-713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13906304#comment-13906304
]
Arpit Agarwal commented on YARN-713:
Is this error in branch-2.4 related?
{code}
WARN: Please see http://www.slf4j.org/codes.html for an explanation.
[ERROR] COMPILATION ERROR :
[ERROR]
/Users/aagarwal/src/commit/branch-2.4/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/Allocation.java:[32,16]
cannot find symbol
symbol : class RecordFactory
location: class
org.apache.hadoop.yarn.server.resourcemanager.scheduler.Allocation
[ERROR]
/Users/aagarwal/src/commit/branch-2.4/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/Allocation.java:[33,6]
cannot find symbol
{code}
Thanks.
> ResourceManager can exit unexpectedly if DNS is unavailable
> ---
>
> Key: YARN-713
> URL: https://issues.apache.org/jira/browse/YARN-713
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 2.1.0-beta
>Reporter: Jason Lowe
>Assignee: Jian He
>Priority: Critical
> Fix For: 2.4.0
>
> Attachments: YARN-713.09052013.1.patch, YARN-713.09062013.1.patch,
> YARN-713.1.patch, YARN-713.2.patch, YARN-713.20130910.1.patch,
> YARN-713.3.patch, YARN-713.4.patch, YARN-713.5.patch, YARN-713.6.patch,
> YARN-713.patch, YARN-713.patch, YARN-713.patch, YARN-713.patch
>
>
> As discussed in MAPREDUCE-5261, there's a possibility that a DNS outage could
> lead to an unhandled exception in the ResourceManager's AsyncDispatcher, and
> that ultimately would cause the RM to exit. The RM should not exit during
> DNS hiccups.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (YARN-1349) yarn.cmd does not support passthrough to any arbitrary class.
[ https://issues.apache.org/jira/browse/YARN-1349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13806421#comment-13806421 ] Arpit Agarwal commented on YARN-1349: - +1 for the patch. > yarn.cmd does not support passthrough to any arbitrary class. > - > > Key: YARN-1349 > URL: https://issues.apache.org/jira/browse/YARN-1349 > Project: Hadoop YARN > Issue Type: Bug > Components: client >Affects Versions: 3.0.0, 2.2.0 >Reporter: Chris Nauroth >Assignee: Chris Nauroth > Attachments: YARN-1349.1.patch, YARN-1349.2.patch > > > The yarn shell script supports passthrough to calling any arbitrary class if > the first argument is not one of the per-defined sub-commands. The > equivalent cmd script does not implement this and instead fails trying to do > a labeled goto to the first argument. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (YARN-1331) yarn.cmd exits with NoClassDefFoundError trying to run rmadmin or logs
[
https://issues.apache.org/jira/browse/YARN-1331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13801396#comment-13801396
]
Arpit Agarwal commented on YARN-1331:
-
+1 for the patch.
Verified both commands manually on a Windows single-node cluster, worked as
expected.
> yarn.cmd exits with NoClassDefFoundError trying to run rmadmin or logs
> --
>
> Key: YARN-1331
> URL: https://issues.apache.org/jira/browse/YARN-1331
> Project: Hadoop YARN
> Issue Type: Bug
> Components: client
>Affects Versions: 2.2.0
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Fix For: 2.2.1
>
> Attachments: YARN-1331.1.patch
>
>
> The yarn shell script was updated so that the rmadmin and logs sub-commands
> launch {{org.apache.hadoop.yarn.client.cli.RMAdminCLI}} and
> {{org.apache.hadoop.yarn.client.cli.LogsCLI}}. The yarn.cmd script also
> needs to be updated so that the commands work on Windows.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (YARN-1025) ResourceManager and NodeManager do not load native libraries on Windows.
[ https://issues.apache.org/jira/browse/YARN-1025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13763614#comment-13763614 ] Arpit Agarwal commented on YARN-1025: - +1 for the change. > ResourceManager and NodeManager do not load native libraries on Windows. > > > Key: YARN-1025 > URL: https://issues.apache.org/jira/browse/YARN-1025 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager, resourcemanager >Affects Versions: 3.0.0, 2.1.1-beta >Reporter: Chris Nauroth > Attachments: YARN-1025.1.patch > > > ResourceManager and NodeManager do not have the correct setting for > java.library.path when launched on Windows. This prevents the processes from > loading native code from hadoop.dll. The native code is required for correct > functioning on Windows (not optional), so this ultimately can cause failures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-491) TestContainerLogsPage fails on Windows
[
https://issues.apache.org/jira/browse/YARN-491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13608079#comment-13608079
]
Arpit Agarwal commented on YARN-491:
+1
Verified on Windows and OS X. Thanks for all the YARN fixes!
> TestContainerLogsPage fails on Windows
> --
>
> Key: YARN-491
> URL: https://issues.apache.org/jira/browse/YARN-491
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 3.0.0
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Attachments: YARN-491.1.patch
>
>
> {{TestContainerLogsPage}} contains some code for initializing a log directory
> that doesn't work correctly on Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-487) TestDiskFailures fails on Windows due to path mishandling
[
https://issues.apache.org/jira/browse/YARN-487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13608061#comment-13608061
]
Arpit Agarwal commented on YARN-487:
+1
Verified on Windows and OS X.
> TestDiskFailures fails on Windows due to path mishandling
> -
>
> Key: YARN-487
> URL: https://issues.apache.org/jira/browse/YARN-487
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 3.0.0
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Attachments: YARN-487.1.patch
>
>
> {{TestDiskFailures#testDirFailuresOnStartup}} fails due to insertion of an
> extra leading '/' on the path within {{LocalDirsHandlerService}} when running
> on Windows. The test assertions also fail to account for the fact that
> {{Path}} normalizes '\' to '/'.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-490) TestDistributedShell fails on Windows
[ https://issues.apache.org/jira/browse/YARN-490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13608049#comment-13608049 ] Arpit Agarwal commented on YARN-490: +1 Verified patch on Windows and OS X. I also verified that eclipse:eclipse builds on both platforms. > TestDistributedShell fails on Windows > - > > Key: YARN-490 > URL: https://issues.apache.org/jira/browse/YARN-490 > Project: Hadoop YARN > Issue Type: Bug > Components: applications/distributed-shell >Affects Versions: 3.0.0 >Reporter: Chris Nauroth >Assignee: Chris Nauroth > Labels: windows > Attachments: YARN-490.1.patch > > > There are a few platform-specific assumption in distributed shell (both main > code and test code) that prevent it from working correctly on Windows. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-490) TestDistributedShell fails on Windows
[ https://issues.apache.org/jira/browse/YARN-490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated YARN-490: --- Labels: windows (was: ) > TestDistributedShell fails on Windows > - > > Key: YARN-490 > URL: https://issues.apache.org/jira/browse/YARN-490 > Project: Hadoop YARN > Issue Type: Bug > Components: applications/distributed-shell >Affects Versions: 3.0.0 >Reporter: Chris Nauroth >Assignee: Chris Nauroth > Labels: windows > Attachments: YARN-490.1.patch > > > There are a few platform-specific assumption in distributed shell (both main > code and test code) that prevent it from working correctly on Windows. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-488) TestContainerManagerSecurity fails on Windows
[ https://issues.apache.org/jira/browse/YARN-488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13607850#comment-13607850 ] Arpit Agarwal commented on YARN-488: +1 Verified on Windows and OSX. > TestContainerManagerSecurity fails on Windows > - > > Key: YARN-488 > URL: https://issues.apache.org/jira/browse/YARN-488 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 3.0.0 >Reporter: Chris Nauroth >Assignee: Chris Nauroth > Attachments: YARN-488.1.patch > > > These tests are failing to launch containers correctly when running on > Windows. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
