[jira] [Updated] (YARN-10265) Upgrade Netty-all dependency to latest version 4.1.50 to fix ARM support issue
[ https://issues.apache.org/jira/browse/YARN-10265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated YARN-10265: Description: In the issue: https://issues.apache.org/jira/browse/YARN-9898 we have added a workaround using an non-official released netty-4.1.48 to fix the ARM support issue. but just a few hours agon, Netty has release the 4.1.50 version which is officially support ARM platform, please see: [https://github.com/netty/netty/pull/9804] netty-4.1.50.Final release: [https://github.com/netty/netty/releases/tag/netty-4.1.50.Final] commits from netty-4.1.48 to netty-4.1.50: [https://github.com/netty/netty/compare/netty-4.1.48.Final...netty-4.1.50.Final] So, now it is better to upgrade the netty-dependency version of Hadoop to 4.1.50 version. was: In the issue: https://issues.apache.org/jira/browse/YARN-9898 we have added a workaround using an non-official released netty-4.1.48 to fix the ARM support issue. but just a few hours agon, Netty has release the 4.1.50 version which is officially support ARM platform, please see: [https://github.com/netty/netty/pull/9804] netty-4.1.50.Final release: [https://github.com/netty/netty/releases/tag/netty-4.1.50.Final] commits from netty-4.1.48 to netty-4.1.50: [https://github.com/netty/netty/compare/netty-4.1.48.Final...netty-4.1.50.Final|https://github.com/netty/netty/compare/netty-4.1.49.Final...netty-4.1.50.Final] So, now it is better to upgrade the netty-dependency version of Hadoop to 4.1.50 version. > Upgrade Netty-all dependency to latest version 4.1.50 to fix ARM support issue > -- > > Key: YARN-10265 > URL: https://issues.apache.org/jira/browse/YARN-10265 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Priority: Major > > In the issue: https://issues.apache.org/jira/browse/YARN-9898 we have added a > workaround using an non-official released netty-4.1.48 to fix the ARM support > issue. but just a few hours agon, Netty has release the 4.1.50 version which > is officially support ARM platform, please see: > [https://github.com/netty/netty/pull/9804] > > netty-4.1.50.Final release: > [https://github.com/netty/netty/releases/tag/netty-4.1.50.Final] > commits from netty-4.1.48 to netty-4.1.50: > [https://github.com/netty/netty/compare/netty-4.1.48.Final...netty-4.1.50.Final] > So, now it is better to upgrade the netty-dependency version of Hadoop to > 4.1.50 version. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10265) Upgrade Netty-all dependency to latest version 4.1.50 to fix ARM support issue
[ https://issues.apache.org/jira/browse/YARN-10265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated YARN-10265: Description: In the issue: https://issues.apache.org/jira/browse/YARN-9898 we have added a workaround using an non-official released netty-4.1.48 to fix the ARM support issue. but just a few hours agon, Netty has release the 4.1.50 version which is officially support ARM platform, please see: [https://github.com/netty/netty/pull/9804] netty-4.1.50.Final release: [https://github.com/netty/netty/releases/tag/netty-4.1.50.Final] commits from netty-4.1.48 to netty-4.1.50: [https://github.com/netty/netty/compare/netty-4.1.48.Final...netty-4.1.50.Final|https://github.com/netty/netty/compare/netty-4.1.49.Final...netty-4.1.50.Final] So, now it is better to upgrade the netty-dependency version of Hadoop to 4.1.50 version. was: In the issue: https://issues.apache.org/jira/browse/YARN-9898 we have added a workaround using an non-official released netty-4.1.49 to fix the ARM support issue. but just a few hours agon, Netty has release the 4.1.50 version which is officially support ARM platform, please see: [https://github.com/netty/netty/pull/9804] netty-4.1.50.Final release: [https://github.com/netty/netty/releases/tag/netty-4.1.50.Final] commits from netty-4.1.49 to netty-4.1.50: [https://github.com/netty/netty/compare/netty-4.1.49.Final...netty-4.1.50.Final] So, now it is better to upgrade the netty-dependency version of Hadoop to 4.1.50 version. > Upgrade Netty-all dependency to latest version 4.1.50 to fix ARM support issue > -- > > Key: YARN-10265 > URL: https://issues.apache.org/jira/browse/YARN-10265 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Priority: Major > > In the issue: https://issues.apache.org/jira/browse/YARN-9898 we have added a > workaround using an non-official released netty-4.1.48 to fix the ARM support > issue. but just a few hours agon, Netty has release the 4.1.50 version which > is officially support ARM platform, please see: > [https://github.com/netty/netty/pull/9804] > > netty-4.1.50.Final release: > [https://github.com/netty/netty/releases/tag/netty-4.1.50.Final] > commits from netty-4.1.48 to netty-4.1.50: > [https://github.com/netty/netty/compare/netty-4.1.48.Final...netty-4.1.50.Final|https://github.com/netty/netty/compare/netty-4.1.49.Final...netty-4.1.50.Final] > So, now it is better to upgrade the netty-dependency version of Hadoop to > 4.1.50 version. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10265) Upgrade Netty-all dependency to latest version 4.1.50 to fix ARM support issue
liusheng created YARN-10265: --- Summary: Upgrade Netty-all dependency to latest version 4.1.50 to fix ARM support issue Key: YARN-10265 URL: https://issues.apache.org/jira/browse/YARN-10265 Project: Hadoop YARN Issue Type: Bug Reporter: liusheng In the issue: https://issues.apache.org/jira/browse/YARN-9898 we have added a workaround using an non-official released netty-4.1.49 to fix the ARM support issue. but just a few hours agon, Netty has release the 4.1.50 version which is officially support ARM platform, please see: [https://github.com/netty/netty/pull/9804] netty-4.1.50.Final release: [https://github.com/netty/netty/releases/tag/netty-4.1.50.Final] commits from netty-4.1.49 to netty-4.1.50: [https://github.com/netty/netty/compare/netty-4.1.49.Final...netty-4.1.50.Final] So, now it is better to upgrade the netty-dependency version of Hadoop to 4.1.50 version. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9898) Dependency netty-all-4.1.27.Final doesn't support ARM platform
[ https://issues.apache.org/jira/browse/YARN-9898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105940#comment-17105940 ] liusheng commented on YARN-9898: The 2 above failed tests I have run locally without this patch applied and still fail > Dependency netty-all-4.1.27.Final doesn't support ARM platform > -- > > Key: YARN-9898 > URL: https://issues.apache.org/jira/browse/YARN-9898 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Assignee: liusheng >Priority: Major > Attachments: YARN-9898.001.patch, YARN-9898.002.patch, > YARN-9898.003.patch, YARN-9898.004.patch > > > Hadoop dependent the Netty package, but the *netty-all-4.1.27.Final* of > io.netty maven repo, cannot support ARM platform. > When run the test *TestCsiClient.testIdentityService* on ARM server, it will > raise error like following: > {code:java} > Caused by: java.io.FileNotFoundException: > META-INF/native/libnetty_transport_native_epoll_aarch_64.so > at > io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161) > ... 45 more > Suppressed: java.lang.UnsatisfiedLinkError: no > netty_transport_native_epoll_aarch_64 in java.library.path > at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) > at java.lang.Runtime.loadLibrary0(Runtime.java:870) > at java.lang.System.loadLibrary(System.java:1122) > at > io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) > at > io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243) > at > io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124) > ... 45 more > Suppressed: java.lang.UnsatisfiedLinkError: no > netty_transport_native_epoll_aarch_64 in java.library.path > at > java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) > at java.lang.Runtime.loadLibrary0(Runtime.java:870) > at java.lang.System.loadLibrary(System.java:1122) > at > io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) > at > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263) > at java.security.AccessController.doPrivileged(Native > Method) > at > io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255) > at > io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233) > ... 46 more > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9898) Dependency netty-all-4.1.27.Final doesn't support ARM platform
[ https://issues.apache.org/jira/browse/YARN-9898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105116#comment-17105116 ] liusheng commented on YARN-9898: I have no idea about the Jenkins results :(.. > Dependency netty-all-4.1.27.Final doesn't support ARM platform > -- > > Key: YARN-9898 > URL: https://issues.apache.org/jira/browse/YARN-9898 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Assignee: liusheng >Priority: Major > Attachments: YARN-9898.001.patch, YARN-9898.002.patch, > YARN-9898.003.patch > > > Hadoop dependent the Netty package, but the *netty-all-4.1.27.Final* of > io.netty maven repo, cannot support ARM platform. > When run the test *TestCsiClient.testIdentityService* on ARM server, it will > raise error like following: > {code:java} > Caused by: java.io.FileNotFoundException: > META-INF/native/libnetty_transport_native_epoll_aarch_64.so > at > io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161) > ... 45 more > Suppressed: java.lang.UnsatisfiedLinkError: no > netty_transport_native_epoll_aarch_64 in java.library.path > at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) > at java.lang.Runtime.loadLibrary0(Runtime.java:870) > at java.lang.System.loadLibrary(System.java:1122) > at > io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) > at > io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243) > at > io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124) > ... 45 more > Suppressed: java.lang.UnsatisfiedLinkError: no > netty_transport_native_epoll_aarch_64 in java.library.path > at > java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) > at java.lang.Runtime.loadLibrary0(Runtime.java:870) > at java.lang.System.loadLibrary(System.java:1122) > at > io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) > at > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263) > at java.security.AccessController.doPrivileged(Native > Method) > at > io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255) > at > io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233) > ... 46 more > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9898) Dependency netty-all-4.1.27.Final doesn't support ARM platform
[ https://issues.apache.org/jira/browse/YARN-9898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17104027#comment-17104027 ] liusheng commented on YARN-9898: I have added a saperated HDFS jira: https://issues.apache.org/jira/browse/HDFS-15098 > Dependency netty-all-4.1.27.Final doesn't support ARM platform > -- > > Key: YARN-9898 > URL: https://issues.apache.org/jira/browse/YARN-9898 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Assignee: liusheng >Priority: Major > Attachments: YARN-9898.001.patch, YARN-9898.002.patch, > YARN-9898.003.patch > > > Hadoop dependent the Netty package, but the *netty-all-4.1.27.Final* of > io.netty maven repo, cannot support ARM platform. > When run the test *TestCsiClient.testIdentityService* on ARM server, it will > raise error like following: > {code:java} > Caused by: java.io.FileNotFoundException: > META-INF/native/libnetty_transport_native_epoll_aarch_64.so > at > io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161) > ... 45 more > Suppressed: java.lang.UnsatisfiedLinkError: no > netty_transport_native_epoll_aarch_64 in java.library.path > at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) > at java.lang.Runtime.loadLibrary0(Runtime.java:870) > at java.lang.System.loadLibrary(System.java:1122) > at > io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) > at > io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243) > at > io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124) > ... 45 more > Suppressed: java.lang.UnsatisfiedLinkError: no > netty_transport_native_epoll_aarch_64 in java.library.path > at > java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) > at java.lang.Runtime.loadLibrary0(Runtime.java:870) > at java.lang.System.loadLibrary(System.java:1122) > at > io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) > at > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263) > at java.security.AccessController.doPrivileged(Native > Method) > at > io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255) > at > io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233) > ... 46 more > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9898) Dependency netty-all-4.1.27.Final doesn't support ARM platform
[ https://issues.apache.org/jira/browse/YARN-9898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17104023#comment-17104023 ] liusheng commented on YARN-9898: [~ayushtkn], thanks, sure, let me try. > Dependency netty-all-4.1.27.Final doesn't support ARM platform > -- > > Key: YARN-9898 > URL: https://issues.apache.org/jira/browse/YARN-9898 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Assignee: liusheng >Priority: Major > Attachments: YARN-9898.001.patch, YARN-9898.002.patch > > > Hadoop dependent the Netty package, but the *netty-all-4.1.27.Final* of > io.netty maven repo, cannot support ARM platform. > When run the test *TestCsiClient.testIdentityService* on ARM server, it will > raise error like following: > {code:java} > Caused by: java.io.FileNotFoundException: > META-INF/native/libnetty_transport_native_epoll_aarch_64.so > at > io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161) > ... 45 more > Suppressed: java.lang.UnsatisfiedLinkError: no > netty_transport_native_epoll_aarch_64 in java.library.path > at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) > at java.lang.Runtime.loadLibrary0(Runtime.java:870) > at java.lang.System.loadLibrary(System.java:1122) > at > io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) > at > io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243) > at > io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124) > ... 45 more > Suppressed: java.lang.UnsatisfiedLinkError: no > netty_transport_native_epoll_aarch_64 in java.library.path > at > java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) > at java.lang.Runtime.loadLibrary0(Runtime.java:870) > at java.lang.System.loadLibrary(System.java:1122) > at > io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) > at > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263) > at java.security.AccessController.doPrivileged(Native > Method) > at > io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255) > at > io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233) > ... 46 more > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9898) Dependency netty-all-4.1.27.Final doesn't support ARM platform
[ https://issues.apache.org/jira/browse/YARN-9898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17103056#comment-17103056 ] liusheng commented on YARN-9898: yes, thank you, I have also tested OK locally before uploading this patch, and this patch have no side affect on x86 functionalities. not sure about the CI results. > Dependency netty-all-4.1.27.Final doesn't support ARM platform > -- > > Key: YARN-9898 > URL: https://issues.apache.org/jira/browse/YARN-9898 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Assignee: liusheng >Priority: Major > Attachments: YARN-9898.001.patch, YARN-9898.002.patch > > > Hadoop dependent the Netty package, but the *netty-all-4.1.27.Final* of > io.netty maven repo, cannot support ARM platform. > When run the test *TestCsiClient.testIdentityService* on ARM server, it will > raise error like following: > {code:java} > Caused by: java.io.FileNotFoundException: > META-INF/native/libnetty_transport_native_epoll_aarch_64.so > at > io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161) > ... 45 more > Suppressed: java.lang.UnsatisfiedLinkError: no > netty_transport_native_epoll_aarch_64 in java.library.path > at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) > at java.lang.Runtime.loadLibrary0(Runtime.java:870) > at java.lang.System.loadLibrary(System.java:1122) > at > io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) > at > io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243) > at > io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124) > ... 45 more > Suppressed: java.lang.UnsatisfiedLinkError: no > netty_transport_native_epoll_aarch_64 in java.library.path > at > java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) > at java.lang.Runtime.loadLibrary0(Runtime.java:870) > at java.lang.System.loadLibrary(System.java:1122) > at > io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) > at > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263) > at java.security.AccessController.doPrivileged(Native > Method) > at > io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255) > at > io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233) > ... 46 more > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9898) Dependency netty-all-4.1.27.Final doesn't support ARM platform
[ https://issues.apache.org/jira/browse/YARN-9898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17102152#comment-17102152 ] liusheng commented on YARN-9898: I have no idea about the jenkins rerults means.. :( > Dependency netty-all-4.1.27.Final doesn't support ARM platform > -- > > Key: YARN-9898 > URL: https://issues.apache.org/jira/browse/YARN-9898 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Assignee: liusheng >Priority: Major > Attachments: YARN-9898.001.patch, YARN-9898.002.patch > > > Hadoop dependent the Netty package, but the *netty-all-4.1.27.Final* of > io.netty maven repo, cannot support ARM platform. > When run the test *TestCsiClient.testIdentityService* on ARM server, it will > raise error like following: > {code:java} > Caused by: java.io.FileNotFoundException: > META-INF/native/libnetty_transport_native_epoll_aarch_64.so > at > io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161) > ... 45 more > Suppressed: java.lang.UnsatisfiedLinkError: no > netty_transport_native_epoll_aarch_64 in java.library.path > at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) > at java.lang.Runtime.loadLibrary0(Runtime.java:870) > at java.lang.System.loadLibrary(System.java:1122) > at > io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) > at > io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243) > at > io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124) > ... 45 more > Suppressed: java.lang.UnsatisfiedLinkError: no > netty_transport_native_epoll_aarch_64 in java.library.path > at > java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) > at java.lang.Runtime.loadLibrary0(Runtime.java:870) > at java.lang.System.loadLibrary(System.java:1122) > at > io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) > at > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263) > at java.security.AccessController.doPrivileged(Native > Method) > at > io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255) > at > io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233) > ... 46 more > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9898) Dependency netty-all-4.1.27.Final doesn't support ARM platform
[ https://issues.apache.org/jira/browse/YARN-9898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated YARN-9898: --- Attachment: (was: YARN-9898.001.patch) > Dependency netty-all-4.1.27.Final doesn't support ARM platform > -- > > Key: YARN-9898 > URL: https://issues.apache.org/jira/browse/YARN-9898 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Assignee: liusheng >Priority: Major > Attachments: YARN-9898.001.patch > > > Hadoop dependent the Netty package, but the *netty-all-4.1.27.Final* of > io.netty maven repo, cannot support ARM platform. > When run the test *TestCsiClient.testIdentityService* on ARM server, it will > raise error like following: > {code:java} > Caused by: java.io.FileNotFoundException: > META-INF/native/libnetty_transport_native_epoll_aarch_64.so > at > io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161) > ... 45 more > Suppressed: java.lang.UnsatisfiedLinkError: no > netty_transport_native_epoll_aarch_64 in java.library.path > at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) > at java.lang.Runtime.loadLibrary0(Runtime.java:870) > at java.lang.System.loadLibrary(System.java:1122) > at > io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) > at > io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243) > at > io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124) > ... 45 more > Suppressed: java.lang.UnsatisfiedLinkError: no > netty_transport_native_epoll_aarch_64 in java.library.path > at > java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) > at java.lang.Runtime.loadLibrary0(Runtime.java:870) > at java.lang.System.loadLibrary(System.java:1122) > at > io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) > at > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263) > at java.security.AccessController.doPrivileged(Native > Method) > at > io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255) > at > io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233) > ... 46 more > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9898) Dependency netty-all-4.1.27.Final doesn't support ARM platform
[ https://issues.apache.org/jira/browse/YARN-9898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated YARN-9898: --- Attachment: (was: YARN-9898.001.patch) > Dependency netty-all-4.1.27.Final doesn't support ARM platform > -- > > Key: YARN-9898 > URL: https://issues.apache.org/jira/browse/YARN-9898 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Assignee: liusheng >Priority: Major > Attachments: YARN-9898.001.patch > > > Hadoop dependent the Netty package, but the *netty-all-4.1.27.Final* of > io.netty maven repo, cannot support ARM platform. > When run the test *TestCsiClient.testIdentityService* on ARM server, it will > raise error like following: > {code:java} > Caused by: java.io.FileNotFoundException: > META-INF/native/libnetty_transport_native_epoll_aarch_64.so > at > io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161) > ... 45 more > Suppressed: java.lang.UnsatisfiedLinkError: no > netty_transport_native_epoll_aarch_64 in java.library.path > at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) > at java.lang.Runtime.loadLibrary0(Runtime.java:870) > at java.lang.System.loadLibrary(System.java:1122) > at > io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) > at > io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243) > at > io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124) > ... 45 more > Suppressed: java.lang.UnsatisfiedLinkError: no > netty_transport_native_epoll_aarch_64 in java.library.path > at > java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) > at java.lang.Runtime.loadLibrary0(Runtime.java:870) > at java.lang.System.loadLibrary(System.java:1122) > at > io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) > at > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263) > at java.security.AccessController.doPrivileged(Native > Method) > at > io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255) > at > io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233) > ... 46 more > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9898) Dependency netty-all-4.1.27.Final doesn't support ARM platform
[ https://issues.apache.org/jira/browse/YARN-9898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated YARN-9898: --- Attachment: (was: YARN-9898.001.patch) > Dependency netty-all-4.1.27.Final doesn't support ARM platform > -- > > Key: YARN-9898 > URL: https://issues.apache.org/jira/browse/YARN-9898 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Assignee: liusheng >Priority: Major > > Hadoop dependent the Netty package, but the *netty-all-4.1.27.Final* of > io.netty maven repo, cannot support ARM platform. > When run the test *TestCsiClient.testIdentityService* on ARM server, it will > raise error like following: > {code:java} > Caused by: java.io.FileNotFoundException: > META-INF/native/libnetty_transport_native_epoll_aarch_64.so > at > io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161) > ... 45 more > Suppressed: java.lang.UnsatisfiedLinkError: no > netty_transport_native_epoll_aarch_64 in java.library.path > at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) > at java.lang.Runtime.loadLibrary0(Runtime.java:870) > at java.lang.System.loadLibrary(System.java:1122) > at > io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) > at > io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243) > at > io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124) > ... 45 more > Suppressed: java.lang.UnsatisfiedLinkError: no > netty_transport_native_epoll_aarch_64 in java.library.path > at > java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) > at java.lang.Runtime.loadLibrary0(Runtime.java:870) > at java.lang.System.loadLibrary(System.java:1122) > at > io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) > at > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263) > at java.security.AccessController.doPrivileged(Native > Method) > at > io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255) > at > io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233) > ... 46 more > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-9898) Dependency netty-all-4.1.27.Final doesn't support ARM platform
[ https://issues.apache.org/jira/browse/YARN-9898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng reassigned YARN-9898: -- Assignee: liusheng > Dependency netty-all-4.1.27.Final doesn't support ARM platform > -- > > Key: YARN-9898 > URL: https://issues.apache.org/jira/browse/YARN-9898 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Assignee: liusheng >Priority: Major > > Hadoop dependent the Netty package, but the *netty-all-4.1.27.Final* of > io.netty maven repo, cannot support ARM platform. > When run the test *TestCsiClient.testIdentityService* on ARM server, it will > raise error like following: > {code:java} > Caused by: java.io.FileNotFoundException: > META-INF/native/libnetty_transport_native_epoll_aarch_64.so > at > io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161) > ... 45 more > Suppressed: java.lang.UnsatisfiedLinkError: no > netty_transport_native_epoll_aarch_64 in java.library.path > at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) > at java.lang.Runtime.loadLibrary0(Runtime.java:870) > at java.lang.System.loadLibrary(System.java:1122) > at > io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) > at > io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243) > at > io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124) > ... 45 more > Suppressed: java.lang.UnsatisfiedLinkError: no > netty_transport_native_epoll_aarch_64 in java.library.path > at > java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) > at java.lang.Runtime.loadLibrary0(Runtime.java:870) > at java.lang.System.loadLibrary(System.java:1122) > at > io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) > at > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263) > at java.security.AccessController.doPrivileged(Native > Method) > at > io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255) > at > io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233) > ... 46 more > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9898) Dependency netty-all-4.1.27.Final doesn't support ARM platform
[ https://issues.apache.org/jira/browse/YARN-9898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17008754#comment-17008754 ] liusheng commented on YARN-9898: we have tried to push netty to support on ARM platform: https://github.com/netty/netty/pull/9804 > Dependency netty-all-4.1.27.Final doesn't support ARM platform > -- > > Key: YARN-9898 > URL: https://issues.apache.org/jira/browse/YARN-9898 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Priority: Major > > Hadoop dependent the Netty package, but the *netty-all-4.1.27.Final* of > io.netty maven repo, cannot support ARM platform. > When run the test *TestCsiClient.testIdentityService* on ARM server, it will > raise error like following: > {code:java} > Caused by: java.io.FileNotFoundException: > META-INF/native/libnetty_transport_native_epoll_aarch_64.so > at > io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161) > ... 45 more > Suppressed: java.lang.UnsatisfiedLinkError: no > netty_transport_native_epoll_aarch_64 in java.library.path > at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) > at java.lang.Runtime.loadLibrary0(Runtime.java:870) > at java.lang.System.loadLibrary(System.java:1122) > at > io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) > at > io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243) > at > io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124) > ... 45 more > Suppressed: java.lang.UnsatisfiedLinkError: no > netty_transport_native_epoll_aarch_64 in java.library.path > at > java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) > at java.lang.Runtime.loadLibrary0(Runtime.java:870) > at java.lang.System.loadLibrary(System.java:1122) > at > io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) > at > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263) > at java.security.AccessController.doPrivileged(Native > Method) > at > io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255) > at > io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233) > ... 46 more > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10042) Uupgrade grpc-xxx depdencies to 1.26.0
[ https://issues.apache.org/jira/browse/YARN-10042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17000521#comment-17000521 ] liusheng commented on YARN-10042: - Thanks a lot for your review [~tangzhankun], [~cheersyang] would you please take a look ? thank you > Uupgrade grpc-xxx depdencies to 1.26.0 > -- > > Key: YARN-10042 > URL: https://issues.apache.org/jira/browse/YARN-10042 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Priority: Major > Attachments: YARN-10042.001.patch, > hadoop_build_aarch64_grpc_1.26.0.log, hadoop_build_x86_64_grpc_1.26.0.log, > yarn_csi_tests_aarch64_grpc_1.26.0.log, yarn_csi_tests_x86_64_grpc_1.26.0.log > > > For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, > grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but > the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the > grpc-java repo has support aarch64 platform and release in 1.26.0 in maven > central. > see: > [https://github.com/grpc/grpc-java/pull/6496] > [https://search.maven.org/search?q=g:io.grpc] > It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 > version. both x86_64 and aarch64 server are building OK accroding to my > testing, please see the attachment, they are: log of building on aarch64, log > of building on x86_64, log of running tests of yarn csi on aarch64, log of > running tests of yarn csi on x86_64. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10042) Uupgrade grpc-xxx depdencies to 1.26.0
[ https://issues.apache.org/jira/browse/YARN-10042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated YARN-10042: Attachment: (was: YARN-10042.001.patch) > Uupgrade grpc-xxx depdencies to 1.26.0 > -- > > Key: YARN-10042 > URL: https://issues.apache.org/jira/browse/YARN-10042 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Priority: Major > Attachments: hadoop_build_aarch64_grpc_1.26.0.log, > hadoop_build_x86_64_grpc_1.26.0.log, yarn_csi_tests_aarch64_grpc_1.26.0.log, > yarn_csi_tests_x86_64_grpc_1.26.0.log > > > For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, > grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but > the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the > grpc-java repo has support aarch64 platform and release in 1.26.0 in maven > central. > see: > [https://github.com/grpc/grpc-java/pull/6496] > [https://search.maven.org/search?q=g:io.grpc] > It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 > version. both x86_64 and aarch64 server are building OK accroding to my > testing, please see the attachment, they are: log of building on aarch64, log > of building on x86_64, log of running tests of yarn csi on aarch64, log of > running tests of yarn csi on x86_64. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10042) Uupgrade grpc-xxx depdencies to 1.26.0
[ https://issues.apache.org/jira/browse/YARN-10042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated YARN-10042: Attachment: (was: YARN-10042.001.patch) > Uupgrade grpc-xxx depdencies to 1.26.0 > -- > > Key: YARN-10042 > URL: https://issues.apache.org/jira/browse/YARN-10042 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Priority: Major > Attachments: YARN-10042.001.patch, > hadoop_build_aarch64_grpc_1.26.0.log, hadoop_build_x86_64_grpc_1.26.0.log, > yarn_csi_tests_aarch64_grpc_1.26.0.log, yarn_csi_tests_x86_64_grpc_1.26.0.log > > > For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, > grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but > the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the > grpc-java repo has support aarch64 platform and release in 1.26.0 in maven > central. > see: > [https://github.com/grpc/grpc-java/pull/6496] > [https://search.maven.org/search?q=g:io.grpc] > It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 > version. both x86_64 and aarch64 server are building OK accroding to my > testing, please see the attachment, they are: log of building on aarch64, log > of building on x86_64, log of running tests of yarn csi on aarch64, log of > running tests of yarn csi on x86_64. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10042) Uupgrade grpc-xxx depdencies to 1.26.0
[ https://issues.apache.org/jira/browse/YARN-10042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated YARN-10042: Attachment: YARN-10042.001.patch > Uupgrade grpc-xxx depdencies to 1.26.0 > -- > > Key: YARN-10042 > URL: https://issues.apache.org/jira/browse/YARN-10042 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Priority: Major > Attachments: YARN-10042.001.patch, YARN-10042.001.patch, > hadoop_build_aarch64_grpc_1.26.0.log, hadoop_build_x86_64_grpc_1.26.0.log, > yarn_csi_tests_aarch64_grpc_1.26.0.log, yarn_csi_tests_x86_64_grpc_1.26.0.log > > > For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, > grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but > the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the > grpc-java repo has support aarch64 platform and release in 1.26.0 in maven > central. > see: > [https://github.com/grpc/grpc-java/pull/6496] > [https://search.maven.org/search?q=g:io.grpc] > It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 > version. both x86_64 and aarch64 server are building OK accroding to my > testing, please see the attachment, they are: log of building on aarch64, log > of building on x86_64, log of running tests of yarn csi on aarch64, log of > running tests of yarn csi on x86_64. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10042) Uupgrade grpc-xxx depdencies to 1.26.0
[ https://issues.apache.org/jira/browse/YARN-10042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated YARN-10042: Attachment: yarn_csi_tests_x86_64_grpc_1.26.0.log > Uupgrade grpc-xxx depdencies to 1.26.0 > -- > > Key: YARN-10042 > URL: https://issues.apache.org/jira/browse/YARN-10042 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Priority: Major > Attachments: hadoop_build_aarch64_grpc_1.26.0.log, > hadoop_build_x86_64_grpc_1.26.0.log, yarn_csi_tests_aarch64_grpc_1.26.0.log, > yarn_csi_tests_x86_64_grpc_1.26.0.log > > > For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, > grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but > the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the > grpc-java repo has support aarch64 platform and release in 1.26.0 in maven > central. > see: > [https://github.com/grpc/grpc-java/pull/6496] > [https://search.maven.org/search?q=g:io.grpc] > It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 > version. both x86_64 and aarch64 server are building OK accroding to my > testing, please see the attachment, they are: log of building on aarch64, log > of building on x86_64, log of running tests of yarn csi on aarch64, log of > running tests of yarn csi on x86_64. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10042) Uupgrade grpc-xxx depdencies to 1.26.0
[ https://issues.apache.org/jira/browse/YARN-10042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated YARN-10042: Attachment: (was: yarn_csi_tests_x86_64_grpc_1.26.0.log) > Uupgrade grpc-xxx depdencies to 1.26.0 > -- > > Key: YARN-10042 > URL: https://issues.apache.org/jira/browse/YARN-10042 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Priority: Major > Attachments: hadoop_build_aarch64_grpc_1.26.0.log, > hadoop_build_x86_64_grpc_1.26.0.log, yarn_csi_tests_aarch64_grpc_1.26.0.log > > > For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, > grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but > the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the > grpc-java repo has support aarch64 platform and release in 1.26.0 in maven > central. > see: > [https://github.com/grpc/grpc-java/pull/6496] > [https://search.maven.org/search?q=g:io.grpc] > It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 > version. both x86_64 and aarch64 server are building OK accroding to my > testing, please see the attachment, they are: log of building on aarch64, log > of building on x86_64, log of running tests of yarn csi on aarch64, log of > running tests of yarn csi on x86_64. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10042) Uupgrade grpc-xxx depdencies to 1.26.0
[ https://issues.apache.org/jira/browse/YARN-10042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated YARN-10042: Attachment: hadoop_build_x86_64_grpc_1.26.0.log > Uupgrade grpc-xxx depdencies to 1.26.0 > -- > > Key: YARN-10042 > URL: https://issues.apache.org/jira/browse/YARN-10042 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Priority: Major > Attachments: hadoop_build_aarch64_grpc_1.26.0.log, > hadoop_build_x86_64_grpc_1.26.0.log, yarn_csi_tests_aarch64_grpc_1.26.0.log, > yarn_csi_tests_x86_64_grpc_1.26.0.log > > > For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, > grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but > the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the > grpc-java repo has support aarch64 platform and release in 1.26.0 in maven > central. > see: > [https://github.com/grpc/grpc-java/pull/6496] > [https://search.maven.org/search?q=g:io.grpc] > It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 > version. both x86_64 and aarch64 server are building OK accroding to my > testing, please see the attachment, they are: log of building on aarch64, log > of building on x86_64, log of running tests of yarn csi on aarch64, log of > running tests of yarn csi on x86_64. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10042) Uupgrade grpc-xxx depdencies to 1.26.0
[ https://issues.apache.org/jira/browse/YARN-10042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated YARN-10042: Attachment: (was: hadoop_build_x86_64_grpc_1.26.0.log.log) > Uupgrade grpc-xxx depdencies to 1.26.0 > -- > > Key: YARN-10042 > URL: https://issues.apache.org/jira/browse/YARN-10042 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Priority: Major > Attachments: hadoop_build_aarch64_grpc_1.26.0.log, > yarn_csi_tests_aarch64_grpc_1.26.0.log, yarn_csi_tests_x86_64_grpc_1.26.0.log > > > For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, > grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but > the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the > grpc-java repo has support aarch64 platform and release in 1.26.0 in maven > central. > see: > [https://github.com/grpc/grpc-java/pull/6496] > [https://search.maven.org/search?q=g:io.grpc] > It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 > version. both x86_64 and aarch64 server are building OK accroding to my > testing, please see the attachment, they are: log of building on aarch64, log > of building on x86_64, log of running tests of yarn csi on aarch64, log of > running tests of yarn csi on x86_64. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10042) Uupgrade grpc-xxx depdencies to 1.26.0
[ https://issues.apache.org/jira/browse/YARN-10042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated YARN-10042: Description: For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the grpc-java repo has support aarch64 platform and release in 1.26.0 in maven central. see: [https://github.com/grpc/grpc-java/pull/6496] [https://search.maven.org/search?q=g:io.grpc] It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 version. both x86_64 and aarch64 server are building OK accroding to my testing, please see the attachment, they are: log of building on aarch64, log of building on x86_64, log of running tests of yarn csi on aarch64, log of running tests of yarn csi on x86_64. was: For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the grpc-java repo has support aarch64 platform and release in 1.26.0 in maven central. see: [https://github.com/grpc/grpc-java/pull/6496] [https://search.maven.org/search?q=g:io.grpc] It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 version. both x86_64 and aarch64 server are building OK accroding to my testing. > Uupgrade grpc-xxx depdencies to 1.26.0 > -- > > Key: YARN-10042 > URL: https://issues.apache.org/jira/browse/YARN-10042 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Priority: Major > Attachments: hadoop_build_aarch64_grpc_1.26.0.log, > hadoop_build_x86_64_grpc_1.26.0.log.log, > yarn_csi_tests_aarch64_grpc_1.26.0.log, yarn_csi_tests_x86_64_grpc_1.26.0.log > > > For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, > grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but > the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the > grpc-java repo has support aarch64 platform and release in 1.26.0 in maven > central. > see: > [https://github.com/grpc/grpc-java/pull/6496] > [https://search.maven.org/search?q=g:io.grpc] > It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 > version. both x86_64 and aarch64 server are building OK accroding to my > testing, please see the attachment, they are: log of building on aarch64, log > of building on x86_64, log of running tests of yarn csi on aarch64, log of > running tests of yarn csi on x86_64. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10042) Uupgrade grpc-xxx depdencies to 1.26.0
[ https://issues.apache.org/jira/browse/YARN-10042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated YARN-10042: Attachment: (was: yarn_csi_tests_aarch64_grpc_1.26.0.log) > Uupgrade grpc-xxx depdencies to 1.26.0 > -- > > Key: YARN-10042 > URL: https://issues.apache.org/jira/browse/YARN-10042 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Priority: Major > Attachments: hadoop_build_aarch64_grpc_1.26.0.log, > hadoop_build_x86_64_grpc_1.26.0.log.log, > yarn_csi_tests_aarch64_grpc_1.26.0.log, yarn_csi_tests_x86_64_grpc_1.26.0.log > > > For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, > grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but > the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the > grpc-java repo has support aarch64 platform and release in 1.26.0 in maven > central. > see: > [https://github.com/grpc/grpc-java/pull/6496] > [https://search.maven.org/search?q=g:io.grpc] > It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 > version. both x86_64 and aarch64 server are building OK accroding to my > testing. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10042) Uupgrade grpc-xxx depdencies to 1.26.0
[ https://issues.apache.org/jira/browse/YARN-10042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated YARN-10042: Attachment: yarn_csi_tests_aarch64_grpc_1.26.0.log > Uupgrade grpc-xxx depdencies to 1.26.0 > -- > > Key: YARN-10042 > URL: https://issues.apache.org/jira/browse/YARN-10042 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Priority: Major > Attachments: hadoop_build_aarch64_grpc_1.26.0.log, > hadoop_build_x86_64_grpc_1.26.0.log.log, > yarn_csi_tests_aarch64_grpc_1.26.0.log, yarn_csi_tests_x86_64_grpc_1.26.0.log > > > For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, > grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but > the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the > grpc-java repo has support aarch64 platform and release in 1.26.0 in maven > central. > see: > [https://github.com/grpc/grpc-java/pull/6496] > [https://search.maven.org/search?q=g:io.grpc] > It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 > version. both x86_64 and aarch64 server are building OK accroding to my > testing. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10042) Uupgrade grpc-xxx depdencies to 1.26.0
[ https://issues.apache.org/jira/browse/YARN-10042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated YARN-10042: Attachment: hadoop_build_aarch64_grpc_1.26.0.log > Uupgrade grpc-xxx depdencies to 1.26.0 > -- > > Key: YARN-10042 > URL: https://issues.apache.org/jira/browse/YARN-10042 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Priority: Major > Attachments: hadoop_build_aarch64_grpc_1.26.0.log, > hadoop_build_x86_64_grpc_1.26.0.log.log, > yarn_csi_tests_aarch64_grpc_1.26.0.log, yarn_csi_tests_x86_64_grpc_1.26.0.log > > > For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, > grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but > the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the > grpc-java repo has support aarch64 platform and release in 1.26.0 in maven > central. > see: > [https://github.com/grpc/grpc-java/pull/6496] > [https://search.maven.org/search?q=g:io.grpc] > It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 > version. both x86_64 and aarch64 server are building OK accroding to my > testing. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10042) Uupgrade grpc-xxx depdencies to 1.26.0
[ https://issues.apache.org/jira/browse/YARN-10042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated YARN-10042: Attachment: yarn_csi_tests_x86_64_grpc_1.26.0.log hadoop_build_x86_64_grpc_1.26.0.log.log yarn_csi_tests_aarch64_grpc_1.26.0.log hadoop_build_aarch64_grpc_1.26.0.log > Uupgrade grpc-xxx depdencies to 1.26.0 > -- > > Key: YARN-10042 > URL: https://issues.apache.org/jira/browse/YARN-10042 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Priority: Major > Attachments: hadoop_build_x86_64_grpc_1.26.0.log.log, > yarn_csi_tests_aarch64_grpc_1.26.0.log, yarn_csi_tests_x86_64_grpc_1.26.0.log > > > For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, > grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but > the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the > grpc-java repo has support aarch64 platform and release in 1.26.0 in maven > central. > see: > [https://github.com/grpc/grpc-java/pull/6496] > [https://search.maven.org/search?q=g:io.grpc] > It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 > version. both x86_64 and aarch64 server are building OK accroding to my > testing. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10042) Uupgrade grpc-xxx depdencies to 1.26.0
[ https://issues.apache.org/jira/browse/YARN-10042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated YARN-10042: Attachment: (was: hadoop_build_aarch64_grpc_1.26.0.log) > Uupgrade grpc-xxx depdencies to 1.26.0 > -- > > Key: YARN-10042 > URL: https://issues.apache.org/jira/browse/YARN-10042 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Priority: Major > Attachments: hadoop_build_x86_64_grpc_1.26.0.log.log, > yarn_csi_tests_aarch64_grpc_1.26.0.log, yarn_csi_tests_x86_64_grpc_1.26.0.log > > > For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, > grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but > the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the > grpc-java repo has support aarch64 platform and release in 1.26.0 in maven > central. > see: > [https://github.com/grpc/grpc-java/pull/6496] > [https://search.maven.org/search?q=g:io.grpc] > It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 > version. both x86_64 and aarch64 server are building OK accroding to my > testing. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10042) Uupgrade grpc-xxx depdencies to 1.26.0
[ https://issues.apache.org/jira/browse/YARN-10042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated YARN-10042: Description: For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the grpc-java repo has support aarch64 platform and release in 1.26.0 in maven central. see: [https://github.com/grpc/grpc-java/pull/6496] [https://search.maven.org/search?q=g:io.grpc] It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 version. both x86_64 and aarch64 server are building OK accroding to my testing. was: For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the grpc-java repo has support aarch64 platform and release in 1.26.0 in maven central. see: [https://github.com/grpc/grpc-java/pull/6496] [https://search.maven.org/search?q=g:io.grpc] It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 version. both x86_64 and aarch64 server are building OK accroding to my testing. > Uupgrade grpc-xxx depdencies to 1.26.0 > -- > > Key: YARN-10042 > URL: https://issues.apache.org/jira/browse/YARN-10042 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Priority: Major > > For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, > grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but > the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the > grpc-java repo has support aarch64 platform and release in 1.26.0 in maven > central. > see: > [https://github.com/grpc/grpc-java/pull/6496] > [https://search.maven.org/search?q=g:io.grpc] > It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 > version. both x86_64 and aarch64 server are building OK accroding to my > testing. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10042) Uupgrade grpc-xxx depdencies to 1.26.0
liusheng created YARN-10042: --- Summary: Uupgrade grpc-xxx depdencies to 1.26.0 Key: YARN-10042 URL: https://issues.apache.org/jira/browse/YARN-10042 Project: Hadoop YARN Issue Type: Bug Reporter: liusheng For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the grpc-java repo has support aarch64 platform and release in 1.26.0 in maven central. see: [https://github.com/grpc/grpc-java/pull/6496] [https://search.maven.org/search?q=g:io.grpc] It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 version. both x86_64 and aarch64 server are building OK accroding to my testing. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9876) TimelineService Hbase tests are failing
[ https://issues.apache.org/jira/browse/YARN-9876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16996906#comment-16996906 ] liusheng commented on YARN-9876: Hi, Anyone can help to care about this issue ? > TimelineService Hbase tests are failing > --- > > Key: YARN-9876 > URL: https://issues.apache.org/jira/browse/YARN-9876 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineservice >Affects Versions: 3.3.0 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > > TimelineService Hbase tests are failing > {code} > hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorageDomain > hadoop.yarn.server.timelineservice.storage.flow.TestHBaseStorageFlowRun > hadoop.yarn.server.timelineservice.storage.flow.TestHBaseStorageFlowActivity > hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorageApps > hadoop.yarn.server.timelineservice.storage.TestTimelineWriterHBaseDown > hadoop.yarn.server.timelineservice.storage.flow.TestHBaseStorageFlowRunCompaction > > > hadoop.yarn.server.timelineservice.reader.TestTimelineReaderWebServicesHBaseStorage > > hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorageSchema > hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorageEntities > hadoop.yarn.server.timelineservice.storage.TestTimelineReaderHBaseDown > {code} > Logs: > {code} > [ERROR] > org.apache.hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorageDomain > Time elapsed: 208.136 s <<< ERROR! > java.io.IOException: Shutting down > at > org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:236) > at > org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:98) > at > org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:1089) > at > org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1045) > at > org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:917) > at > org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:911) > at > org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:840) > at > org.apache.hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorageDomain.setupBeforeClass(TestHBaseTimelineStorageDomain.java:55) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) > at > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) > at org.junit.runners.ParentRunner.run(ParentRunner.java:363) > at > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365) > at > org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273) > at > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238) > at > org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159) > at > org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384) > at > org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345) > at > org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126) > at > org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418) > Caused by: java.lang.RuntimeException: Master not initialized after 20ms > seconds > at > org.apache.hadoop.hbase.util.JVMClusterUtil.startup(JVMClusterUtil.java:230) > at > org.apache.hadoop.hbase.LocalHBaseCluster.startup(LocalHBaseCluster.java:414) > at > org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:229) > ... 25 more > 2019-10-06 22:27:04,233 FATAL [192.168.1.180:52250.activeMasterManager] > master.HMaster (HMaster.java:run(2039)) - Failed to become active master > java.lang.NoClassDefFoundError: com/google/protobuf/LiteralByteString > at java.lang.ClassLoader.defineClass1(Native Method) > at java.lang.ClassLoader.defineCla
[jira] [Commented] (YARN-9511) [JDK11] TestAuxServices#testRemoteAuxServiceClassPath YarnRuntimeException: The remote jarfile should not be writable by group or others. The current Permission is 436
[ https://issues.apache.org/jira/browse/YARN-9511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16959353#comment-16959353 ] liusheng commented on YARN-9511: Hi [~adam.antal], I am using a Ubuntu 16.04 server with JDK 8(for building and testing), {code:java} zuul@ubuntu:~$ java -version openjdk version "1.8.0_222" OpenJDK Runtime Environment (build 1.8.0_222-8u222-b10-1ubuntu1~16.04.1-b10) OpenJDK 64-Bit Server VM (build 25.222-b10, mixed mode) {code} The *umask* is for determining the default permissions(read,write, executable) of newly created file/directory. it take effect on current login session if set by 'umask' command. > [JDK11] TestAuxServices#testRemoteAuxServiceClassPath YarnRuntimeException: > The remote jarfile should not be writable by group or others. The current > Permission is 436 > --- > > Key: YARN-9511 > URL: https://issues.apache.org/jira/browse/YARN-9511 > Project: Hadoop YARN > Issue Type: Bug > Components: test >Reporter: Siyao Meng >Assignee: Szilard Nemeth >Priority: Major > > Found in maven JDK 11 unit test run. Compiled on JDK 8. > {code} > [ERROR] > testRemoteAuxServiceClassPath(org.apache.hadoop.yarn.server.nodemanager.containermanager.TestAuxServices) > Time elapsed: 0.551 s <<< > ERROR!org.apache.hadoop.yarn.exceptions.YarnRuntimeException: The remote > jarfile should not be writable by group or others. The current Permission is > 436 > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:202) > at > org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.TestAuxServices.testRemoteAuxServiceClassPath(TestAuxServices.java:268) > at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.base/java.lang.reflect.Method.invoke(Method.java:566) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) > at org.junit.runners.ParentRunner.run(ParentRunner.java:309) > at > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365) > at > org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273) > at > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238) > at > org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159) > at > org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384) > at > org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345) > at > org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126) > at > org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-9897) Add an Aarch64 CI for YARN
[ https://issues.apache.org/jira/browse/YARN-9897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16957484#comment-16957484 ] liusheng edited comment on YARN-9897 at 10/23/19 1:34 AM: -- Hi [~eyang], I have tried your the two tests suggested in your comment, looks like both are success. {code:java} [INFO] C M A K E B U I L D E RT E S T [INFO] --- [INFO] cetest: running /home/zuul/src/github.com/liusheng/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/test/cetest --gtest_filter=-Perf. --gtest_output=xml:/home/zuul/src/github.com/liusheng/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/surefire-reports/TEST-cetest.xml [INFO] with extra environment variables {} [INFO] STATUS: SUCCESS after 154 millisecond(s). [INFO] --- [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 01:01 min [INFO] Finished at: 2019-10-23T01:29:41Z [INFO] {code} {code:java} [INFO] --- [INFO] C M A K E B U I L D E RT E S T [INFO] --- [INFO] test-container-executor: running /home/zuul/src/github.com/liusheng/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/target/usr/local/bin/test-container-executor [INFO] with extra environment variables {} [INFO] STATUS: SUCCESS after 5968 millisecond(s). [INFO] --- [INFO] [INFO] --- hadoop-maven-plugins:3.3.0-SNAPSHOT:cmake-test (cetest) @ hadoop-yarn-server-nodemanager --- [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 01:07 min [INFO] Finished at: 2019-10-23T01:32:28Z [INFO] {code} Does this look good to you ? was (Author: seanlau): Hi [~eyang], Looks like both these two tests are OK, see: {code:java} [INFO] C M A K E B U I L D E RT E S T [INFO] --- [INFO] cetest: running /home/zuul/src/github.com/liusheng/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/test/cetest --gtest_filter=-Perf. --gtest_output=xml:/home/zuul/src/github.com/liusheng/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/surefire-reports/TEST-cetest.xml [INFO] with extra environment variables {} [INFO] STATUS: SUCCESS after 154 millisecond(s). [INFO] --- [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 01:01 min [INFO] Finished at: 2019-10-23T01:29:41Z [INFO] {code} > Add an Aarch64 CI for YARN > -- > > Key: YARN-9897 > URL: https://issues.apache.org/jira/browse/YARN-9897 > Project: Hadoop YARN > Issue Type: Improvement > Components: build, test >Reporter: Zhenyu Zheng >Priority: Major > Attachments: hadoop_build.log > > > As YARN is the resource manager of Hadoop and there are large number of other > software that also uses YARN for resource management. The capability of > running YARN on platforms with different architecture and managing hardware > resources with different architecture could be very important and useful. > Aarch64(ARM) architecture is currently the dominate architecture in small > devices like phone, IOT devices, security cameras, drones etc. With the > increasing compuiting capability and the increasing connection speed like 5G > network, there could be greate posibility and opportunity for world chaging > inovations and new market if we can managing and make use of those devices as > well. > Currently, all YARN CIs are based on x86 architecture and we have been > performing tests on Aarch64 and proposing possible solutions for problems we > have meet, like: > https://issues.apache.org/jira/browse/HADOOP-16614 > we have done all YARN tests and it turns out there are only a few problems, > and we can provide possible solutions for discussion. > We want to propose to add an
[jira] [Commented] (YARN-9897) Add an Aarch64 CI for YARN
[ https://issues.apache.org/jira/browse/YARN-9897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16957484#comment-16957484 ] liusheng commented on YARN-9897: Hi [~eyang], Looks like both these two tests are OK, see: {code:java} [INFO] C M A K E B U I L D E RT E S T [INFO] --- [INFO] cetest: running /home/zuul/src/github.com/liusheng/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/test/cetest --gtest_filter=-Perf. --gtest_output=xml:/home/zuul/src/github.com/liusheng/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/surefire-reports/TEST-cetest.xml [INFO] with extra environment variables {} [INFO] STATUS: SUCCESS after 154 millisecond(s). [INFO] --- [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 01:01 min [INFO] Finished at: 2019-10-23T01:29:41Z [INFO] {code} > Add an Aarch64 CI for YARN > -- > > Key: YARN-9897 > URL: https://issues.apache.org/jira/browse/YARN-9897 > Project: Hadoop YARN > Issue Type: Improvement > Components: build, test >Reporter: Zhenyu Zheng >Priority: Major > Attachments: hadoop_build.log > > > As YARN is the resource manager of Hadoop and there are large number of other > software that also uses YARN for resource management. The capability of > running YARN on platforms with different architecture and managing hardware > resources with different architecture could be very important and useful. > Aarch64(ARM) architecture is currently the dominate architecture in small > devices like phone, IOT devices, security cameras, drones etc. With the > increasing compuiting capability and the increasing connection speed like 5G > network, there could be greate posibility and opportunity for world chaging > inovations and new market if we can managing and make use of those devices as > well. > Currently, all YARN CIs are based on x86 architecture and we have been > performing tests on Aarch64 and proposing possible solutions for problems we > have meet, like: > https://issues.apache.org/jira/browse/HADOOP-16614 > we have done all YARN tests and it turns out there are only a few problems, > and we can provide possible solutions for discussion. > We want to propose to add an Aarch64 CI for YARN to promote the support for > YARN on Aarch64 platforms. We are willing to provide machines to the current > CI system and manpower to mananging the CI and fxing problems that occours. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9897) Add an Aarch64 CI for YARN
[ https://issues.apache.org/jira/browse/YARN-9897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16956640#comment-16956640 ] liusheng commented on YARN-9897: Hi [~eyang], Yes, we have built Hadoop with *-Pnative* flag, without the container-executor.additional_cflags, seems everything is OK, here is my building command: {code:java} mvn clean install -e -B -Pdist,native -Dtar -DskipTests -Dmaven.javadoc.skip {code} and attach my building log > Add an Aarch64 CI for YARN > -- > > Key: YARN-9897 > URL: https://issues.apache.org/jira/browse/YARN-9897 > Project: Hadoop YARN > Issue Type: Improvement > Components: build, test >Reporter: Zhenyu Zheng >Priority: Major > > As YARN is the resource manager of Hadoop and there are large number of other > software that also uses YARN for resource management. The capability of > running YARN on platforms with different architecture and managing hardware > resources with different architecture could be very important and useful. > Aarch64(ARM) architecture is currently the dominate architecture in small > devices like phone, IOT devices, security cameras, drones etc. With the > increasing compuiting capability and the increasing connection speed like 5G > network, there could be greate posibility and opportunity for world chaging > inovations and new market if we can managing and make use of those devices as > well. > Currently, all YARN CIs are based on x86 architecture and we have been > performing tests on Aarch64 and proposing possible solutions for problems we > have meet, like: > https://issues.apache.org/jira/browse/HADOOP-16614 > we have done all YARN tests and it turns out there are only a few problems, > and we can provide possible solutions for discussion. > We want to propose to add an Aarch64 CI for YARN to promote the support for > YARN on Aarch64 platforms. We are willing to provide machines to the current > CI system and manpower to mananging the CI and fxing problems that occours. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9511) [JDK11] TestAuxServices#testRemoteAuxServiceClassPath YarnRuntimeException: The remote jarfile should not be writable by group or others. The current Permission is 436
[ https://issues.apache.org/jira/browse/YARN-9511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16956634#comment-16956634 ] liusheng commented on YARN-9511: Hi [~adam.antal], Here are my steps: 1. Confirm current *umask* value of system, mine is *0002* {code:java} $ umask 0002{code} 2. Run unit tests of *TestAuxServices*, and will reproduce the problems described in this issue. {code:java} $ cd hadoop-yarn-project/ $ mvn test -Dtest=TestAuxServices{code} 3. modify the permissions of hadoop source code directory to remove other and group wirte permission and the system's umask to *022* {code:java} $ chmod go-w hadoop/ -R $ umask 022 $ umask 0022{code} 4. Re-run the unitests of *TestAuxServices* again, all the unit tests can pass {code:java} $ cd hadoop-yarn-project/ $ mvn test -Dtest=TestAuxServices{code} > [JDK11] TestAuxServices#testRemoteAuxServiceClassPath YarnRuntimeException: > The remote jarfile should not be writable by group or others. The current > Permission is 436 > --- > > Key: YARN-9511 > URL: https://issues.apache.org/jira/browse/YARN-9511 > Project: Hadoop YARN > Issue Type: Bug > Components: test >Reporter: Siyao Meng >Assignee: Szilard Nemeth >Priority: Major > > Found in maven JDK 11 unit test run. Compiled on JDK 8. > {code} > [ERROR] > testRemoteAuxServiceClassPath(org.apache.hadoop.yarn.server.nodemanager.containermanager.TestAuxServices) > Time elapsed: 0.551 s <<< > ERROR!org.apache.hadoop.yarn.exceptions.YarnRuntimeException: The remote > jarfile should not be writable by group or others. The current Permission is > 436 > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:202) > at > org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.TestAuxServices.testRemoteAuxServiceClassPath(TestAuxServices.java:268) > at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.base/java.lang.reflect.Method.invoke(Method.java:566) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) > at org.junit.runners.ParentRunner.run(ParentRunner.java:309) > at > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365) > at > org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273) > at > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238) > at > org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159) > at > org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384) > at > org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345) > at > org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126) > at > org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9898) Dependency netty-all-4.1.27.Final doesn't support ARM platform
[ https://issues.apache.org/jira/browse/YARN-9898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated YARN-9898: --- Description: Hadoop dependent the Netty package, but the *netty-all-4.1.27.Final* of io.netty maven repo, cannot support ARM platform. When run the test *TestCsiClient.testIdentityService* on ARM server, it will raise error like following: {code:java} Caused by: java.io.FileNotFoundException: META-INF/native/libnetty_transport_native_epoll_aarch_64.so at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161) ... 45 more Suppressed: java.lang.UnsatisfiedLinkError: no netty_transport_native_epoll_aarch_64 in java.library.path at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) at java.lang.Runtime.loadLibrary0(Runtime.java:870) at java.lang.System.loadLibrary(System.java:1122) at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243) at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124) ... 45 more Suppressed: java.lang.UnsatisfiedLinkError: no netty_transport_native_epoll_aarch_64 in java.library.path at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) at java.lang.Runtime.loadLibrary0(Runtime.java:870) at java.lang.System.loadLibrary(System.java:1122) at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263) at java.security.AccessController.doPrivileged(Native Method) at io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255) at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233) ... 46 more {code} was: Hadoop dependent the Netty package, but the *netty-all-4.1.27.Final* of io.netty maven repo, cannot support ARM platform. When run the test *TestCsiClient.testIdentityService*, it will raise error like following: {code:java} Caused by: java.io.FileNotFoundException: META-INF/native/libnetty_transport_native_epoll_aarch_64.so at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161) ... 45 more Suppressed: java.lang.UnsatisfiedLinkError: no netty_transport_native_epoll_aarch_64 in java.library.path at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) at java.lang.Runtime.loadLibrary0(Runtime.java:870) at java.lang.System.loadLibrary(System.java:1122) at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243) at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124) ... 45 more Suppressed: java.lang.UnsatisfiedLinkError: no netty_transport_native_epoll_aarch_64 in java.library.path at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) at java.lang.Runtime.loadLibrary0(Runtime.java:870) at java.lang.System.loadLibrary(System.java:1122) at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263) at java.security.AccessController.doPrivileged(Native Method) at io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255) at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLi
[jira] [Updated] (YARN-9898) Dependency netty-all-4.1.27.Final doesn't support ARM platform
[ https://issues.apache.org/jira/browse/YARN-9898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated YARN-9898: --- Description: Hadoop dependent the Netty package, but the *netty-all-4.1.27.Final* of io.netty maven repo, cannot support ARM platform. When run the test *TestCsiClient.testIdentityService*, it will raise error like following: {code:java} Caused by: java.io.FileNotFoundException: META-INF/native/libnetty_transport_native_epoll_aarch_64.so at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161) ... 45 more Suppressed: java.lang.UnsatisfiedLinkError: no netty_transport_native_epoll_aarch_64 in java.library.path at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) at java.lang.Runtime.loadLibrary0(Runtime.java:870) at java.lang.System.loadLibrary(System.java:1122) at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243) at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124) ... 45 more Suppressed: java.lang.UnsatisfiedLinkError: no netty_transport_native_epoll_aarch_64 in java.library.path at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) at java.lang.Runtime.loadLibrary0(Runtime.java:870) at java.lang.System.loadLibrary(System.java:1122) at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263) at java.security.AccessController.doPrivileged(Native Method) at io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255) at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233) ... 46 more {code} was: {code:java} Caused by: java.io.FileNotFoundException: META-INF/native/libnetty_transport_native_epoll_aarch_64.so at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161) ... 45 more Suppressed: java.lang.UnsatisfiedLinkError: no netty_transport_native_epoll_aarch_64 in java.library.path at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) at java.lang.Runtime.loadLibrary0(Runtime.java:870) at java.lang.System.loadLibrary(System.java:1122) at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243) at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124) ... 45 more Suppressed: java.lang.UnsatisfiedLinkError: no netty_transport_native_epoll_aarch_64 in java.library.path at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) at java.lang.Runtime.loadLibrary0(Runtime.java:870) at java.lang.System.loadLibrary(System.java:1122) at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263) at java.security.AccessController.doPrivileged(Native Method) at io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255) at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233) ... 46 more {code} Summary: Dependency netty-all-4.1.27.Final doesn't support ARM platform (was: Dependency netty-all-4.1.27.Final doesn) > Dependency netty-all
[jira] [Created] (YARN-9898) Dependency netty-all-4.1.27.Final doesn
liusheng created YARN-9898: -- Summary: Dependency netty-all-4.1.27.Final doesn Key: YARN-9898 URL: https://issues.apache.org/jira/browse/YARN-9898 Project: Hadoop YARN Issue Type: Bug Reporter: liusheng {code:java} Caused by: java.io.FileNotFoundException: META-INF/native/libnetty_transport_native_epoll_aarch_64.so at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161) ... 45 more Suppressed: java.lang.UnsatisfiedLinkError: no netty_transport_native_epoll_aarch_64 in java.library.path at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) at java.lang.Runtime.loadLibrary0(Runtime.java:870) at java.lang.System.loadLibrary(System.java:1122) at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243) at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124) ... 45 more Suppressed: java.lang.UnsatisfiedLinkError: no netty_transport_native_epoll_aarch_64 in java.library.path at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) at java.lang.Runtime.loadLibrary0(Runtime.java:870) at java.lang.System.loadLibrary(System.java:1122) at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263) at java.security.AccessController.doPrivileged(Native Method) at io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255) at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233) ... 46 more {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-9511) [JDK11] TestAuxServices#testRemoteAuxServiceClassPath YarnRuntimeException: The remote jarfile should not be writable by group or others. The current Permission is
[ https://issues.apache.org/jira/browse/YARN-9511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16949890#comment-16949890 ] liusheng edited comment on YARN-9511 at 10/12/19 3:51 AM: -- Hi [~snemeth] [~adam.antal], Thank you both for care about this issue. I have take some time tried to find the reason of this issue. This issue will effect the tests of *TestAuxServices*, and will cause *2 Errors 9 Failures*, see: {code:java} Failures: TestAuxServices.testAuxServiceRecoverySetup:717 expected:<2> but was:<0> TestAuxServices.testAuxServicesManifestPermissions:874 expected:<2> but was:<0> TestAuxServices.testAuxServicesMeta:638 Invalid mix of services expected:<6> but was:<1> TestAuxServices.testAuxServices:610 Invalid mix of services expected:<6> but was:<1> TestAuxServices.testCustomizedAuxServiceClassPath:416 TestAuxServices.testManualReload:919 expected:<2> but was:<0> TestAuxServices.testRemoteAuxServiceClassPath:313 The permission of the jar is wrong.Should throw out exception. TestAuxServices.testRemoveManifest:897 expected:<2> but was:<0> TestAuxServices.testValidAuxServiceName:698 Should receive the exception. Errors: TestAuxServices.testAuxUnexpectedStop:664 » NoSuchElement TestAuxServices.testRemoteAuxServiceClassPath:334 » YarnRuntime The remote jar... {code} After debuging, I found all these issues are directly or indirectly related with files permissions. there are two situations when running these tests: # *useManifest* enabled, when running tests with useManifest enabled, the tests will check and use the manifest file: {code:java} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/test-dir/TestAuxServices/manifest.txt{code} this file and all its parents directories must not be writable by group or others, see: {code:java} private boolean checkManifestPermissions(FileStatus status) throws IOException { if ((status.getPermission().toShort() & 0022) != 0) { LOG.error("Manifest file and parents must not be writable by group or " + "others. The current Permission of " + status.getPath() + " is " + status.getPermission()); return false; } Path parent = status.getPath().getParent(); if (parent == null) { return true; } return checkManifestPermissions(manifestFS.getFileStatus(parent)); }{code} # *useManifest not* enabled, when running tests with useManifest enabled, tests will use a *test-runjar.jar* file {code:java} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/test-dir/TestAuxServices/test-runjar.jar {code} related code for checking its permission: {code:java} private Path maybeDownloadJars(String sName, String className, String remoteFile, AuxServiceFile.TypeEnum type, Configuration conf) throws IOException { // load AuxiliaryService from remote classpath FileContext localLFS = getLocalFileContext(conf); // create NM aux-service dir in NM localdir if it does not exist. Path nmAuxDir = dirsHandler.getLocalPathForWrite("." + Path.SEPARATOR + NM_AUX_SERVICE_DIR); if (!localLFS.util().exists(nmAuxDir)) { try { localLFS.mkdir(nmAuxDir, NM_AUX_SERVICE_DIR_PERM, true); } catch (IOException ex) { throw new YarnRuntimeException("Fail to create dir:" + nmAuxDir.toString(), ex); } } Path src = new Path(remoteFile); FileContext remoteLFS = getRemoteFileContext(src.toUri(), conf); FileStatus scFileStatus = remoteLFS.getFileStatus(src); if (!scFileStatus.getOwner().equals( this.userUGI.getShortUserName())) { throw new YarnRuntimeException("The remote jarfile owner:" + scFileStatus.getOwner() + " is not the same as the NM user:" + this.userUGI.getShortUserName() + "."); } if ((scFileStatus.getPermission().toShort() & 0022) != 0) { throw new YarnRuntimeException("The remote jarfile should not " + "be writable by group or others. " + "The current Permission is " + scFileStatus.getPermission().toShort()); } {code} According to the above reasons, I have tried to change *manifest.txt* file parents directories without *writeable* permission of group and others. and change the *umask to 022*, which will effect new created file and directories permissions, because the *manifest.txt* and *run-tests.jar* will be new created when running tests. {code:java} chmod go-w yourpath/hadoop/ -R umask 022 umask {code} After doing above and re-run tests of *TestAuxServices*, all the tests can pass. Actually, I am a new comer to Hadoop, so I am not sure whether this is a bug of Hadoop or not. could you please give some suggestions? Thanks. was (Author: seanlau): Hi [~snemeth] [~adam.antal], Thank you both for care about this issue. I have take some time tried to find the reason of this issue. This issue will effect th
[jira] [Comment Edited] (YARN-9511) [JDK11] TestAuxServices#testRemoteAuxServiceClassPath YarnRuntimeException: The remote jarfile should not be writable by group or others. The current Permission is
[ https://issues.apache.org/jira/browse/YARN-9511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16949890#comment-16949890 ] liusheng edited comment on YARN-9511 at 10/12/19 3:51 AM: -- Hi [~snemeth] [~adam.antal], Thank you both for care about this issue. I have take some time tried to find the reason of this issue. This issue will effect the tests of *TestAuxServices*, and will cause *2 Errors 9 Failures*, see: {code:java} Failures: TestAuxServices.testAuxServiceRecoverySetup:717 expected:<2> but was:<0> TestAuxServices.testAuxServicesManifestPermissions:874 expected:<2> but was:<0> TestAuxServices.testAuxServicesMeta:638 Invalid mix of services expected:<6> but was:<1> TestAuxServices.testAuxServices:610 Invalid mix of services expected:<6> but was:<1> TestAuxServices.testCustomizedAuxServiceClassPath:416 TestAuxServices.testManualReload:919 expected:<2> but was:<0> TestAuxServices.testRemoteAuxServiceClassPath:313 The permission of the jar is wrong.Should throw out exception. TestAuxServices.testRemoveManifest:897 expected:<2> but was:<0> TestAuxServices.testValidAuxServiceName:698 Should receive the exception. Errors: TestAuxServices.testAuxUnexpectedStop:664 » NoSuchElement TestAuxServices.testRemoteAuxServiceClassPath:334 » YarnRuntime The remote jar... {code} After debuging, I found all these issues are directly or indirectly related with files permissions. there are two situations when running these tests: # *useManifest* enabled, when running tests with useManifest enabled, the tests will check and use the manifest file: {code:java} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/test-dir/TestAuxServices/manifest.txt{code} this file and all its parents directories must not be writable by group or others, see: {code:java} private boolean checkManifestPermissions(FileStatus status) throws IOException { if ((status.getPermission().toShort() & 0022) != 0) { LOG.error("Manifest file and parents must not be writable by group or " + "others. The current Permission of " + status.getPath() + " is " + status.getPermission()); return false; } Path parent = status.getPath().getParent(); if (parent == null) { return true; } return checkManifestPermissions(manifestFS.getFileStatus(parent)); }{code} # *useManifest not* enabled, when running tests with useManifest enabled, tests will use a *test-runjar.jar* file {code:java} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/test-dir/TestAuxServices/test-runjar.jar {code} related code for checking its permission: {code:java} private Path maybeDownloadJars(String sName, String className, String remoteFile, AuxServiceFile.TypeEnum type, Configuration conf) throws IOException { // load AuxiliaryService from remote classpath FileContext localLFS = getLocalFileContext(conf); // create NM aux-service dir in NM localdir if it does not exist. Path nmAuxDir = dirsHandler.getLocalPathForWrite("." + Path.SEPARATOR + NM_AUX_SERVICE_DIR); if (!localLFS.util().exists(nmAuxDir)) { try { localLFS.mkdir(nmAuxDir, NM_AUX_SERVICE_DIR_PERM, true); } catch (IOException ex) { throw new YarnRuntimeException("Fail to create dir:" + nmAuxDir.toString(), ex); } } Path src = new Path(remoteFile); FileContext remoteLFS = getRemoteFileContext(src.toUri(), conf); FileStatus scFileStatus = remoteLFS.getFileStatus(src); if (!scFileStatus.getOwner().equals( this.userUGI.getShortUserName())) { throw new YarnRuntimeException("The remote jarfile owner:" + scFileStatus.getOwner() + " is not the same as the NM user:" + this.userUGI.getShortUserName() + "."); } if ((scFileStatus.getPermission().toShort() & 0022) != 0) { throw new YarnRuntimeException("The remote jarfile should not " + "be writable by group or others. " + "The current Permission is " + scFileStatus.getPermission().toShort()); } {code} According to the above reasons, I have tried to change *manifest.txt* file parents directories without *writeable* permission of group and others. and change the *umask to 022*, which will effect new created file and directories permissions, because the *manifest.txt* and *run-tests.jar* will be new created when running tests. {code:java} chmod go-w yourpath/hadoop/ -R umask 022 umask {code} After doing above and re-run tests of *TestAuxServices*, all the tests can pass. Actually, I am a new comer to Hadoop, so I am not sure whether this is a bug of Hadoop or not. could you please give some suggestions? Thanks. was (Author: seanlau): Hi [~snemeth] [~adam.antal], Thank you both for care about this issue. I have take some time tried to find the reason of this issue. This issue will effect th
[jira] [Comment Edited] (YARN-9511) [JDK11] TestAuxServices#testRemoteAuxServiceClassPath YarnRuntimeException: The remote jarfile should not be writable by group or others. The current Permission is
[ https://issues.apache.org/jira/browse/YARN-9511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16949890#comment-16949890 ] liusheng edited comment on YARN-9511 at 10/12/19 3:50 AM: -- Hi [~snemeth] [~adam.antal], Thank you both for care about this issue. I have take some time tried to find the reason of this issue. This issue will effect the tests of *TestAuxServices*, and will cause *2 Errors 9 Failures*, see: {code:java} Failures: TestAuxServices.testAuxServiceRecoverySetup:717 expected:<2> but was:<0> TestAuxServices.testAuxServicesManifestPermissions:874 expected:<2> but was:<0> TestAuxServices.testAuxServicesMeta:638 Invalid mix of services expected:<6> but was:<1> TestAuxServices.testAuxServices:610 Invalid mix of services expected:<6> but was:<1> TestAuxServices.testCustomizedAuxServiceClassPath:416 TestAuxServices.testManualReload:919 expected:<2> but was:<0> TestAuxServices.testRemoteAuxServiceClassPath:313 The permission of the jar is wrong.Should throw out exception. TestAuxServices.testRemoveManifest:897 expected:<2> but was:<0> TestAuxServices.testValidAuxServiceName:698 Should receive the exception. Errors: TestAuxServices.testAuxUnexpectedStop:664 » NoSuchElement TestAuxServices.testRemoteAuxServiceClassPath:334 » YarnRuntime The remote jar... {code} After debuging, I found all these issues are directly or indirectly related with files permissions. there are two situations when running these tests: # *useManifest* enabled, when running tests with useManifest enabled, the tests will check and use the manifest file: {code:java} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/test-dir/TestAuxServices/manifest.txt{code} this file and all its parents directories must not be writable by group or others, see: {code:java} private boolean checkManifestPermissions(FileStatus status) throws IOException { if ((status.getPermission().toShort() & 0022) != 0) { LOG.error("Manifest file and parents must not be writable by group or " + "others. The current Permission of " + status.getPath() + " is " + status.getPermission()); return false; } Path parent = status.getPath().getParent(); if (parent == null) { return true; } return checkManifestPermissions(manifestFS.getFileStatus(parent)); }{code} ** # *useManifest not* enabled, when running tests with useManifest enabled, tests will use a *test-runjar.jar* file {code:java} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/test-dir/TestAuxServices/test-runjar.jar {code} related code for checking its permission: {code:java} private Path maybeDownloadJars(String sName, String className, String remoteFile, AuxServiceFile.TypeEnum type, Configuration conf) throws IOException { // load AuxiliaryService from remote classpath FileContext localLFS = getLocalFileContext(conf); // create NM aux-service dir in NM localdir if it does not exist. Path nmAuxDir = dirsHandler.getLocalPathForWrite("." + Path.SEPARATOR + NM_AUX_SERVICE_DIR); if (!localLFS.util().exists(nmAuxDir)) { try { localLFS.mkdir(nmAuxDir, NM_AUX_SERVICE_DIR_PERM, true); } catch (IOException ex) { throw new YarnRuntimeException("Fail to create dir:" + nmAuxDir.toString(), ex); } } Path src = new Path(remoteFile); FileContext remoteLFS = getRemoteFileContext(src.toUri(), conf); FileStatus scFileStatus = remoteLFS.getFileStatus(src); if (!scFileStatus.getOwner().equals( this.userUGI.getShortUserName())) { throw new YarnRuntimeException("The remote jarfile owner:" + scFileStatus.getOwner() + " is not the same as the NM user:" + this.userUGI.getShortUserName() + "."); } if ((scFileStatus.getPermission().toShort() & 0022) != 0) { throw new YarnRuntimeException("The remote jarfile should not " + "be writable by group or others. " + "The current Permission is " + scFileStatus.getPermission().toShort()); } {code} According to the above reasons, I have tried to change *manifest.txt* file parents directories without *writeable* permission of group and others. and change the *umask to 022*, which will effect new created file and directories permissions, because the *manifest.txt* and *run-tests.jar* will be new created when running tests. {code:java} chmod go-w yourpath/hadoop/ -R umask 022 umask {code} After doing above and re-run tests of *TestAuxServices*, all the tests can pass. Actually, I am a new comer to Hadoop, so I am not sure whether this is a bug of Hadoop or not. could you please give some suggestions? Thanks. was (Author: seanlau): Hi [~snemeth] [~adam.antal], Thank you both for care about this issue. I have take some time tried to find the reason of this issue. This issue will effect t
[jira] [Comment Edited] (YARN-9511) [JDK11] TestAuxServices#testRemoteAuxServiceClassPath YarnRuntimeException: The remote jarfile should not be writable by group or others. The current Permission is
[ https://issues.apache.org/jira/browse/YARN-9511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16949890#comment-16949890 ] liusheng edited comment on YARN-9511 at 10/12/19 3:49 AM: -- Hi [~snemeth] [~adam.antal], Thank you both for care about this issue. I have take some time tried to find the reason of this issue. This issue will effect the tests of *TestAuxServices*, and will cause *2 Errors 9 Failures*, see: {code:java} Failures: TestAuxServices.testAuxServiceRecoverySetup:717 expected:<2> but was:<0> TestAuxServices.testAuxServicesManifestPermissions:874 expected:<2> but was:<0> TestAuxServices.testAuxServicesMeta:638 Invalid mix of services expected:<6> but was:<1> TestAuxServices.testAuxServices:610 Invalid mix of services expected:<6> but was:<1> TestAuxServices.testCustomizedAuxServiceClassPath:416 TestAuxServices.testManualReload:919 expected:<2> but was:<0> TestAuxServices.testRemoteAuxServiceClassPath:313 The permission of the jar is wrong.Should throw out exception. TestAuxServices.testRemoveManifest:897 expected:<2> but was:<0> TestAuxServices.testValidAuxServiceName:698 Should receive the exception. Errors: TestAuxServices.testAuxUnexpectedStop:664 » NoSuchElement TestAuxServices.testRemoteAuxServiceClassPath:334 » YarnRuntime The remote jar... {code} After debuging, I found all these issues are directly or indirectly related with files permissions. there are two situations when running these tests: # *useManifest* enabled, when running tests with useManifest enabled, the tests will check and use the manifest file: {code:java} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/test-dir/TestAuxServices/manifest.txt{code} this file and all its parents directories must not be writable by group or others, see: {code:java} private boolean checkManifestPermissions(FileStatus status) throws IOException { if ((status.getPermission().toShort() & 0022) != 0) { LOG.error("Manifest file and parents must not be writable by group or " + "others. The current Permission of " + status.getPath() + " is " + status.getPermission()); return false; } Path parent = status.getPath().getParent(); if (parent == null) { return true; } return checkManifestPermissions(manifestFS.getFileStatus(parent)); }{code} ** # *useManifest not* enabled, when running tests with useManifest enabled, tests will use a *test-runjar.jar* file {code:java} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/test-dir/TestAuxServices/test-runjar.jar {code} related code for checking its permission: {code:java} private Path maybeDownloadJars(String sName, String className, String remoteFile, AuxServiceFile.TypeEnum type, Configuration conf) throws IOException { // load AuxiliaryService from remote classpath FileContext localLFS = getLocalFileContext(conf); // create NM aux-service dir in NM localdir if it does not exist. Path nmAuxDir = dirsHandler.getLocalPathForWrite("." + Path.SEPARATOR + NM_AUX_SERVICE_DIR); if (!localLFS.util().exists(nmAuxDir)) { try { localLFS.mkdir(nmAuxDir, NM_AUX_SERVICE_DIR_PERM, true); } catch (IOException ex) { throw new YarnRuntimeException("Fail to create dir:" + nmAuxDir.toString(), ex); } } Path src = new Path(remoteFile); FileContext remoteLFS = getRemoteFileContext(src.toUri(), conf); FileStatus scFileStatus = remoteLFS.getFileStatus(src); if (!scFileStatus.getOwner().equals( this.userUGI.getShortUserName())) { throw new YarnRuntimeException("The remote jarfile owner:" + scFileStatus.getOwner() + " is not the same as the NM user:" + this.userUGI.getShortUserName() + "."); } if ((scFileStatus.getPermission().toShort() & 0022) != 0) { throw new YarnRuntimeException("The remote jarfile should not " + "be writable by group or others. " + "The current Permission is " + scFileStatus.getPermission().toShort()); } {code} According to the above reasons, I have tried to change *manifest.txt* file parents directories without *writeable* permission of group and others. and change the *umask to 077*, which will effect new created file and directories permissions, because the *manifest.txt* and *run-tests.jar* will be new created when running tests. {code:java} chmod go-w yourpath/hadoop/ -R umask 022 umask {code} After doing above and re-run tests of *TestAuxServices*, all the tests can pass. Actually, I am a new comer to Hadoop, so I am not sure whether this is a bug of Hadoop or not. could you please give some suggestions? Thanks. was (Author: seanlau): Hi [~snemeth] [~adam.antal], Thank you both for care about this issue. I have take some time tried to find the reason of this issue. This issue will effect th
[jira] [Commented] (YARN-9511) [JDK11] TestAuxServices#testRemoteAuxServiceClassPath YarnRuntimeException: The remote jarfile should not be writable by group or others. The current Permission is 436
[ https://issues.apache.org/jira/browse/YARN-9511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16949890#comment-16949890 ] liusheng commented on YARN-9511: Hi [~snemeth] [~adam.antal], Thank you both for care about this issue. I have take some time tried to find the reason of this issue. This issue will effect the tests of *TestAuxServices*, and will cause *2 Errors 9 Failures*, see: {code:java} Failures: TestAuxServices.testAuxServiceRecoverySetup:717 expected:<2> but was:<0> TestAuxServices.testAuxServicesManifestPermissions:874 expected:<2> but was:<0> TestAuxServices.testAuxServicesMeta:638 Invalid mix of services expected:<6> but was:<1> TestAuxServices.testAuxServices:610 Invalid mix of services expected:<6> but was:<1> TestAuxServices.testCustomizedAuxServiceClassPath:416 TestAuxServices.testManualReload:919 expected:<2> but was:<0> TestAuxServices.testRemoteAuxServiceClassPath:313 The permission of the jar is wrong.Should throw out exception. TestAuxServices.testRemoveManifest:897 expected:<2> but was:<0> TestAuxServices.testValidAuxServiceName:698 Should receive the exception. Errors: TestAuxServices.testAuxUnexpectedStop:664 » NoSuchElement TestAuxServices.testRemoteAuxServiceClassPath:334 » YarnRuntime The remote jar... {code} After debuging, I found all these issues are directly or indirectly related with files permissions. there are two situations when running these tests: # *useManifest* enabled, when running tests with useManifest enabled, the tests will check and use the manifest file: {code:java} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/test-dir/TestAuxServices/manifest.txt{code} this file and all its parents directories must not be writable by group or others, see: {code:java} private boolean checkManifestPermissions(FileStatus status) throws IOException { if ((status.getPermission().toShort() & 0022) != 0) { LOG.error("Manifest file and parents must not be writable by group or " + "others. The current Permission of " + status.getPath() + " is " + status.getPermission()); return false; } Path parent = status.getPath().getParent(); if (parent == null) { return true; } return checkManifestPermissions(manifestFS.getFileStatus(parent)); }{code} # *useManifest not* enabled, when running tests with useManifest enabled, tests will use a test-jar.jar file {code:java} private Path maybeDownloadJars(String sName, String className, String remoteFile, AuxServiceFile.TypeEnum type, Configuration conf) throws IOException { // load AuxiliaryService from remote classpath FileContext localLFS = getLocalFileContext(conf); // create NM aux-service dir in NM localdir if it does not exist. Path nmAuxDir = dirsHandler.getLocalPathForWrite("." + Path.SEPARATOR + NM_AUX_SERVICE_DIR); if (!localLFS.util().exists(nmAuxDir)) { try { localLFS.mkdir(nmAuxDir, NM_AUX_SERVICE_DIR_PERM, true); } catch (IOException ex) { throw new YarnRuntimeException("Fail to create dir:" + nmAuxDir.toString(), ex); } } Path src = new Path(remoteFile); FileContext remoteLFS = getRemoteFileContext(src.toUri(), conf); FileStatus scFileStatus = remoteLFS.getFileStatus(src); if (!scFileStatus.getOwner().equals( this.userUGI.getShortUserName())) { throw new YarnRuntimeException("The remote jarfile owner:" + scFileStatus.getOwner() + " is not the same as the NM user:" + this.userUGI.getShortUserName() + "."); } if ((scFileStatus.getPermission().toShort() & 0022) != 0) { throw new YarnRuntimeException("The remote jarfile should not " + "be writable by group or others. " + "The current Permission is " + scFileStatus.getPermission().toShort()); } {code} According to the above reasons, I have tried to change *manifest.txt* file parents directories without *writeable* permission of group and others. and change the *umask to 077*, which will effect new created file and directories permissions, because the *manifest.txt* and *run-tests.jar* will be new created when running tests. {code:java} chmod go-w yourpath/hadoop/ -R umask 022 umask {code} After doing above and re-run tests of *TestAuxServices*, all the tests can pass. Actually, I am a new comer to Hadoop, so I am not sure whether this is a bug of Hadoop or not. could you please give some suggestions? Thanks. > [JDK11] TestAuxServices#testRemoteAuxServiceClassPath YarnRuntimeException: > The remote jarfile should not be writable by group or others. The current > Permission is 436 > --- > > Key: YARN-9511 > URL: https://issues.apache.
[jira] [Commented] (YARN-9511) [JDK11] TestAuxServices#testRemoteAuxServiceClassPath YarnRuntimeException: The remote jarfile should not be writable by group or others. The current Permission is 436
[ https://issues.apache.org/jira/browse/YARN-9511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16922211#comment-16922211 ] liusheng commented on YARN-9511: Hi, Any update about this issue ? > [JDK11] TestAuxServices#testRemoteAuxServiceClassPath YarnRuntimeException: > The remote jarfile should not be writable by group or others. The current > Permission is 436 > --- > > Key: YARN-9511 > URL: https://issues.apache.org/jira/browse/YARN-9511 > Project: Hadoop YARN > Issue Type: Bug > Components: test >Reporter: Siyao Meng >Assignee: Szilard Nemeth >Priority: Major > > Found in maven JDK 11 unit test run. Compiled on JDK 8. > {code} > [ERROR] > testRemoteAuxServiceClassPath(org.apache.hadoop.yarn.server.nodemanager.containermanager.TestAuxServices) > Time elapsed: 0.551 s <<< > ERROR!org.apache.hadoop.yarn.exceptions.YarnRuntimeException: The remote > jarfile should not be writable by group or others. The current Permission is > 436 > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:202) > at > org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.TestAuxServices.testRemoteAuxServiceClassPath(TestAuxServices.java:268) > at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.base/java.lang.reflect.Method.invoke(Method.java:566) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) > at org.junit.runners.ParentRunner.run(ParentRunner.java:309) > at > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365) > at > org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273) > at > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238) > at > org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159) > at > org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384) > at > org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345) > at > org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126) > at > org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418) > {code} -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org