[jira] [Updated] (YARN-2720) Windows: Wildcard classpath variables not expanded against resources contained in archives
[ https://issues.apache.org/jira/browse/YARN-2720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated YARN-2720: Hadoop Flags: Reviewed +1 for the patch, pending Jenkins run. I've verified that this works in my environment with a few test runs. Thank you for fixing this, Craig. > Windows: Wildcard classpath variables not expanded against resources > contained in archives > -- > > Key: YARN-2720 > URL: https://issues.apache.org/jira/browse/YARN-2720 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Craig Welch >Assignee: Craig Welch > Attachments: YARN-2720.2.patch, YARN-2720.3.patch, YARN-2720.4.patch > > > On windows there are limitations to the length of command lines and > environment variables which prevent placing all classpath resources into > these elements. Instead, a jar containing only a classpath manifest is > created to provide the classpath. During this process wildcard references > are expanded by inspecting the filesystem. Since archives are extracted to a > different location and linked into the final location after the classpath jar > is created, resources referred to via wildcards which exist in localized > archives (.zip, tar.gz) are not added to the classpath manifest jar. Since > these entries are removed from the final classpath for the container they are > not on the container's classpath as they should be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2720) Windows: Wildcard classpath variables not expanded against resources contained in archives
[ https://issues.apache.org/jira/browse/YARN-2720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Welch updated YARN-2720: -- Attachment: YARN-2720.4.patch Updated version with unit test modification and white space fixes > Windows: Wildcard classpath variables not expanded against resources > contained in archives > -- > > Key: YARN-2720 > URL: https://issues.apache.org/jira/browse/YARN-2720 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Craig Welch >Assignee: Craig Welch > Attachments: YARN-2720.2.patch, YARN-2720.3.patch, YARN-2720.4.patch > > > On windows there are limitations to the length of command lines and > environment variables which prevent placing all classpath resources into > these elements. Instead, a jar containing only a classpath manifest is > created to provide the classpath. During this process wildcard references > are expanded by inspecting the filesystem. Since archives are extracted to a > different location and linked into the final location after the classpath jar > is created, resources referred to via wildcards which exist in localized > archives (.zip, tar.gz) are not added to the classpath manifest jar. Since > these entries are removed from the final classpath for the container they are > not on the container's classpath as they should be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2720) Windows: Wildcard classpath variables not expanded against resources contained in archives
[ https://issues.apache.org/jira/browse/YARN-2720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Welch updated YARN-2720: -- Attachment: YARN-2720.3.patch It's hacky and not always possible to pass back the additional classpath info using the environment, so change the createJarWithClassPath signature to return an array > Windows: Wildcard classpath variables not expanded against resources > contained in archives > -- > > Key: YARN-2720 > URL: https://issues.apache.org/jira/browse/YARN-2720 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Craig Welch >Assignee: Craig Welch > Attachments: YARN-2720.2.patch, YARN-2720.3.patch > > > On windows there are limitations to the length of command lines and > environment variables which prevent placing all classpath resources into > these elements. Instead, a jar containing only a classpath manifest is > created to provide the classpath. During this process wildcard references > are expanded by inspecting the filesystem. Since archives are extracted to a > different location and linked into the final location after the classpath jar > is created, resources referred to via wildcards which exist in localized > archives (.zip, tar.gz) are not added to the classpath manifest jar. Since > these entries are removed from the final classpath for the container they are > not on the container's classpath as they should be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2720) Windows: Wildcard classpath variables not expanded against resources contained in archives
[ https://issues.apache.org/jira/browse/YARN-2720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Welch updated YARN-2720: -- Attachment: YARN-2720.2.patch Patch which tracks unexpanded wildcard classpath entries and adds them to the final classpath which is used when the container is launched > Windows: Wildcard classpath variables not expanded against resources > contained in archives > -- > > Key: YARN-2720 > URL: https://issues.apache.org/jira/browse/YARN-2720 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Craig Welch >Assignee: Craig Welch > Attachments: YARN-2720.2.patch > > > On windows there are limitations to the length of command lines and > environment variables which prevent placing all classpath resources into > these elements. Instead, a jar containing only a classpath manifest is > created to provide the classpath. During this process wildcard references > are expanded by inspecting the filesystem. Since archives are extracted to a > different location and linked into the final location after the classpath jar > is created, resources referred to via wildcards which exist in localized > archives (.zip, tar.gz) are not added to the classpath manifest jar. Since > these entries are removed from the final classpath for the container they are > not on the container's classpath as they should be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2720) Windows: Wildcard classpath variables not expanded against resources contained in archives
[ https://issues.apache.org/jira/browse/YARN-2720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated YARN-2720: Component/s: nodemanager Target Version/s: 2.6.0 > Windows: Wildcard classpath variables not expanded against resources > contained in archives > -- > > Key: YARN-2720 > URL: https://issues.apache.org/jira/browse/YARN-2720 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Craig Welch >Assignee: Craig Welch > > On windows there are limitations to the length of command lines and > environment variables which prevent placing all classpath resources into > these elements. Instead, a jar containing only a classpath manifest is > created to provide the classpath. During this process wildcard references > are expanded by inspecting the filesystem. Since archives are extracted to a > different location and linked into the final location after the classpath jar > is created, resources referred to via wildcards which exist in localized > archives (.zip, tar.gz) are not added to the classpath manifest jar. Since > these entries are removed from the final classpath for the container they are > not on the container's classpath as they should be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
