[jira] [Commented] (HADOOP-13835) Move Google Test Framework code from mapreduce to hadoop-common
[ https://issues.apache.org/jira/browse/HADOOP-13835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190836#comment-16190836 ] Varun Vasudev commented on HADOOP-13835: [~ajisakaa] - any chance you can take a look at this today? I'd like to get this into the 2.9.0 release. Thanks! > Move Google Test Framework code from mapreduce to hadoop-common > --- > > Key: HADOOP-13835 > URL: https://issues.apache.org/jira/browse/HADOOP-13835 > Project: Hadoop Common > Issue Type: Task > Components: test >Reporter: Varun Vasudev >Assignee: Varun Vasudev > Fix For: 3.0.0-alpha2 > > Attachments: HADOOP-13835.001.patch, HADOOP-13835.002.patch, > HADOOP-13835.003.patch, HADOOP-13835.004.patch, HADOOP-13835.005.patch, > HADOOP-13835.006.patch, HADOOP-13835.007.patch, > HADOOP-13835.branch-2.007.patch, HADOOP-13835.branch-2.008.patch > > > The mapreduce project has Google Test Framework code to allow testing of > native libraries. This should be moved to hadoop-common so that other > projects can use it as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13835) Move Google Test Framework code from mapreduce to hadoop-common
[ https://issues.apache.org/jira/browse/HADOOP-13835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189401#comment-16189401 ] Varun Vasudev commented on HADOOP-13835: [~ajisakaa] - can you please review the latest patch for branch-2? It addresses your review comments. Also, if it looks good to you, can you commit it to branch-2, branch-2.9, and branch-2.8? Thanks! > Move Google Test Framework code from mapreduce to hadoop-common > --- > > Key: HADOOP-13835 > URL: https://issues.apache.org/jira/browse/HADOOP-13835 > Project: Hadoop Common > Issue Type: Task > Components: test >Reporter: Varun Vasudev >Assignee: Varun Vasudev > Fix For: 3.0.0-alpha2 > > Attachments: HADOOP-13835.001.patch, HADOOP-13835.002.patch, > HADOOP-13835.003.patch, HADOOP-13835.004.patch, HADOOP-13835.005.patch, > HADOOP-13835.006.patch, HADOOP-13835.007.patch, > HADOOP-13835.branch-2.007.patch, HADOOP-13835.branch-2.008.patch > > > The mapreduce project has Google Test Framework code to allow testing of > native libraries. This should be moved to hadoop-common so that other > projects can use it as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13835) Move Google Test Framework code from mapreduce to hadoop-common
[ https://issues.apache.org/jira/browse/HADOOP-13835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Vasudev updated HADOOP-13835: --- Attachment: HADOOP-13835.branch-2.008.patch Uploaded patch for branch-2. > Move Google Test Framework code from mapreduce to hadoop-common > --- > > Key: HADOOP-13835 > URL: https://issues.apache.org/jira/browse/HADOOP-13835 > Project: Hadoop Common > Issue Type: Task > Components: test >Reporter: Varun Vasudev >Assignee: Varun Vasudev > Fix For: 3.0.0-alpha2 > > Attachments: HADOOP-13835.001.patch, HADOOP-13835.002.patch, > HADOOP-13835.003.patch, HADOOP-13835.004.patch, HADOOP-13835.005.patch, > HADOOP-13835.006.patch, HADOOP-13835.007.patch, > HADOOP-13835.branch-2.007.patch, HADOOP-13835.branch-2.008.patch > > > The mapreduce project has Google Test Framework code to allow testing of > native libraries. This should be moved to hadoop-common so that other > projects can use it as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13835) Move Google Test Framework code from mapreduce to hadoop-common
[ https://issues.apache.org/jira/browse/HADOOP-13835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16136787#comment-16136787 ] Varun Vasudev commented on HADOOP-13835: [~leftnoteasy] - did you get a chance to do this? Can you backport to branch-2.8 as well? Thanks! > Move Google Test Framework code from mapreduce to hadoop-common > --- > > Key: HADOOP-13835 > URL: https://issues.apache.org/jira/browse/HADOOP-13835 > Project: Hadoop Common > Issue Type: Task > Components: test >Reporter: Varun Vasudev >Assignee: Varun Vasudev > Fix For: 3.0.0-alpha2 > > Attachments: HADOOP-13835.001.patch, HADOOP-13835.002.patch, > HADOOP-13835.003.patch, HADOOP-13835.004.patch, HADOOP-13835.005.patch, > HADOOP-13835.006.patch, HADOOP-13835.007.patch, > HADOOP-13835.branch-2.007.patch > > > The mapreduce project has Google Test Framework code to allow testing of > native libraries. This should be moved to hadoop-common so that other > projects can use it as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-14465) LdapGroupsMapping - support user and group search base
[ https://issues.apache.org/jira/browse/HADOOP-14465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Vasudev reassigned HADOOP-14465: -- Assignee: Shwetha G S > LdapGroupsMapping - support user and group search base > -- > > Key: HADOOP-14465 > URL: https://issues.apache.org/jira/browse/HADOOP-14465 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Reporter: Shwetha G S >Assignee: Shwetha G S > Attachments: > 0001-HADOOP-14465-LdapGroupsMapping-support-user-and-grou.patch > > > org.apache.hadoop.security.LdapGroupsMapping currently supports > hadoop.security.group.mapping.ldap.base as search base for both user and > group searches. However, this doesn't work when user and group search bases > are different like ou=Users,dc=xxx,dc=com and ou=Groups,dc=xxx,dc=com. > Expose different configs for user and group search base which defaults to the > existing hadoop.security.group.mapping.ldap.base config -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13835) Move Google Test Framework code from mapreduce to hadoop-common
[ https://issues.apache.org/jira/browse/HADOOP-13835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15724622#comment-15724622 ] Varun Vasudev commented on HADOOP-13835: Thanks for the review and commit [~ajisakaa]! > Move Google Test Framework code from mapreduce to hadoop-common > --- > > Key: HADOOP-13835 > URL: https://issues.apache.org/jira/browse/HADOOP-13835 > Project: Hadoop Common > Issue Type: Task > Components: test >Reporter: Varun Vasudev >Assignee: Varun Vasudev > Fix For: 3.0.0-alpha2 > > Attachments: HADOOP-13835.001.patch, HADOOP-13835.002.patch, > HADOOP-13835.003.patch, HADOOP-13835.004.patch, HADOOP-13835.005.patch, > HADOOP-13835.006.patch, HADOOP-13835.007.patch > > > The mapreduce project has Google Test Framework code to allow testing of > native libraries. This should be moved to hadoop-common so that other > projects can use it as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13835) Move Google Test Framework code from mapreduce to hadoop-common
[ https://issues.apache.org/jira/browse/HADOOP-13835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Vasudev updated HADOOP-13835: --- Attachment: HADOOP-13835.007.patch Fix path for system dirs in CMakeLists.txt > Move Google Test Framework code from mapreduce to hadoop-common > --- > > Key: HADOOP-13835 > URL: https://issues.apache.org/jira/browse/HADOOP-13835 > Project: Hadoop Common > Issue Type: Task >Reporter: Varun Vasudev >Assignee: Varun Vasudev > Attachments: HADOOP-13835.001.patch, HADOOP-13835.002.patch, > HADOOP-13835.003.patch, HADOOP-13835.004.patch, HADOOP-13835.005.patch, > HADOOP-13835.006.patch, HADOOP-13835.007.patch > > > The mapreduce project has Google Test Framework code to allow testing of > native libraries. This should be moved to hadoop-common so that other > projects can use it as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13835) Move Google Test Framework code from mapreduce to hadoop-common
[ https://issues.apache.org/jira/browse/HADOOP-13835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Vasudev updated HADOOP-13835: --- Attachment: HADOOP-13835.006.patch -006 1) Fix a typo in CMakeLists.txt 2) Fix path for adding gtest files to system libs in CMakeLists.txt > Move Google Test Framework code from mapreduce to hadoop-common > --- > > Key: HADOOP-13835 > URL: https://issues.apache.org/jira/browse/HADOOP-13835 > Project: Hadoop Common > Issue Type: Task >Reporter: Varun Vasudev >Assignee: Varun Vasudev > Attachments: HADOOP-13835.001.patch, HADOOP-13835.002.patch, > HADOOP-13835.003.patch, HADOOP-13835.004.patch, HADOOP-13835.005.patch, > HADOOP-13835.006.patch > > > The mapreduce project has Google Test Framework code to allow testing of > native libraries. This should be moved to hadoop-common so that other > projects can use it as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13835) Move Google Test Framework code from mapreduce to hadoop-common
[ https://issues.apache.org/jira/browse/HADOOP-13835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15713913#comment-15713913 ] Varun Vasudev commented on HADOOP-13835: [~ajisakaa] - I think the latest patch is ready for review. Can you take a look? Thanks! > Move Google Test Framework code from mapreduce to hadoop-common > --- > > Key: HADOOP-13835 > URL: https://issues.apache.org/jira/browse/HADOOP-13835 > Project: Hadoop Common > Issue Type: Task >Reporter: Varun Vasudev >Assignee: Varun Vasudev > Attachments: HADOOP-13835.001.patch, HADOOP-13835.002.patch, > HADOOP-13835.003.patch, HADOOP-13835.004.patch, HADOOP-13835.005.patch > > > The mapreduce project has Google Test Framework code to allow testing of > native libraries. This should be moved to hadoop-common so that other > projects can use it as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13835) Move Google Test Framework code from mapreduce to hadoop-common
[ https://issues.apache.org/jira/browse/HADOOP-13835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Vasudev updated HADOOP-13835: --- Attachment: HADOOP-13835.005.patch -005 Fix the compile errors in mapreduce. > Move Google Test Framework code from mapreduce to hadoop-common > --- > > Key: HADOOP-13835 > URL: https://issues.apache.org/jira/browse/HADOOP-13835 > Project: Hadoop Common > Issue Type: Task >Reporter: Varun Vasudev >Assignee: Varun Vasudev > Attachments: HADOOP-13835.001.patch, HADOOP-13835.002.patch, > HADOOP-13835.003.patch, HADOOP-13835.004.patch, HADOOP-13835.005.patch > > > The mapreduce project has Google Test Framework code to allow testing of > native libraries. This should be moved to hadoop-common so that other > projects can use it as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13835) Move Google Test Framework code from mapreduce to hadoop-common
[ https://issues.apache.org/jira/browse/HADOOP-13835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Vasudev updated HADOOP-13835: --- Attachment: HADOOP-13835.004.patch Thanks for the review [~ajisakaa]! I've addressed your feedback in the latest patch. > Move Google Test Framework code from mapreduce to hadoop-common > --- > > Key: HADOOP-13835 > URL: https://issues.apache.org/jira/browse/HADOOP-13835 > Project: Hadoop Common > Issue Type: Task >Reporter: Varun Vasudev >Assignee: Varun Vasudev > Attachments: HADOOP-13835.001.patch, HADOOP-13835.002.patch, > HADOOP-13835.003.patch, HADOOP-13835.004.patch > > > The mapreduce project has Google Test Framework code to allow testing of > native libraries. This should be moved to hadoop-common so that other > projects can use it as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13835) Move Google Test Framework code from mapreduce to hadoop-common
[ https://issues.apache.org/jira/browse/HADOOP-13835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15711073#comment-15711073 ] Varun Vasudev commented on HADOOP-13835: [~ajisakaa] - can you please review the latest patch? Thanks! > Move Google Test Framework code from mapreduce to hadoop-common > --- > > Key: HADOOP-13835 > URL: https://issues.apache.org/jira/browse/HADOOP-13835 > Project: Hadoop Common > Issue Type: Task >Reporter: Varun Vasudev >Assignee: Varun Vasudev > Attachments: HADOOP-13835.001.patch, HADOOP-13835.002.patch, > HADOOP-13835.003.patch > > > The mapreduce project has Google Test Framework code to allow testing of > native libraries. This should be moved to hadoop-common so that other > projects can use it as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13835) Move Google Test Framework code from mapreduce to hadoop-common
[ https://issues.apache.org/jira/browse/HADOOP-13835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15708839#comment-15708839 ] Varun Vasudev commented on HADOOP-13835: The ASF license warnings and the test failures are unrelated to the patch. The cc warnings are due to the google test code. [~aw] - is there a way to suppress the cc warnings? > Move Google Test Framework code from mapreduce to hadoop-common > --- > > Key: HADOOP-13835 > URL: https://issues.apache.org/jira/browse/HADOOP-13835 > Project: Hadoop Common > Issue Type: Task >Reporter: Varun Vasudev >Assignee: Varun Vasudev > Attachments: HADOOP-13835.001.patch, HADOOP-13835.002.patch, > HADOOP-13835.003.patch > > > The mapreduce project has Google Test Framework code to allow testing of > native libraries. This should be moved to hadoop-common so that other > projects can use it as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13835) Move Google Test Framework code from mapreduce to hadoop-common
[ https://issues.apache.org/jira/browse/HADOOP-13835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Vasudev updated HADOOP-13835: --- Attachment: HADOOP-13835.003.patch Uploaded a new patch after MAPREDUCE-6743 > Move Google Test Framework code from mapreduce to hadoop-common > --- > > Key: HADOOP-13835 > URL: https://issues.apache.org/jira/browse/HADOOP-13835 > Project: Hadoop Common > Issue Type: Task >Reporter: Varun Vasudev >Assignee: Varun Vasudev > Attachments: HADOOP-13835.001.patch, HADOOP-13835.002.patch, > HADOOP-13835.003.patch > > > The mapreduce project has Google Test Framework code to allow testing of > native libraries. This should be moved to hadoop-common so that other > projects can use it as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13835) Move Google Test Framework code from mapreduce to hadoop-common
[ https://issues.apache.org/jira/browse/HADOOP-13835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15701033#comment-15701033 ] Varun Vasudev commented on HADOOP-13835: [~aw] - sure. I can wait for MAPREDUCE-6743 to go in. > Move Google Test Framework code from mapreduce to hadoop-common > --- > > Key: HADOOP-13835 > URL: https://issues.apache.org/jira/browse/HADOOP-13835 > Project: Hadoop Common > Issue Type: Task >Reporter: Varun Vasudev >Assignee: Varun Vasudev > Attachments: HADOOP-13835.001.patch, HADOOP-13835.002.patch > > > The mapreduce project has Google Test Framework code to allow testing of > native libraries. This should be moved to hadoop-common so that other > projects can use it as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13835) Move Google Test Framework code from mapreduce to hadoop-common
[ https://issues.apache.org/jira/browse/HADOOP-13835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Vasudev updated HADOOP-13835: --- Attachment: HADOOP-13835.002.patch Thanks for the catch [~ajisakaa]! Uploaded a new patch with the fix. > Move Google Test Framework code from mapreduce to hadoop-common > --- > > Key: HADOOP-13835 > URL: https://issues.apache.org/jira/browse/HADOOP-13835 > Project: Hadoop Common > Issue Type: Task >Reporter: Varun Vasudev >Assignee: Varun Vasudev > Attachments: HADOOP-13835.001.patch, HADOOP-13835.002.patch > > > The mapreduce project has Google Test Framework code to allow testing of > native libraries. This should be moved to hadoop-common so that other > projects can use it as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13835) Move Google Test Framework code from mapreduce to hadoop-common
[ https://issues.apache.org/jira/browse/HADOOP-13835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Vasudev updated HADOOP-13835: --- Status: Patch Available (was: Open) > Move Google Test Framework code from mapreduce to hadoop-common > --- > > Key: HADOOP-13835 > URL: https://issues.apache.org/jira/browse/HADOOP-13835 > Project: Hadoop Common > Issue Type: Task >Reporter: Varun Vasudev >Assignee: Varun Vasudev > Attachments: HADOOP-13835.001.patch > > > The mapreduce project has Google Test Framework code to allow testing of > native libraries. This should be moved to hadoop-common so that other > projects can use it as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13835) Move Google Test Framework code from mapreduce to hadoop-common
[ https://issues.apache.org/jira/browse/HADOOP-13835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Vasudev updated HADOOP-13835: --- Attachment: HADOOP-13835.001.patch Patch attached. It does 2 things - 1) Moves the gtest directory from hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/main/native/ to hadoop-common-project/hadoop-common/src/main/native 2) Update the CMakeLists.txt in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src to point to the new location of gtest. > Move Google Test Framework code from mapreduce to hadoop-common > --- > > Key: HADOOP-13835 > URL: https://issues.apache.org/jira/browse/HADOOP-13835 > Project: Hadoop Common > Issue Type: Task >Reporter: Varun Vasudev >Assignee: Varun Vasudev > Attachments: HADOOP-13835.001.patch > > > The mapreduce project has Google Test Framework code to allow testing of > native libraries. This should be moved to hadoop-common so that other > projects can use it as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13835) Move Google Test Framework code from mapreduce to hadoop-common
Varun Vasudev created HADOOP-13835: -- Summary: Move Google Test Framework code from mapreduce to hadoop-common Key: HADOOP-13835 URL: https://issues.apache.org/jira/browse/HADOOP-13835 Project: Hadoop Common Issue Type: Task Reporter: Varun Vasudev Assignee: Varun Vasudev The mapreduce project has Google Test Framework code to allow testing of native libraries. This should be moved to hadoop-common so that other projects can use it as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13008) Add XFS Filter for UIs to Hadoop Common
[ https://issues.apache.org/jira/browse/HADOOP-13008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15295523#comment-15295523 ] Varun Vasudev commented on HADOOP-13008: [~lmccay] - sorry I haven't filed a JIRA; I wasn't planning to work on the issue. > Add XFS Filter for UIs to Hadoop Common > --- > > Key: HADOOP-13008 > URL: https://issues.apache.org/jira/browse/HADOOP-13008 > Project: Hadoop Common > Issue Type: New Feature > Components: security >Reporter: Larry McCay >Assignee: Larry McCay > Fix For: 2.8.0 > > Attachments: HADOOP-13008-001.patch, HADOOP-13008-002.patch, > HADOOP-13008-003.patch, HADOOP-13008-004.patch > > > Cross Frame Scripting (XFS) prevention for UIs can be provided through a > common servlet filter. This filter will set the X-Frame-Options HTTP header > to DENY unless configured to another valid setting. > There are a number of UIs that could just add this to their filters as well > as the Yarn webapp proxy which could add it for all it's proxied UIs - if > appropriate. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-13008) Add XFS Filter for UIs to Hadoop Common
[ https://issues.apache.org/jira/browse/HADOOP-13008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15289320#comment-15289320 ] Varun Vasudev edited comment on HADOOP-13008 at 5/18/16 5:37 PM: - I prefer the generic XFS filter based approach to the QuotingInputFilter - it's more flexible. The only reason I realized this is that I was testing a patch for YARN integration and noticed that the header was being set for all responses and I wasn't sure why. I defer to Chris and you on what to do going forward. was (Author: vvasudev): I prefer the filter based approach - it's more flexible. The only reason I realized this is that I was testing a patch for YARN integration and noticed that the header was being set for all responses and I wasn't sure why. I defer to Chris and you on what to do going forward. > Add XFS Filter for UIs to Hadoop Common > --- > > Key: HADOOP-13008 > URL: https://issues.apache.org/jira/browse/HADOOP-13008 > Project: Hadoop Common > Issue Type: New Feature > Components: security >Reporter: Larry McCay >Assignee: Larry McCay > Fix For: 2.8.0 > > Attachments: HADOOP-13008-001.patch, HADOOP-13008-002.patch, > HADOOP-13008-003.patch, HADOOP-13008-004.patch > > > Cross Frame Scripting (XFS) prevention for UIs can be provided through a > common servlet filter. This filter will set the X-Frame-Options HTTP header > to DENY unless configured to another valid setting. > There are a number of UIs that could just add this to their filters as well > as the Yarn webapp proxy which could add it for all it's proxied UIs - if > appropriate. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13008) Add XFS Filter for UIs to Hadoop Common
[ https://issues.apache.org/jira/browse/HADOOP-13008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15289320#comment-15289320 ] Varun Vasudev commented on HADOOP-13008: I prefer the filter based approach - it's more flexible. The only reason I realized this is that I was testing a patch for YARN integration and noticed that the header was being set for all responses and I wasn't sure why. I defer to Chris and you on what to do going forward. > Add XFS Filter for UIs to Hadoop Common > --- > > Key: HADOOP-13008 > URL: https://issues.apache.org/jira/browse/HADOOP-13008 > Project: Hadoop Common > Issue Type: New Feature > Components: security >Reporter: Larry McCay >Assignee: Larry McCay > Fix For: 2.8.0 > > Attachments: HADOOP-13008-001.patch, HADOOP-13008-002.patch, > HADOOP-13008-003.patch, HADOOP-13008-004.patch > > > Cross Frame Scripting (XFS) prevention for UIs can be provided through a > common servlet filter. This filter will set the X-Frame-Options HTTP header > to DENY unless configured to another valid setting. > There are a number of UIs that could just add this to their filters as well > as the Yarn webapp proxy which could add it for all it's proxied UIs - if > appropriate. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13008) Add XFS Filter for UIs to Hadoop Common
[ https://issues.apache.org/jira/browse/HADOOP-13008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15289148#comment-15289148 ] Varun Vasudev commented on HADOOP-13008: [~lmccay], [~cnauroth] - HADOOP-12964 has also added XFS support. There seems to some amount of duplicate code between the two issues. Can you please let me know if this is intended? > Add XFS Filter for UIs to Hadoop Common > --- > > Key: HADOOP-13008 > URL: https://issues.apache.org/jira/browse/HADOOP-13008 > Project: Hadoop Common > Issue Type: New Feature > Components: security >Reporter: Larry McCay >Assignee: Larry McCay > Fix For: 2.8.0 > > Attachments: HADOOP-13008-001.patch, HADOOP-13008-002.patch, > HADOOP-13008-003.patch, HADOOP-13008-004.patch > > > Cross Frame Scripting (XFS) prevention for UIs can be provided through a > common servlet filter. This filter will set the X-Frame-Options HTTP header > to DENY unless configured to another valid setting. > There are a number of UIs that could just add this to their filters as well > as the Yarn webapp proxy which could add it for all it's proxied UIs - if > appropriate. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-12749) Create a threadpoolexecutor that overrides afterExecute to log uncaught exceptions/errors
[ https://issues.apache.org/jira/browse/HADOOP-12749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Vasudev updated HADOOP-12749: --- Resolution: Fixed Fix Version/s: 2.9.0 Status: Resolved (was: Patch Available) Committed to trunk and branch-2. Thanks [~sidharta-s]! > Create a threadpoolexecutor that overrides afterExecute to log uncaught > exceptions/errors > - > > Key: HADOOP-12749 > URL: https://issues.apache.org/jira/browse/HADOOP-12749 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Sidharta Seethana >Assignee: Sidharta Seethana > Fix For: 2.9.0 > > Attachments: HADOOP-12749.001.patch, HADOOP-12749.002.patch, > HADOOP-12749.003.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (HADOOP-12749) Create a threadpoolexecutor that overrides afterExecute to log uncaught exceptions/errors
[ https://issues.apache.org/jira/browse/HADOOP-12749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Vasudev reopened HADOOP-12749: Doh! Reopening ticket because the new files didn't get added to the repository in the commit. My apologies [~sidharta-s] > Create a threadpoolexecutor that overrides afterExecute to log uncaught > exceptions/errors > - > > Key: HADOOP-12749 > URL: https://issues.apache.org/jira/browse/HADOOP-12749 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Sidharta Seethana >Assignee: Sidharta Seethana > Fix For: 2.9.0 > > Attachments: HADOOP-12749.001.patch, HADOOP-12749.002.patch, > HADOOP-12749.003.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12749) Create a threadpoolexecutor that overrides afterExecute to log uncaught exceptions/errors
[ https://issues.apache.org/jira/browse/HADOOP-12749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15137026#comment-15137026 ] Varun Vasudev commented on HADOOP-12749: Added missing files. > Create a threadpoolexecutor that overrides afterExecute to log uncaught > exceptions/errors > - > > Key: HADOOP-12749 > URL: https://issues.apache.org/jira/browse/HADOOP-12749 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Sidharta Seethana >Assignee: Sidharta Seethana > Fix For: 2.9.0 > > Attachments: HADOOP-12749.001.patch, HADOOP-12749.002.patch, > HADOOP-12749.003.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HADOOP-12749) Create a threadpoolexecutor that overrides afterExecute to log uncaught exceptions/errors
[ https://issues.apache.org/jira/browse/HADOOP-12749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Vasudev resolved HADOOP-12749. Resolution: Fixed > Create a threadpoolexecutor that overrides afterExecute to log uncaught > exceptions/errors > - > > Key: HADOOP-12749 > URL: https://issues.apache.org/jira/browse/HADOOP-12749 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Sidharta Seethana >Assignee: Sidharta Seethana > Fix For: 2.9.0 > > Attachments: HADOOP-12749.001.patch, HADOOP-12749.002.patch, > HADOOP-12749.003.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12749) Create a threadpoolexecutor that overrides afterExecute to log uncaught exceptions/errors
[
https://issues.apache.org/jira/browse/HADOOP-12749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15132095#comment-15132095
]
Varun Vasudev commented on HADOOP-12749:
Thanks for the patch [~sidharta-s]. One minor fix - in
HadoopScheduledThreadPoolExecutor.java,
{code}
private static final Log LOG = LogFactory
.getLog(HadoopScheduledThreadPoolExecutor.class);
{code}
is never used. Can you remove it?
> Create a threadpoolexecutor that overrides afterExecute to log uncaught
> exceptions/errors
> -
>
> Key: HADOOP-12749
> URL: https://issues.apache.org/jira/browse/HADOOP-12749
> Project: Hadoop Common
> Issue Type: Sub-task
>Reporter: Sidharta Seethana
>Assignee: Sidharta Seethana
> Attachments: HADOOP-12749.001.patch, HADOOP-12749.002.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12749) Create a threadpoolexecutor that overrides afterExecute to log uncaught exceptions/errors
[ https://issues.apache.org/jira/browse/HADOOP-12749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15132639#comment-15132639 ] Varun Vasudev commented on HADOOP-12749: +1 pending Jenkins. > Create a threadpoolexecutor that overrides afterExecute to log uncaught > exceptions/errors > - > > Key: HADOOP-12749 > URL: https://issues.apache.org/jira/browse/HADOOP-12749 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Sidharta Seethana >Assignee: Sidharta Seethana > Attachments: HADOOP-12749.001.patch, HADOOP-12749.002.patch, > HADOOP-12749.003.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12712) Fix some cmake plugin and native build warnings
[ https://issues.apache.org/jira/browse/HADOOP-12712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15110401#comment-15110401 ] Varun Vasudev commented on HADOOP-12712: Thanks for catching this [~cmccabe]. My apologies for letting this slip through. Like Andrew, I suspect some implicit return was carried out by the compiler. I've cherry-picked the patch to 2.8 as well. > Fix some cmake plugin and native build warnings > --- > > Key: HADOOP-12712 > URL: https://issues.apache.org/jira/browse/HADOOP-12712 > Project: Hadoop Common > Issue Type: Bug > Components: native >Affects Versions: 2.4.0 >Reporter: Colin Patrick McCabe >Assignee: Colin Patrick McCabe >Priority: Minor > Fix For: 2.8.0, 2.9.0 > > Attachments: HADOOP-12712.001.patch, HADOOP-12712.002.patch, > HADOOP-12712.003.patch > > > Fix some native build warnings -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12712) Fix some cmake plugin and native build warnings
[ https://issues.apache.org/jira/browse/HADOOP-12712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Vasudev updated HADOOP-12712: --- Fix Version/s: 2.8.0 > Fix some cmake plugin and native build warnings > --- > > Key: HADOOP-12712 > URL: https://issues.apache.org/jira/browse/HADOOP-12712 > Project: Hadoop Common > Issue Type: Bug > Components: native >Affects Versions: 2.4.0 >Reporter: Colin Patrick McCabe >Assignee: Colin Patrick McCabe >Priority: Minor > Fix For: 2.8.0, 2.9.0 > > Attachments: HADOOP-12712.001.patch, HADOOP-12712.002.patch, > HADOOP-12712.003.patch > > > Fix some native build warnings -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12687) SecureUtil#getByName should also try to resolve direct hostname, incase multiple loopback addresses are present in /etc/hosts
[
https://issues.apache.org/jira/browse/HADOOP-12687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15087185#comment-15087185
]
Varun Vasudev commented on HADOOP-12687:
[~rohithsharma] - I think you need to revert this patch. If essentially undoes
the security check in getByExactName(). When doing hostname lookups, the
hostname must be rooted(“.” added to the end to avoid the security hole in RFC
1535). This patch undoes that check.
> SecureUtil#getByName should also try to resolve direct hostname, incase
> multiple loopback addresses are present in /etc/hosts
> -
>
> Key: HADOOP-12687
> URL: https://issues.apache.org/jira/browse/HADOOP-12687
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Junping Du
>Assignee: Sunil G
> Labels: security
> Fix For: 2.9.0
>
> Attachments: 0001-YARN-4352.patch, 0002-YARN-4352.patch,
> 0003-HADOOP-12687.patch, 0004-HADOOP-12687.patch
>
>
> From
> https://builds.apache.org/job/PreCommit-YARN-Build/9661/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_79.txt,
> we can see the tests in TestYarnClient, TestAMRMClient and TestNMClient get
> timeout which can be reproduced locally.
> When {{/etc/hosts}} has multiple loopback entries,
> {{InetAddress.getByName(null)}} will be returning the first entry present in
> etc/hosts. Hence its possible that machine hostname can be second in list and
> cause {{UnKnownHostException}}.
> Suggesting a direct resolve for such hostname scenarios.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12366) expose calculated paths
[ https://issues.apache.org/jira/browse/HADOOP-12366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14993450#comment-14993450 ] Varun Vasudev commented on HADOOP-12366: +1 for the latest patch for trunk. [~aw] - the branch-2 version hasn't gone through yetus. Do you want to run that through yetus as well? > expose calculated paths > --- > > Key: HADOOP-12366 > URL: https://issues.apache.org/jira/browse/HADOOP-12366 > Project: Hadoop Common > Issue Type: New Feature >Affects Versions: 3.0.0 >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer > Attachments: HADOOP-12366.00.patch, HADOOP-12366.01.patch, > HADOOP-12366.02.patch, HADOOP-12366.branch-2.02.patch > > > It would be useful for 3rd party apps to know the locations of things when > hadoop is running without explicit path env vars set. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-10787) Rename/remove non-HADOOP_*, etc from the shell scripts
[ https://issues.apache.org/jira/browse/HADOOP-10787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Vasudev updated HADOOP-10787: --- Resolution: Fixed Status: Resolved (was: Patch Available) +1. Committed to trunk. Thanks [~aw]! > Rename/remove non-HADOOP_*, etc from the shell scripts > -- > > Key: HADOOP-10787 > URL: https://issues.apache.org/jira/browse/HADOOP-10787 > Project: Hadoop Common > Issue Type: Improvement > Components: scripts >Affects Versions: 3.0.0 >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer >Priority: Blocker > Labels: scripts > Attachments: HADOOP-10787.00.patch, HADOOP-10787.01.patch, > HADOOP-10787.02.patch, HADOOP-10787.03.patch, HADOOP-10787.04.patch, > HADOOP-10787.05.patch > > > We should make an effort to clean up the shell env var name space by removing > unsafe variables. See comments for list. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-10787) Rename/remove non-HADOOP_*, etc from the shell scripts
[
https://issues.apache.org/jira/browse/HADOOP-10787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980422#comment-14980422
]
Varun Vasudev commented on HADOOP-10787:
[~aw] - the issue still exists on my mac and the linux vm.
On my mac -
{code}
hadoop-common vvasudev$ pwd
/Users/vvasudev/Workspace/apache/hadoop/hadoop-common-project/hadoop-common
hadoop-common vvasudev$ mvn clean package install -DskipTests
-Dmaven.javadoc.skip=true
{code}
results in
{code}
[exec] 1..3
[exec] not ok 1 hadoop_add_common_to_classpath (negative)
[exec] # (from function `freetheclasses' in file
hadoop_add_common_to_classpath.bats, line 27,
[exec] # in test file hadoop_add_common_to_classpath.bats, line 45)
[exec] # `freetheclasses' failed
[exec] # bindir:
/Users/vvasudev/Workspace/apache/hadoop/hadoop-common-project/hadoop-common/src/test/scripts
[exec] # /var/folders/n1/klkrb91135d8s9xl7m4gcsgwgp/T/bats.97469.src:
line 27: unset: `/Users/vvasudev/Workspace/current-hadoop': not a valid
identifier
[exec] not ok 2 hadoop_add_common_to_classpath (positive)
[exec] # (from function `freetheclasses' in file
hadoop_add_common_to_classpath.bats, line 27,
[exec] # in test file hadoop_add_common_to_classpath.bats, line 53)
[exec] # `freetheclasses' failed
[exec] # bindir:
/Users/vvasudev/Workspace/apache/hadoop/hadoop-common-project/hadoop-common/src/test/scripts
[exec] # /var/folders/n1/klkrb91135d8s9xl7m4gcsgwgp/T/bats.97469.src:
line 27: unset: `/Users/vvasudev/Workspace/current-hadoop': not a valid
identifier
[exec] not ok 3 hadoop_add_common_to_classpath (build paths)
[exec] # (from function `freetheclasses' in file
hadoop_add_common_to_classpath.bats, line 27,
[exec] # in test file hadoop_add_common_to_classpath.bats, line 63)
[exec] # `freetheclasses' failed
[exec] # bindir:
/Users/vvasudev/Workspace/apache/hadoop/hadoop-common-project/hadoop-common/src/test/scripts
[exec] # /var/folders/n1/klkrb91135d8s9xl7m4gcsgwgp/T/bats.97469.src:
line 27: unset: `/Users/vvasudev/Workspace/current-hadoop': not a valid
identifier
[exec] Running bats -t hadoop_add_javalibpath.bats
{code}
which means that I can't run a build with bats enabled(even if I skip tests).
In my .profile, I have
{code}
export HADOOP_INSTALL=/Users/vvasudev/Workspace/current-hadoop
{code}
The bash version is -
{code}
hadoop-common vvasudev$ bash --version
GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin14)
Copyright (C) 2007 Free Software Foundation, Inc.
hadoop-common vvasudev$
{code}
The root cause seems to the function freetheclasses in
hadoop_add_common_to_classpath.bats - the unset in that function gives the
error.
> Rename/remove non-HADOOP_*, etc from the shell scripts
> --
>
> Key: HADOOP-10787
> URL: https://issues.apache.org/jira/browse/HADOOP-10787
> Project: Hadoop Common
> Issue Type: Improvement
> Components: scripts
>Affects Versions: 3.0.0
>Reporter: Allen Wittenauer
>Assignee: Allen Wittenauer
>Priority: Blocker
> Labels: scripts
> Attachments: HADOOP-10787.00.patch, HADOOP-10787.01.patch,
> HADOOP-10787.02.patch, HADOOP-10787.03.patch, HADOOP-10787.04.patch
>
>
> We should make an effort to clean up the shell env var name space by removing
> unsafe variables. See comments for list.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-10787) Rename/remove non-HADOOP_*, etc from the shell scripts
[
https://issues.apache.org/jira/browse/HADOOP-10787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978072#comment-14978072
]
Varun Vasudev commented on HADOOP-10787:
[~aw] - any idea what's happening with the failing test?
I tried to run it on my machine and here's what I get(all these runs were with
the patch applied) -
1) On my mac
{code}
[exec] 1..3
[exec] not ok 1 hadoop_add_common_to_classpath (negative)
[exec] # (from function `freetheclasses' in file
hadoop_add_common_to_classpath.bats, line 27,
[exec] # in test file hadoop_add_common_to_classpath.bats, line 45)
[exec] # `freetheclasses' failed
[exec] # bindir:
/Users/vvasudev/Workspace/apache/hadoop/hadoop-common-project/hadoop-common/src/test/scripts
[exec] # /var/folders/n1/klkrb91135d8s9xl7m4gcsgwgp/T/bats.39050.src:
line 27: unset: `/Users/vvasudev/Workspace/current-hadoop': not a valid
identifier
[exec] not ok 2 hadoop_add_common_to_classpath (positive)
[exec] # (from function `freetheclasses' in file
hadoop_add_common_to_classpath.bats, line 27,
[exec] # in test file hadoop_add_common_to_classpath.bats, line 53)
[exec] # `freetheclasses' failed
[exec] # bindir:
/Users/vvasudev/Workspace/apache/hadoop/hadoop-common-project/hadoop-common/src/test/scripts
[exec] # /var/folders/n1/klkrb91135d8s9xl7m4gcsgwgp/T/bats.39050.src:
line 27: unset: `/Users/vvasudev/Workspace/current-hadoop': not a valid
identifier
[exec] not ok 3 hadoop_add_common_to_classpath (build paths)
[exec] # (from function `freetheclasses' in file
hadoop_add_common_to_classpath.bats, line 27,
[exec] # in test file hadoop_add_common_to_classpath.bats, line 63)
[exec] # `freetheclasses' failed
[exec] # bindir:
/Users/vvasudev/Workspace/apache/hadoop/hadoop-common-project/hadoop-common/src/test/scripts
[exec] # /var/folders/n1/klkrb91135d8s9xl7m4gcsgwgp/T/bats.39050.src:
line 27: unset: `/Users/vvasudev/Workspace/current-hadoop': not a valid
identifier
[exec] Running bats -t hadoop_add_javalibpath.bats
{code}
2) On a Linux VM
{code}
[exec] Running bats -t hadoop_add_to_classpath_toolspath.bats
[exec] 1..5
[exec] ok 1 hadoop_add_to_classpath_toolspath (nothing)
[exec] ok 2 hadoop_add_to_classpath_toolspath (none)
[exec] not ok 3 hadoop_add_to_classpath_toolspath (only)
[exec] # (from function `hadoop_add_to_classpath_toolspath' in file
../../main/bin/hadoop-functions.sh, line 970,
[exec] # in test file hadoop_add_to_classpath_toolspath.bats, line 52)
[exec] # `hadoop_add_to_classpath_toolspath' failed
[exec] # bindir:
/home/varun/hadoop/hadoop-common-project/hadoop-common/src/test/scripts
[exec] # DEBUG: Adding HADOOP_TOOLS_PATH to CLASSPATH
[exec] not ok 4 hadoop_add_to_classpath_toolspath (1+1)
[exec] # (from function `hadoop_add_to_classpath_toolspath' in file
../../main/bin/hadoop-functions.sh, line 970,
[exec] # in test file hadoop_add_to_classpath_toolspath.bats, line 61)
[exec] # `hadoop_add_to_classpath_toolspath' failed
[exec] # bindir:
/home/varun/hadoop/hadoop-common-project/hadoop-common/src/test/scripts
[exec] # DEBUG: Adding HADOOP_TOOLS_PATH to CLASSPATH
[exec] ok 5 hadoop_add_to_classpath_toolspath (3+2)
[exec] Running bats -t hadoop_add_to_classpath_userpath.bats
[exec] 1..7
[exec] not ok 1 hadoop_add_to_classpath_userpath (nothing)
[exec] # (from function `freetheclasses' in file
hadoop_add_to_classpath_userpath.bats, line 25,
[exec] # in test file hadoop_add_to_classpath_userpath.bats, line 38)
[exec] # `freetheclasses' failed
[exec] # bindir:
/home/varun/hadoop/hadoop-common-project/hadoop-common/src/test/scripts
[exec] # /tmp/bats.24440.src: line 25: unset:
`/var/hadoop/tez-conf:/var/hadoop/tez-0.7.0-minimal/*:/var/hadoop/tez-0.7.0-minimal/lib/*':
not a valid identifier
[exec] not ok 2 hadoop_add_to_classpath_userpath (none)
[exec] # (from function `freetheclasses' in file
hadoop_add_to_classpath_userpath.bats, line 25,
[exec] # in test file hadoop_add_to_classpath_userpath.bats, line 44)
[exec] # `freetheclasses' failed
[exec] # bindir:
/home/varun/hadoop/hadoop-common-project/hadoop-common/src/test/scripts
[exec] # /tmp/bats.24440.src: line 25: unset:
`/var/hadoop/tez-conf:/var/hadoop/tez-0.7.0-minimal/*:/var/hadoop/tez-0.7.0-minimal/lib/*':
not a valid identifier
[exec] not ok 3 hadoop_add_to_classpath_userpath (only)
[exec] # (from function `freetheclasses' in file
hadoop_add_to_classpath_userpath.bats, line 25,
[exec] # in test file hadoop_add_to_classpath_userpath.bats, line 51)
[exec] # `freetheclasses' failed
[exec] # bindir:
/home/varun/hadoop/hadoop-common-project/hadoop-common/src/test/scripts
[exec] # /tmp/bats.24440.src: line 25:
[jira] [Commented] (HADOOP-12366) expose calculated paths
[ https://issues.apache.org/jira/browse/HADOOP-12366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14976046#comment-14976046 ] Varun Vasudev commented on HADOOP-12366: Thanks for the patch [~aw]! Couple of things - 1) The patch doesn't apply cleanly to trunk any more. I see issues with 'hdfs'. 2.) Some of the variables printed out(such as JAVA_HOME, HADOOP_COMMON_HOME and HADOOP_CONF_DIR) have absolute paths while some(such as HADOOP_COMMON_DIR, HADOOP_COMMON_LIB_JARS_DIR and HADOOP_COMMON_LIB_NATIVE_DIR) have relative paths. Do you think the difference matters or do we expect users to know what the individual variables are referring to? The rest of the patch looks good. > expose calculated paths > --- > > Key: HADOOP-12366 > URL: https://issues.apache.org/jira/browse/HADOOP-12366 > Project: Hadoop Common > Issue Type: New Feature >Affects Versions: 3.0.0 >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer > Attachments: HADOOP-12366.00.patch > > > It would be useful for 3rd party apps to know the locations of things when > hadoop is running without explicit path env vars set. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-10787) Rename/remove non-HADOOP_*, etc from the shell scripts
[
https://issues.apache.org/jira/browse/HADOOP-10787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14976118#comment-14976118
]
Varun Vasudev commented on HADOOP-10787:
Thank for the patches [~aw]. The latest patch doesn't apply to trunk. Can you
please take a look?
Also, mapred still has a reference to TOOL_PATH
{code}
archive-logs)
CLASS=org.apache.hadoop.tools.HadoopArchiveLogs
hadoop_debug "Injecting TOOL_PATH into CLASSPATH"
hadoop_add_classpath "${TOOL_PATH}"
hadoop_debug "Appending HADOOP_CLIENT_OPTS onto HADOOP_OPTS"
HADOOP_OPTS="${HADOOP_OPTS} ${HADOOP_CLIENT_OPTS}"
{code}
> Rename/remove non-HADOOP_*, etc from the shell scripts
> --
>
> Key: HADOOP-10787
> URL: https://issues.apache.org/jira/browse/HADOOP-10787
> Project: Hadoop Common
> Issue Type: Improvement
> Components: scripts
>Affects Versions: 3.0.0
>Reporter: Allen Wittenauer
>Assignee: Allen Wittenauer
>Priority: Blocker
> Labels: scripts
> Attachments: HADOOP-10787.00.patch, HADOOP-10787.01.patch,
> HADOOP-10787.02.patch
>
>
> We should make an effort to clean up the shell env var name space by removing
> unsafe variables. See comments for list.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-11872) hadoop dfs command prints message about using yarn jar on Windows(branch-2 only)
[
https://issues.apache.org/jira/browse/HADOOP-11872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Vasudev updated HADOOP-11872:
---
Status: Patch Available (was: Open)
hadoop dfs command prints message about using yarn jar on
Windows(branch-2 only)
Key: HADOOP-11872
URL: https://issues.apache.org/jira/browse/HADOOP-11872
Project: Hadoop Common
Issue Type: Bug
Components: scripts
Reporter: Varun Vasudev
Assignee: Varun Vasudev
Priority: Minor
Attachments: HADOOP-11872-branch-2.001.patch
Using the hadoop dfs command on a branch-2 build prints a message about
using yarn jar.
{noformat}
C:\hadoop\hadoop-common-project\hadoop-common\src\main\bin hadoop.cmd dfs -ls
note: please use yarn jar to launch
YARN applications, not this command.
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-11872) hadoop dfs command prints message about using yarn jar on Windows(branch-2 only)
[
https://issues.apache.org/jira/browse/HADOOP-11872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Vasudev updated HADOOP-11872:
---
Attachment: HADOOP-11872-branch-2.001.patch
Uploaded a patch with the fix.
hadoop dfs command prints message about using yarn jar on
Windows(branch-2 only)
Key: HADOOP-11872
URL: https://issues.apache.org/jira/browse/HADOOP-11872
Project: Hadoop Common
Issue Type: Bug
Components: scripts
Reporter: Varun Vasudev
Assignee: Varun Vasudev
Priority: Minor
Attachments: HADOOP-11872-branch-2.001.patch
Using the hadoop dfs command on a branch-2 build prints a message about
using yarn jar.
{noformat}
C:\hadoop\hadoop-common-project\hadoop-common\src\main\bin hadoop.cmd dfs -ls
note: please use yarn jar to launch
YARN applications, not this command.
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HADOOP-11872) hadoop dfs command prints message about using yarn jar on Windows(branch-2 only)
Varun Vasudev created HADOOP-11872:
--
Summary: hadoop dfs command prints message about using yarn
jar on Windows(branch-2 only)
Key: HADOOP-11872
URL: https://issues.apache.org/jira/browse/HADOOP-11872
Project: Hadoop Common
Issue Type: Bug
Components: scripts
Reporter: Varun Vasudev
Assignee: Varun Vasudev
Priority: Minor
Using the hadoop dfs command on a branch-2 build prints a message about using
yarn jar.
{noformat}
C:\hadoop\hadoop-common-project\hadoop-common\src\main\bin hadoop.cmd dfs -ls
note: please use yarn jar to launch
YARN applications, not this command.
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HADOOP-11253) Hadoop streaming test TestStreamXmlMultipleRecords fails on Windows
Varun Vasudev created HADOOP-11253:
--
Summary: Hadoop streaming test TestStreamXmlMultipleRecords fails
on Windows
Key: HADOOP-11253
URL: https://issues.apache.org/jira/browse/HADOOP-11253
Project: Hadoop Common
Issue Type: Test
Components: tools
Reporter: Varun Vasudev
Assignee: Varun Vasudev
All the tests in TestStreamXmlMultipleRecords fail on Windows with errors
similar to this -
{noformat}
java.lang.AssertionError: null
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at
org.apache.hadoop.streaming.TestStreaming.assertOutput(TestStreaming.java:177)
at
org.apache.hadoop.streaming.TestStreaming.checkOutput(TestStreaming.java:166)
at
org.apache.hadoop.streaming.TestStreaming.testCommandLine(TestStreaming.java:201)
at
org.apache.hadoop.streaming.TestStreamXmlMultipleRecords.testStreamXmlMultiInnerFast(TestStreamXmlMultipleRecords.java:113)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-11253) Hadoop streaming test TestStreamXmlMultipleRecords fails on Windows
[
https://issues.apache.org/jira/browse/HADOOP-11253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Vasudev updated HADOOP-11253:
---
Status: Patch Available (was: Open)
Hadoop streaming test TestStreamXmlMultipleRecords fails on Windows
---
Key: HADOOP-11253
URL: https://issues.apache.org/jira/browse/HADOOP-11253
Project: Hadoop Common
Issue Type: Test
Components: tools
Reporter: Varun Vasudev
Assignee: Varun Vasudev
Attachments: apache-hadoop-11253.0.patch
All the tests in TestStreamXmlMultipleRecords fail on Windows with errors
similar to this -
{noformat}
java.lang.AssertionError: null
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at
org.apache.hadoop.streaming.TestStreaming.assertOutput(TestStreaming.java:177)
at
org.apache.hadoop.streaming.TestStreaming.checkOutput(TestStreaming.java:166)
at
org.apache.hadoop.streaming.TestStreaming.testCommandLine(TestStreaming.java:201)
at
org.apache.hadoop.streaming.TestStreamXmlMultipleRecords.testStreamXmlMultiInnerFast(TestStreamXmlMultipleRecords.java:113)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11241) TestNMSimulator fails sometimes due to timing issue
[
https://issues.apache.org/jira/browse/HADOOP-11241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14188059#comment-14188059
]
Varun Vasudev commented on HADOOP-11241:
Thanks for the build fix [~cmccabe].
TestNMSimulator fails sometimes due to timing issue
---
Key: HADOOP-11241
URL: https://issues.apache.org/jira/browse/HADOOP-11241
Project: Hadoop Common
Issue Type: Bug
Reporter: Varun Vasudev
Assignee: Varun Vasudev
Attachments: apache-hadoop-11241.0.patch
TestNMSimulator fails sometimes due to timing issues. From a failure -
{noformat}
2014-10-16 23:21:42,343 INFO resourcemanager.ResourceTrackerService
(ResourceTrackerService.java:registerNodeManager(337)) - NodeManager from
node node1(cmPort: 0 httpPort: 80) registered with capability: memory:10240,
vCores:10, assigned nodeId node1:0
2014-10-16 23:21:42,397 ERROR delegation.AbstractDelegationTokenSecretManager
(AbstractDelegationTokenSecretManager.java:run(642)) - ExpiredTokenRemover
received java.lang.InterruptedException: sleep interrupted
2014-10-16 23:21:42,400 INFO rmnode.RMNodeImpl (RMNodeImpl.java:handle(423))
- node1:0 Node Transitioned from NEW to RUNNING
2014-10-16 23:21:42,404 INFO fair.FairScheduler
(FairScheduler.java:addNode(825)) - Added node node1:0 cluster capacity:
memory:10240, vCores:10
2014-10-16 23:21:42,407 INFO mortbay.log (Slf4jLog.java:info(67)) - Stopped
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:18088
2014-10-16 23:21:42,409 ERROR delegation.AbstractDelegationTokenSecretManager
(AbstractDelegationTokenSecretManager.java:run(642)) - ExpiredTokenRemover
received java.lang.InterruptedException: sleep interrupted
2014-10-16 23:21:42,410 INFO ipc.Server (Server.java:stop(2437)) - Stopping
server on 18032
2014-10-16 23:21:42,412 INFO ipc.Server (Server.java:run(706)) - Stopping
IPC Server listener on 18032
2014-10-16 23:21:42,412 INFO ipc.Server (Server.java:run(832)) - Stopping
IPC Server Responder
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-11241) TestNMSimulator fails sometimes due to timing issue
[
https://issues.apache.org/jira/browse/HADOOP-11241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Vasudev updated HADOOP-11241:
---
Status: Open (was: Patch Available)
TestNMSimulator fails sometimes due to timing issue
---
Key: HADOOP-11241
URL: https://issues.apache.org/jira/browse/HADOOP-11241
Project: Hadoop Common
Issue Type: Bug
Reporter: Varun Vasudev
Assignee: Varun Vasudev
Attachments: apache-yarn-2763.0.patch
TestNMSimulator fails sometimes due to timing issues. From a failure -
{noformat}
2014-10-16 23:21:42,343 INFO resourcemanager.ResourceTrackerService
(ResourceTrackerService.java:registerNodeManager(337)) - NodeManager from
node node1(cmPort: 0 httpPort: 80) registered with capability: memory:10240,
vCores:10, assigned nodeId node1:0
2014-10-16 23:21:42,397 ERROR delegation.AbstractDelegationTokenSecretManager
(AbstractDelegationTokenSecretManager.java:run(642)) - ExpiredTokenRemover
received java.lang.InterruptedException: sleep interrupted
2014-10-16 23:21:42,400 INFO rmnode.RMNodeImpl (RMNodeImpl.java:handle(423))
- node1:0 Node Transitioned from NEW to RUNNING
2014-10-16 23:21:42,404 INFO fair.FairScheduler
(FairScheduler.java:addNode(825)) - Added node node1:0 cluster capacity:
memory:10240, vCores:10
2014-10-16 23:21:42,407 INFO mortbay.log (Slf4jLog.java:info(67)) - Stopped
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:18088
2014-10-16 23:21:42,409 ERROR delegation.AbstractDelegationTokenSecretManager
(AbstractDelegationTokenSecretManager.java:run(642)) - ExpiredTokenRemover
received java.lang.InterruptedException: sleep interrupted
2014-10-16 23:21:42,410 INFO ipc.Server (Server.java:stop(2437)) - Stopping
server on 18032
2014-10-16 23:21:42,412 INFO ipc.Server (Server.java:run(706)) - Stopping
IPC Server listener on 18032
2014-10-16 23:21:42,412 INFO ipc.Server (Server.java:run(832)) - Stopping
IPC Server Responder
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Moved] (HADOOP-11241) TestNMSimulator fails sometimes due to timing issue
[
https://issues.apache.org/jira/browse/HADOOP-11241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Vasudev moved YARN-2763 to HADOOP-11241:
--
Key: HADOOP-11241 (was: YARN-2763)
Project: Hadoop Common (was: Hadoop YARN)
TestNMSimulator fails sometimes due to timing issue
---
Key: HADOOP-11241
URL: https://issues.apache.org/jira/browse/HADOOP-11241
Project: Hadoop Common
Issue Type: Bug
Reporter: Varun Vasudev
Assignee: Varun Vasudev
Attachments: apache-yarn-2763.0.patch
TestNMSimulator fails sometimes due to timing issues. From a failure -
{noformat}
2014-10-16 23:21:42,343 INFO resourcemanager.ResourceTrackerService
(ResourceTrackerService.java:registerNodeManager(337)) - NodeManager from
node node1(cmPort: 0 httpPort: 80) registered with capability: memory:10240,
vCores:10, assigned nodeId node1:0
2014-10-16 23:21:42,397 ERROR delegation.AbstractDelegationTokenSecretManager
(AbstractDelegationTokenSecretManager.java:run(642)) - ExpiredTokenRemover
received java.lang.InterruptedException: sleep interrupted
2014-10-16 23:21:42,400 INFO rmnode.RMNodeImpl (RMNodeImpl.java:handle(423))
- node1:0 Node Transitioned from NEW to RUNNING
2014-10-16 23:21:42,404 INFO fair.FairScheduler
(FairScheduler.java:addNode(825)) - Added node node1:0 cluster capacity:
memory:10240, vCores:10
2014-10-16 23:21:42,407 INFO mortbay.log (Slf4jLog.java:info(67)) - Stopped
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:18088
2014-10-16 23:21:42,409 ERROR delegation.AbstractDelegationTokenSecretManager
(AbstractDelegationTokenSecretManager.java:run(642)) - ExpiredTokenRemover
received java.lang.InterruptedException: sleep interrupted
2014-10-16 23:21:42,410 INFO ipc.Server (Server.java:stop(2437)) - Stopping
server on 18032
2014-10-16 23:21:42,412 INFO ipc.Server (Server.java:run(706)) - Stopping
IPC Server listener on 18032
2014-10-16 23:21:42,412 INFO ipc.Server (Server.java:run(832)) - Stopping
IPC Server Responder
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-11241) TestNMSimulator fails sometimes due to timing issue
[
https://issues.apache.org/jira/browse/HADOOP-11241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Vasudev updated HADOOP-11241:
---
Status: Patch Available (was: Open)
TestNMSimulator fails sometimes due to timing issue
---
Key: HADOOP-11241
URL: https://issues.apache.org/jira/browse/HADOOP-11241
Project: Hadoop Common
Issue Type: Bug
Reporter: Varun Vasudev
Assignee: Varun Vasudev
Attachments: apache-yarn-2763.0.patch
TestNMSimulator fails sometimes due to timing issues. From a failure -
{noformat}
2014-10-16 23:21:42,343 INFO resourcemanager.ResourceTrackerService
(ResourceTrackerService.java:registerNodeManager(337)) - NodeManager from
node node1(cmPort: 0 httpPort: 80) registered with capability: memory:10240,
vCores:10, assigned nodeId node1:0
2014-10-16 23:21:42,397 ERROR delegation.AbstractDelegationTokenSecretManager
(AbstractDelegationTokenSecretManager.java:run(642)) - ExpiredTokenRemover
received java.lang.InterruptedException: sleep interrupted
2014-10-16 23:21:42,400 INFO rmnode.RMNodeImpl (RMNodeImpl.java:handle(423))
- node1:0 Node Transitioned from NEW to RUNNING
2014-10-16 23:21:42,404 INFO fair.FairScheduler
(FairScheduler.java:addNode(825)) - Added node node1:0 cluster capacity:
memory:10240, vCores:10
2014-10-16 23:21:42,407 INFO mortbay.log (Slf4jLog.java:info(67)) - Stopped
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:18088
2014-10-16 23:21:42,409 ERROR delegation.AbstractDelegationTokenSecretManager
(AbstractDelegationTokenSecretManager.java:run(642)) - ExpiredTokenRemover
received java.lang.InterruptedException: sleep interrupted
2014-10-16 23:21:42,410 INFO ipc.Server (Server.java:stop(2437)) - Stopping
server on 18032
2014-10-16 23:21:42,412 INFO ipc.Server (Server.java:run(706)) - Stopping
IPC Server listener on 18032
2014-10-16 23:21:42,412 INFO ipc.Server (Server.java:run(832)) - Stopping
IPC Server Responder
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-11241) TestNMSimulator fails sometimes due to timing issue
[
https://issues.apache.org/jira/browse/HADOOP-11241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Vasudev updated HADOOP-11241:
---
Attachment: (was: apache-yarn-2763.0.patch)
TestNMSimulator fails sometimes due to timing issue
---
Key: HADOOP-11241
URL: https://issues.apache.org/jira/browse/HADOOP-11241
Project: Hadoop Common
Issue Type: Bug
Reporter: Varun Vasudev
Assignee: Varun Vasudev
TestNMSimulator fails sometimes due to timing issues. From a failure -
{noformat}
2014-10-16 23:21:42,343 INFO resourcemanager.ResourceTrackerService
(ResourceTrackerService.java:registerNodeManager(337)) - NodeManager from
node node1(cmPort: 0 httpPort: 80) registered with capability: memory:10240,
vCores:10, assigned nodeId node1:0
2014-10-16 23:21:42,397 ERROR delegation.AbstractDelegationTokenSecretManager
(AbstractDelegationTokenSecretManager.java:run(642)) - ExpiredTokenRemover
received java.lang.InterruptedException: sleep interrupted
2014-10-16 23:21:42,400 INFO rmnode.RMNodeImpl (RMNodeImpl.java:handle(423))
- node1:0 Node Transitioned from NEW to RUNNING
2014-10-16 23:21:42,404 INFO fair.FairScheduler
(FairScheduler.java:addNode(825)) - Added node node1:0 cluster capacity:
memory:10240, vCores:10
2014-10-16 23:21:42,407 INFO mortbay.log (Slf4jLog.java:info(67)) - Stopped
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:18088
2014-10-16 23:21:42,409 ERROR delegation.AbstractDelegationTokenSecretManager
(AbstractDelegationTokenSecretManager.java:run(642)) - ExpiredTokenRemover
received java.lang.InterruptedException: sleep interrupted
2014-10-16 23:21:42,410 INFO ipc.Server (Server.java:stop(2437)) - Stopping
server on 18032
2014-10-16 23:21:42,412 INFO ipc.Server (Server.java:run(706)) - Stopping
IPC Server listener on 18032
2014-10-16 23:21:42,412 INFO ipc.Server (Server.java:run(832)) - Stopping
IPC Server Responder
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-11241) TestNMSimulator fails sometimes due to timing issue
[
https://issues.apache.org/jira/browse/HADOOP-11241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Vasudev updated HADOOP-11241:
---
Status: Patch Available (was: Open)
TestNMSimulator fails sometimes due to timing issue
---
Key: HADOOP-11241
URL: https://issues.apache.org/jira/browse/HADOOP-11241
Project: Hadoop Common
Issue Type: Bug
Reporter: Varun Vasudev
Assignee: Varun Vasudev
Attachments: apache-hadoop-11241.0.patch
TestNMSimulator fails sometimes due to timing issues. From a failure -
{noformat}
2014-10-16 23:21:42,343 INFO resourcemanager.ResourceTrackerService
(ResourceTrackerService.java:registerNodeManager(337)) - NodeManager from
node node1(cmPort: 0 httpPort: 80) registered with capability: memory:10240,
vCores:10, assigned nodeId node1:0
2014-10-16 23:21:42,397 ERROR delegation.AbstractDelegationTokenSecretManager
(AbstractDelegationTokenSecretManager.java:run(642)) - ExpiredTokenRemover
received java.lang.InterruptedException: sleep interrupted
2014-10-16 23:21:42,400 INFO rmnode.RMNodeImpl (RMNodeImpl.java:handle(423))
- node1:0 Node Transitioned from NEW to RUNNING
2014-10-16 23:21:42,404 INFO fair.FairScheduler
(FairScheduler.java:addNode(825)) - Added node node1:0 cluster capacity:
memory:10240, vCores:10
2014-10-16 23:21:42,407 INFO mortbay.log (Slf4jLog.java:info(67)) - Stopped
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:18088
2014-10-16 23:21:42,409 ERROR delegation.AbstractDelegationTokenSecretManager
(AbstractDelegationTokenSecretManager.java:run(642)) - ExpiredTokenRemover
received java.lang.InterruptedException: sleep interrupted
2014-10-16 23:21:42,410 INFO ipc.Server (Server.java:stop(2437)) - Stopping
server on 18032
2014-10-16 23:21:42,412 INFO ipc.Server (Server.java:run(706)) - Stopping
IPC Server listener on 18032
2014-10-16 23:21:42,412 INFO ipc.Server (Server.java:run(832)) - Stopping
IPC Server Responder
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-11241) TestNMSimulator fails sometimes due to timing issue
[
https://issues.apache.org/jira/browse/HADOOP-11241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Vasudev updated HADOOP-11241:
---
Status: Open (was: Patch Available)
TestNMSimulator fails sometimes due to timing issue
---
Key: HADOOP-11241
URL: https://issues.apache.org/jira/browse/HADOOP-11241
Project: Hadoop Common
Issue Type: Bug
Reporter: Varun Vasudev
Assignee: Varun Vasudev
Attachments: apache-hadoop-11241.0.patch
TestNMSimulator fails sometimes due to timing issues. From a failure -
{noformat}
2014-10-16 23:21:42,343 INFO resourcemanager.ResourceTrackerService
(ResourceTrackerService.java:registerNodeManager(337)) - NodeManager from
node node1(cmPort: 0 httpPort: 80) registered with capability: memory:10240,
vCores:10, assigned nodeId node1:0
2014-10-16 23:21:42,397 ERROR delegation.AbstractDelegationTokenSecretManager
(AbstractDelegationTokenSecretManager.java:run(642)) - ExpiredTokenRemover
received java.lang.InterruptedException: sleep interrupted
2014-10-16 23:21:42,400 INFO rmnode.RMNodeImpl (RMNodeImpl.java:handle(423))
- node1:0 Node Transitioned from NEW to RUNNING
2014-10-16 23:21:42,404 INFO fair.FairScheduler
(FairScheduler.java:addNode(825)) - Added node node1:0 cluster capacity:
memory:10240, vCores:10
2014-10-16 23:21:42,407 INFO mortbay.log (Slf4jLog.java:info(67)) - Stopped
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:18088
2014-10-16 23:21:42,409 ERROR delegation.AbstractDelegationTokenSecretManager
(AbstractDelegationTokenSecretManager.java:run(642)) - ExpiredTokenRemover
received java.lang.InterruptedException: sleep interrupted
2014-10-16 23:21:42,410 INFO ipc.Server (Server.java:stop(2437)) - Stopping
server on 18032
2014-10-16 23:21:42,412 INFO ipc.Server (Server.java:run(706)) - Stopping
IPC Server listener on 18032
2014-10-16 23:21:42,412 INFO ipc.Server (Server.java:run(832)) - Stopping
IPC Server Responder
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-11241) TestNMSimulator fails sometimes due to timing issue
[
https://issues.apache.org/jira/browse/HADOOP-11241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Vasudev updated HADOOP-11241:
---
Attachment: apache-hadoop-11241.0.patch
Renaming patch file
TestNMSimulator fails sometimes due to timing issue
---
Key: HADOOP-11241
URL: https://issues.apache.org/jira/browse/HADOOP-11241
Project: Hadoop Common
Issue Type: Bug
Reporter: Varun Vasudev
Assignee: Varun Vasudev
Attachments: apache-hadoop-11241.0.patch
TestNMSimulator fails sometimes due to timing issues. From a failure -
{noformat}
2014-10-16 23:21:42,343 INFO resourcemanager.ResourceTrackerService
(ResourceTrackerService.java:registerNodeManager(337)) - NodeManager from
node node1(cmPort: 0 httpPort: 80) registered with capability: memory:10240,
vCores:10, assigned nodeId node1:0
2014-10-16 23:21:42,397 ERROR delegation.AbstractDelegationTokenSecretManager
(AbstractDelegationTokenSecretManager.java:run(642)) - ExpiredTokenRemover
received java.lang.InterruptedException: sleep interrupted
2014-10-16 23:21:42,400 INFO rmnode.RMNodeImpl (RMNodeImpl.java:handle(423))
- node1:0 Node Transitioned from NEW to RUNNING
2014-10-16 23:21:42,404 INFO fair.FairScheduler
(FairScheduler.java:addNode(825)) - Added node node1:0 cluster capacity:
memory:10240, vCores:10
2014-10-16 23:21:42,407 INFO mortbay.log (Slf4jLog.java:info(67)) - Stopped
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:18088
2014-10-16 23:21:42,409 ERROR delegation.AbstractDelegationTokenSecretManager
(AbstractDelegationTokenSecretManager.java:run(642)) - ExpiredTokenRemover
received java.lang.InterruptedException: sleep interrupted
2014-10-16 23:21:42,410 INFO ipc.Server (Server.java:stop(2437)) - Stopping
server on 18032
2014-10-16 23:21:42,412 INFO ipc.Server (Server.java:run(706)) - Stopping
IPC Server listener on 18032
2014-10-16 23:21:42,412 INFO ipc.Server (Server.java:run(832)) - Stopping
IPC Server Responder
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11181) o.a.h.security.token.delegation.DelegationTokenManager should be more generalized to handle other DelegationTokenIdentifier
[
https://issues.apache.org/jira/browse/HADOOP-11181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14169861#comment-14169861
]
Varun Vasudev commented on HADOOP-11181:
Patch looks good to me.
o.a.h.security.token.delegation.DelegationTokenManager should be more
generalized to handle other DelegationTokenIdentifier
---
Key: HADOOP-11181
URL: https://issues.apache.org/jira/browse/HADOOP-11181
Project: Hadoop Common
Issue Type: Bug
Components: security
Reporter: Zhijie Shen
Assignee: Zhijie Shen
Attachments: HADOOP-11181.1.patch
While DelegationTokenManager can set external secretManager, it have the
assumption that the token is going to be
o.a.h.security.token.delegation.DelegationTokenIdentifier, and use
DelegationTokenIdentifier method to decode a token.
{code}
@SuppressWarnings(unchecked)
public UserGroupInformation verifyToken(TokenDelegationTokenIdentifier
token) throws IOException {
ByteArrayInputStream buf = new
ByteArrayInputStream(token.getIdentifier());
DataInputStream dis = new DataInputStream(buf);
DelegationTokenIdentifier id = new DelegationTokenIdentifier(tokenKind);
id.readFields(dis);
dis.close();
secretManager.verifyToken(id, token.getPassword());
return id.getUser();
}
{code}
It's not going to work it the token kind is other than
web.DelegationTokenIdentifier. For example, RM want to reuse it but hook it
to RMDelegationTokenSecretManager and RMDelegationTokenIdentifier, which has
the customized way to decode a token.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HADOOP-10832) Add support for passing delegation tokens via headers for web services
[ https://issues.apache.org/jira/browse/HADOOP-10832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Vasudev resolved HADOOP-10832. Resolution: Duplicate Duplicate of HADOOP-10880 Add support for passing delegation tokens via headers for web services -- Key: HADOOP-10832 URL: https://issues.apache.org/jira/browse/HADOOP-10832 Project: Hadoop Common Issue Type: Task Reporter: Varun Vasudev Assignee: Varun Vasudev HADOOP-10799 refactors the WebHDFS code to handle delegation tokens a part of hadoop-common. We should add support to pass delegation tokens as a header instead of passing it as part of the url. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HADOOP-10799) Refactor HTTP delegation token logic from httpfs into reusable code in hadoop-common.
[ https://issues.apache.org/jira/browse/HADOOP-10799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068429#comment-14068429 ] Varun Vasudev commented on HADOOP-10799: [~tucu00] I'm fine with that. I've filed YARN-2290 for 3. Refactor HTTP delegation token logic from httpfs into reusable code in hadoop-common. - Key: HADOOP-10799 URL: https://issues.apache.org/jira/browse/HADOOP-10799 Project: Hadoop Common Issue Type: Sub-task Components: security Affects Versions: 3.0.0 Reporter: Alejandro Abdelnur Assignee: Alejandro Abdelnur Attachments: HADOOP-10799.patch, HADOOP-10799.patch, HADOOP-10799.patch, HADOOP-10799.patch, HADOOP-10799.patch, HADOOP-10799.patch, HADOOP-10799.patch, HADOOP-10799.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Moved] (HADOOP-10832) Add support for passing delegation tokens via headers for web services
[ https://issues.apache.org/jira/browse/HADOOP-10832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Vasudev moved YARN-2290 to HADOOP-10832: -- Assignee: (was: Varun Vasudev) Issue Type: Task (was: Bug) Key: HADOOP-10832 (was: YARN-2290) Project: Hadoop Common (was: Hadoop YARN) Add support for passing delegation tokens via headers for web services -- Key: HADOOP-10832 URL: https://issues.apache.org/jira/browse/HADOOP-10832 Project: Hadoop Common Issue Type: Task Reporter: Varun Vasudev HADOOP-10799 refactors the WebHDFS code to handle delegation tokens a part of hadoop-common. We should add support to pass delegation tokens as a header instead of passing it as part of the url. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (HADOOP-10832) Add support for passing delegation tokens via headers for web services
[ https://issues.apache.org/jira/browse/HADOOP-10832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Vasudev reassigned HADOOP-10832: -- Assignee: Varun Vasudev Add support for passing delegation tokens via headers for web services -- Key: HADOOP-10832 URL: https://issues.apache.org/jira/browse/HADOOP-10832 Project: Hadoop Common Issue Type: Task Reporter: Varun Vasudev Assignee: Varun Vasudev HADOOP-10799 refactors the WebHDFS code to handle delegation tokens a part of hadoop-common. We should add support to pass delegation tokens as a header instead of passing it as part of the url. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HADOOP-10832) Add support for passing delegation tokens via headers for web services
[
https://issues.apache.org/jira/browse/HADOOP-10832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062362#comment-14062362
]
Varun Vasudev commented on HADOOP-10832:
The reason for the
change(https://issues.apache.org/jira/browse/HADOOP-10799?focusedCommentId=14059272page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14059272):
{quote}
Alejandro Abdelnur I'm not sure if this is the right ticket but it would be
preferable to pass tokens as headers instead of URL parameters. URLs can get
logged and passed on as part of the referrer header which exposes the
delegation token. In addition, users can pass around links with delegation
tokens by mistake. YARN-2247(waiting to be reviewed) also implements auth using
delegation tokens for the RM web services but passes the tokens as a header. My
plan was to file a ticket to shift the TimelineServer auth to the header model
once YARN-2247 got committed. I'd be happy to hear your thoughts.
{quote}
Add support for passing delegation tokens via headers for web services
--
Key: HADOOP-10832
URL: https://issues.apache.org/jira/browse/HADOOP-10832
Project: Hadoop Common
Issue Type: Task
Reporter: Varun Vasudev
Assignee: Varun Vasudev
HADOOP-10799 refactors the WebHDFS code to handle delegation tokens a part of
hadoop-common. We should add support to pass delegation tokens as a header
instead of passing it as part of the url.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HADOOP-10799) Refactor HTTP delegation token logic from httpfs into reusable code in hadoop-common.
[ https://issues.apache.org/jira/browse/HADOOP-10799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14059272#comment-14059272 ] Varun Vasudev commented on HADOOP-10799: [~tucu00] I'm not sure if this is the right ticket but it would be preferable to pass tokens as headers instead of URL parameters. URLs can get logged and passed on as part of the referrer header which exposes the delegation token. In addition, users can pass around links with delegation tokens by mistake. YARN-2247(waiting to be reviewed) also implements auth using delegation tokens for the RM web services but passes the tokens as a header. My plan was to file a ticket to shift the TimelineServer auth to the header model once YARN-2247 got committed. I'd be happy to hear your thoughts. Refactor HTTP delegation token logic from httpfs into reusable code in hadoop-common. - Key: HADOOP-10799 URL: https://issues.apache.org/jira/browse/HADOOP-10799 Project: Hadoop Common Issue Type: Sub-task Components: security Affects Versions: 3.0.0 Reporter: Alejandro Abdelnur Assignee: Alejandro Abdelnur Attachments: HADOOP-10799.patch, HADOOP-10799.patch, HADOOP-10799.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HADOOP-10799) Refactor HTTP delegation token logic from httpfs into reusable code in hadoop-common.
[ https://issues.apache.org/jira/browse/HADOOP-10799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14059274#comment-14059274 ] Varun Vasudev commented on HADOOP-10799: I forgot to mention, there's a lot of similar code between YARN-2247 and YARN-2100. Refactor HTTP delegation token logic from httpfs into reusable code in hadoop-common. - Key: HADOOP-10799 URL: https://issues.apache.org/jira/browse/HADOOP-10799 Project: Hadoop Common Issue Type: Sub-task Components: security Affects Versions: 3.0.0 Reporter: Alejandro Abdelnur Assignee: Alejandro Abdelnur Attachments: HADOOP-10799.patch, HADOOP-10799.patch, HADOOP-10799.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HADOOP-10799) Refactor HTTP delegation token logic from httpfs into reusable code in hadoop-common.
[ https://issues.apache.org/jira/browse/HADOOP-10799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14059276#comment-14059276 ] Varun Vasudev commented on HADOOP-10799: Sorry I hit send too soon, I meant similar code between YARN-2247 and the current TimelineServer delegation token auth. Refactor HTTP delegation token logic from httpfs into reusable code in hadoop-common. - Key: HADOOP-10799 URL: https://issues.apache.org/jira/browse/HADOOP-10799 Project: Hadoop Common Issue Type: Sub-task Components: security Affects Versions: 3.0.0 Reporter: Alejandro Abdelnur Assignee: Alejandro Abdelnur Attachments: HADOOP-10799.patch, HADOOP-10799.patch, HADOOP-10799.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HADOOP-10799) Refactor HTTP delegation token logic from httpfs into reusable code in hadoop-common.
[ https://issues.apache.org/jira/browse/HADOOP-10799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056473#comment-14056473 ] Varun Vasudev commented on HADOOP-10799: Sorry I missed the parent ticket. My apologies. Refactor HTTP delegation token logic from httpfs into reusable code in hadoop-common. - Key: HADOOP-10799 URL: https://issues.apache.org/jira/browse/HADOOP-10799 Project: Hadoop Common Issue Type: Sub-task Components: security Affects Versions: 3.0.0 Reporter: Alejandro Abdelnur Assignee: Alejandro Abdelnur Attachments: HADOOP-10799.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HADOOP-10799) Refactor HTTP delegation token logic from httpfs into reusable code in hadoop-common.
[ https://issues.apache.org/jira/browse/HADOOP-10799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14055860#comment-14055860 ] Varun Vasudev commented on HADOOP-10799: [~tucu00] just curious - any particular reason for this or simply refactoring? Refactor HTTP delegation token logic from httpfs into reusable code in hadoop-common. - Key: HADOOP-10799 URL: https://issues.apache.org/jira/browse/HADOOP-10799 Project: Hadoop Common Issue Type: Sub-task Components: security Affects Versions: 3.0.0 Reporter: Alejandro Abdelnur Assignee: Alejandro Abdelnur Attachments: HADOOP-10799.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
