Getting close to a vote on a merge of S3Guard., HADOOP-13345

2017-08-16 Thread Steve Loughran
FYI, We're getting ready for a patch to merge the current S3Guard branch, HADOOP-13345, via a patch https://issues.apache.org/jira/browse/HADOOP-13998 After that's done, we do plan to have a second iteration, work on a 0-rename committer (HADOOP-13786) with all the other tuning and improvements

[jira] [Created] (HADOOP-14777) S3Guard premerge changes: java 7 build & test tuning

2017-08-16 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-14777: --- Summary: S3Guard premerge changes: java 7 build & test tuning Key: HADOOP-14777 URL: https://issues.apache.org/jira/browse/HADOOP-14777 Project: Hadoop Common

[jira] [Created] (HADOOP-14778) AzureNativeFileSystemStore.getDirectorySet() to use getTrimmedStrings to get directory listings

2017-08-16 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-14778: --- Summary: AzureNativeFileSystemStore.getDirectorySet() to use getTrimmedStrings to get directory listings Key: HADOOP-14778 URL: https://issues.apache.org/jira/browse/HADOOP-

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

2017-08-16 Thread Apache Jenkins Server
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/494/ [Aug 15, 2017 8:48:49 AM] (yqlin) HDFS-11696. Fix warnings from Spotbugs in hadoop-hdfs. Contributed by [Aug 15, 2017 2:38:43 PM] (weichiu) HDFS-12054. FSNamesystem#addErasureCodingPolicies should call [Au

[DISCUSS] Merging YARN-5355 (Timeline Service v.2) to trunk

2017-08-16 Thread Vrushali Channapattan
Hi All I’d like to open a discussion for merging Timeline Service v.2 (YARN-5355) to trunk in a few weeks. We have previously completed one merge onto trunk [1] and Timeline Service v2 has been part of Hadoop release 3.0.0-alpha1. Since then, we have been working on extending the capabilities of

[jira] [Created] (HADOOP-14779) Refactor decryptEncryptedKey in KeyProviderCryptoExtension

2017-08-16 Thread Xiao Chen (JIRA)
Xiao Chen created HADOOP-14779: -- Summary: Refactor decryptEncryptedKey in KeyProviderCryptoExtension Key: HADOOP-14779 URL: https://issues.apache.org/jira/browse/HADOOP-14779 Project: Hadoop Common

Re: [DISCUSS] Merging YARN-5355 (Timeline Service v.2) to trunk

2017-08-16 Thread Vrushali Channapattan
Looks like some of the hyperlinks appear messed up, my apologies, resending the same email with hopefully better looking content: Hi All, I'd like to open a discussion for merging Timeline Service v2 (YARN-5355) to trunk in a few weeks. We have previously completed one merge onto trunk [1] and T

Re: [DISCUSS] Merge yarn-native-services branch into trunk

2017-08-16 Thread Andrew Wang
Hi Jian, Hadoop 3.0.0-beta1 is planned for mid-September. If the plan is to merge in hopefully the next two weeks, that's very, very close to the goal release date. We've already got a pile of blockers and criticals to resolve before then. Could you comment on testing and API stability for this b

Re: Getting close to a vote on a merge of S3Guard., HADOOP-13345

2017-08-16 Thread Andrew Wang
Hi Steve, What's the target release vehicle, and the timeline for merging this? The target date for beta1 is mid-September, so any large code movements make me nervous. Could you comment on testing and API stability of this branch? I'm trusting the judgement of the contributors involved, since th

Re: [DISCUSS] Merging YARN-5355 (Timeline Service v.2) to trunk

2017-08-16 Thread Andrew Wang
Hi Vrushali, Glad to hear this major dev milestone is nearing completion! Repeating my request on other merge [DISCUSS] threads, could you comment on testing and API stability of this merge? Our timeline for beta1 is about a month out, so there's not much time to fix things beforehand. Looking a

Re: [DISCUSS] Merging YARN-5355 (Timeline Service v.2) to trunk

2017-08-16 Thread Vrushali Channapattan
Hi Andrew, Thanks for your response! There have been no changes to existing APIs since alpha1. We at Twitter have tested the feature to demonstrate it works at what we consider moderate scale but this did not include the security related testing. The security testing is in progress at present by

Re: [DISCUSS] Merging YARN-5355 (Timeline Service v.2) to trunk

2017-08-16 Thread Andrew Wang
Great, thanks Vrushali! Sounds good to me. I have a few procedural release notes comments I'll put on YARN-5355, to make sure we advertise this to our users appropriately. On Wed, Aug 16, 2017 at 11:32 AM, Vrushali Channapattan < vrushal...@gmail.com> wrote: > Hi Andrew, > > Thanks for your resp

[jira] [Created] (HADOOP-14780) Investigate and move shared resources to member variables on DefaultCryptoExtension

2017-08-16 Thread Xiao Chen (JIRA)
Xiao Chen created HADOOP-14780: -- Summary: Investigate and move shared resources to member variables on DefaultCryptoExtension Key: HADOOP-14780 URL: https://issues.apache.org/jira/browse/HADOOP-14780 Pro

Re: Getting close to a vote on a merge of S3Guard., HADOOP-13345

2017-08-16 Thread Aaron Fabbri
On Wed, Aug 16, 2017 at 1:39 PM, Andrew Wang wrote: > Hi Steve, > > What's the target release vehicle, and the timeline for merging this? The > target date for beta1 is mid-September, so any large code movements make me > nervous. > I think this is ready to get in before beta1. Most of upstream

[jira] [Created] (HADOOP-14781) Clarify that HADOOP_CONF_DIR shouldn't actually be set in hadoop-env.sh

2017-08-16 Thread Allen Wittenauer (JIRA)
Allen Wittenauer created HADOOP-14781: - Summary: Clarify that HADOOP_CONF_DIR shouldn't actually be set in hadoop-env.sh Key: HADOOP-14781 URL: https://issues.apache.org/jira/browse/HADOOP-14781 P

[jira] [Created] (HADOOP-14782) Added sw_64 CPU architecture

2017-08-16 Thread Xing (JIRA)
Xing created HADOOP-14782: - Summary: Added sw_64 CPU architecture Key: HADOOP-14782 URL: https://issues.apache.org/jira/browse/HADOOP-14782 Project: Hadoop Common Issue Type: Improvement Env

Re: HADOOP-14163 proposal for new hadoop.apache.org

2017-08-16 Thread Akira Ajisaka
Thanks a lot! First I will work on the 1. and 2. in HADOOP-14163. On 2017/08/09 5:03, Allen Wittenauer wrote: On Aug 8, 2017, at 12:36 AM, Akira Ajisaka wrote: Now I'm okay with not creating another repo. I'm thinking the following procedures may work: 1. Create ./asf-site directory 2. Add

Re: Getting close to a vote on a merge of S3Guard., HADOOP-13345

2017-08-16 Thread Andrew Wang
Thanks for the detailed explanation Aaron. Given that this has gone through Cloudera's QA cycle and is run in production, that adds a lot of confidence in the feature. Looking forward to having this in 3.0.0-beta1! Best, Andrew On Wed, Aug 16, 2017 at 2:17 PM, Aaron Fabbri wrote: > > > On Wed,