Apache Hadoop qbt Report: trunk+JDK9 on Linux/x86

For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java9-linux-x86/2/ [Oct 20, 2017 9:24:17 PM] (haibochen) YARN-7372. [Oct 20, 2017 9:27:04 PM] (stevel) HADOOP-14942. DistCp#cleanup() should check whether jobFS is null. [Oct 20, 2017 10:54:15 PM] (wangda) YARN-7318. Fix shell

Re: [DISCUSS] Feature Branch Merge and Security Audits

Hi Eric - Thanks for the additional item suggestions! "We might want to start a security section for Hadoop wiki for each of the services and components. This helps to track what has been completed." Do you mean to keep the audit checklist for each service and component there? Interesting idea,

2.9.0 status update (10/20/2017)

Today was the feature freeze date and we are glad to inform that all the major planned features are merged in branch-2: https://cwiki.apache.org/confluence/display/HADOOP/Roadmap#Roadmap-Plannedfeatures : Kudos to everyone who pulled through multiple blockers and made this happen. Special

Re: [DISCUSS] Feature Branch Merge and Security Audits

The check list looks good. Some more items to add: Kerberos TGT renewal SPNEGO support Delegation token Proxy User ACL CVE tracking list We might want to start a security section for Hadoop wiki for each of the services and components. This helps to track what has been completed. How

[jira] [Resolved] (HADOOP-14961) Docker failed to build yetus/hadoop:0de40f0: Oracle JDK 8 is NOT installed

[ https://issues.apache.org/jira/browse/HADOOP-14961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HADOOP-14961. --- Resolution: Fixed > Docker failed to build yetus/hadoop:0de40f0: Oracle JDK 8 is

[jira] [Created] (HADOOP-14971) Merge S3A committers into trunk

Steve Loughran created HADOOP-14971: --- Summary: Merge S3A committers into trunk Key: HADOOP-14971 URL: https://issues.apache.org/jira/browse/HADOOP-14971 Project: Hadoop Common Issue Type:

Re: 答复: [DISCUSSION] Merging HDFS-7240 Object Store (Ozone) to trunk

All - I broke this list of questions out into a separate DISCUSS thread where we can iterate over how a security audit process at merge time might look and whether it is even something that we want to take on. I will try and continue discussion on that thread and drive that to some conclusion

Re: [VOTE] Release Apache Hadoop 2.8.2 (RC1)

Thanks, Junping. +1 (non-binding) - Build from hadoop-2.8.2-src.tar.gz (skipped test) and also tried hadoop-2.8.2.tar.gz - Setup a 30-node HDFS cluster - Run some basic HDFS/webhdfs commands (read/write/mkdirs/listStatus/getFileInfo) - Check NN Web UI and operations through the UI - Run a simple

Re: [VOTE] Release Apache Hadoop 2.8.2 (RC1)

Hi Junping, Thanks for preparing the 2.8.2-RC1 release. Verified the following: - Built from source on Mac OS X 10.11.6 with Java 1.7.0_79 - Deployed binary to a 3-node docker cluster - Sanity checks - Basic dfs operations - MapReduce Wordcount & Grep +1 (non-binding)

2017-10-20 Hadoop 3 release status update

https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3+release+status+updates 2017-10-20 Apologies for skipping the update last week. Here's how we're tracking for GA. Highlights: - Merge of HDFS router-based federation and API-based scheduler configuration with no reported

Re: [DISCUSS] Feature Branch Merge and Security Audits

Adding security@hadoop list as well... On Fri, Oct 20, 2017 at 2:29 PM, larry mccay wrote: > All - > > Given the maturity of Hadoop at this point, I would like to propose that > we start doing explicit security audits of features at merge time. > > There are a few reasons

[DISCUSS] Feature Branch Merge and Security Audits

All - Given the maturity of Hadoop at this point, I would like to propose that we start doing explicit security audits of features at merge time. There are a few reasons that I think this is a good place/time to do the review: 1. It represents a specific snapshot of where the feature stands as

[jira] [Created] (HADOOP-14970) MiniHadoopClusterManager doesn't respect lack of format option

Erik Krogen created HADOOP-14970: Summary: MiniHadoopClusterManager doesn't respect lack of format option Key: HADOOP-14970 URL: https://issues.apache.org/jira/browse/HADOOP-14970 Project: Hadoop

Re: 答复: [DISCUSSION] Merging HDFS-7240 Object Store (Ozone) to trunk

I previously sent this same email from my work email and it doesn't seem to have gone through - resending from apache account (apologizing up from for the length) For such sizable merges in Hadoop, I would like to start doing security audits in order to have an initial idea of the attack

[jira] [Resolved] (HADOOP-7553) hadoop-common tries to find hadoop-assemblies:jar:0.23.0-SNAPSHOT in http://snapshots.repository.codehaus.org

[ https://issues.apache.org/jira/browse/HADOOP-7553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Bokor resolved HADOOP-7553. -- Resolution: Invalid It is no longer an issue. > hadoop-common tries to find

Re: 答复: [DISCUSSION] Merging HDFS-7240 Object Store (Ozone) to trunk

Hi Steve, In addition to everything Weiwei mentioned (chapter 3 of user guide), if you really want to drill down to REST protocol you might want to apply this patch and build ozone. https://issues.apache.org/jira/browse/HDFS-12690 This will generate an Open API (https://www.openapis.org ,

答复: [DISCUSSION] Merging HDFS-7240 Object Store (Ozone) to trunk

Hi Steve The code is available in HDFS-7240 feature branch, public git repo here. I am not sure if there is a "public" API for object stores, but the design doc uses

Re: [DISCUSSION] Merging HDFS-7240 Object Store (Ozone) to trunk

Wow, big piece of work 1. Where is a PR/branch on github with rendered docs for us to look at? 2. Have you made any public APi changes related to object stores? That's probably something I'll have opinions on more than implementation details. thanks > On 19 Oct 2017, at 02:54, Yang Weiwei

Jars in Maven Central Repo

Hi, I am wondering how the jar files like javadoc, sources, tests, tests-sources and so on get to Central Repo. As I am checking different versions I see different set of jar files. E.g: - 2.8.0 has only tests and the binary jar but 2.8.1 has all the necessary jars - 3.0 alpha1 has all other