Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

[Apache Jenkins Server]

For more details, see 
https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/152/

[Mar 2, 2018 4:08:32 PM] (jlowe) YARN-7511. NPE in ContainerLocalizer when 
localization failed for
[Mar 3, 2018 12:22:20 AM] (aajisaka) MAPREDUCE-7023. 
TestHadoopArchiveLogs.testCheckFilesAndSeedApps fails on




-1 overall


The following subsystems voted -1:
docker


Powered by Apache Yetus 0.8.0-SNAPSHOT   http://yetus.apache.org

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Re: [VOTE] Merging branch HDFS-7240 to trunk

[Anu Engineer]

Hi Owen,

  >> 1. It is hard to tell what has changed. git rebase -i tells me the
  >> branch has 722 commits. The rebase failed with a conflict. It would really
   >> help if you rebased to current trunk.

Thanks for the comments. I have merged trunk to HDFS-7240 branch. 
Hopefully, this makes it easy to look at the changes; I have committed the 
change required to fix the conflict as a separate commit to make it easy for 
you to see.

Thanks
Anu


On 3/2/18, 4:42 PM, "Wangda Tan"  wrote:

I like the idea of same source / same release and put Ozone's source under
a different directory.

Like Owen mentioned, It gonna be important for all parties to keep a
regular and shorter release cycle for Hadoop, e.g. 3-4 months between minor
releases. Users can try features and give feedbacks to stabilize feature
earlier; developers can be happier since efforts will be consumed by users
soon after features get merged. In addition to this, if features merged to
trunk after reasonable tests/review, Andrew's concern may not be a problem
anymore:

bq. Finally, I earnestly believe that Ozone/HDSL itself would benefit from
being a separate project. Ozone could release faster and iterate more
quickly if it wasn't hampered by Hadoop's release schedule and security and
compatibility requirements.

Thanks,
Wangda


On Fri, Mar 2, 2018 at 4:24 PM, Owen O'Malley 
wrote:

> On Thu, Mar 1, 2018 at 11:03 PM, Andrew Wang 
> wrote:
>
> Owen mentioned making a Hadoop subproject; we'd have to
> > hash out what exactly this means (I assume a separate repo still managed
> by
> > the Hadoop project), but I think we could make this work if it's more
> > attractive than incubation or a new TLP.
>
>
> Ok, there are multiple levels of sub-projects that all make sense:
>
>- Same source tree, same releases - examples like HDFS & YARN
>- Same master branch, separate releases and release branches - Hive's
>Storage API vs Hive. It is in the source tree for the master branch, 
but
>has distinct releases and release branches.
>- Separate source, separate release - Apache Commons.
>
> There are advantages and disadvantages to each. I'd propose that we use 
the
> same source, same release pattern for Ozone. Note that we tried and later
> reverted doing Common, HDFS, and YARN as separate source, separate release
> because it was too much trouble. I like Daryn's idea of putting it as a 
top
> level directory in Hadoop and making sure that nothing in Common, HDFS, or
> YARN depend on it. That way if a Release Manager doesn't think it is ready
> for release, it can be trivially removed before the release.
>
> One thing about using the same releases, Sanjay and Jitendra are signing 
up
> to make much more regular bugfix and minor releases in the near future. 
For
> example, they'll need to make 3.2 relatively soon to get it released and
> then 3.3 somewhere in the next 3 to 6 months. That would be good for the
> project. Hadoop needs more regular releases and fewer big bang releases.
>
> .. Owen
>

Re: [VOTE] Merging branch HDFS-7240 to trunk

[Wangda Tan]

I like the idea of same source / same release and put Ozone's source under
a different directory.

Like Owen mentioned, It gonna be important for all parties to keep a
regular and shorter release cycle for Hadoop, e.g. 3-4 months between minor
releases. Users can try features and give feedbacks to stabilize feature
earlier; developers can be happier since efforts will be consumed by users
soon after features get merged. In addition to this, if features merged to
trunk after reasonable tests/review, Andrew's concern may not be a problem
anymore:

bq. Finally, I earnestly believe that Ozone/HDSL itself would benefit from
being a separate project. Ozone could release faster and iterate more
quickly if it wasn't hampered by Hadoop's release schedule and security and
compatibility requirements.

Thanks,
Wangda


On Fri, Mar 2, 2018 at 4:24 PM, Owen O'Malley 
wrote:

> On Thu, Mar 1, 2018 at 11:03 PM, Andrew Wang 
> wrote:
>
> Owen mentioned making a Hadoop subproject; we'd have to
> > hash out what exactly this means (I assume a separate repo still managed
> by
> > the Hadoop project), but I think we could make this work if it's more
> > attractive than incubation or a new TLP.
>
>
> Ok, there are multiple levels of sub-projects that all make sense:
>
>- Same source tree, same releases - examples like HDFS & YARN
>- Same master branch, separate releases and release branches - Hive's
>Storage API vs Hive. It is in the source tree for the master branch, but
>has distinct releases and release branches.
>- Separate source, separate release - Apache Commons.
>
> There are advantages and disadvantages to each. I'd propose that we use the
> same source, same release pattern for Ozone. Note that we tried and later
> reverted doing Common, HDFS, and YARN as separate source, separate release
> because it was too much trouble. I like Daryn's idea of putting it as a top
> level directory in Hadoop and making sure that nothing in Common, HDFS, or
> YARN depend on it. That way if a Release Manager doesn't think it is ready
> for release, it can be trivially removed before the release.
>
> One thing about using the same releases, Sanjay and Jitendra are signing up
> to make much more regular bugfix and minor releases in the near future. For
> example, they'll need to make 3.2 relatively soon to get it released and
> then 3.3 somewhere in the next 3 to 6 months. That would be good for the
> project. Hadoop needs more regular releases and fewer big bang releases.
>
> .. Owen
>

Re: [VOTE] Merging branch HDFS-7240 to trunk

[Owen O'Malley]

On Thu, Mar 1, 2018 at 11:03 PM, Andrew Wang 
wrote:

Owen mentioned making a Hadoop subproject; we'd have to
> hash out what exactly this means (I assume a separate repo still managed by
> the Hadoop project), but I think we could make this work if it's more
> attractive than incubation or a new TLP.


Ok, there are multiple levels of sub-projects that all make sense:

   - Same source tree, same releases - examples like HDFS & YARN
   - Same master branch, separate releases and release branches - Hive's
   Storage API vs Hive. It is in the source tree for the master branch, but
   has distinct releases and release branches.
   - Separate source, separate release - Apache Commons.

There are advantages and disadvantages to each. I'd propose that we use the
same source, same release pattern for Ozone. Note that we tried and later
reverted doing Common, HDFS, and YARN as separate source, separate release
because it was too much trouble. I like Daryn's idea of putting it as a top
level directory in Hadoop and making sure that nothing in Common, HDFS, or
YARN depend on it. That way if a Release Manager doesn't think it is ready
for release, it can be trivially removed before the release.

One thing about using the same releases, Sanjay and Jitendra are signing up
to make much more regular bugfix and minor releases in the near future. For
example, they'll need to make 3.2 relatively soon to get it released and
then 3.3 somewhere in the next 3 to 6 months. That would be good for the
project. Hadoop needs more regular releases and fewer big bang releases.

.. Owen

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

[Apache Jenkins Server]

For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/709/

[Mar 1, 2018 11:36:10 PM] (rchiang) HADOOP-14652. Update metrics-core version 
to 3.2.4. (rchiang)
[Mar 2, 2018 3:49:12 AM] (yqlin) HDFS-13210. Fix the typo in MiniDFSCluster 
class. Contributed by fang




-1 overall


The following subsystems voted -1:
findbugs unit xml


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

FindBugs :

   module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api 
   org.apache.hadoop.yarn.api.records.Resource.getResources() may expose 
internal representation by returning Resource.resources At Resource.java:by 
returning Resource.resources At Resource.java:[line 234] 

Failed junit tests :

   hadoop.fs.shell.TestCopyPreserveFlag 
   hadoop.crypto.key.kms.server.TestKMS 
   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure 
   hadoop.hdfs.web.TestWebHdfsTimeouts 
   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure 
   hadoop.fs.http.server.TestHttpFSServerWebServer 
   hadoop.yarn.server.nodemanager.webapp.TestContainerLogsPage 
   hadoop.yarn.applications.distributedshell.TestDistributedShell 
  

   cc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/709/artifact/out/diff-compile-cc-root.txt
  [4.0K]

   javac:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/709/artifact/out/diff-compile-javac-root.txt
  [280K]

   checkstyle:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/709/artifact/out/diff-checkstyle-root.txt
  [17M]

   pylint:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/709/artifact/out/diff-patch-pylint.txt
  [24K]

   shellcheck:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/709/artifact/out/diff-patch-shellcheck.txt
  [20K]

   shelldocs:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/709/artifact/out/diff-patch-shelldocs.txt
  [12K]

   whitespace:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/709/artifact/out/whitespace-eol.txt
  [9.2M]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/709/artifact/out/whitespace-tabs.txt
  [288K]

   xml:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/709/artifact/out/xml.txt
  [4.0K]

   findbugs:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/709/artifact/out/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-warnings.html
  [8.0K]

   javadoc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/709/artifact/out/diff-javadoc-javadoc-root.txt
  [760K]

   unit:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/709/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt
  [184K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/709/artifact/out/patch-unit-hadoop-common-project_hadoop-kms.txt
  [12K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/709/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
  [316K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/709/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-httpfs.txt
  [24K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/709/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
  [48K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/709/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-applications-distributedshell.txt
  [12K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/709/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt
  [84K]

Powered by Apache Yetus 0.8.0-SNAPSHOT   http://yetus.apache.org

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org