date:20181113

[VOTE] Release Apache Hadoop Ozone 0.3.0-alpha (RC0)

2018-11-13 Thread Elek, Marton

Hi all,

I've created the first release candidate (RC0) for Apache Hadoop Ozone
0.3.0-alpha according to the plans shared here previously.

This is the second release of Apache Hadoop Ozone. Notable changes since
the first release:

* A new S3 compatible rest server is added. Ozone can be used from any
S3 compatible tools (HDDS-434)
* Ozone Hadoop file system URL prefix is renamed from o3:// to o3fs://
(HDDS-651)
* Extensive testing and stability improvements of OzoneFs.
* Spark, YARN and Hive support and stability improvements.
* Improved Pipeline handling and recovery.
* Separated/dedicated classpath definitions for all the Ozone
components. (HDDS-447)

The RC artifacts are available from:
https://home.apache.org/~elek/ozone-0.3.0-alpha-rc0/

The RC tag in git is: ozone-0.3.0-alpha-RC0 (dc661083683)

Please try it out, vote, or just give us feedback.

The vote will run for 5 days, ending on November 18, 2018 13:00 UTC.


Thank you very much,
Marton

PS:

The easiest way to try it out is:

1. Download the binary artifact
2. Read the docs from ./docs/index.html
3. TLDR; cd compose/ozone && docker-compose up -d
4. open localhost:9874 or localhost:9876



The easiest way to try it out from the source:

1. mvn  install -DskipTests -Pdist -Dmaven.javadoc.skip=true -Phdds
-DskipShade -am -pl :hadoop-ozone-dist
2. cd hadoop-ozone/dist/target/ozone-0.3.0-alpha && docker-compose up -d



The easiest way to test basic functionality (with acceptance tests):

1. mvn  install -DskipTests -Pdist -Dmaven.javadoc.skip=true -Phdds
-DskipShade -am -pl :hadoop-ozone-dist
2. cd hadoop-ozone/dist/target/ozone-0.3.0-alpha/smoketest
3. ./test.sh

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Re: Hadoop 3.2 Release Plan proposal

2018-11-13 Thread Sunil G

Hi Folks,

All blockers are closed by last weekend and corrected jiras as well.
Preparing RC0 now. Facing some issue with shaded jars file size. Hence
respinning again.
Planning to complete this by end of this week.

Thanks
Sunil

On Fri, Oct 26, 2018 at 2:34 AM Konstantin Shvachko 
wrote:

> Another thing is that I see a bunch of jiras under HDFS-8707, which don't
> have the Fix Version field listing 3.2, some have it just empty.
> This means they will not be populated into release notes.
>
> Thanks,
> --Konstantin
>
> On Thu, Oct 25, 2018 at 7:59 AM Sunil G  wrote:
>
> > Thanks Konstantin for pointing out.
> > As 3.2 is pretty much on RC level, its better we try to find a good
> > solution to this issue.
> >
> > I ll follow up on this in the jira.
> >
> > - Sunil
> >
> > On Thu, Oct 25, 2018 at 11:35 AM Konstantin Shvachko <
> shv.had...@gmail.com>
> > wrote:
> >
> >> I've tried to attract attention to an incompatibility issue through the
> >> jira, but it didn't work. So pitching in in this thread.
> >> https://issues.apache.org/jira/browse/HDFS-12026
> >> It introduced binary incompatibility, which will prevent people from
> >> upgrading from 3.1 to 3.2.
> >> I think it can get messy if we release anything with this feature.
> >>
> >> Thanks,
> >> --Konstantin
> >>
> >> On Mon, Oct 22, 2018 at 5:01 AM Steve Loughran 
> >> wrote:
> >>
> >>> its in.
> >>>
> >>> good catch!
> >>>
> >>> On 20 Oct 2018, at 01:35, Wei-Chiu Chuang   >>> weic...@cloudera.com>> wrote:
> >>>
> >>> Thanks Sunil G for driving the release,
> >>> I filed HADOOP-15866<
> https://issues.apache.org/jira/browse/HADOOP-15866>
> >>> for a compat fix. If any one has cycle please review it, as I think it
> is
> >>> needed for 3.2.0.
> >>>
> >>> On Thu, Oct 18, 2018 at 4:43 AM Sunil G  >>> sun...@apache.org>> wrote:
> >>> Hi Folks,
> >>>
> >>> As we previously communicated for 3.2.0 release, we have delayed due to
> >>> few
> >>> blockers in our gate.
> >>>
> >>> I just cut branch-3.2.0 for release purpose. branch-3.2 will be open
> for
> >>> all bug fixes.
> >>>
> >>> - Sunil
> >>>
> >>>
> >>> On Tue, Oct 16, 2018 at 8:59 AM Sunil G  >>> sun...@apache.org>> wrote:
> >>>
> >>> > Hi Folks,
> >>> >
> >>> > We are now close to RC as other blocker issues are now merged to
> trunk
> >>> and
> >>> > branch-3.2. Last 2 critical issues are closer to merge and will be
> >>> > committed in few hours.
> >>> > With this, I will be creating 3.2.0 branch today and will go ahead
> >>> with RC
> >>> > related process.
> >>> >
> >>> > - Sunil
> >>> >
> >>> > On Mon, Oct 15, 2018 at 11:43 PM Jonathan Bender <
> jonben...@stripe.com
> >>> >
> >>> > wrote:
> >>> >
> >>> >> Hello, were there any updates around the 3.2.0 RC timing? All I see
> in
> >>> >> the current blockers are related to the new Submarine subproject,
> >>> wasn't
> >>> >> sure if that is what is holding things up.
> >>> >>
> >>> >> Cheers,
> >>> >> Jon
> >>> >>
> >>> >> On Tue, Oct 2, 2018 at 7:13 PM, Sunil G  >>> sun...@apache.org>> wrote:
> >>> >>
> >>> >>> Thanks Robert and Haibo for quickly correcting same.
> >>> >>> Sigh, I somehow missed one file while committing the change. Sorry
> >>> for
> >>> >>> the
> >>> >>> trouble.
> >>> >>>
> >>> >>> - Sunil
> >>> >>>
> >>> >>> On Wed, Oct 3, 2018 at 5:22 AM Robert Kanter  >>> >
> >>> >>> wrote:
> >>> >>>
> >>> >>> > Looks like there's two that weren't updated:
> >>> >>> > >> [115] 16:32 : hadoop-common (trunk) :: grep "3.2.0-SNAPSHOT" .
> >>> -r
> >>> >>> > --include=pom.xml
> >>> >>> > ./hadoop-project/pom.xml:
> >>> >>> >
> >>> 3.2.0-SNAPSHOT
> >>> >>> > ./pom.xml:3.2.0-SNAPSHOT
> >>> >>> >
> >>> >>> > I've just pushed in an addendum commit to fix those.
> >>> >>> > In the future, please make sure to do a sanity compile when
> >>> updating
> >>> >>> poms.
> >>> >>> >
> >>> >>> > thanks
> >>> >>> > - Robert
> >>> >>> >
> >>> >>> > On Tue, Oct 2, 2018 at 11:44 AM Aaron Fabbri
> >>> >>> mailto:fab...@cloudera.com.invalid>>
> >>> >>> > wrote:
> >>> >>> >
> >>> >>> >> Trunk is not building for me.. Did you miss a 3.2.0-SNAPSHOT in
> >>> the
> >>> >>> >> top-level pom.xml?
> >>> >>> >>
> >>> >>> >>
> >>> >>> >> On Tue, Oct 2, 2018 at 10:16 AM Sunil G  >>> > wrote:
> >>> >>> >>
> >>> >>> >> > Hi All
> >>> >>> >> >
> >>> >>> >> > As mentioned in earlier mail, I have cut branch-3.2 and reset
> >>> trunk
> >>> >>> to
> >>> >>> >> > 3.3.0-SNAPSHOT. I will share the RC details sooner once all
> >>> >>> necessary
> >>> >>> >> > patches are pulled into branch-3.2.
> >>> >>> >> >
> >>> >>> >> > Thank You
> >>> >>> >> > - Sunil
> >>> >>> >> >
> >>> >>> >> >
> >>> >>> >> > On Mon, Sep 24, 2018 at 2:00 PM Sunil G  >>> > wrote:
> >>> >>> >> >
> >>> >>> >> > > Hi All
> >>> >>> >> > >
> >>> >>> >> > > We are now down to the last Blocker and HADOOP-15407 is
> >>> merged to
> >>> >>> >> trunk.
> >>> >>> >> > > Thanks for the support.
> >>> >>> >> > >
>

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

2018-11-13 Thread Apache Jenkins Server

For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/956/

[Nov 12, 2018 6:27:02 AM] (sunilg) YARN-8902. [CSI] Add volume manager that 
manages CSI volume lifecycle.
[Nov 12, 2018 9:54:41 AM] (elek) HDDS-767. OM should not search for STDOUT root 
logger for audit logging.
[Nov 12, 2018 10:18:23 AM] (wwei) YARN-8987. Usability improvements 
node-attributes CLI. Contributed by 
[Nov 12, 2018 12:58:05 PM] (stevel) HADOOP-15110. Gauges are getting logged in 
exceptions from
[Nov 12, 2018 3:39:30 PM] (sunilg) YARN-8877. [CSI] Extend service spec to 
allow setting resource
[Nov 12, 2018 6:03:11 PM] (nanda) HDDS-576. Move ContainerWithPipeline creation 
to RPC endpoint.
[Nov 12, 2018 6:42:30 PM] (billie) YARN-8776. Implement Container Exec feature 
in LinuxContainerExecutor.
[Nov 12, 2018 10:08:39 PM] (jitendra) HDDS-709. Modify Close Container handling 
sequence on datanodes.
[Nov 12, 2018 11:06:43 PM] (gifuma) YARN-8997. [Submarine] Small refactors of 
modifier, condition check and
[Nov 12, 2018 11:31:42 PM] (arp) HDFS-14065. Failed Storage Locations shows 
nothing in the Datanode
[Nov 13, 2018 12:53:10 AM] (eyang) YARN-8761. Service AM support for 
decommissioning component instances.  




-1 overall


The following subsystems voted -1:
asflicense findbugs hadolint pathlen unit


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

Failed junit tests :

   hadoop.util.TestDiskCheckerWithDiskIo 
   hadoop.util.TestReadWriteDiskValidator 
   hadoop.hdfs.web.TestWebHdfsTimeouts 
   hadoop.yarn.server.resourcemanager.TestCapacitySchedulerMetrics 
   
hadoop.yarn.server.timelineservice.reader.TestTimelineReaderWebServicesHBaseStorage
 
   hadoop.yarn.sls.TestSLSRunner 
   hadoop.yarn.sls.TestSLSStreamAMSynth 
  

   cc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/956/artifact/out/diff-compile-cc-root.txt
  [4.0K]

   javac:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/956/artifact/out/diff-compile-javac-root.txt
  [324K]

   checkstyle:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/956/artifact/out/diff-checkstyle-root.txt
  [17M]

   hadolint:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/956/artifact/out/diff-patch-hadolint.txt
  [4.0K]

   pathlen:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/956/artifact/out/pathlen.txt
  [12K]

   pylint:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/956/artifact/out/diff-patch-pylint.txt
  [40K]

   shellcheck:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/956/artifact/out/diff-patch-shellcheck.txt
  [68K]

   shelldocs:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/956/artifact/out/diff-patch-shelldocs.txt
  [12K]

   whitespace:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/956/artifact/out/whitespace-eol.txt
  [9.3M]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/956/artifact/out/whitespace-tabs.txt
  [1.1M]

   findbugs:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/956/artifact/out/branch-findbugs-hadoop-hdds_client.txt
  [8.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/956/artifact/out/branch-findbugs-hadoop-hdds_container-service.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/956/artifact/out/branch-findbugs-hadoop-hdds_framework.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/956/artifact/out/branch-findbugs-hadoop-hdds_server-scm.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/956/artifact/out/branch-findbugs-hadoop-hdds_tools.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/956/artifact/out/branch-findbugs-hadoop-ozone_client.txt
  [8.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/956/artifact/out/branch-findbugs-hadoop-ozone_common.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/956/artifact/out/branch-findbugs-hadoop-ozone_objectstore-service.txt
  [8.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/956/artifact/out/branch-findbugs-hadoop-ozone_ozone-manager.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/956/artifact/out/branch-findbugs-hadoop-ozone_ozonefs.txt
  [12K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/956/artifact/out/branch-findbugs-hadoop-ozone_s3gateway.txt
  [4.0K]

Re: branch-2.9.2 is almost closed for commit

2018-11-13 Thread Akira Ajisaka

Hi all,

I wanted to push release-2.9.2-RC0 tag and wrongly pushed all the
local tags by 'git push --tags'. I deleted some tags but I couldn't
delete 'rel/release-' tag because all the tags start with 'rel/' are
protected. I'll ask infra to delete this tag.
Sorry for pushing the wrong tags.

-Akira

2018年11月13日(火) 16:07 Akira Ajisaka :
>
> Recently I hit the gpg-agent cache ttl issue while creating 2.9.2 RC0,
> and the issue was fixed by HADOOP-15923.
> I'll create RC0 and start the vote by the end of this week. Sorry for the 
> delay.
>
> Thanks,
> Akira
> 2018年11月6日(火) 13:51 Akira Ajisaka :
> >
> > Hi folks,
> >
> > Now there is only 1 critical issues targeted for 2.9.2 (YARN-8233), so
> > I'd like to close branch-2.9.2 except YARN-8233.
> > I create RC0 as soon as YARN-8233 is committed to branch-2.9.2.
> >
> > Thanks,
> > Akira

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

[VOTE] Release Apache Hadoop 2.9.2 (RC0)

2018-11-13 Thread Akira Ajisaka

Hi folks,

I have put together a release candidate (RC0) for Hadoop 2.9.2. It
includes 204 bug fixes and improvements since 2.9.1. [1]

The RC is available at http://home.apache.org/~aajisaka/hadoop-2.9.2-RC0/
Git signed tag is release-2.9.2-RC0 and the checksum is
826afbeae31ca687bc2f8471dc841b66ed2c6704
The maven artifacts are staged at
https://repository.apache.org/content/repositories/orgapachehadoop-1166/

You can find my public key at:
https://dist.apache.org/repos/dist/release/hadoop/common/KEYS

Please try the release and vote. The vote will run for 5 days.

[1] https://s.apache.org/2.9.2-fixed-jiras

Thanks,
Akira

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

[VOTE] Release Apache Hadoop Ozone 0.3.0-alpha (RC0)

Re: Hadoop 3.2 Release Plan proposal

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

Re: branch-2.9.2 is almost closed for commit

[VOTE] Release Apache Hadoop 2.9.2 (RC0)

5 matches

Site Navigation

Mail list logo

Footer information