Re: [VOTE] Propose to start new Hadoop sub project "submarine"

[Prabhu Josephraj]

+1 (non-binding)

On Tue, Feb 5, 2019 at 10:30 AM Naganarasimha Garla <
naganarasimha...@apache.org> wrote:

> +1
>
> On Sat, 2 Feb 2019, 09:51 Rohith Sharma K S  wrote:
>
> > +1
> >
> > On Sat, Feb 2, 2019, 3:54 AM Wangda Tan  wrote:
> >
> > > Hi all,
> > >
> > > According to positive feedbacks from the thread [1]
> > >
> > > This is vote thread to start a new subproject named "hadoop-submarine"
> > > which follows the release process already established for ozone.
> > >
> > > The vote runs for usual 7 days, which ends at Feb 8th 5 PM PDT.
> > >
> > > Thanks,
> > > Wangda Tan
> > >
> > > [1]
> > >
> > >
> >
> https://lists.apache.org/thread.html/f864461eb188bd12859d51b0098ec38942c4429aae7e4d001a633d96@%3Cyarn-dev.hadoop.apache.org%3E
> > >
> >
>

Re: [VOTE] Release Apache Hadoop 3.1.2 - RC1

[Suma Shivaprasad]

+1

- Built from source
- Deployed a single node cluster
- Ran few MR and Distributed Shell Jobs
- Check YARN UI2 flow with Yarn services and jobs

Thanks
Suma

On Mon, Feb 4, 2019 at 7:51 PM Wilfred Spiegelenburg
 wrote:

> +1 (non binding)
>
> - build from source on MacOSX 10.14.2, 1.8.0u181
> - successful native build on Ubuntu 16.04.3
> - deployed a single node cluster  (openjdk 1.8u191 / centos 7.5)
> - uploaded the MR framework
> - configured YARN with the FS
> - ran multiple MR jobs
>
> > On 29 Jan 2019, at 17:19, Sunil G  wrote:
> >
> > Hi Folks,
> >
> > On behalf of Wangda, we have an RC1 for Apache Hadoop 3.1.2.
> >
> > The artifacts are available here:
> > http://home.apache.org/~sunilg/hadoop-3.1.2-RC1/
> >
> > The RC tag in git is release-3.1.2-RC1:
> > https://github.com/apache/hadoop/commits/release-3.1.2-RC1
> >
> > The maven artifacts are available via repository.apache.org at
> > https://repository.apache.org/content/repositories/orgapachehadoop-1215
> >
> > This vote will run 5 days from now.
> >
> > 3.1.2 contains 325 [1] fixed JIRA issues since 3.1.1.
> >
> > We have done testing with a pseudo cluster and distributed shell job.
> >
> > My +1 to start.
> >
> > Best,
> > Wangda Tan and Sunil Govindan
> >
> > [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.1.2)
> > ORDER BY priority DESC
>
>
> Wilfred Spiegelenburg | Software Engineer
> cloudera.com 
>
>
>
>
>
>
>
>

Re: [VOTE] Release Apache Hadoop 3.1.2 - RC1

[Wilfred Spiegelenburg]

+1 (non binding)

- build from source on MacOSX 10.14.2, 1.8.0u181
- successful native build on Ubuntu 16.04.3
- deployed a single node cluster  (openjdk 1.8u191 / centos 7.5)
- uploaded the MR framework
- configured YARN with the FS
- ran multiple MR jobs

> On 29 Jan 2019, at 17:19, Sunil G  wrote:
> 
> Hi Folks,
> 
> On behalf of Wangda, we have an RC1 for Apache Hadoop 3.1.2.
> 
> The artifacts are available here:
> http://home.apache.org/~sunilg/hadoop-3.1.2-RC1/
> 
> The RC tag in git is release-3.1.2-RC1:
> https://github.com/apache/hadoop/commits/release-3.1.2-RC1
> 
> The maven artifacts are available via repository.apache.org at
> https://repository.apache.org/content/repositories/orgapachehadoop-1215
> 
> This vote will run 5 days from now.
> 
> 3.1.2 contains 325 [1] fixed JIRA issues since 3.1.1.
> 
> We have done testing with a pseudo cluster and distributed shell job.
> 
> My +1 to start.
> 
> Best,
> Wangda Tan and Sunil Govindan
> 
> [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.1.2)
> ORDER BY priority DESC


Wilfred Spiegelenburg | Software Engineer
cloudera.com 

Re: [VOTE] Moving branch-2 precommit/nightly test builds to java 8

[Zhe Zhang]

+1

On Mon, Feb 4, 2019 at 6:14 PM Jonathan Hung  wrote:

> Hello,
>
> Starting a vote based on the discuss thread [1] for moving branch-2
> precommit/nightly test builds to openjdk8. After this change, the test
> phase for precommit builds [2] and branch-2 nightly build [3] will run on
> openjdk8. To maintain source compatibility, these builds will still run
> their compile phase for branch-2 on openjdk7 as they do now (in addition to
> compiling on openjdk8).
>
> Vote will run for three business days until Thursday Feb 7 6:00PM PDT.
>
> [1]
>
> https://lists.apache.org/thread.html/7e6fb28fc67560f83a2eb62752df35a8d58d86b2a3df4cacb5d738ca@%3Ccommon-dev.hadoop.apache.org%3E
>
> [2]
> https://builds.apache.org/view/H-L/view/Hadoop/job/PreCommit-HADOOP-Build/
> https://builds.apache.org/view/H-L/view/Hadoop/job/PreCommit-HDFS-Build/
> https://builds.apache.org/view/H-L/view/Hadoop/job/PreCommit-YARN-Build/
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/PreCommit-MAPREDUCE-Build/
>
> [3]
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/hadoop-qbt-branch2-java7-linux-x86/
>
> Jonathan Hung
>
-- 
Zhe Zhang
Apache Hadoop Committer
http://zhe-thoughts.github.io/about/ | @oldcap

[VOTE] Moving branch-2 precommit/nightly test builds to java 8

[Jonathan Hung]

Hello,

Starting a vote based on the discuss thread [1] for moving branch-2
precommit/nightly test builds to openjdk8. After this change, the test
phase for precommit builds [2] and branch-2 nightly build [3] will run on
openjdk8. To maintain source compatibility, these builds will still run
their compile phase for branch-2 on openjdk7 as they do now (in addition to
compiling on openjdk8).

Vote will run for three business days until Thursday Feb 7 6:00PM PDT.

[1]
https://lists.apache.org/thread.html/7e6fb28fc67560f83a2eb62752df35a8d58d86b2a3df4cacb5d738ca@%3Ccommon-dev.hadoop.apache.org%3E

[2]
https://builds.apache.org/view/H-L/view/Hadoop/job/PreCommit-HADOOP-Build/
https://builds.apache.org/view/H-L/view/Hadoop/job/PreCommit-HDFS-Build/
https://builds.apache.org/view/H-L/view/Hadoop/job/PreCommit-YARN-Build/
https://builds.apache.org/view/H-L/view/Hadoop/job/PreCommit-MAPREDUCE-Build/

[3]
https://builds.apache.org/view/H-L/view/Hadoop/job/hadoop-qbt-branch2-java7-linux-x86/

Jonathan Hung

Re: [VOTE] Release Apache Hadoop 3.1.2 - RC1

[Arpit Agarwal]

+1 (binding)

- Verified signatures
- Verified checksums
- Built from source
- Verified Maven artifacts on staging repo
- Deployed 3 node cluster
- Tried out HDFS commands, MapReduce jobs.

Confirmed the issues Billie pointed out. Not sure if you need to spin up a new 
RC or you can update the tarball - contents of the git tag look fine.


> On Jan 28, 2019, at 10:19 PM, Sunil G  wrote:
> 
> Hi Folks,
> 
> On behalf of Wangda, we have an RC1 for Apache Hadoop 3.1.2.
> 
> The artifacts are available here:
> http://home.apache.org/~sunilg/hadoop-3.1.2-RC1/
> 
> The RC tag in git is release-3.1.2-RC1:
> https://github.com/apache/hadoop/commits/release-3.1.2-RC1
> 
> The maven artifacts are available via repository.apache.org at
> https://repository.apache.org/content/repositories/orgapachehadoop-1215
> 
> This vote will run 5 days from now.
> 
> 3.1.2 contains 325 [1] fixed JIRA issues since 3.1.1.
> 
> We have done testing with a pseudo cluster and distributed shell job.
> 
> My +1 to start.
> 
> Best,
> Wangda Tan and Sunil Govindan
> 
> [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.1.2)
> ORDER BY priority DESC


-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: [VOTE] Propose to start new Hadoop sub project "submarine"

[Chandni Singh]

+1 (non-binding). Thanks Wangda!

On Mon, Feb 4, 2019 at 2:48 PM Hanisha Koneru 
wrote:

> +1 (non-binding).
>
> Thanks,
> Hanisha
>
>
>
>
>
>
>
>
>
> On 2/4/19, 10:16 AM, "Weiwei Yang"  wrote:
>
> >+1
> >
> >Weiwei
> >
> >--
> >Weiwei
> >On Feb 5, 2019, 2:11 AM +0800, Steve Loughran ,
> wrote:
> >> +1, binding
> >>
> >> > On 1 Feb 2019, at 22:15, Wangda Tan  wrote:
> >> >
> >> > Hi all,
> >> >
> >> > According to positive feedbacks from the thread [1]
> >> >
> >> > This is vote thread to start a new subproject named "hadoop-submarine"
> >> > which follows the release process already established for ozone.
> >> >
> >> > The vote runs for usual 7 days, which ends at Feb 8th 5 PM PDT.
> >> >
> >> > Thanks,
> >> > Wangda Tan
> >> >
> >> > [1]
> >> >
> https://lists.apache.org/thread.html/f864461eb188bd12859d51b0098ec38942c4429aae7e4d001a633d96@%3Cyarn-dev.hadoop.apache.org%3E
> >>
> >>
> >> -
> >> To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
> >> For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
> >>
>

Re: [VOTE] Release Apache Hadoop 3.1.2 - RC1

[Hanisha Koneru]

Thanks Sunil and Wangda for putting up the RC.

+1 (non-binding)
Verified the following:
- Built from source
- Deployed binary to a 3-node docker cluster
- Sanity checks - basic dfs operations



Thanks,
Hanisha








On 1/28/19, 10:19 PM, "Sunil G"  wrote:

>Hi Folks,
>
>On behalf of Wangda, we have an RC1 for Apache Hadoop 3.1.2.
>
>The artifacts are available here:
>http://home.apache.org/~sunilg/hadoop-3.1.2-RC1/
>
>The RC tag in git is release-3.1.2-RC1:
>https://github.com/apache/hadoop/commits/release-3.1.2-RC1
>
>The maven artifacts are available via repository.apache.org at
>https://repository.apache.org/content/repositories/orgapachehadoop-1215
>
>This vote will run 5 days from now.
>
>3.1.2 contains 325 [1] fixed JIRA issues since 3.1.1.
>
>We have done testing with a pseudo cluster and distributed shell job.
>
>My +1 to start.
>
>Best,
>Wangda Tan and Sunil Govindan
>
>[1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.1.2)
>ORDER BY priority DESC

Re: [VOTE] Release Apache Hadoop 3.1.2 - RC1

[Billie Rinaldi]

Hey Sunil and Wangda, thanks for the RC. The source tarball has a
patchprocess directory with some yetus code in it. Also, the file
dev-support/bin/create-release file has the following line added:
  export GPG_AGENT_INFO="/home/sunilg/.gnupg/S.gpg-agent:$(pgrep
gpg-agent):1"

I think we are probably due for an overall review of LICENSE and NOTICE. I
saw some idiosyncrasies there but nothing that looked like a blocker.

On Mon, Jan 28, 2019 at 10:20 PM Sunil G  wrote:

> Hi Folks,
>
> On behalf of Wangda, we have an RC1 for Apache Hadoop 3.1.2.
>
> The artifacts are available here:
> http://home.apache.org/~sunilg/hadoop-3.1.2-RC1/
>
> The RC tag in git is release-3.1.2-RC1:
> https://github.com/apache/hadoop/commits/release-3.1.2-RC1
>
> The maven artifacts are available via repository.apache.org at
> https://repository.apache.org/content/repositories/orgapachehadoop-1215
>
> This vote will run 5 days from now.
>
> 3.1.2 contains 325 [1] fixed JIRA issues since 3.1.1.
>
> We have done testing with a pseudo cluster and distributed shell job.
>
> My +1 to start.
>
> Best,
> Wangda Tan and Sunil Govindan
>
> [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.1.2)
> ORDER BY priority DESC
>

Re: [VOTE] Release Apache Hadoop 3.1.2 - RC1

[Masatake Iwasaki]

Thanks for putting this up, Wangda Tan and Sunil Govindan

+1(non-binding)

* verified signature and mds
* built from source on CentOS 7 and OpenJDK 8 with native profile,
* deployed 3-nodes ha cluster and ran some example jobs.
* ran kms, put a file into encrypted zone and cat it.
* built site documentation and skimmed the contents.

I felt HADOOP-14445 and HADOOP-15996 are relatively big change for a 
point release,

while those looks working or disabled by default.

Masatake Iwasaki

On 1/28/19 22:19, Sunil G wrote:

Hi Folks,

On behalf of Wangda, we have an RC1 for Apache Hadoop 3.1.2.

The artifacts are available here:
http://home.apache.org/~sunilg/hadoop-3.1.2-RC1/

The RC tag in git is release-3.1.2-RC1:
https://github.com/apache/hadoop/commits/release-3.1.2-RC1

The maven artifacts are available via repository.apache.org at
https://repository.apache.org/content/repositories/orgapachehadoop-1215

This vote will run 5 days from now.

3.1.2 contains 325 [1] fixed JIRA issues since 3.1.1.

We have done testing with a pseudo cluster and distributed shell job.

My +1 to start.

Best,
Wangda Tan and Sunil Govindan

[1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.1.2)
ORDER BY priority DESC




-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.1.2 - RC1

[Chandni Singh]

+1 non-binding
- Built from source
- Launched sleeper service
- Upgraded sleeper containers
- Canceled upgrade
- Upgraded the containers again and finalized the upgrade.



On Wed, Jan 30, 2019 at 7:58 PM Rohith Sharma K S 
wrote:

> +1(binding)
> - Built from source for -Dhbase.profile=2.0 and installed RM HA non-secure
> cluster with 2 nodes. Configured ATSv2 in cluster.
> - Ran sample jobs with MR and DS.
> - Verified UI2 for for flow activity page and data validation.
> - Verified for ATSv2 REST API's
>
> -Rohith Sharma K S
>
>
> On Tue, 29 Jan 2019 at 11:50, Sunil G  wrote:
>
> > Hi Folks,
> >
> > On behalf of Wangda, we have an RC1 for Apache Hadoop 3.1.2.
> >
> > The artifacts are available here:
> > http://home.apache.org/~sunilg/hadoop-3.1.2-RC1/
> >
> > The RC tag in git is release-3.1.2-RC1:
> > https://github.com/apache/hadoop/commits/release-3.1.2-RC1
> >
> > The maven artifacts are available via repository.apache.org at
> > https://repository.apache.org/content/repositories/orgapachehadoop-1215
> >
> > This vote will run 5 days from now.
> >
> > 3.1.2 contains 325 [1] fixed JIRA issues since 3.1.1.
> >
> > We have done testing with a pseudo cluster and distributed shell job.
> >
> > My +1 to start.
> >
> > Best,
> > Wangda Tan and Sunil Govindan
> >
> > [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.1.2)
> > ORDER BY priority DESC
> >
>

Re: [VOTE] Propose to start new Hadoop sub project "submarine"

[Hanisha Koneru]

+1 (non-binding).

Thanks,
Hanisha









On 2/4/19, 10:16 AM, "Weiwei Yang"  wrote:

>+1
>
>Weiwei
>
>--
>Weiwei
>On Feb 5, 2019, 2:11 AM +0800, Steve Loughran , wrote:
>> +1, binding
>>
>> > On 1 Feb 2019, at 22:15, Wangda Tan  wrote:
>> >
>> > Hi all,
>> >
>> > According to positive feedbacks from the thread [1]
>> >
>> > This is vote thread to start a new subproject named "hadoop-submarine"
>> > which follows the release process already established for ozone.
>> >
>> > The vote runs for usual 7 days, which ends at Feb 8th 5 PM PDT.
>> >
>> > Thanks,
>> > Wangda Tan
>> >
>> > [1]
>> > https://lists.apache.org/thread.html/f864461eb188bd12859d51b0098ec38942c4429aae7e4d001a633d96@%3Cyarn-dev.hadoop.apache.org%3E
>>
>>
>> -
>> To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
>> For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
>>

Re: [VOTE] Propose to start new Hadoop sub project "submarine"

[Zhe Zhang]

+1, binding

On Mon, Feb 4, 2019 at 1:20 PM Takanobu Asanuma 
wrote:

> +1 (non-binding). Thanks, Wangda!
>
> - Takanobu
>
> on 2019/02/02 7:24, "Wangda Tan" wrote:
>
> Hi all,
>
> According to positive feedbacks from the thread [1]
>
> This is vote thread to start a new subproject named "hadoop-submarine"
> which follows the release process already established for ozone.
>
> The vote runs for usual 7 days, which ends at Feb 8th 5 PM PDT.
>
> Thanks,
> Wangda Tan
>
> [1]
>
> https://lists.apache.org/thread.html/f864461eb188bd12859d51b0098ec38942c4429aae7e4d001a633d96@%3Cyarn-dev.hadoop.apache.org%3E
>
>
> --
Zhe Zhang
Apache Hadoop Committer
http://zhe-thoughts.github.io/about/ | @oldcap

Re: [VOTE] Propose to start new Hadoop sub project "submarine"

[Takanobu Asanuma]

+1 (non-binding). Thanks, Wangda!

- Takanobu

on 2019/02/02 7:24, "Wangda Tan" wrote:

Hi all,

According to positive feedbacks from the thread [1]

This is vote thread to start a new subproject named "hadoop-submarine"
which follows the release process already established for ozone.

The vote runs for usual 7 days, which ends at Feb 8th 5 PM PDT.

Thanks,
Wangda Tan

[1]

https://lists.apache.org/thread.html/f864461eb188bd12859d51b0098ec38942c4429aae7e4d001a633d96@%3Cyarn-dev.hadoop.apache.org%3E

Re: [DISCUSS] Moving branch-2 to java 8

[Jonathan Hung]

Hi Anu, we will configure precommit jobs to continue compiling on openjdk7.
If there's incompatible source changes then the precommit job will catch
this. The change proposed here is only for the *test* phase of branch-2
precommit executions (and branch-2 nightly job) to run on openjdk8 only.

Jonathan Hung


On Mon, Feb 4, 2019 at 10:45 AM Anu Engineer 
wrote:

> Konstantin,
>
> Just a nitpicky thought, if we move this branch to Java-8 on Jenkins, but
> still hope to release code that can run on Java 7, how will we detect
> Java 8 only changes? I am asking because till now whenever I checked in
> Java 8 features in branch-2 Jenkins would catch that issue.
>
> With this approach, we might not find it out the issues till the release
> time when the release manager decides to compile with Java 7.
> It might be more pragmatic to say that your Java 7 mileage may vary once
> this goes in, since we will have no visibility to Java 7 compatibility
> until it is too late.
>
> Another approach could be that we create a read-only 2.x branch, then we
> know that code will work with Java 7 since the last snapshot was known to
> work with Java 7.
>
>
> Thanks
> Anu
>
>
>
> On 2/1/19, 5:04 PM, "Konstantin Shvachko"  wrote:
>
> Just to make sure we are on the same page, as the subject of this
> thread is
> too generic and confusing.
> *The proposal is to move branch-2 Jenkins builds such as precommit to
> run
> tests on openJDK-8.*
> We do not want to break Java 7 source compatibility. The sources and
> releases will still depend on Java 7.
> We don't see test failures discussed in HADOOP-15711 when we run them
> locally with Oracle Java 7.
>
> Thanks,
> --Konst
>
> On Fri, Feb 1, 2019 at 12:44 PM Jonathan Hung 
> wrote:
>
> > Thanks Vinod and Steve, agreed about java7 compile compatibility. At
> least
> > for now, we should be able to maintain java7 source compatibility
> and run
> > tests on java8. There's a test run here:
> >
> https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86-jhung/46/
> > which calls a java8 specific API, installs both openjdk7/openjdk8 in
> the
> > dockerfile, compiles on both versions, and tests on just java8 (via
> >
> >
> --multijdkdirs=/usr/lib/jvm/java-7-openjdk-amd64,/usr/lib/jvm/java-8-openjdk-amd64
> > and --multijdktests=compile). If we eventually decide it's too much
> of a
> > pain to maintain java7 source compatibility we can do that at a later
> > point.
> >
> > Also based on discussion with others in the community at the
> contributors
> > meetup this past Wednesday, seems we are generally in favor of
> testing
> > against java8. I'll start a vote soon.
> >
> > Jonathan Hung
> >
> >
> > On Tue, Jan 29, 2019 at 4:11 AM Steve Loughran <
> ste...@hortonworks.com>
> > wrote:
> >
> > > branch-2 is the JDK 7 branch, but for a long time I (and presumably
> > > others) have relied on jenkins to keep us honest by doing that
> build and
> > > test
> > >
> > > right now, we can't do that any more, due to jdk7 bugs which will
> never
> > be
> > > fixed by oracle, or at least, not in a public release.
> > >
> > > If we can still do the compile in java 7 language and link to java
> 7 JDK,
> > > then that bit of the release is good -then java 8 can be used for
> that
> > test
> > >
> > > Ultimately, we're going to be forced onto java 8 just because all
> our
> > > dependencies have moved onto it, and some CVE will force us to
> move.
> > >
> > > At which point, I think its time to declare branch-2 dead. It's
> had a
> > > great life, but trying to keep java 7 support alive isn't
> sustainable.
> > Not
> > > just in this testing, but
> > > cherrypicking patches back gets more and more difficult -branch-3
> has
> > > moved on in both use of java 8 language, and in the codebase in
> general.
> > >
> > > > On 28 Jan 2019, at 20:18, Vinod Kumar Vavilapalli <
> vino...@apache.org>
> > > wrote:
> > > >
> > > > The community made a decision long time ago that we'd like to
> keep the
> > > compatibility & so tie branch-2 to Java 7, but do Java 8+ only
> work on
> > 3.x.
> > > >
> > > > I always assumed that most (all?) downstream users build
> branch-2 on
> > JDK
> > > 7 only, can anyone confirm? If so, there may be an easier way to
> address
> > > these test issues.
> > > >
> > > > +Vinod
> > > >
> > > >> On Jan 28, 2019, at 11:24 AM, Jonathan Hung <
> jyhung2...@gmail.com>
> > > wrote:
> > > >>
> > > >> Hi folks,
> > > >>
> > > >> Forking a discussion based on HADOOP-15711. To summarize, there
> are
> > > issues
> > > >> with branch-2 tests running on java 7 (openjdk) which don't
> exist on
> > > java
> > > >> 8. From our testing, the build can pass with openjdk 8.
> > > >>
>   

Re: [DISCUSS] Moving branch-2 to java 8

[Anu Engineer]

Konstantin, 

Just a nitpicky thought, if we move this branch to Java-8 on Jenkins, but still 
hope to release code that can run on Java 7, how will we detect
Java 8 only changes? I am asking because till now whenever I checked in Java 8 
features in branch-2 Jenkins would catch that issue.

With this approach, we might not find it out the issues till the release time 
when the release manager decides to compile with Java 7.
It might be more pragmatic to say that your Java 7 mileage may vary once this 
goes in, since we will have no visibility to Java 7 compatibility until it is 
too late.

Another approach could be that we create a read-only 2.x branch, then we know 
that code will work with Java 7 since the last snapshot was known to work with 
Java 7.


Thanks
Anu



On 2/1/19, 5:04 PM, "Konstantin Shvachko"  wrote:

Just to make sure we are on the same page, as the subject of this thread is
too generic and confusing.
*The proposal is to move branch-2 Jenkins builds such as precommit to run
tests on openJDK-8.*
We do not want to break Java 7 source compatibility. The sources and
releases will still depend on Java 7.
We don't see test failures discussed in HADOOP-15711 when we run them
locally with Oracle Java 7.

Thanks,
--Konst

On Fri, Feb 1, 2019 at 12:44 PM Jonathan Hung  wrote:

> Thanks Vinod and Steve, agreed about java7 compile compatibility. At least
> for now, we should be able to maintain java7 source compatibility and run
> tests on java8. There's a test run here:
> https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86-jhung/46/
> which calls a java8 specific API, installs both openjdk7/openjdk8 in the
> dockerfile, compiles on both versions, and tests on just java8 (via
>
> 
--multijdkdirs=/usr/lib/jvm/java-7-openjdk-amd64,/usr/lib/jvm/java-8-openjdk-amd64
> and --multijdktests=compile). If we eventually decide it's too much of a
> pain to maintain java7 source compatibility we can do that at a later
> point.
>
> Also based on discussion with others in the community at the contributors
> meetup this past Wednesday, seems we are generally in favor of testing
> against java8. I'll start a vote soon.
>
> Jonathan Hung
>
>
> On Tue, Jan 29, 2019 at 4:11 AM Steve Loughran 
> wrote:
>
> > branch-2 is the JDK 7 branch, but for a long time I (and presumably
> > others) have relied on jenkins to keep us honest by doing that build and
> > test
> >
> > right now, we can't do that any more, due to jdk7 bugs which will never
> be
> > fixed by oracle, or at least, not in a public release.
> >
> > If we can still do the compile in java 7 language and link to java 7 
JDK,
> > then that bit of the release is good -then java 8 can be used for that
> test
> >
> > Ultimately, we're going to be forced onto java 8 just because all our
> > dependencies have moved onto it, and some CVE will force us to move.
> >
> > At which point, I think its time to declare branch-2 dead. It's had a
> > great life, but trying to keep java 7 support alive isn't sustainable.
> Not
> > just in this testing, but
> > cherrypicking patches back gets more and more difficult -branch-3 has
> > moved on in both use of java 8 language, and in the codebase in general.
> >
> > > On 28 Jan 2019, at 20:18, Vinod Kumar Vavilapalli 
> > wrote:
> > >
> > > The community made a decision long time ago that we'd like to keep the
> > compatibility & so tie branch-2 to Java 7, but do Java 8+ only work on
> 3.x.
> > >
> > > I always assumed that most (all?) downstream users build branch-2 on
> JDK
> > 7 only, can anyone confirm? If so, there may be an easier way to address
> > these test issues.
> > >
> > > +Vinod
> > >
> > >> On Jan 28, 2019, at 11:24 AM, Jonathan Hung 
> > wrote:
> > >>
> > >> Hi folks,
> > >>
> > >> Forking a discussion based on HADOOP-15711. To summarize, there are
> > issues
> > >> with branch-2 tests running on java 7 (openjdk) which don't exist on
> > java
> > >> 8. From our testing, the build can pass with openjdk 8.
> > >>
> > >> For branch-3, the work to move the build to use java 8 was done in
> > >> HADOOP-14816 as part of the Dockerfile OS version change. 
HADOOP-16053
> > was
> > >> filed to backport this OS version change to branch-2 (but without the
> > java
> > >> 7 -> java 8 change). So my proposal is to also make the java 7 ->
> java 8
> > >> version change in branch-2.
> > >>
> > >> As mentioned in HADOOP-15711, the main issue is around source and
> binary
> > >> compatibility. I don't currently have a great answer, but one initial
> > >> thought is to build source/binary against java 7 to ensure
> compatibility
> > >> 

Re: [DISCUSS] Moving branch-2 to java 8

[Jonathan Hung]

Yeah, it's possible with yetus, there's one example here

which
runs compilation on openjdk7 (and openjdk8), and runs tests on openjdk8
only.

Jonathan Hung


On Mon, Feb 4, 2019 at 10:11 AM Steve Loughran 
wrote:

>
>
> On 2 Feb 2019, at 00:57, Konstantin Shvachko  wrote:
>
> Just to make sure we are on the same page, as the subject of this thread
> is too generic and confusing.
> *The proposal is to move branch-2 Jenkins builds such as precommit to run
> tests on openJDK-8.*
> We do not want to break Java 7 source compatibility. The sources and
> releases will still depend on Java 7.
> We don't see test failures discussed in HADOOP-15711 when we run them
> locally with Oracle Java 7.
>
> Thanks,
> --Konst
>
>
> Given the tests aren't working today, the risk that an openjdk 8 test run
> hides a problem which would show up on openjdk 7 has to consider that at
> least openjdk8 will run the tests.
>
> One thing I would like to be confident is that at least the compile phase
> of all the source (including generated source) is on jdk7, and its only the
> test run which switches JVM. Can we do that?
>

Re: [VOTE] Propose to start new Hadoop sub project "submarine"

[Steve Loughran]

+1, binding

> On 1 Feb 2019, at 22:15, Wangda Tan  wrote:
> 
> Hi all,
> 
> According to positive feedbacks from the thread [1]
> 
> This is vote thread to start a new subproject named "hadoop-submarine"
> which follows the release process already established for ozone.
> 
> The vote runs for usual 7 days, which ends at Feb 8th 5 PM PDT.
> 
> Thanks,
> Wangda Tan
> 
> [1]
> https://lists.apache.org/thread.html/f864461eb188bd12859d51b0098ec38942c4429aae7e4d001a633d96@%3Cyarn-dev.hadoop.apache.org%3E


-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: [DISCUSS] Moving branch-2 to java 8

[Steve Loughran]

On 2 Feb 2019, at 00:57, Konstantin Shvachko 
mailto:shv.had...@gmail.com>> wrote:

Just to make sure we are on the same page, as the subject of this thread is too 
generic and confusing.
The proposal is to move branch-2 Jenkins builds such as precommit to run tests 
on openJDK-8.
We do not want to break Java 7 source compatibility. The sources and releases 
will still depend on Java 7.
We don't see test failures discussed in HADOOP-15711 when we run them locally 
with Oracle Java 7.

Thanks,
--Konst

Given the tests aren't working today, the risk that an openjdk 8 test run hides 
a problem which would show up on openjdk 7 has to consider that at least 
openjdk8 will run the tests.

One thing I would like to be confident is that at least the compile phase of 
all the source (including generated source) is on jdk7, and its only the test 
run which switches JVM. Can we do that?

Re: [VOTE] Propose to start new Hadoop sub project "submarine"

[Sandeep Nemuri]

+1 (non-binding)

On Mon, Feb 4, 2019 at 10:24 PM Xiaoyu Yao  wrote:

> +1
>
> -Xiaoyu
>
> Sent from my iPhone
>
> > On Feb 4, 2019, at 7:54 AM, Suma Shivaprasad <
> sumasai.shivapra...@gmail.com> wrote:
> >
> > +1 ( non binding)
> >
> > On Mon, Feb 4, 2019 at 7:23 AM Shashikant Banerjee <
> > sbaner...@hortonworks.com> wrote:
> >
> >> +1 (non-binding)
> >>
> >> Thanks
> >> Shashi
> >>
> >> On 2/4/19, 8:27 PM, "Elek, Marton"  wrote:
> >>
> >>+1 (non-binding)
> >>
> >>(my arguments are in the discuss thread. small move, huge benefit)
> >>
> >>Thanks,
> >>Marton
> >>
> >>>On 2/1/19 11:15 PM, Wangda Tan wrote:
> >>> Hi all,
> >>>
> >>> According to positive feedbacks from the thread [1]
> >>>
> >>> This is vote thread to start a new subproject named
> >> "hadoop-submarine"
> >>> which follows the release process already established for ozone.
> >>>
> >>> The vote runs for usual 7 days, which ends at Feb 8th 5 PM PDT.
> >>>
> >>> Thanks,
> >>> Wangda Tan
> >>>
> >>> [1]
> >>>
> >>
> https://lists.apache.org/thread.html/f864461eb188bd12859d51b0098ec38942c4429aae7e4d001a633d96@%3Cyarn-dev.hadoop.apache.org%3E
> >>>
> >>
> >>-
> >>To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> >>For additional commands, e-mail: common-dev-h...@hadoop.apache.org
> >>
> >>
> >>
> >>
>
> -
> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>


-- 
*  Regards*
*  Sandeep Nemuri*

[jira] [Resolved] (HADOOP-15991) testMultipartUpload timing out

[Steve Loughran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-15991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Loughran resolved HADOOP-15991.
-
   Resolution: Duplicate
 Assignee: Steve Loughran  (was: lqjacklee)
Fix Version/s: 3.2.1
   3.3.0

> testMultipartUpload timing out
> --
>
> Key: HADOOP-15991
> URL: https://issues.apache.org/jira/browse/HADOOP-15991
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: fs/s3, test
>Affects Versions: 3.2.0
>Reporter: lqjacklee
>Assignee: Steve Loughran
>Priority: Minor
> Fix For: 3.3.0, 3.2.1
>
>
> timeout of S3 mpu tests



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: [VOTE] Propose to start new Hadoop sub project "submarine"

[Xiaoyu Yao]

+1

-Xiaoyu

Sent from my iPhone

> On Feb 4, 2019, at 7:54 AM, Suma Shivaprasad  
> wrote:
> 
> +1 ( non binding)
> 
> On Mon, Feb 4, 2019 at 7:23 AM Shashikant Banerjee <
> sbaner...@hortonworks.com> wrote:
> 
>> +1 (non-binding)
>> 
>> Thanks
>> Shashi
>> 
>> On 2/4/19, 8:27 PM, "Elek, Marton"  wrote:
>> 
>>+1 (non-binding)
>> 
>>(my arguments are in the discuss thread. small move, huge benefit)
>> 
>>Thanks,
>>Marton
>> 
>>>On 2/1/19 11:15 PM, Wangda Tan wrote:
>>> Hi all,
>>> 
>>> According to positive feedbacks from the thread [1]
>>> 
>>> This is vote thread to start a new subproject named
>> "hadoop-submarine"
>>> which follows the release process already established for ozone.
>>> 
>>> The vote runs for usual 7 days, which ends at Feb 8th 5 PM PDT.
>>> 
>>> Thanks,
>>> Wangda Tan
>>> 
>>> [1]
>>> 
>> https://lists.apache.org/thread.html/f864461eb188bd12859d51b0098ec38942c4429aae7e4d001a633d96@%3Cyarn-dev.hadoop.apache.org%3E
>>> 
>> 
>>-
>>To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
>>For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>> 
>> 
>> 
>> 

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: [VOTE] Propose to start new Hadoop sub project "submarine"

[Suma Shivaprasad]

+1 ( non binding)

On Mon, Feb 4, 2019 at 7:23 AM Shashikant Banerjee <
sbaner...@hortonworks.com> wrote:

> +1 (non-binding)
>
> Thanks
> Shashi
>
> On 2/4/19, 8:27 PM, "Elek, Marton"  wrote:
>
> +1 (non-binding)
>
> (my arguments are in the discuss thread. small move, huge benefit)
>
> Thanks,
> Marton
>
> On 2/1/19 11:15 PM, Wangda Tan wrote:
> > Hi all,
> >
> > According to positive feedbacks from the thread [1]
> >
> > This is vote thread to start a new subproject named
> "hadoop-submarine"
> > which follows the release process already established for ozone.
> >
> > The vote runs for usual 7 days, which ends at Feb 8th 5 PM PDT.
> >
> > Thanks,
> > Wangda Tan
> >
> > [1]
> >
> https://lists.apache.org/thread.html/f864461eb188bd12859d51b0098ec38942c4429aae7e4d001a633d96@%3Cyarn-dev.hadoop.apache.org%3E
> >
>
> -
> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>
>
>
>

Re: [VOTE] Propose to start new Hadoop sub project "submarine"

[Shashikant Banerjee]

+1 (non-binding)

Thanks
Shashi

On 2/4/19, 8:27 PM, "Elek, Marton"  wrote:

+1 (non-binding)

(my arguments are in the discuss thread. small move, huge benefit)

Thanks,
Marton

On 2/1/19 11:15 PM, Wangda Tan wrote:
> Hi all,
> 
> According to positive feedbacks from the thread [1]
> 
> This is vote thread to start a new subproject named "hadoop-submarine"
> which follows the release process already established for ozone.
> 
> The vote runs for usual 7 days, which ends at Feb 8th 5 PM PDT.
> 
> Thanks,
> Wangda Tan
> 
> [1]
> 
https://lists.apache.org/thread.html/f864461eb188bd12859d51b0098ec38942c4429aae7e4d001a633d96@%3Cyarn-dev.hadoop.apache.org%3E
> 

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: [VOTE] Propose to start new Hadoop sub project "submarine"

[Elek, Marton]

+1 (non-binding)

(my arguments are in the discuss thread. small move, huge benefit)

Thanks,
Marton

On 2/1/19 11:15 PM, Wangda Tan wrote:
> Hi all,
> 
> According to positive feedbacks from the thread [1]
> 
> This is vote thread to start a new subproject named "hadoop-submarine"
> which follows the release process already established for ozone.
> 
> The vote runs for usual 7 days, which ends at Feb 8th 5 PM PDT.
> 
> Thanks,
> Wangda Tan
> 
> [1]
> https://lists.apache.org/thread.html/f864461eb188bd12859d51b0098ec38942c4429aae7e4d001a633d96@%3Cyarn-dev.hadoop.apache.org%3E
> 

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

[Apache Jenkins Server]

For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1037/

[Feb 4, 2019 1:12:09 AM] (tasanuma) HDFS-14232. libhdfs is not included in 
binary tarball. Contributed by




-1 overall


The following subsystems voted -1:
asflicense findbugs hadolint pathlen unit


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

Failed junit tests :

   hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks 
   hadoop.hdfs.web.TestWebHdfsTimeouts 
   hadoop.hdfs.web.TestWebHDFS 
   hadoop.yarn.client.api.impl.TestAMRMClient 
  

   cc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1037/artifact/out/diff-compile-cc-root.txt
  [4.0K]

   javac:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1037/artifact/out/diff-compile-javac-root.txt
  [336K]

   checkstyle:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1037/artifact/out/diff-checkstyle-root.txt
  [17M]

   hadolint:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1037/artifact/out/diff-patch-hadolint.txt
  [8.0K]

   pathlen:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1037/artifact/out/pathlen.txt
  [12K]

   pylint:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1037/artifact/out/diff-patch-pylint.txt
  [104K]

   shellcheck:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1037/artifact/out/diff-patch-shellcheck.txt
  [20K]

   shelldocs:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1037/artifact/out/diff-patch-shelldocs.txt
  [12K]

   whitespace:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1037/artifact/out/whitespace-eol.txt
  [9.3M]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1037/artifact/out/whitespace-tabs.txt
  [1.1M]

   findbugs:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1037/artifact/out/branch-findbugs-hadoop-hdds_client.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1037/artifact/out/branch-findbugs-hadoop-hdds_container-service.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1037/artifact/out/branch-findbugs-hadoop-hdds_framework.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1037/artifact/out/branch-findbugs-hadoop-hdds_server-scm.txt
  [8.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1037/artifact/out/branch-findbugs-hadoop-hdds_tools.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1037/artifact/out/branch-findbugs-hadoop-ozone_client.txt
  [8.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1037/artifact/out/branch-findbugs-hadoop-ozone_common.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1037/artifact/out/branch-findbugs-hadoop-ozone_objectstore-service.txt
  [8.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1037/artifact/out/branch-findbugs-hadoop-ozone_ozone-manager.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1037/artifact/out/branch-findbugs-hadoop-ozone_ozonefs.txt
  [8.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1037/artifact/out/branch-findbugs-hadoop-ozone_s3gateway.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1037/artifact/out/branch-findbugs-hadoop-ozone_tools.txt
  [8.0K]

   javadoc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1037/artifact/out/diff-javadoc-javadoc-root.txt
  [752K]

   unit:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1037/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
  [332K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1037/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
  [84K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1037/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
  [16K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1037/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt
  [84K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1037/artifact/out/patch-unit-hadoop-hdds_client.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1037/artifact/out/patch-unit-hadoop-hdds_container-service.txt
  [4.0K]
   

[jira] [Resolved] (HADOOP-15573) s3guard set-capacity to not retry on an access denied exception

[Steve Loughran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-15573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Loughran resolved HADOOP-15573.
-
   Resolution: Duplicate
 Assignee: Steve Loughran
Fix Version/s: 3.2.0

> s3guard set-capacity to not retry on an access denied exception
> ---
>
> Key: HADOOP-15573
> URL: https://issues.apache.org/jira/browse/HADOOP-15573
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: fs/s3
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Minor
> Fix For: 3.2.0
>
>
> when you call {{hadoop s3guard set-capacity}} with restricted access, you are 
> (correctly) blocked by AWS, but the client keeps retrying. It should fail 
> fast on a 400/AccessDenied



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org