Re: [VOTE] Move Submarine source code, documentation, etc. to a separate Apache Git repo

[Sunil Govindan]

+1

- Sunil

On Sat, Aug 24, 2019 at 7:36 AM Wangda Tan  wrote:

> Hi devs,
>
> This is a voting thread to move Submarine source code, documentation from
> Hadoop repo to a separate Apache Git repo. Which is based on discussions of
>
> https://lists.apache.org/thread.html/e49d60b2e0e021206e22bb2d430f4310019a8b29ee5020f3eea3bd95@%3Cyarn-dev.hadoop.apache.org%3E
>
> Contributors who have permissions to push to Hadoop Git repository will
> have permissions to push to the new Submarine repository.
>
> This voting thread will run for 7 days and will end at Aug 30th.
>
> Please let me know if you have any questions.
>
> Thanks,
> Wangda Tan
>

[jira] [Created] (HADOOP-16533) Update jackson-databind to 2.9.9.3

[Akira Ajisaka (Jira)]

Akira Ajisaka created HADOOP-16533:
--

 Summary: Update jackson-databind to 2.9.9.3
 Key: HADOOP-16533
 URL: https://issues.apache.org/jira/browse/HADOOP-16533
 Project: Hadoop Common
  Issue Type: Improvement
Reporter: Akira Ajisaka


Now the latest version is 2.9.9.3 and that fixes regression in 2.9.9.2
https://github.com/FasterXML/jackson-databind/issues/2395



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: [VOTE] Move Submarine source code, documentation, etc. to a separate Apache Git repo

[简会洋]

+1

Thanks
Huiyang Jian

Weiwei Yang  于2019年8月27日周二 上午10:05写道：

> +1
>
> Thanks
> Weiwei
> On Aug 27, 2019, 9:40 AM +0800, Xun Liu , wrote:
> > +1 (non-binding)
> > This is the best news.
> >
> > Peter Bacsko  于2019年8月27日周二 上午4:59写道：
> >
> > > +1 (non-binding)
> > >
> > > On Sat, Aug 24, 2019 at 4:06 AM Wangda Tan 
> wrote:
> > >
> > > > Hi devs,
> > > >
> > > > This is a voting thread to move Submarine source code, documentation
> from
> > > > Hadoop repo to a separate Apache Git repo. Which is based on
> discussions
> > > of
> > > >
> > > >
> > >
> https://lists.apache.org/thread.html/e49d60b2e0e021206e22bb2d430f4310019a8b29ee5020f3eea3bd95@%3Cyarn-dev.hadoop.apache.org%3E
> > > >
> > > > Contributors who have permissions to push to Hadoop Git repository
> will
> > > > have permissions to push to the new Submarine repository.
> > > >
> > > > This voting thread will run for 7 days and will end at Aug 30th.
> > > >
> > > > Please let me know if you have any questions.
> > > >
> > > > Thanks,
> > > > Wangda Tan
> > > >
> > >
>

Re: [VOTE] Move Submarine source code, documentation, etc. to a separate Apache Git repo

[Weiwei Yang]

+1

Thanks
Weiwei
On Aug 27, 2019, 9:40 AM +0800, Xun Liu , wrote:
> +1 (non-binding)
> This is the best news.
>
> Peter Bacsko  于2019年8月27日周二 上午4:59写道：
>
> > +1 (non-binding)
> >
> > On Sat, Aug 24, 2019 at 4:06 AM Wangda Tan  wrote:
> >
> > > Hi devs,
> > >
> > > This is a voting thread to move Submarine source code, documentation from
> > > Hadoop repo to a separate Apache Git repo. Which is based on discussions
> > of
> > >
> > >
> > https://lists.apache.org/thread.html/e49d60b2e0e021206e22bb2d430f4310019a8b29ee5020f3eea3bd95@%3Cyarn-dev.hadoop.apache.org%3E
> > >
> > > Contributors who have permissions to push to Hadoop Git repository will
> > > have permissions to push to the new Submarine repository.
> > >
> > > This voting thread will run for 7 days and will end at Aug 30th.
> > >
> > > Please let me know if you have any questions.
> > >
> > > Thanks,
> > > Wangda Tan
> > >
> >

[DISCUSS] ARM/aarch64 support for Hadoop

[Vinayakumar B]

Hi Folks,

ARM is becoming famous lately in its processing capability and has got the
potential to run Bigdata workloads.
Many users have been moving to ARM machines due to its low cost.

In the past there were attempts to compile Hadoop on ARM (Rasberry PI) for
experimental purposes. Today ARM architecture is taking some of the
serverside processing as well. So there will be/is a real need of Hadoop to
support ARM architecture as well.

There are bunch of users who are trying out building Hadoop on ARM, trying
to add ARM CI to hadoop and facing issues[1]. Also some

As of today, Hadoop does not compile on ARM due to below issues, found from
testing done in openlab in [2].

1. Protobuf :
---
 Hadoop project (also some downstream projects) stuck to protobuf 2.5.0
version, due to backward compatibility reasons. Protobuf-2.5.0 is not being
maintained in the community. While protobuf 3.x is being actively adopted
widely, still protobuf 3.x provides wire compatibility for proto2 messages.
Due to some compilation issues in the generated java code, which can induce
problems in downstream. Due to this reason protobuf upgrade from 2.5.0 was
not taken up.
In 3.0.0 onwards, hadoop supports shading of libraries to avoid classpath
problem in downstream projects.
There are patches available to fix compilation in Hadoop. But need to
find a way to upgrade protobuf to latest version and still maintain the
downstream's classpath using shading feature of Hadoop build.

 There is a Jira for protobuf upgrade[3] created even before shade
support was added to Hadoop. Now need to revisit the Jira and continue
explore possibilities.

2. leveldbjni:
---
Current leveldbjni used in YARN doesnot support ARM architecture, need
to check whether any of the future versions support ARM and can hadoop
upgrade to that version.


3. hadoop-yarn-csi's dependency 'protoc-gen-grpc-java:1.15.1'
-
'protoc-gen-grpc-java:1.15.1' does not provide ARM executable by default in
the maven repository. Workaround is to build it locally and keep in local
maven repository.
Need to check whether any future versions of 'protoc-gen-grpc-java' is
having ARM executable and whether hadoop-yarn-csi can upgrade it?


Once the compilation issues are solved, then there might be many native
code related issues due to different architectures.
So to explore everything, need to join hands together and proceed.


Let us discuss and check, whether any body else out there who also need the
support of Hadoop on ARM architectures and ready to lend their hands and
time in this work.


[1] https://issues.apache.org/jira/browse/HADOOP-16358
[2]
https://issues.apache.org/jira/browse/HADOOP-16358?focusedCommentId=16904887=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16904887
[3] https://issues.apache.org/jira/browse/HADOOP-13363

-Vinay

Re: [VOTE] Merge YARN-8200 to branch-2 and branch-3.0

[Zhe Zhang]

+1 (binding)

As Jonathan said this feature in branch-2 has been running stably for over
a year. Therefore I’m supportive to the merge

On Mon, Aug 26, 2019 at 2:00 PM Jim Brennan 
wrote:

> +1 (non-binding).
> I have built branch-2 with the latest YARN-8200 patch
> (YARN-8200-branch-2.003.patch).  I ran all of the NM/RM tests and ran a few
> test jobs on a one-node cluster with default settings.
>
>
> On Mon, Aug 26, 2019 at 3:51 PM Oliver Hu  wrote:
>
> > +1 (non-binding)
> >
> > We have used this patch internally for more than a year to acquire GPU
> > reliably at LinkedIn. I don't think it is necessary to merge this to
> > branch-3.0 tho, as we are EOLing that branch.
> >
> > On Thu, Aug 22, 2019 at 4:43 PM Jonathan Hung 
> > wrote:
> >
> > > Hi folks,
> > >
> > > As per [1], starting a vote to merge YARN-8200 (and YARN-8200.branch3)
> > > feature branch to branch-2 (and branch-3.0).
> > >
> > > Vote runs for 7 days, to Thursday, Aug 29 5:00PM PDT.
> > >
> > > Thanks.
> > >
> > > [1]
> > >
> > >
> >
> http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201908.mbox/%3cCAHzWLgcX7f5Tr3q=csrqgysvpdf7mh-iu17femgx89dhr+1...@mail.gmail.com%3e
> > >
> > > Jonathan Hung
> > >
> >
>

Re: [VOTE] Merge YARN-8200 to branch-2 and branch-3.0

[Jim Brennan]

+1 (non-binding).
I have built branch-2 with the latest YARN-8200 patch
(YARN-8200-branch-2.003.patch).  I ran all of the NM/RM tests and ran a few
test jobs on a one-node cluster with default settings.


On Mon, Aug 26, 2019 at 3:51 PM Oliver Hu  wrote:

> +1 (non-binding)
>
> We have used this patch internally for more than a year to acquire GPU
> reliably at LinkedIn. I don't think it is necessary to merge this to
> branch-3.0 tho, as we are EOLing that branch.
>
> On Thu, Aug 22, 2019 at 4:43 PM Jonathan Hung 
> wrote:
>
> > Hi folks,
> >
> > As per [1], starting a vote to merge YARN-8200 (and YARN-8200.branch3)
> > feature branch to branch-2 (and branch-3.0).
> >
> > Vote runs for 7 days, to Thursday, Aug 29 5:00PM PDT.
> >
> > Thanks.
> >
> > [1]
> >
> >
> http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201908.mbox/%3cCAHzWLgcX7f5Tr3q=csrqgysvpdf7mh-iu17femgx89dhr+1...@mail.gmail.com%3e
> >
> > Jonathan Hung
> >
>

Re: [VOTE] Move Submarine source code, documentation, etc. to a separate Apache Git repo

[Peter Bacsko]

+1 (non-binding)

On Sat, Aug 24, 2019 at 4:06 AM Wangda Tan  wrote:

> Hi devs,
>
> This is a voting thread to move Submarine source code, documentation from
> Hadoop repo to a separate Apache Git repo. Which is based on discussions of
>
> https://lists.apache.org/thread.html/e49d60b2e0e021206e22bb2d430f4310019a8b29ee5020f3eea3bd95@%3Cyarn-dev.hadoop.apache.org%3E
>
> Contributors who have permissions to push to Hadoop Git repository will
> have permissions to push to the new Submarine repository.
>
> This voting thread will run for 7 days and will end at Aug 30th.
>
> Please let me know if you have any questions.
>
> Thanks,
> Wangda Tan
>

Re: [VOTE] Move Submarine source code, documentation, etc. to a separate Apache Git repo

[Sandeep Nemuri]

+1 (non-binding)

On Mon, Aug 26, 2019 at 9:59 PM Wei-Chiu Chuang  wrote:

> +1
> given the scope of the future of Submarine, I think that makes sense.
> I am also happy to see Hadoop contributors being able to commit to
> Submarine repo. I think that'll help grow the Submarine community in the
> long run.
>
> On Mon, Aug 26, 2019 at 9:26 AM Ayush Saxena  wrote:
>
> > +1
> >
> > -Ayush
> > > On 26-Aug-2019, at 9:45 PM, Bharat Viswanadham <
> > bviswanad...@cloudera.com.invalid> wrote:
> > >
> > > +1 (non-binding)
> > >
> > > Thanks,
> > > Bharat
> > >
> > > On Mon, Aug 26, 2019 at 9:13 AM Sean Busbey
>  > >
> > > wrote:
> > >
> > >> +1 (non-binding)
> > >>
> > >>> On Fri, Aug 23, 2019 at 9:06 PM Wangda Tan 
> > wrote:
> > >>>
> > >>> Hi devs,
> > >>>
> > >>> This is a voting thread to move Submarine source code, documentation
> > from
> > >>> Hadoop repo to a separate Apache Git repo. Which is based on
> > discussions
> > >> of
> > >>>
> > >>
> >
> https://lists.apache.org/thread.html/e49d60b2e0e021206e22bb2d430f4310019a8b29ee5020f3eea3bd95@%3Cyarn-dev.hadoop.apache.org%3E
> > >>>
> > >>> Contributors who have permissions to push to Hadoop Git repository
> will
> > >>> have permissions to push to the new Submarine repository.
> > >>>
> > >>> This voting thread will run for 7 days and will end at Aug 30th.
> > >>>
> > >>> Please let me know if you have any questions.
> > >>>
> > >>> Thanks,
> > >>> Wangda Tan
> > >>
> > >>
> > >>
> > >> --
> > >> busbey
> > >>
> > >> -
> > >> To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
> > >> For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
> > >>
> > >>
> >
>


-- 
*  Regards*
*  Sandeep Nemuri*

[DISCUSS] Hadoop 2.10.0 release plan

[Jonathan Hung]

Hi folks,

As discussed previously (e.g. [1], [2]) we'd like to do a 2.10.0 release
soon. Some features/big-items we're targeting for this release:

   - YARN resource types/GPU support (YARN-8200
   )
   - Selective wire encryption (HDFS-13541
   )
   - Rolling upgrade support from 2.x to 3.x (e.g. HDFS-14509
   )

Per [3] sounds like there's concern around upgrading dependencies as well.

We created a public jira filter here (
https://issues.apache.org/jira/issues/?filter=12346975) marking all
blockers for 2.10.0 release. If you have other jiras that should be 2.10.0
blockers, please mark "Target Version/s" as "2.10.0" and add label
"release-blocker" so we can track it through this filter.

We're targeting a release at end of September.

Please share any thoughts you have about this. Thanks!

[1] https://www.mail-archive.com/yarn-dev@hadoop.apache.org/msg29461.html
[2]
https://www.mail-archive.com/mapreduce-dev@hadoop.apache.org/msg21293.html
[3] https://www.mail-archive.com/yarn-dev@hadoop.apache.org/msg33440.html


Jonathan Hung

Re: [VOTE] Move Submarine source code, documentation, etc. to a separate Apache Git repo

[Wei-Chiu Chuang]

+1
given the scope of the future of Submarine, I think that makes sense.
I am also happy to see Hadoop contributors being able to commit to
Submarine repo. I think that'll help grow the Submarine community in the
long run.

On Mon, Aug 26, 2019 at 9:26 AM Ayush Saxena  wrote:

> +1
>
> -Ayush
> > On 26-Aug-2019, at 9:45 PM, Bharat Viswanadham <
> bviswanad...@cloudera.com.invalid> wrote:
> >
> > +1 (non-binding)
> >
> > Thanks,
> > Bharat
> >
> > On Mon, Aug 26, 2019 at 9:13 AM Sean Busbey  >
> > wrote:
> >
> >> +1 (non-binding)
> >>
> >>> On Fri, Aug 23, 2019 at 9:06 PM Wangda Tan 
> wrote:
> >>>
> >>> Hi devs,
> >>>
> >>> This is a voting thread to move Submarine source code, documentation
> from
> >>> Hadoop repo to a separate Apache Git repo. Which is based on
> discussions
> >> of
> >>>
> >>
> https://lists.apache.org/thread.html/e49d60b2e0e021206e22bb2d430f4310019a8b29ee5020f3eea3bd95@%3Cyarn-dev.hadoop.apache.org%3E
> >>>
> >>> Contributors who have permissions to push to Hadoop Git repository will
> >>> have permissions to push to the new Submarine repository.
> >>>
> >>> This voting thread will run for 7 days and will end at Aug 30th.
> >>>
> >>> Please let me know if you have any questions.
> >>>
> >>> Thanks,
> >>> Wangda Tan
> >>
> >>
> >>
> >> --
> >> busbey
> >>
> >> -
> >> To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
> >> For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
> >>
> >>
>

Re: [VOTE] Mark 2.6, 2.7, 3.0 release lines EOL

[Wei-Chiu Chuang]

This is starting to diverge from the original topic :)

But I would like to say
(1) If we are treating branch-2/2.0 as the final, last 2.x release, we
should be proactive in updating dependencies. I did a quick search and
found a lot of dependencies in branch-2 are very old, and some are even
susceptible to known vulnerabilities. We did a lot in 3.x, some are even
backward-incompatible. But I guess we didn't imagine people are still on
2.x a few years after 3.x went GA.

(2) I wrote a summary for the last Hadoop Contributor Meetup, and
circulated it internally. It wasn't meant for public consumption, but if it
helps, I can publish it here in the mailing list.
We will also have an upcoming blog post on the meetup at Cloudera's
website, too.

On Mon, Aug 26, 2019 at 9:16 AM Sean Busbey 
wrote:

> On Mon, Aug 26, 2019 at 9:20 AM Eric Badger 
> wrote:
> >
> > - Stuff has been going into branch-2 sporadically but I don't who is
> > actively
> > using that code other than as part of a cherrypick backwards strategy.
> >
> > - Should we do a 2.10.x release? Or just say "time to upgrade?"
> >
> > We have talked at a few different Hadoop contributors meetups and
> community
> > syncs and agreed that 2.10 should serve as a "bridge" release for 3.x so
> > that 2.x clients can talk to 3.x servers and vice versa without
> > compatibility issues. At the last meeting where we discussed this it
> seemed
> > that there were 3 major compatibility issues. The biggest one was in the
> > edit logs.
> >
>
>
> Just a quick friendly reminder to everyone that stuff has to make it
> to the mailing lists for Hadoop for it to be considered a discussion
> that happened "in the community". Especially decisions and the
> discussions that lead to them have to happen on our mailing lists and
> not at in person events.
>
> --
> busbey
>
> -
> To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
>
>

Re: [VOTE] Move Submarine source code, documentation, etc. to a separate Apache Git repo

[Ayush Saxena]

+1

-Ayush
> On 26-Aug-2019, at 9:45 PM, Bharat Viswanadham 
>  wrote:
> 
> +1 (non-binding)
> 
> Thanks,
> Bharat
> 
> On Mon, Aug 26, 2019 at 9:13 AM Sean Busbey 
> wrote:
> 
>> +1 (non-binding)
>> 
>>> On Fri, Aug 23, 2019 at 9:06 PM Wangda Tan  wrote:
>>> 
>>> Hi devs,
>>> 
>>> This is a voting thread to move Submarine source code, documentation from
>>> Hadoop repo to a separate Apache Git repo. Which is based on discussions
>> of
>>> 
>> https://lists.apache.org/thread.html/e49d60b2e0e021206e22bb2d430f4310019a8b29ee5020f3eea3bd95@%3Cyarn-dev.hadoop.apache.org%3E
>>> 
>>> Contributors who have permissions to push to Hadoop Git repository will
>>> have permissions to push to the new Submarine repository.
>>> 
>>> This voting thread will run for 7 days and will end at Aug 30th.
>>> 
>>> Please let me know if you have any questions.
>>> 
>>> Thanks,
>>> Wangda Tan
>> 
>> 
>> 
>> --
>> busbey
>> 
>> -
>> To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
>> For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
>> 
>> 

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: [VOTE] Mark 2.6, 2.7, 3.0 release lines EOL

[Sean Busbey]

On Mon, Aug 26, 2019 at 9:20 AM Eric Badger  wrote:
>
> - Stuff has been going into branch-2 sporadically but I don't who is
> actively
> using that code other than as part of a cherrypick backwards strategy.
>
> - Should we do a 2.10.x release? Or just say "time to upgrade?"
>
> We have talked at a few different Hadoop contributors meetups and community
> syncs and agreed that 2.10 should serve as a "bridge" release for 3.x so
> that 2.x clients can talk to 3.x servers and vice versa without
> compatibility issues. At the last meeting where we discussed this it seemed
> that there were 3 major compatibility issues. The biggest one was in the
> edit logs.
>


Just a quick friendly reminder to everyone that stuff has to make it
to the mailing lists for Hadoop for it to be considered a discussion
that happened "in the community". Especially decisions and the
discussions that lead to them have to happen on our mailing lists and
not at in person events.

-- 
busbey

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: [VOTE] Move Submarine source code, documentation, etc. to a separate Apache Git repo

[Sean Busbey]

+1 (non-binding)

On Fri, Aug 23, 2019 at 9:06 PM Wangda Tan  wrote:
>
> Hi devs,
>
> This is a voting thread to move Submarine source code, documentation from
> Hadoop repo to a separate Apache Git repo. Which is based on discussions of
> https://lists.apache.org/thread.html/e49d60b2e0e021206e22bb2d430f4310019a8b29ee5020f3eea3bd95@%3Cyarn-dev.hadoop.apache.org%3E
>
> Contributors who have permissions to push to Hadoop Git repository will
> have permissions to push to the new Submarine repository.
>
> This voting thread will run for 7 days and will end at Aug 30th.
>
> Please let me know if you have any questions.
>
> Thanks,
> Wangda Tan



-- 
busbey

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-16532) TestViewFsTrash uses home directory of real fs; brittle

[Steve Loughran (Jira)]

Steve Loughran created HADOOP-16532:
---

 Summary: TestViewFsTrash uses home directory of real fs; brittle
 Key: HADOOP-16532
 URL: https://issues.apache.org/jira/browse/HADOOP-16532
 Project: Hadoop Common
  Issue Type: Bug
  Components: fs, test
Affects Versions: 3.3.0
Reporter: Steve Loughran


the test {{TestViewFsTrash}} uses the .Trash directory under the current user's 
home dir so

* fails in test setups which block writing to it (jenkins)
* fails when users have real trash in there
* may fail if there are parallel test runs.

the home dir should be under some test path of the build.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: [VOTE] Move Submarine source code, documentation, etc. to a separate Apache Git repo

[Dinesh Chitlangia]

+1 (non-binding)

-Dinesh




On Mon, Aug 26, 2019 at 11:42 AM Wanqiang Ji  wrote:

> +1 (non-binding)
>
> Thanks,
> Wanqiang Ji
>
> On Sat, Aug 24, 2019 at 10:06 AM Wangda Tan  wrote:
>
> > Hi devs,
> >
> > This is a voting thread to move Submarine source code, documentation from
> > Hadoop repo to a separate Apache Git repo. Which is based on discussions
> of
> >
> >
> https://lists.apache.org/thread.html/e49d60b2e0e021206e22bb2d430f4310019a8b29ee5020f3eea3bd95@%3Cyarn-dev.hadoop.apache.org%3E
> >
> > Contributors who have permissions to push to Hadoop Git repository will
> > have permissions to push to the new Submarine repository.
> >
> > This voting thread will run for 7 days and will end at Aug 30th.
> >
> > Please let me know if you have any questions.
> >
> > Thanks,
> > Wangda Tan
> >
>

Re: [VOTE] Move Submarine source code, documentation, etc. to a separate Apache Git repo

[Wanqiang Ji]

+1 (non-binding)

Thanks,
Wanqiang Ji

On Sat, Aug 24, 2019 at 10:06 AM Wangda Tan  wrote:

> Hi devs,
>
> This is a voting thread to move Submarine source code, documentation from
> Hadoop repo to a separate Apache Git repo. Which is based on discussions of
>
> https://lists.apache.org/thread.html/e49d60b2e0e021206e22bb2d430f4310019a8b29ee5020f3eea3bd95@%3Cyarn-dev.hadoop.apache.org%3E
>
> Contributors who have permissions to push to Hadoop Git repository will
> have permissions to push to the new Submarine repository.
>
> This voting thread will run for 7 days and will end at Aug 30th.
>
> Please let me know if you have any questions.
>
> Thanks,
> Wangda Tan
>

Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

[Apache Jenkins Server]

For more details, see 
https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/425/

No changes


[Error replacing 'FILE' - Workspace is not accessible]

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

[Apache Jenkins Server]

For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1240/

No changes


[Error replacing 'FILE' - Workspace is not accessible]

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: [VOTE] Mark 2.6, 2.7, 3.0 release lines EOL

[Eric Badger]

- Stuff has been going into branch-2 sporadically but I don't who is
actively
using that code other than as part of a cherrypick backwards strategy.

- Should we do a 2.10.x release? Or just say "time to upgrade?"

We have talked at a few different Hadoop contributors meetups and community
syncs and agreed that 2.10 should serve as a "bridge" release for 3.x so
that 2.x clients can talk to 3.x servers and vice versa without
compatibility issues. At the last meeting where we discussed this it seemed
that there were 3 major compatibility issues. The biggest one was in the
edit logs.

Verizon Media is currently working on upgrading to branch-2 from
branch-2.8. I believe LinkedIn is also interested in branch-2 and working
towards a 2.10 release.

Eric

On Mon, Aug 26, 2019 at 9:03 AM Steve Loughran 
wrote:

> On Sat, Aug 24, 2019 at 2:25 AM Wangda Tan  wrote:
>
> > Hi Steve,
> >
> > The proposal is to EOL for the following branches:
> >
> > [2.0.x - 2.7.x]
> > [3.0.x]
> >
> > 2.8.x, 2.9.x, 2.10.x (not released yet), 3.1.x (and later) are not EOL.
> >
>
> one final 2.8.x, 1+ for 2.9, and then we have to start thinking 2.10 as a
> "last ever branch- release".
>
> Stuff has been going into branch-2 sporadically but I don't who is actively
> using that code other than as part of a cherrypick backwards strategy.
>
> Should we do a 2.10.x release? Or just say "time to upgrade?"
>
>
>
> > Thoughts?
> >
> > On Sat, Aug 24, 2019 at 1:40 AM Steve Loughran 
> > wrote:
> >
> >>
> >>
> >> On Wed, Aug 21, 2019 at 4:03 AM Wangda Tan  wrote:
> >>
> >>> Hi all,
> >>>
> >>> This is a vote thread to mark any versions smaller than 2.7
> (inclusive),
> >>> and 3.0 EOL. This is based on discussions of [1]
> >>>
> >>
> >> 3.0 inclusive? i.e the non EOl ones being 2.8+ and 3.1+?
> >>
> >
>

Re: [VOTE] Move Submarine source code, documentation, etc. to a separate Apache Git repo

[Steve Loughran]

+1

On Sat, Aug 24, 2019 at 3:06 AM Wangda Tan  wrote:

> Hi devs,
>
> This is a voting thread to move Submarine source code, documentation from
> Hadoop repo to a separate Apache Git repo. Which is based on discussions of
>
> https://lists.apache.org/thread.html/e49d60b2e0e021206e22bb2d430f4310019a8b29ee5020f3eea3bd95@%3Cyarn-dev.hadoop.apache.org%3E
>
> Contributors who have permissions to push to Hadoop Git repository will
> have permissions to push to the new Submarine repository.
>
> This voting thread will run for 7 days and will end at Aug 30th.
>
> Please let me know if you have any questions.
>
> Thanks,
> Wangda Tan
>

Re: [VOTE] Mark 2.6, 2.7, 3.0 release lines EOL

[Steve Loughran]

On Sat, Aug 24, 2019 at 2:25 AM Wangda Tan  wrote:

> Hi Steve,
>
> The proposal is to EOL for the following branches:
>
> [2.0.x - 2.7.x]
> [3.0.x]
>
> 2.8.x, 2.9.x, 2.10.x (not released yet), 3.1.x (and later) are not EOL.
>

one final 2.8.x, 1+ for 2.9, and then we have to start thinking 2.10 as a
"last ever branch- release".

Stuff has been going into branch-2 sporadically but I don't who is actively
using that code other than as part of a cherrypick backwards strategy.

Should we do a 2.10.x release? Or just say "time to upgrade?"



> Thoughts?
>
> On Sat, Aug 24, 2019 at 1:40 AM Steve Loughran 
> wrote:
>
>>
>>
>> On Wed, Aug 21, 2019 at 4:03 AM Wangda Tan  wrote:
>>
>>> Hi all,
>>>
>>> This is a vote thread to mark any versions smaller than 2.7 (inclusive),
>>> and 3.0 EOL. This is based on discussions of [1]
>>>
>>
>> 3.0 inclusive? i.e the non EOl ones being 2.8+ and 3.1+?
>>
>

[jira] [Reopened] (HADOOP-16193) add extra S3A MPU test to see what happens if a file is created during the MPU

[Ewan Higgs (Jira)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-16193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ewan Higgs reopened HADOOP-16193:
-

I must have made a mistake in testing this. Re-reviewing it, it's not possible 
for this to work and indeed it fails when I test it again.

When writing to S3, the 'winner' will be the 
last-started-upload-that-completed-successfully. In the case of MPU this start 
is at init time, not complete time. So when the transient file comes in and is 
deleted then the MPU becomes moot.

In the case of a versioned bucket, then the transient file doesn't unroll the 
latest version to the MPU because all the 'latest' knowledge is based on init 
time, not complete time.

I will revert the commit.

> add extra S3A MPU test to see what happens if a file is created during the MPU
> --
>
> Key: HADOOP-16193
> URL: https://issues.apache.org/jira/browse/HADOOP-16193
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: fs/s3
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Minor
> Fix For: 3.1.3
>
>
> Proposed extra test for the S3A MPU: if you create and then delete a file 
> while an MPU is in progress, when you finally complete the MPU the new data 
> is present.
> This verifies that the other FS operations don't somehow cancel the 
> in-progress upload, and that eventual consistency brings the latest value out.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: [VOTE] Move Submarine source code, documentation, etc. to a separate Apache Git repo

[kevin su]

+1 (non-binding)

Thanks,
Kevin

Akira Ajisaka  於 2019年8月26日 週一 下午1:09寫道：

> +1
>
> Thanks,
> Akira
>
> On Sat, Aug 24, 2019 at 11:06 AM Wangda Tan  wrote:
> >
> > Hi devs,
> >
> > This is a voting thread to move Submarine source code, documentation from
> > Hadoop repo to a separate Apache Git repo. Which is based on discussions
> of
> >
> https://lists.apache.org/thread.html/e49d60b2e0e021206e22bb2d430f4310019a8b29ee5020f3eea3bd95@%3Cyarn-dev.hadoop.apache.org%3E
> >
> > Contributors who have permissions to push to Hadoop Git repository will
> > have permissions to push to the new Submarine repository.
> >
> > This voting thread will run for 7 days and will end at Aug 30th.
> >
> > Please let me know if you have any questions.
> >
> > Thanks,
> > Wangda Tan
>
> -
> To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
>
>