Really looking forward to getting this in.
Couple of questions:
* Can you maybe comment a bit on the type of scale testing done ?
Specifically, the number of resources tested with and any point where it is
discovered that performance might take a hit. Also, given that we do not
have AM's that curr
Hi all,
I've written up a status report for the current state of Hadoop 3 on the
wiki. I've also pasted it below for your convenience.
Cheers,
Andrew
https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3+release+status+updates
2017-08-25
Another month flew by without an update. This is a
Jonathan, thanks for the heads up. I don't have much familiarity with YARN,
but gave the PBs and pom changes a look, and left a few small comments on
the umbrella JIRA.
This seems like a smaller change than some of the other branch merges we're
discussing, but I'm again reticent about adding scope
Here's a summary of some 1-on-1 conversations I had with contributors of
the different features I'm tracking.
Storage Policy Satisfier (Uma Gangumalla)
* Target version: 3.1.0, maybe 3.0.0 GA
* We're resolving some design discussions on JIRA. Plan is to do some MVP
work on the API to get this into
Hi Subru,
Thanks for your vote and your response!
Regarding your question about merging to branch2, I think, after the merge
to trunk is done, it will be good to merge to branch-2 as soon as we can.
A lot of testing with trunk has been done presently and it will be good to
repeat the same with th
> From a release management perspective, it's *extremely* reasonable to block
> the inclusion of new features a month from the planned release date. A
> typical software development lifecycle includes weeks of feature freeze and
> weeks of code freeze. It is no knock on any developer or any feat
Thanks Subru for voting.
> What are the timelines you are looking for getting this into branch-2?
We haven't yet decided on it and were thinking of discussing this in detail
within the team after merge to trunk.
The timelines would depend on whether we release whatever we merge to trunk
in 2.9 o
+1 (binding).
I have been following the effort and had few design discussions around the
team especially about how it integrates with Federation. Overall I feel
it's a welcome improvement to YARN.
What are the timelines you are looking for getting this into branch-2?
Thanks,
Subru
On Fri, Aug 2
Allen Wittenauer wrote:
> Doesn't this place an undue burden on the contributor with the first
> incompatible patch to prove worthiness? What happens if it is decided that
> it's not good enough?
It is a burden for that first, "this can't go anywhere else but 4.x"
change, but arguably that sho
Plan looks good to me.
+1
Regards,
Uma
On 8/25/17, 10:36 AM, "Andrew Wang" wrote:
>Hi folks,
>
>With 3.0.0-beta1 fast approaching, I wanted to go over the proposed
>branching strategy.
>
>In the early 2.x days, moving trunk immediately to 3.0.0 was a mistake.
>branch-2 and trunk were virtually
Hi Andrew,
Thanks for updating the proposal, +1.
- Wangda
On Fri, Aug 25, 2017 at 12:16 PM, Allen Wittenauer wrote:
>
> > On Aug 25, 2017, at 10:36 AM, Andrew Wang
> wrote:
>
> > Until we need to make incompatible changes, there's no need for
> > a Hadoop 4.0 version.
>
> Some questions:
>
>
Hi Andrew,
Thanks for starting the discussion - we have a feature YARN-5734 for API
based scheduler configuration that I feel is pretty close to merge (also "a
few weeks"). It's almost completely code and API additions and we were
careful to design it so that it's compatible (feature is also turne
Thank you everyone for reviewing and voting on the S3Guard feature branch
merge.
It looks like the Vote was a success. We have six binding +1's (Steve
Loughran, Sean Mackrory, Mingliang Liu, Sanjay Radia, Kihwal Lee, and Lei
(Eddy) Xu) and zero -1's.
I will coordinate w/ Steve L to get this commi
> On Aug 25, 2017, at 10:36 AM, Andrew Wang wrote:
> Until we need to make incompatible changes, there's no need for
> a Hadoop 4.0 version.
Some questions:
Doesn't this place an undue burden on the contributor with the first
incompatible patch to prove worthiness? What happens if it
> On Aug 25, 2017, at 10:00 AM, Steve Loughran wrote:
>
> Catching up on this. Looks like I don't have a hadoop-aws profile, which
> explains a lot, doesn't it.
Yes. This is exactly the type of failure I'd expect.
> How do those profiles get created/copied in?
Maven kludgery.
+1 from me too for the branching proposal.
On Fri, Aug 25, 2017 at 12:56 PM, Eric Payne <
eric.payne1...@yahoo.com.invalid> wrote:
> +1 for this branching proposal.-Eric
>
>
> From: Andrew Wang
> To: "common-dev@hadoop.apache.org" ; "
> mapreduce-...@hadoop.apache.org" ; "
> hdfs-...@hado
Resource profile is similar to TSv2, the feature is:
- Alpha feature, we will not freeze new added APIs. And all added APIs are
explicitly marked to @Unstable.
- Allow rolling upgrade from branch-2.
- Touched existing code, but we have, and will continue tests to make sure
changes are safe.
Discus
+1 for this branching proposal.-Eric
From: Andrew Wang
To: "common-dev@hadoop.apache.org" ;
"mapreduce-...@hadoop.apache.org" ;
"hdfs-...@hadoop.apache.org" ;
"yarn-...@hadoop.apache.org"
Sent: Friday, August 25, 2017 12:36 PM
Subject: [DISCUSS] Branches and versions for Hadoop 3
On 25 August 2017 at 22:39, Andrew Wang wrote:
> Hi Rohith,
>
> Given that we're advertising TSv2 as an alpha feature, I think we're
> allowed to break compatibility. Let's make sure this is clear in the
> release notes and documentation.
>
> That said, with TSv2 phase 2, is the API going to be
Hi folks,
With 3.0.0-beta1 fast approaching, I wanted to go over the proposed
branching strategy.
In the early 2.x days, moving trunk immediately to 3.0.0 was a mistake.
branch-2 and trunk were virtually identical, and increased backport
complexity. Until we need to make incompatible changes, the
Hi Jason,
I agree with this proposal. I'll start another email thread spelling this
out, and gather additional feedback.
Best,
Andrew
On Fri, Aug 25, 2017 at 6:27 AM, Jason Lowe wrote:
> Andrew Wang wrote:
>
>
>> This means I'll cut branch-3 and
>> branch-3.0, and move trunk to 4.0.0 before th
Hi Rohith,
Given that we're advertising TSv2 as an alpha feature, I think we're
allowed to break compatibility. Let's make sure this is clear in the
release notes and documentation.
That said, with TSv2 phase 2, is the API going to be frozen? The umbrella
JIRA refers to "TSv2 alpha2" which indica
+1 (binding)
I've built the current branch, and checked out a few basic areas including
documentation. Also perused the most recent changes that went in.
Thanks much for the great team work! I look forward to seeing it in action.
Regards,
Sangjin
On Fri, Aug 25, 2017 at 9:27 AM, Haibo Chen wro
On 22 Aug 2017, at 17:24, Allen Wittenauer
mailto:a...@effectivemachines.com>> wrote:
Ugly error, but still no CNFE. So at least out of the box with a build from
last week. I guess this is working? At this point, it’d probably be worthwhile
to make sure that the libexec/shellprofile.d/hadoop-
John Zhuge created HADOOP-14808:
---
Summary: Hadoop keychain
Key: HADOOP-14808
URL: https://issues.apache.org/jira/browse/HADOOP-14808
Project: Hadoop Common
Issue Type: New Feature
R
For more details, see
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/503/
[Aug 24, 2017 7:26:37 AM] (jzhuge) HDFS-12318. Fix IOException condition for
openInfo in DFSInputStream.
[Aug 24, 2017 11:04:38 AM] (bibinchundatt) YARN-7074. Fix NM state store update
comment. Contributed
+1 from my side.
More from the perspective of ensuring there is no impact of ATSv2 when it
is off (by default), I deployed the latest YARN-5355 bits into a few
clusters and ran internal Smoke tests. The tests shows no impact when ATSv2
is off.
Best,
Haibo
On Thu, Aug 24, 2017 at 7:51 AM, Sunil G
Andrew Wang wrote:
> This means I'll cut branch-3 and
> branch-3.0, and move trunk to 4.0.0 before these VOTEs end. This will open
> up development for Hadoop 3.1.0 and 4.0.0.
I can see a need for branch-3.0, but please do not create branch-3. Doing
so will relegate trunk back to the "patch pu
28 matches
Mail list logo
