Thanks Jim.
Dhruba, can we move
https://issues.apache.org/jira/browse/HADOOP-4379
to 0.21.0?
Nige
On Feb 26, 2009, at 10:37 AM, Jim Kellerman (POWERSET) wrote:
With the availability of HADOOP-5332 I remove my objection.
-----Original Message-----
From: Dhruba Borthakur [mailto:[email protected]]
Sent: Wednesday, February 25, 2009 9:32 PM
To: [email protected]
Subject: Re: Hadoop 0.20.0
I posted a patch for HADOOP-5332. I am suggesting that this patch be
applied
into the 0.19, 0.20 and trunk. This patch switches off "append" by
default,
but it can be switched on by setting the config parameter
dfs.support.append. This does not mean that "append" is bug free in
the
code, it just allows developers to continue testing with append
functionality till the bugs are fixed.
thanks,
dhruba
On Wed, Feb 25, 2009 at 9:05 PM, Hemanth Yamijala <yhema...@yahoo-
inc.com>wrote:
+1 for HADOOP-5332. I am in the same position as Brian, as an
outside
observer. This will help us to move on Hadoop 0.20 which has a lot
of
other
features as well that users are asking for.
Thanks
hemanth
On Feb 25, 2009, at 10:20 PM, Nigel Daley wrote:
On Feb 25, 2009, at 7:52 PM, Dhruba Borthakur wrote:
"Whipping out a patch" says nothing about its reliability.
i would like some focus from the developer's community to
properly
fix
this
issue. I am willing to spend as much as time it takes ot get it
fixed
the
right way, I but I would like even more constructive engagement
from
more
people to get this one right. May I request you to see if you can
volunteer
to spend some time testing some of this code at scale ?(I have
access to
10
machines only for testing).
Dhruba, can you define "testing some of this code at scale"? Do you
simply
need access or folks who can run challenging jobs? Scaring up
access
to the
cluster can be easy, but admin / user time isn't really available.
Sorry, I can't commit any time/resources to this right now. Perhaps
some
hbase folks can. In the meantime, can we make append
configurable in
0.19.2
and 0.20.0? I filed
https://issues.apache.org/jira/browse/HADOOP-5332
As an outside, irrelevant observer, I think this is a really good
compromise. Helps out HBase but also would help prevent rushing.
Brian
Cheers,
Nige
thanks
dhruba
On Wed, Feb 25, 2009 at 7:34 PM, Nigel Daley <[email protected]
>
wrote:
On Feb 24, 2009, at 9:28 PM, Dhruba Borthakur wrote:
Hi Jim,
I can understand your problem. I can probably whip out a fix
for
HADOOP-4663 and HADOOP-4379 by the end of this week. It would
be
nice
if
somebody else (Hairong, Sanjay, Konstantin?) can volunteer to
discuss
and
review the patches/fixes.
"Whipping out a patch" doesn't give me any confidence that this
feature
will be fixed properly. We're building a file system. Data
reliability
and
accuracy are absolutely key. We know that this feature has been
very
lightly tested.
Nigel: wht is the proposed deadline for 0.20?
March 6.
Nige
thanks,
dhruba
On Tue, Feb 24, 2009 at 4:25 PM, Jim Kellerman (POWERSET) <
[email protected]> wrote:
--1
HBase really needs 4379. My testing to date indicates that it
does
work
(although I have a bit more testing to do).
I was ok with not putting it into 0.19.1 provided it was in
0.19.2
and
0.20.0.
It's a big problem for us now and is hurting our ability to
keep
our
community alive. (They will go to Cassandra or something
else to
ensure
reliability).
-----Original Message-----
From: Nigel Daley [mailto:[email protected]]
Sent: Tuesday, February 24, 2009 4:02 PM
To: [email protected]
Subject: Hadoop 0.20.0
Folks,
Hadoop 0.19.1 is now available with the file append feature
disabled.
It's time to talk about a Hadoop 0.20.0 release.
Hadoop 0.20.0 feature freeze date was almost 3 months ago.
The
last
few blockers are now almost fixed (should be next week)
except
for
HADOOP-4379. HADOOP-4379 is work that is needed to properly
implement
file append.
*** I propose we move HADOOP-4379 off to release 0.21.0 and
apply
the
same disabling of file append in Hadoop 0.20.0 that we put in
place
to
get 0.19.1 released (HADOOP-5224 and HADOOP-5225).
I will call a vote for 0.20.0 when blockers are fixed.
Cheers,
Nigel
Folks,
Some Hadoop deployments have upgraded to 0.19.0. Clearly,
the
0.19
branch has issues and a 0.19.1 release is needed.
Quality issues in the changes made for the file append
feature
have
prevented some from deploying Hadoop 0.19. One of these
changes
(sync) has now been "fixed" by reducing its semantics in
Hadoop
0.18.3 (HADOOP-4997). This was necessary to stabilize the
0.18
branch.
I would like to propose that we apply this same "fix" to
sync
in
0.19.1 and 0.20.0. Since append requires the full
semantics of
sync, I propose we also disable append (perhaps throw
UnsupportedOperationException from API?). Yes, this would
unfortunately be an incompatible change between 0.19.0 and
0.19.1.
We can then take the time needed to fix append properly in
0.21.0.
I will call a vote for 0.19.1 and 0.20.0 when blockers are
fixed.
Nigel
