stack wrote:
The below proposes pushing a working append out 6 months or so.
Can we have pointers as to what is meant by 'quality issues' in the below so
we can make a more informed vote or is it just the hdfs issue posted against
0.19.1?
Is it there also that we would find what is involved making append
work in 0.19.1?
If one knew what is enough to fix properly, it would be easy. But over
last couple of months, there have been many fixes (some of these jiras
are listed in one Konstantins HADOOP-4663). The discussions are still
bringing up more cases where the implementation or algorithm should
change. But these are improvements for sure. But doubt if I would be
ready to call it is 'completely fixed'. It needs time and a lot of
testing in large clusters.
Personally I am +1 for getting these into 0.19 branch. Most importantly
even clusters and application not using append or sync were also
affected, thats why extra caution.
my 2 cents. hope this does not digress too much from the main topic.
Raghu.
Thanks,
St.Ack
On Fri, Jan 30, 2009 at 2:36 AM, Steve Loughran <ste...@apache.org> wrote:
Nigel Daley wrote:
Folks,
Some Hadoop deployments have upgraded to 0.19.0. Clearly, the 0.19 branch
has issues and a 0.19.1 release is needed.
Quality issues in the changes made for the file append feature have
prevented some from deploying Hadoop 0.19. One of these changes (sync) has
now been "fixed" by reducing its semantics in Hadoop 0.18.3 (HADOOP-4997).
This was necessary to stabilize the 0.18 branch.
I would like to propose that we apply this same "fix" to sync in 0.19.1
and 0.20.0. Since append requires the full semantics of sync, I propose we
also disable append (perhaps throw UnsupportedOperationException from API?).
Yes, this would unfortunately be an incompatible change between 0.19.0 and
0.19.1. We can then take the time needed to fix append properly in 0.21.0.
I can see some people being unhappy about this, but giving them a choice
between having the filesystem work or not, hopefully they will see the
merits of the change. And I am +1 to taking time to fix things; fast fixes
often create new problems
