Appending on 1.x releases is available but not tested/supported and can be toggled to be disabled completely. Appending works better on 2.x releases.
On Mon, Mar 18, 2013 at 9:14 PM, Tony Burton <[email protected]> wrote: > Hi list, > > > > I’m using Hadoop 1.0.3 for a MapReduce task and I thought it might be a > simple job to append a Counter value and some text to the end of a file > (which ultimately will be in AWS S3). How wrong I was :) > > > > I’ve been reading about o.a.h.fs.FileSystem.append and whether it does or > doesn’t work in my version of hadoop, and I'm not sure if it does or not. > Can anyone say for definite either way? I'm going with the conclusion that > it can't be trusted to perform as expected. > > > > What I'd like to do is this: Perform a daily MapReduce job and add up some > some values along the way, and as well as the output of the MR job, add that > day's date with the Counter value to the end of a file (in a different > location), so the file grows by one line daily. Assuming > o.a.h.fs.FileSystem.append is currently out of favour, do you have any > suggestions for achieving this? > > > > Thanks! > > > > Tony > > > > > ***************************************************************************** > P Please consider the environment before printing this email or attachments > > > This email and any attachments are confidential, protected by copyright and > may be legally privileged. If you are not the intended recipient, then the > dissemination or copying of this email is prohibited. If you have received > this in error, please notify the sender by replying by email and then delete > the email completely from your system. Neither Sporting Index nor the sender > accepts responsibility for any virus, or any other defect which might affect > any computer or IT system into which the email is received and/or opened. It > is the responsibility of the recipient to scan the email and no > responsibility is accepted for any loss or damage arising in any way from > receipt or use of this email. Sporting Index Ltd is a company registered in > England and Wales with company number 2636842, whose registered office is at > Gateway House, Milverton Street, London, SE11 4AP. Sporting Index Ltd is > authorised and regulated by the UK Financial Services Authority (reg. no. > 150404) and Gambling Commission (reg. no. 000-027343-R-308898-001). Any > financial promotion contained herein has been issued and approved by > Sporting Index Ltd. > > > Outbound email has been scanned for viruses and SPAM -- Harsh J
