*merge* and sort them to only one file on *local fs*. <src> is kept. Are you sure that you want to merge several HDFS files into one LOCAL file? Local file would be in your local file system. The simples way is to use union in pig and union existig files in HDFS with new one generated by pig script. The other way is to write pig script with one reduce action
2013/7/18 Bhavesh Shah <[email protected]> > Thanks for reply. :) > > I just came across one command -getmerge > > > > -getmerge <src> <localdst>: Get all the files in the directories that > match the source file pattern and merge and sort them to only > one file on local fs. <src> is kept. > > > > I am thinking if I STORE the data in some other file say TMP_Name > > and later If I use this command to dump the data in the required file. > > > > Is it possible to merge the data using this command in PIG? If yes, then > is it good way to achieve my goal? > > Please let me know. > > > > > > Many Thanks, > > Bhavesh Shah > > > > > > > > Date: Thu, 18 Jul 2013 15:49:47 +0400 > > Subject: Re: Want to add data in same file in Apache PIG? > > From: [email protected] > > To: [email protected] > > CC: [email protected] > > > > it's not possible. It's HDFS. > > > > > > 2013/7/18 Bhavesh Shah <[email protected]> > > > > > Hello, > > > > > > Actually I have a use case in which I will receive the data from some > > > source and I have to dump it in the same file after every regular > interval > > > and use that file for further operation. I tried to search on it, but I > > > didn't see the anything related to this. > > > > > > I am using STORE function, but STORE function always create new file > with > > > specified name and gives error if the specified file already exists. > > > How should I do store the data in same file? Is it possible in Pig or > have > > > some work around for it? > > > Please suggest me some solution over this. > > > > > > > > > Thanks, > > > Bhavesh Shah > >
