Re: How to get large data into HDFS

2011-03-29 Thread Eric
Hi Will, In theory, your only bottleneck is the network and the amount of datanodes you have running, so it should scale quite well. I'm very interested to hear about your experiences after adding more writers. Thanks, Eric 2011/3/29 Will Maier > Hi Eric- > > On Tue, Mar 29, 2011 at 03:20:38PM

Re: How to get large data into HDFS

2011-03-29 Thread Will Maier
Hi Eric- On Tue, Mar 29, 2011 at 03:20:38PM +0200, Eric wrote: > I'm interested in hearing how you get data into and out of HDFS. Are you > using tools like Flume? Are you using fuse_dfs? Are you putting files on > HDFS with "hadoop dfs -put ..."? > And how does your method scale? Can you move ter