How many files do you have? Each time a new file is to be selected, a file listing is done. If you a lot of files, this could be slow.
Thanks, Hari On Thu, Oct 9, 2014 at 10:41 PM, [email protected] <[email protected]> wrote: > i'm currently using CDH5's flume-ng. i've been testing an agent whose sink is > hdfs and source is spooldir. when i put files around the size of 30m into the > spooldir, this agent worked perfectly well, but when i put files around 80m > the agent is still working but the transfer speed is ridiculously slow. maybe > have to spend an hour to tranfer a 80m file. does anyone have any clue why > this is happening??? thx. > [email protected]
