Or as Brock said, you can refer to the link he posted and use the example from the user guide instead, then you'll need to include this:
agent.sources = avro-AppSrv-sourceagent.sinks = hdfs-Cluster1-sinkagent.channels = mem-channel-1 ... but that example uses an Avro source so you'll likely need to start an avro-client to test (or use Flume SDK). Or just change the source type to SEQ. Cheers, Will On Mon, Jul 23, 2012 at 6:07 PM, mardan Khan <[email protected]> wrote: > > > > Thanks Brocks, > > I have just gone through the posted link and just copy past the one of > configuration file and change the hdfs path as below: > > > > # properties of avro-AppSrv-source > agent.sources.avro-AppSrv-source.type = avro > agent.sources.avro-AppSrv-source.bind = localhost > agent.sources.avro-AppSrv-source.port = 10000 > > # properties of mem-channel-1 > agent.channels.mem-channel-1.type = memory > agent.channels.mem-channel-1.capacity = 1000 > agent.channels.mem-channel-1.transactionCapacity = 100 > > # properties of hdfs-Cluster1-sink > agent.sinks.hdfs-Cluster1-sink.type = hdfs > agent.sinks.hdfs-Cluster1-sink.hdfs.path = hdfs:// > 134.83.35.24/user/mardan/flume/ > > > apply the following command: > > $ /usr/bin/flume-ng agent -n agent -c conf -f > /usr/lib/flume-ng/conf/flume.conf > > > and got the following error. Most of the time of getting this error > > 12/07/24 01:54:43 ERROR properties.PropertiesFileConfigurationProvider: > Failed to load configuration data. Exception follows. > java.lang.NullPointerException > at > org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.loadSources(PropertiesFileConfigurationProvider.java:324) > at > org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.load(PropertiesFileConfigurationProvider.java:222) > at > org.apache.flume.conf.file.AbstractFileConfigurationProvider.doLoad(AbstractFileConfigurationProvider.java:123) > at > org.apache.flume.conf.file.AbstractFileConfigurationProvider.access$300(AbstractFileConfigurationProvider.java:38) > at > org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:202) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) > at > java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) > > I think some thing wrong in the configuration file. I am using flume1.x > version and installed in /usr/lib/flume-ng/ > > Could you please check the command and configuration file. > > Thanks > > > > > > > > On Tue, Jul 24, 2012 at 1:33 AM, Brock Noland <[email protected]> wrote: > >> Yes, you can do that. In fact that is the most common case. The >> documents which should help you do so are here: >> >> https://cwiki.apache.org/confluence/display/FLUME/Flume+1.x+Documentation >> >> Brock >> >> On Mon, Jul 23, 2012 at 7:26 PM, mardan Khan <[email protected]> >> wrote: >> > Hi, >> > >> > I am just doing testing. I am generating the sequence and want to upload >> > into hdfs. My configuration file as: >> > >> > agent2.channels = c1 >> > agent2.sources = r1 >> > agent2.sinks = k1 >> > >> > agent2.channels.c1.type = MEMORY >> > >> > agent2.sources.r1.channels = c1 >> > agent2.sources.r1.type = SEQ >> > >> > agent2.sinks.k1.channel = c1 >> > agent2.sinks.k1.type = LOGGER >> > >> > >> > Is it possible to upload into hdfs, if possible then how I can make the >> > changes in configuration file. >> > >> > >> > Many thanks >> > >> >> >> >> -- >> Apache MRUnit - Unit testing MapReduce - >> http://incubator.apache.org/mrunit/ >> > >
