Thanks Brocks,
I have just gone through the posted link and just copy past the one of
configuration file and change the hdfs path as below:
# properties of avro-AppSrv-source
agent.sources.avro-AppSrv-source.type = avro
agent.sources.avro-AppSrv-source.bind = localhost
agent.sources.avro-AppSrv-source.port = 10000
# properties of mem-channel-1
agent.channels.mem-channel-1.type = memory
agent.channels.mem-channel-1.capacity = 1000
agent.channels.mem-channel-1.transactionCapacity = 100
# properties of hdfs-Cluster1-sink
agent.sinks.hdfs-Cluster1-sink.type = hdfs
agent.sinks.hdfs-Cluster1-sink.hdfs.path = hdfs://
134.83.35.24/user/mardan/flume/
apply the following command:
$ /usr/bin/flume-ng agent -n agent -c conf -f
/usr/lib/flume-ng/conf/flume.conf
and got the following error. Most of the time of getting this error
12/07/24 01:54:43 ERROR properties.PropertiesFileConfigurationProvider:
Failed to load configuration data. Exception follows.
java.lang.NullPointerException
at
org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.loadSources(PropertiesFileConfigurationProvider.java:324)
at
org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.load(PropertiesFileConfigurationProvider.java:222)
at
org.apache.flume.conf.file.AbstractFileConfigurationProvider.doLoad(AbstractFileConfigurationProvider.java:123)
at
org.apache.flume.conf.file.AbstractFileConfigurationProvider.access$300(AbstractFileConfigurationProvider.java:38)
at
org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:202)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
I think some thing wrong in the configuration file. I am using flume1.x
version and installed in /usr/lib/flume-ng/
Could you please check the command and configuration file.
Thanks
On Tue, Jul 24, 2012 at 1:33 AM, Brock Noland <[email protected]> wrote:
> Yes, you can do that. In fact that is the most common case. The
> documents which should help you do so are here:
>
> https://cwiki.apache.org/confluence/display/FLUME/Flume+1.x+Documentation
>
> Brock
>
> On Mon, Jul 23, 2012 at 7:26 PM, mardan Khan <[email protected]> wrote:
> > Hi,
> >
> > I am just doing testing. I am generating the sequence and want to upload
> > into hdfs. My configuration file as:
> >
> > agent2.channels = c1
> > agent2.sources = r1
> > agent2.sinks = k1
> >
> > agent2.channels.c1.type = MEMORY
> >
> > agent2.sources.r1.channels = c1
> > agent2.sources.r1.type = SEQ
> >
> > agent2.sinks.k1.channel = c1
> > agent2.sinks.k1.type = LOGGER
> >
> >
> > Is it possible to upload into hdfs, if possible then how I can make the
> > changes in configuration file.
> >
> >
> > Many thanks
> >
>
>
>
> --
> Apache MRUnit - Unit testing MapReduce -
> http://incubator.apache.org/mrunit/
>
