Hello list,
After updating to Flume v0.9.4-cdh3u1, I do meet this error:
WARN com.cloudera.flume.conf.FlumeBuilder: Deprecated syntax: Expected a
format spec but instead had a (String) raw
As a result, collector does not save events on storage backend
Before the updating, my plug-ins worked completely well. Below is my typical
config:
exec config node1 test 'tail("/path/to/test.txt",startFromEnd="true")'
'{deco_modiy_event_body()=>autoDFOChain}'
Any thoughts?
Thanks,
~Thai
On Sat, Sep 10, 2011 at 4:14 AM, Jeff Hansen <[email protected]> wrote:
> Sorry, I should have also mentioned that the output itself is coming
> out in the raw format (consistent with the xml config rather than the
> runtime config)
>
> On Fri, Sep 9, 2011 at 4:10 PM, Jeff Hansen <[email protected]> wrote:
> > When I configure a collectorSink to use a format different from the
> > default specified in the flume-conf.xml file, it continues to use the
> > default. I get the following error
> >
> > 2011-09-09 16:02:29,059 [Roll-TriggerThread-0] WARN conf.FlumeBuilder:
> > Deprecated syntax: Expected a format spec but instead had a (String)
> > raw
> >
> > even though I'm using the following combination of configurations. Is
> > the xml file configuration supposed to supercede that of the runtime
> > configuration?
> >
> > I'm running version "Flume 0.9.4-cdh3u1"
> >
> > Here's the complete startup script I'm using:
> >
> > #!/bin/sh
> > gnome-terminal -e "flume master"
> > sleep 10
> > flume shell -c localhost -e "exec config agent
> > 'tail(\"/var/log/apache2/access.log\")' '[console,
> > collectorSink(\"hdfs://localhost/flume/avro/\",\"log\",60000,avrojson)]'"
> > gnome-terminal -e "flume node -n agent"
> >
> > Here's the appropriate entry from my flume-conf.xml file:
> > <property>
> > <name>flume.collector.output.format</name>
> > <value>raw</value>
> > <description>The output format for the data written by a Flume
> > collector node. There are several formats available:
> > syslog - outputs events in a syslog-like format
> > log4j - outputs events in a pattern similar to Hadoop's log4j
> pattern
> > raw - Event body only. This is most similar to copying a file but
> > does not preserve any uniqifying metadata like
> host/timestamp/nanos.
> > avro - Avro Native file format. Default currently is uncompressed.
> > avrojson - this outputs data as json encoded by avro
> > avrodata - this outputs data as a avro binary encoded data
> > debug - used only for debugging
> > </description>
> > </property>
> >
>
