Can you enable debug logging for Flume? I want to see what the configuration looks like.
On Thu, Jan 22, 2015 at 8:05 PM, Carlotta Hicks <[email protected]> wrote: > Here is the code change in EmbeddedAgentConfiguration: > > > > /** > > * Avro sink which can send events to a downstream avro source. This is > the > > * only supported sink for Embedded Agents. > > */ > > public static final String SINK_TYPE_AVRO = SinkType.AVRO.name(); > > public static final String SINK_TYPE_HBASE = SinkType.HBASE.name(); > > public static final String SINK_TYPE_HDFS = SinkType.HDFS.name(); > > > > private static final String[] ALLOWED_SINKS = { > > SINK_TYPE_AVRO, > > SINK_TYPE_HBASE, > > SINK_TYPE_HDFS > > }; > > > > This is my code with configurations: > > > > Map<String, String> properties = new HashMap<String, String>(); > > > > properties.put("channel.type", "file"); > > properties.put("sinks", "hdfs1"); > > properties.put("hdfs1.type", "hdfs"); > > properties.put("hdfs1.path", "hdfs://lax07.unx.sas.com:8020/tmp"); > > properties.put("processor.type", "default"); > > > > agent = new EmbeddedAgent(“myAgent”); > > > > agent.configure(properties); ß This statement causes > the exception below > > > > org.apache.flume.FlumeException: Expected one source and got 0 > > at > org.apache.flume.agent.embedded.EmbeddedAgent.doConfigure(EmbeddedAgent.java:160) > > at > org.apache.flume.agent.embedded.EmbeddedAgent.configure(EmbeddedAgent.java:95) > > > > > > > > *From:* Hari Shreedharan [mailto:[email protected]] > *Sent:* Tuesday, January 20, 2015 3:23 PM > *To:* [email protected] > *Cc:* [email protected] > *Subject:* RE: Java API for Flume Agent > > > > Can you send the code you used? > > > Thanks, > > Hari > > > > On Tue, Jan 20, 2015 at 8:48 AM, Carlotta Hicks <[email protected]> > wrote: > > I added in an HDFS sink. But, I am getting this FlumeException: Expected > one source and got 0 . > > The only source for the embedded agent is 'embedded', correct? Not sure > how to proceed from here. > > Need help. > > -----Original Message----- > From: Ashish [mailto:[email protected] <[email protected]>] > Sent: Saturday, January 17, 2015 7:58 AM > To: [email protected] > Subject: Re: Java API for Flume Agent > > Try a bit of hacking in > org.apache.flume.agent.embedded.EmbeddedAgentConfiguration class, and add > HBase Sink to allowed sinks in the section > > private static final String[] ALLOWED_SINKS = { > SINK_TYPE_AVRO, > // Add Hbase sink here <<< > > }; > > See if it works and let us know. > > HTH! > > On Sat, Jan 17, 2015 at 10:19 AM, Carlotta Hicks <[email protected]> > wrote: > > Thanks Joey! I would love to use the embedded agent. It is exactly what > I am looking for. However, I can't use the embedded agent because with the > embedded agent, the sink must be avro. I need the sink to be hbase. > > I am assuming that requirements has not changed. Please correct me if I > am wrong. > > ________________________________________ > > From: Joey Echeverria <[email protected]> > > Sent: Friday, January 16, 2015 3:47 PM > > To: [email protected] > > Subject: Re: Java API for Flume Agent > > > > You could use an embedded flume agent[1]. This will do the same thing > > that the Flume flow does but run flume inside of another Java > > application. > > > > Let me know if that meets your needs. > > > > -Joey > > > > [1] http://flume.apache.org/FlumeDeveloperGuide.html#embedded-agent > > > > > > On Fri, Jan 16, 2015 at 10:30 AM, Carlotta Hicks <[email protected]> > wrote: > >> I need to create an agent using java API to load an HBase table using > >> a CSV file. I am able to accomplish this using flume-ng and an agent > >> with a spooldir directory source, file channel and HBase sink. > >> > >> > >> > >> I need information on how to accomplish this via a java API. > >> > >> > >> > >> Any help would be appreciated. > >> > >> > >> > >> -CM > >> > >> > > > > > > > > -- > > Joey Echeverria > > > > -- > thanks > ashish > > Blog: http://www.ashishpaliwal.com/blog > My Photo Galleries: http://www.pbase.com/ashishpaliwal > > >
