Som, on Flume NG you must run a source, channel, and sink all in the same JVM. An Avro source accepts connections and events from an Avro sink.
Try taking a look @ the user guide: https://people.apache.org/~mpercy/flume/flume-1.2.0-incubating-SNAPSHOT/docs/FlumeUserGuide.html Regards, Mike On Mon, Jul 16, 2012 at 11:01 PM, shekhar sharma <[email protected]>wrote: > Hello, > I am running on flume-ng (flume-1.2.0.SNAPSHOT) and i am trying to see > whether the events are being captured by avrosource or not!!!.. > > When i am running avro source only programmitically it is throwing me this > error...( I am trying to imitate TestAvroSource.java test) > > But if i configure both source and sink in the agent file and then send > the events, i am not getting any error > > > Well my actuall use case is like this: > > machine1---> avro source and memory channel > machine2--->sink and memory channel > > i am trying to achieve this., > > Regards, > Som Shekhar > > > > > > On Tue, Jul 17, 2012 at 3:53 AM, Mike Percy <[email protected]> wrote: > >> (adding flume users list, bcc: avro users list) >> >> Shekhar, >> Can you please provide more information about what you are doing when you >> see this message and the version of Flume that you are running? >> >> Regards, >> Mike >> >> >> >> On Mon, Jul 16, 2012 at 3:12 PM, Doug Cutting <[email protected]> wrote: >> >>> From the stack trace you sent this looks like an error thrown by >>> Flume, not by Avro. >>> >>> Doug >>> >>> On Fri, Jul 13, 2012 at 11:53 AM, shekhar sharma <[email protected]> >>> wrote: >>> > Hello, >>> > i am using avro source to send the events to Flume Avro source. >>> Connection >>> > is fine but when i am sending the events it is throwing Avro remote >>> > exception. >>> > I have an application which uses RPC client factory get >>> > connected to the avro source. RpcClient is successfuly connected to >>> > the Avro Source but when i am sending the events to the Avro source i >>> > am getting AvroRemoteException. >>> > >>> > This is the following code snippet am using: >>> > AvroSource source = new AvroSource(); >>> > Channel channel = new MemoryChannel(); >>> > List<Channel> ch = new ArrayList(); >>> > ch.add(channel); >>> > ChannelSelector cs = new ReplicatingChannelSelector(); >>> > cs.setChannels(ch); >>> > Configurables.configure(cs, new Context()); >>> > >>> > ChannelProcessor cp = new ChannelProcessor(cs); >>> > Context context = new Context(); >>> > context.put("bind","localhost" >>> > ); >>> > context.put("port", "12345"); >>> > source.configure(context); >>> > source.setChannelProcessor(cp); >>> > source.start(); >>> > //Server server = new NettyServer(new >>> > SpecificResponder(AvroSource.class, source), new >>> > InetSocketAddress("localhost",12345)); >>> > //server.start(); >>> > >>> > AvroSourceProtocol client = >>> > SpecificRequestor.getClient(AvroSourceProtocol.class, new >>> > NettyTransceiver(new InetSocketAddress(12345))); >>> > AvroFlumeEvent avroEvent = new AvroFlumeEvent(); >>> > >>> > avroEvent.setHeaders(new HashMap<CharSequence, >>> CharSequence>()); >>> > avroEvent.setBody(ByteBuffer.wrap("Hello avro".getBytes())); >>> > Status status = client.append(avroEvent); >>> > >>> > >>> > Transaction transaction = channel.getTransaction(); >>> > transaction.begin(); >>> > >>> > Event event1 = channel.take(); >>> > transaction.commit(); >>> > transaction.close(); >>> > >>> > And i am getting the following exception: >>> > >>> > [New I/O server worker #1-1] WARN org.apache.avro.ipc.Responder - user >>> error >>> > java.lang.NullPointerException >>> > at >>> > >>> org.apache.flume.channel.ChannelProcessor.processEvent(ChannelProcessor.java:183) >>> > at >>> org.apache.flume.source.AvroSource.append(AvroSource.java:179) >>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> > at >>> > >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >>> > at >>> > >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >>> > at java.lang.reflect.Method.invoke(Method.java:597) >>> > at >>> > >>> org.apache.avro.ipc.specific.SpecificResponder.respond(SpecificResponder.java:88) >>> > at org.apache.avro.ipc.Responder.respond(Responder.java:149) >>> > at >>> > >>> org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.messageReceived(NettyServer.java:140) >>> > at >>> > >>> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80) >>> > at >>> > >>> org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream(NettyServer.java:125) >>> > at >>> > >>> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) >>> > at >>> > >>> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:783) >>> > at >>> > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:302) >>> > at >>> > >>> org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:317) >>> > at >>> > >>> org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:299) >>> > at >>> > >>> org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:216) >>> > at >>> > >>> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80) >>> > at >>> > >>> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) >>> > at >>> > >>> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) >>> > at >>> > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:274) >>> > at >>> > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:261) >>> > at >>> > org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:351) >>> > at >>> > >>> org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:282) >>> > at >>> > org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:202) >>> > at >>> > >>> org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) >>> > at >>> > >>> org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44) >>> > at >>> > >>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) >>> > at >>> > >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) >>> > at java.lang.Thread.run(Thread.java:662) >>> >> >> >
