Re: using avro flume source in nifi

[Chris Herssens]

Hello Joe,

Currently we use Flume for sending avro message to our big data platform.
Now I want to replace flume with NIfi. Since the sender is a  flume agent,
I need to use the flume source.
If I configure this sourcs for receiving netcat, I can access the tcp port.
Using avro as type doesn't work. If I set "run schedule" ti 60 sec, then it
work, I can telnet to the configured tcp port.

Regards,

Chris

On Fri, Aug 11, 2017 at 4:03 AM, Joe Witt  wrote:

> Chris,
>
> I dont believe there are many in the community here familiar enough
> with Flume so that might make getting help here a bit tougher.  Are
> you wanting NiFi to act as a TCP socket listener on a given port
> receiving Avro objects?
>
> Thanks
>
> On Thu, Aug 3, 2017 at 11:22 PM, Chris Herssens
>  wrote:
> > Hello All,
> >
> > I want to configure a flume source processor using avro as flume source.
> > When I start the ExecuteFlumeSource, I don't get any errors, but I can't
> > connect to the source port.
> >
> > telnet localhost 5151 gives connection refused.
> >
> > The configuration is as follows
> >
> > type = avro
> >
> > agent name = a1
> >
> > source name = batchSource
> >
> > Flume configuration
> >
> >a1.sources.batchSource.type = avro
> >
> >a1.sources.batchSource.bind = 0.0.0.0
> >
> >a1.sources.batchSource.port=5151
> >
> >
> > Using netcat as flume source works.
>

Re: using avro flume source in nifi

[Joe Witt]

Chris,

I dont believe there are many in the community here familiar enough
with Flume so that might make getting help here a bit tougher.  Are
you wanting NiFi to act as a TCP socket listener on a given port
receiving Avro objects?

Thanks

On Thu, Aug 3, 2017 at 11:22 PM, Chris Herssens
 wrote:
> Hello All,
>
> I want to configure a flume source processor using avro as flume source.
> When I start the ExecuteFlumeSource, I don't get any errors, but I can't
> connect to the source port.
>
> telnet localhost 5151 gives connection refused.
>
> The configuration is as follows
>
> type = avro
>
> agent name = a1
>
> source name = batchSource
>
> Flume configuration
>
>a1.sources.batchSource.type = avro
>
>a1.sources.batchSource.bind = 0.0.0.0
>
>a1.sources.batchSource.port=5151
>
>
> Using netcat as flume source works.