Stackoverflow question: Moving data from one RDB to another through NiFi

[Sivaprasanna]

I had a chance to attempt a question raised on stackoverflow regarding
moving data from SQL Server to MySQL using NiFi. The user is using
GenerateTableFetch to read data from SQL Server and then try to use LOAD
DATA command in ExecuteSQL but this involves writing the read SQL Server
data to filesystem and then load it, which is a performance hit, I
suggested the user to try PutDatabaseRecord but I have never tried the
approach myself and going by the docs, I think it won't show any
performance benefit than LOAD DATA because the former reads from file and
inserts at a high speed while the latter reads content and parses it
according to the configured Record Reader and insert the rows as a single
batch. Confused, I wanted to get the community's opinion/thoughts on this.
Please attempt the questions, if you have better suggestions.

Links:

   -
   
https://stackoverflow.com/questions/49400447/bulk-load-sql-server-data-into-mysql-apache-nifi?noredirect=1#comment85843021_49400447
   -
   
https://stackoverflow.com/questions/49380307/flowfile-absolute-path-nifi/49398500?noredirect=1#comment85805848_49398500

Thanks,

Sivaprasanna

Re: [discuss] nifi 1.6.0 release

[Joe Witt]

Team,

An update.  After a couple important findings mentioned in NIFI-4995
I've got the RC staged now. Will do some final validation in the
morning and send out the RC vote assuming all is well.

I've created 1.7.0 for any new commits to master to be set to in JIRA.

Thanks
Joe

On Tue, Mar 20, 2018 at 9:16 AM, Joe Witt  wrote:
> Alrighty team.  Looks like we're at a good point to cut a 1.6.0
> release.  I'm starting the RC process now.  The only thing 'new' in
> terms of process is I think we're dropping one of the message digest
> types.  Will look into that as I get to that point.
>
> On Tue, Mar 13, 2018 at 2:09 PM, Michael Moser  wrote:
>> Looks like NIFI-4774 [1] isn't quite done.  It has one committed PR [2]
>> which changed the default FlowFileRepository implementation to
>> SequentialAccessWriteAheadLog.  It has another PR [3] that allows users to
>> opt into the old FFR implementation, and this seems really close to being
>> ready.
>>
>> [1] - https://issues.apache.org/jira/browse/NIFI-4774
>> [2] - https://github.com/apache/nifi/pull/2416
>> [3] - https://github.com/apache/nifi/pull/2487
>>
>> Thanks.
>>
>>
>> On Tue, Mar 13, 2018 at 6:40 AM, Joe Witt  wrote:
>>
>>> i plan to start rm processes early next week.  ill send a note with the
>>> commit ill base off of but if anything pops up in particular pls advise.
>>>
>>> thanks
>>>
>>> On Mar 12, 2018 10:34 PM, "Andy LoPresto"  wrote:
>>>
>>> > I’d like to finish reviewing PR 2085 [1] for OAuth2 controller service
>>> > inclusion as well as NIFI-4942 [2]. What is the expected timeframe for a
>>> > release candidate?
>>> >
>>> > [1] https://github.com/apache/nifi/pull/2085
>>> > [2] https://issues.apache.org/jira/browse/NIFI-4942
>>> >
>>> > Andy LoPresto
>>> > alopre...@apache.org
>>> > *alopresto.apa...@gmail.com *
>>> > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>> >
>>> > On Mar 12, 2018, at 11:01 AM, Mike Thomsen 
>>> wrote:
>>> >
>>> > Joe,
>>> >
>>> > Nope, not a 1.6 blocker, just a nice to have.
>>> >
>>> > On Mon, Mar 12, 2018 at 1:47 PM, Joe Witt  wrote:
>>> >
>>> > Mike, Michael,
>>> >
>>> > Both items noted seem fair for inclusion in whatever release vehicle
>>> > follows them being merged but neither look like they'd impact release
>>> > timing.  Please advise if I'm misreading either of those.
>>> >
>>> >
>>> > Thanks
>>> > Joe
>>> >
>>> > On Mon, Mar 12, 2018 at 5:42 PM, Michael Moser 
>>> wrote:
>>> >
>>> > I would like to finish up NIFI-3599
>>> > , which just needs some
>>> > contributer / review agreement.
>>> >
>>> > -- Another Mike
>>> >
>>> > On Mon, Mar 12, 2018 at 1:28 PM, Mike Thomsen 
>>> > wrote:
>>> >
>>> > If we could get this patch for HBase visibility labels merged before,
>>> >
>>> > I'd
>>> >
>>> > appreciate it.
>>> >
>>> > https://github.com/apache/nifi/pull/2518
>>> >
>>> > If anyone else has patches they'd like to see merged, feel to ping me on
>>> > GitHub, and I'll help out with a code review.
>>> >
>>> > Thanks,
>>> >
>>> > Mike
>>> >
>>> > On Mon, Mar 12, 2018 at 12:56 PM, Joe Witt  wrote:
>>> >
>>> > Team,
>>> >
>>> > Looks like a ton of great work already in JIRA making a nifi 1.6.0
>>> > release pretty easily justified.  Lots of nice bug fixes/improvements
>>> > and notable we relaxed the http header corruption restrictions which
>>> > fixed a potential security risk but also created operations
>>> > challenges.
>>> >
>>> > Any glaring things to hold up on?  Doing this soon doesn't preclude
>>> > doing another release soon but it feels like we need to get out a
>>> > release again.
>>> >
>>> > I'm happy to RM unless someone is really looking to do so.
>>> >
>>> > Thanks
>>> >
>>> >
>>> >
>>> >
>>> >
>>>

NIFI-4360 Need reviewers - Support For ADLS Processors

[Milan Chandna]

Hi,

Need reviewers for PR: https://github.com/apache/nifi/pull/2158

Please pitch in for review or recommend someone who can give insights.

ADLS is gaining attention among NIFI users lately.
Just to quote an example, Animesh yesterday asked for this on user group.

P.S: ADLS is Azure Data Lake Store, which is HDFS like filesystem.

Thanks,
Milan.

RE: Apache NiFi: Processor for Azure Data Lake(ADL)

[Milan Chandna]

Hi Animesh,

There is an ADLS connector underway but is lacking traction.
Here is the PR: https://github.com/apache/nifi/pull/2158
Probably you can self build this and use it.
Will be great if you come back and provide some feedback on this.

Thanks,
Milan.

-Original Message-
From: Animesh Jain  
Sent: Wednesday, March 21, 2018 6:39 PM
To: us...@nifi.apache.org; dev@nifi.apache.org
Subject: Apache NiFi: Processor for Azure Data Lake(ADL)

Hi Everyone,

I am looking for a processor for Azure Data Lake which can work in cluster mode 
setup with separate zookeeper instance.

If any one has worked on a similar use case, appreciate your help here. I see 
that there is no inbuilt ADL processor which comes up with Nifi. What is the 
recommendations here ?

Have a good day!

Thanks,
Animesh

Re: [RESULT][VOTE] Establish Fluid Design System, a sub-project of Apache NiFi

[Joe Witt]

Team: I did receive confirmation that the github mirroring is being
setup and it may take up to 24 hours.

So I think we're all set with this.  Thanks

On Wed, Mar 21, 2018 at 2:03 PM, Sivaprasanna  wrote:
> Great news. Wish this new initiative achieves the same success that NiFi
> has achieved. With a community as good as ours, I have no doubt in it. All
> the very best!
>
> -
> Sivaprasanna
>
> On Wed, 21 Mar 2018 at 11:25 PM, Joe Witt  wrote:
>
>> I've initiated the request for the nifi-fds.git repository in the ASF
>> git-wip.  Should be ready in about an hour.
>>
>> I *think* it has github mirroring by default but if not we'll file a
>> JIRA with asf-infra to enable that.
>>
>> Thanks
>>
>> On Mon, Mar 12, 2018 at 4:28 PM, Scott Aslan 
>> wrote:
>> > All,
>> >
>> > This vote has ended with:
>> >
>> > 15 binding +1s
>> > 2 non-binding +1s
>> > 0 <= 0
>> >
>> > The vote to establish Fluid Design System as a sub-project of the Apache
>> > NiFi TLP
>> > has passed.
>> >
>> > Thank you all for your participation in this important vote. I will
>> > submit the necessary information to infra to get the ball rolling
>> >
>> > Thanks,
>> >
>> > Scott
>>

Re: [RESULT][VOTE] Establish Fluid Design System, a sub-project of Apache NiFi

[Sivaprasanna]

Great news. Wish this new initiative achieves the same success that NiFi
has achieved. With a community as good as ours, I have no doubt in it. All
the very best!

-
Sivaprasanna

On Wed, 21 Mar 2018 at 11:25 PM, Joe Witt  wrote:

> I've initiated the request for the nifi-fds.git repository in the ASF
> git-wip.  Should be ready in about an hour.
>
> I *think* it has github mirroring by default but if not we'll file a
> JIRA with asf-infra to enable that.
>
> Thanks
>
> On Mon, Mar 12, 2018 at 4:28 PM, Scott Aslan 
> wrote:
> > All,
> >
> > This vote has ended with:
> >
> > 15 binding +1s
> > 2 non-binding +1s
> > 0 <= 0
> >
> > The vote to establish Fluid Design System as a sub-project of the Apache
> > NiFi TLP
> > has passed.
> >
> > Thank you all for your participation in this important vote. I will
> > submit the necessary information to infra to get the ball rolling
> >
> > Thanks,
> >
> > Scott
>

Re: [RESULT][VOTE] Establish Fluid Design System, a sub-project of Apache NiFi

[Joe Witt]

I've initiated the request for the nifi-fds.git repository in the ASF
git-wip.  Should be ready in about an hour.

I *think* it has github mirroring by default but if not we'll file a
JIRA with asf-infra to enable that.

Thanks

On Mon, Mar 12, 2018 at 4:28 PM, Scott Aslan  wrote:
> All,
>
> This vote has ended with:
>
> 15 binding +1s
> 2 non-binding +1s
> 0 <= 0
>
> The vote to establish Fluid Design System as a sub-project of the Apache
> NiFi TLP
> has passed.
>
> Thank you all for your participation in this important vote. I will
> submit the necessary information to infra to get the ball rolling
>
> Thanks,
>
> Scott

Re: [ANNOUNCE] New Apache NiFi Committer Mike Thomsen

[Joe Percivall]

Thank you for your prior and continued contributions Mike, and congrats!

On Wed, Mar 21, 2018 at 12:59 PM, Marco Gaido 
wrote:

> Congrats Mike! Well deserved!
>
> On Wed, 21 Mar 2018, 16:53 Sivaprasanna, 
> wrote:
>
> > Congratulations, Mike :)
> >
> > On Wed, 21 Mar 2018 at 8:57 PM, Jeff  wrote:
> >
> > > Congrats Mike!
> > >
> > > On Wed, Mar 21, 2018 at 11:20 AM Scott Aslan 
> > > wrote:
> > >
> > > > Congrats!
> > > >
> > > > On Wed, Mar 21, 2018 at 10:15 AM, Pierre Villard <
> > > > pierre.villard...@gmail.com> wrote:
> > > >
> > > > > Congrats Mike, well deserved!
> > > > >
> > > > > 2018-03-21 14:55 GMT+01:00 Mike Thomsen :
> > > > >
> > > > > > Thanks everyone!
> > > > > >
> > > > > > On Wed, Mar 21, 2018 at 9:55 AM, Joe Witt 
> > > wrote:
> > > > > >
> > > > > > > Mike
> > > > > > >
> > > > > > > Thanks for all the great contributions and reviews and
> > discussions
> > > > and
> > > > > > > congratulations on the well deserved commit bit!
> > > > > > >
> > > > > > > Thanks
> > > > > > > Joe
> > > > > > >
> > > > > > > On Wed, Mar 21, 2018 at 1:53 PM, Kevin Doran <
> kdo...@apache.org>
> > > > > wrote:
> > > > > > > > Congrats, Mike!
> > > > > > > >
> > > > > > > > On 3/21/18, 09:42, "Tony Kurc"  wrote:
> > > > > > > >
> > > > > > > > On behalf of the Apache NiFI PMC, I am very pleased to
> > > announce
> > > > > > that
> > > > > > > Mike
> > > > > > > > has accepted the PMC's invitation to become a committer
> on
> > > the
> > > > > > > Apache NiFi
> > > > > > > > project. We greatly appreciate all of Mike's hard work
> and
> > > > > generous
> > > > > > > > contributions to the project. We look forward to his
> > > continued
> > > > > > > involvement
> > > > > > > > in the project.
> > > > > > > >
> > > > > > > > Mike has been contributing to the project for quite a
> > while,
> > > > > > > contributing
> > > > > > > > features such as  enhancements to MongoDB and HBase
> > > processors
> > > > as
> > > > > > > well as
> > > > > > > > things like improvements to Expression Language and
> > > > > LookupService.
> > > > > > > I'm sure
> > > > > > > > many of you have interacted with Mike on the mailing
> lists
> > > > where
> > > > > he
> > > > > > > has
> > > > > > > > been giving great input on both the project and
> community.
> > We
> > > > > also
> > > > > > > > appreciate his work reviewing and providing feedback on
> new
> > > > > > > contributions.
> > > > > > > >
> > > > > > > > Welcome and congratulations!
> > > > > > > > Tony
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>



-- 
*Joe Percivall*
linkedin.com/in/Percivall
e: jperciv...@apache.com

Java 10

[Mike Thomsen]

Looks like Oracle just dropped Java 10 on us:

http://www.oracle.com/technetwork/java/javase/downloads/jdk10-downloads-4416644.html

Announcement for anyone interested:

http://mail.openjdk.java.net/pipermail/announce/2018-March/000247.html

Re: [ANNOUNCE] New Apache NiFi Committer Mike Thomsen

[Marco Gaido]

Congrats Mike! Well deserved!

On Wed, 21 Mar 2018, 16:53 Sivaprasanna,  wrote:

> Congratulations, Mike :)
>
> On Wed, 21 Mar 2018 at 8:57 PM, Jeff  wrote:
>
> > Congrats Mike!
> >
> > On Wed, Mar 21, 2018 at 11:20 AM Scott Aslan 
> > wrote:
> >
> > > Congrats!
> > >
> > > On Wed, Mar 21, 2018 at 10:15 AM, Pierre Villard <
> > > pierre.villard...@gmail.com> wrote:
> > >
> > > > Congrats Mike, well deserved!
> > > >
> > > > 2018-03-21 14:55 GMT+01:00 Mike Thomsen :
> > > >
> > > > > Thanks everyone!
> > > > >
> > > > > On Wed, Mar 21, 2018 at 9:55 AM, Joe Witt 
> > wrote:
> > > > >
> > > > > > Mike
> > > > > >
> > > > > > Thanks for all the great contributions and reviews and
> discussions
> > > and
> > > > > > congratulations on the well deserved commit bit!
> > > > > >
> > > > > > Thanks
> > > > > > Joe
> > > > > >
> > > > > > On Wed, Mar 21, 2018 at 1:53 PM, Kevin Doran 
> > > > wrote:
> > > > > > > Congrats, Mike!
> > > > > > >
> > > > > > > On 3/21/18, 09:42, "Tony Kurc"  wrote:
> > > > > > >
> > > > > > > On behalf of the Apache NiFI PMC, I am very pleased to
> > announce
> > > > > that
> > > > > > Mike
> > > > > > > has accepted the PMC's invitation to become a committer on
> > the
> > > > > > Apache NiFi
> > > > > > > project. We greatly appreciate all of Mike's hard work and
> > > > generous
> > > > > > > contributions to the project. We look forward to his
> > continued
> > > > > > involvement
> > > > > > > in the project.
> > > > > > >
> > > > > > > Mike has been contributing to the project for quite a
> while,
> > > > > > contributing
> > > > > > > features such as  enhancements to MongoDB and HBase
> > processors
> > > as
> > > > > > well as
> > > > > > > things like improvements to Expression Language and
> > > > LookupService.
> > > > > > I'm sure
> > > > > > > many of you have interacted with Mike on the mailing lists
> > > where
> > > > he
> > > > > > has
> > > > > > > been giving great input on both the project and community.
> We
> > > > also
> > > > > > > appreciate his work reviewing and providing feedback on new
> > > > > > contributions.
> > > > > > >
> > > > > > > Welcome and congratulations!
> > > > > > > Tony
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [ANNOUNCE] New Apache NiFi Committer Mike Thomsen

[Sivaprasanna]

Congratulations, Mike :)

On Wed, 21 Mar 2018 at 8:57 PM, Jeff  wrote:

> Congrats Mike!
>
> On Wed, Mar 21, 2018 at 11:20 AM Scott Aslan 
> wrote:
>
> > Congrats!
> >
> > On Wed, Mar 21, 2018 at 10:15 AM, Pierre Villard <
> > pierre.villard...@gmail.com> wrote:
> >
> > > Congrats Mike, well deserved!
> > >
> > > 2018-03-21 14:55 GMT+01:00 Mike Thomsen :
> > >
> > > > Thanks everyone!
> > > >
> > > > On Wed, Mar 21, 2018 at 9:55 AM, Joe Witt 
> wrote:
> > > >
> > > > > Mike
> > > > >
> > > > > Thanks for all the great contributions and reviews and discussions
> > and
> > > > > congratulations on the well deserved commit bit!
> > > > >
> > > > > Thanks
> > > > > Joe
> > > > >
> > > > > On Wed, Mar 21, 2018 at 1:53 PM, Kevin Doran 
> > > wrote:
> > > > > > Congrats, Mike!
> > > > > >
> > > > > > On 3/21/18, 09:42, "Tony Kurc"  wrote:
> > > > > >
> > > > > > On behalf of the Apache NiFI PMC, I am very pleased to
> announce
> > > > that
> > > > > Mike
> > > > > > has accepted the PMC's invitation to become a committer on
> the
> > > > > Apache NiFi
> > > > > > project. We greatly appreciate all of Mike's hard work and
> > > generous
> > > > > > contributions to the project. We look forward to his
> continued
> > > > > involvement
> > > > > > in the project.
> > > > > >
> > > > > > Mike has been contributing to the project for quite a while,
> > > > > contributing
> > > > > > features such as  enhancements to MongoDB and HBase
> processors
> > as
> > > > > well as
> > > > > > things like improvements to Expression Language and
> > > LookupService.
> > > > > I'm sure
> > > > > > many of you have interacted with Mike on the mailing lists
> > where
> > > he
> > > > > has
> > > > > > been giving great input on both the project and community. We
> > > also
> > > > > > appreciate his work reviewing and providing feedback on new
> > > > > contributions.
> > > > > >
> > > > > > Welcome and congratulations!
> > > > > > Tony
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [ANNOUNCE] New Apache NiFi Committer Mike Thomsen

[Jeff]

Congrats Mike!

On Wed, Mar 21, 2018 at 11:20 AM Scott Aslan  wrote:

> Congrats!
>
> On Wed, Mar 21, 2018 at 10:15 AM, Pierre Villard <
> pierre.villard...@gmail.com> wrote:
>
> > Congrats Mike, well deserved!
> >
> > 2018-03-21 14:55 GMT+01:00 Mike Thomsen :
> >
> > > Thanks everyone!
> > >
> > > On Wed, Mar 21, 2018 at 9:55 AM, Joe Witt  wrote:
> > >
> > > > Mike
> > > >
> > > > Thanks for all the great contributions and reviews and discussions
> and
> > > > congratulations on the well deserved commit bit!
> > > >
> > > > Thanks
> > > > Joe
> > > >
> > > > On Wed, Mar 21, 2018 at 1:53 PM, Kevin Doran 
> > wrote:
> > > > > Congrats, Mike!
> > > > >
> > > > > On 3/21/18, 09:42, "Tony Kurc"  wrote:
> > > > >
> > > > > On behalf of the Apache NiFI PMC, I am very pleased to announce
> > > that
> > > > Mike
> > > > > has accepted the PMC's invitation to become a committer on the
> > > > Apache NiFi
> > > > > project. We greatly appreciate all of Mike's hard work and
> > generous
> > > > > contributions to the project. We look forward to his continued
> > > > involvement
> > > > > in the project.
> > > > >
> > > > > Mike has been contributing to the project for quite a while,
> > > > contributing
> > > > > features such as  enhancements to MongoDB and HBase processors
> as
> > > > well as
> > > > > things like improvements to Expression Language and
> > LookupService.
> > > > I'm sure
> > > > > many of you have interacted with Mike on the mailing lists
> where
> > he
> > > > has
> > > > > been giving great input on both the project and community. We
> > also
> > > > > appreciate his work reviewing and providing feedback on new
> > > > contributions.
> > > > >
> > > > > Welcome and congratulations!
> > > > > Tony
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
>

Re: [ANNOUNCE] New Apache NiFi Committer Mike Thomsen

[Pierre Villard]

Congrats Mike, well deserved!

2018-03-21 14:55 GMT+01:00 Mike Thomsen :

> Thanks everyone!
>
> On Wed, Mar 21, 2018 at 9:55 AM, Joe Witt  wrote:
>
> > Mike
> >
> > Thanks for all the great contributions and reviews and discussions and
> > congratulations on the well deserved commit bit!
> >
> > Thanks
> > Joe
> >
> > On Wed, Mar 21, 2018 at 1:53 PM, Kevin Doran  wrote:
> > > Congrats, Mike!
> > >
> > > On 3/21/18, 09:42, "Tony Kurc"  wrote:
> > >
> > > On behalf of the Apache NiFI PMC, I am very pleased to announce
> that
> > Mike
> > > has accepted the PMC's invitation to become a committer on the
> > Apache NiFi
> > > project. We greatly appreciate all of Mike's hard work and generous
> > > contributions to the project. We look forward to his continued
> > involvement
> > > in the project.
> > >
> > > Mike has been contributing to the project for quite a while,
> > contributing
> > > features such as  enhancements to MongoDB and HBase processors as
> > well as
> > > things like improvements to Expression Language and LookupService.
> > I'm sure
> > > many of you have interacted with Mike on the mailing lists where he
> > has
> > > been giving great input on both the project and community. We also
> > > appreciate his work reviewing and providing feedback on new
> > contributions.
> > >
> > > Welcome and congratulations!
> > > Tony
> > >
> > >
> > >
> >
>

Re: [ANNOUNCE] New Apache NiFi Committer Mike Thomsen

[Mike Thomsen]

Thanks everyone!

On Wed, Mar 21, 2018 at 9:55 AM, Joe Witt  wrote:

> Mike
>
> Thanks for all the great contributions and reviews and discussions and
> congratulations on the well deserved commit bit!
>
> Thanks
> Joe
>
> On Wed, Mar 21, 2018 at 1:53 PM, Kevin Doran  wrote:
> > Congrats, Mike!
> >
> > On 3/21/18, 09:42, "Tony Kurc"  wrote:
> >
> > On behalf of the Apache NiFI PMC, I am very pleased to announce that
> Mike
> > has accepted the PMC's invitation to become a committer on the
> Apache NiFi
> > project. We greatly appreciate all of Mike's hard work and generous
> > contributions to the project. We look forward to his continued
> involvement
> > in the project.
> >
> > Mike has been contributing to the project for quite a while,
> contributing
> > features such as  enhancements to MongoDB and HBase processors as
> well as
> > things like improvements to Expression Language and LookupService.
> I'm sure
> > many of you have interacted with Mike on the mailing lists where he
> has
> > been giving great input on both the project and community. We also
> > appreciate his work reviewing and providing feedback on new
> contributions.
> >
> > Welcome and congratulations!
> > Tony
> >
> >
> >
>

Re: [ANNOUNCE] New Apache NiFi Committer Mike Thomsen

[Joe Witt]

Mike

Thanks for all the great contributions and reviews and discussions and
congratulations on the well deserved commit bit!

Thanks
Joe

On Wed, Mar 21, 2018 at 1:53 PM, Kevin Doran  wrote:
> Congrats, Mike!
>
> On 3/21/18, 09:42, "Tony Kurc"  wrote:
>
> On behalf of the Apache NiFI PMC, I am very pleased to announce that Mike
> has accepted the PMC's invitation to become a committer on the Apache NiFi
> project. We greatly appreciate all of Mike's hard work and generous
> contributions to the project. We look forward to his continued involvement
> in the project.
>
> Mike has been contributing to the project for quite a while, contributing
> features such as  enhancements to MongoDB and HBase processors as well as
> things like improvements to Expression Language and LookupService. I'm 
> sure
> many of you have interacted with Mike on the mailing lists where he has
> been giving great input on both the project and community. We also
> appreciate his work reviewing and providing feedback on new contributions.
>
> Welcome and congratulations!
> Tony
>
>
>

Re: [ANNOUNCE] New Apache NiFi Committer Mike Thomsen

[Kevin Doran]

Congrats, Mike!

On 3/21/18, 09:42, "Tony Kurc"  wrote:

On behalf of the Apache NiFI PMC, I am very pleased to announce that Mike
has accepted the PMC's invitation to become a committer on the Apache NiFi
project. We greatly appreciate all of Mike's hard work and generous
contributions to the project. We look forward to his continued involvement
in the project.

Mike has been contributing to the project for quite a while, contributing
features such as  enhancements to MongoDB and HBase processors as well as
things like improvements to Expression Language and LookupService. I'm sure
many of you have interacted with Mike on the mailing lists where he has
been giving great input on both the project and community. We also
appreciate his work reviewing and providing feedback on new contributions.

Welcome and congratulations!
Tony

[ANNOUNCE] New Apache NiFi Committer Mike Thomsen

[Tony Kurc]

On behalf of the Apache NiFI PMC, I am very pleased to announce that Mike
has accepted the PMC's invitation to become a committer on the Apache NiFi
project. We greatly appreciate all of Mike's hard work and generous
contributions to the project. We look forward to his continued involvement
in the project.

Mike has been contributing to the project for quite a while, contributing
features such as  enhancements to MongoDB and HBase processors as well as
things like improvements to Expression Language and LookupService. I'm sure
many of you have interacted with Mike on the mailing lists where he has
been giving great input on both the project and community. We also
appreciate his work reviewing and providing feedback on new contributions.

Welcome and congratulations!
Tony

Re: ExecuteHQL failed to reading table from Hive

[Matt Burgess]

This sounds like a bug in HiveJdbcCommon, we should be able to support
a Hive SMALLINT into an Avro integer or something.

On Tue, Mar 20, 2018 at 11:53 AM, Rohit  wrote:
> Hi Matt,
>
> Issue got resolved after casting smallint to int, as avro not supported
> smallint datatype.
>
> Thanks.
>
>
>
> --
> Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

Re: Nifi copy from cluster to cluster

[Joe Witt]

Atul,

If you're finding distcp to be a good enough tool for the job then I
would definitely use it.  For the things distcp works for it is very
stable/very to the point/and if your use case fits squarely inside
there you'll be in a good position.

If you need more than direct file replication from one environment to
another that is when a flow management solution might make sense to
bring into the picture.

You'd have to share a lot more details about your nifi configuration
for anyone to help comment on duplicates.  It should be extremely
unlikely for duplicates to occur in a fairly straightforward
configuration.  Raw performance wise distcp doesn't make an
intermediary copy of the data like NiFi would so it *should* be a bit
faster in a pure sense for this particular use case but then see above
comments.

Thanks

On Wed, Mar 21, 2018 at 12:14 PM, atul gupta  wrote:
> Hi team ,
>
> We are using nifi copy framework to copy data across clusters but are
> facing issues in terms of duplicates ( can nifi stops or check if table is
> already present in the destination cluster ?) . Moreover if volume is more
> than 1 tb we are finding distcp a better option as nifi flow is taking more
> time or failing .Please advise .
>
> Thanks & Regards,
> Atul Gupta

Apache NiFi: Processor for Azure Data Lake(ADL)

[Animesh Jain]

Hi Everyone,

I am looking for a processor for Azure Data Lake which can work in cluster mode 
setup with separate zookeeper instance.

If any one has worked on a similar use case, appreciate your help here. I see 
that there is no inbuilt ADL processor which comes up with Nifi. What is the 
recommendations here ?

Have a good day!

Thanks,
Animesh

Nifi copy from cluster to cluster

[atul gupta]

Hi team ,

We are using nifi copy framework to copy data across clusters but are
facing issues in terms of duplicates ( can nifi stops or check if table is
already present in the destination cluster ?) . Moreover if volume is more
than 1 tb we are finding distcp a better option as nifi flow is taking more
time or failing .Please advise .

Thanks & Regards,
Atul Gupta

Re: "Method not supported" error facing GenerateTableFetch and QueryDatabaseTable processor

[Rohit]

Thanks Matt for reply...

So is there any other way to read data from have table into batches because
my table having millions of row and i am using NiFi 1.3

I tried with below processor:-

1. ExecuteHQL(Customer processor of Kylo) -- creating single file to
downstream.
2. SelectHiveQL - "Max Rows Per Flow File" property not supporting NiFi
1.3.0
3. GenerateTableFetch and QueryDatabaseTable



--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/