Re: Stackoverflow question: Moving data from one RDB to another through NiFi

that you use ? That would be new to me and I would be > interested. > > Jorge Machado > > > > > > > On 22 Mar 2018, at 08:22, Brett Ryan <brett.r...@gmail.com> wrote: > > > > Sure it does, I’m using it for postgres and MariaDB (which is > essentiall

Re: Stackoverflow question: Moving data from one RDB to another through NiFi

achado > > > > > >> On 22 Mar 2018, at 08:13, Brett Ryan <brett.r...@gmail.com> wrote: >> >> Could Sqoop [1] be an option? >> >> [1]: http://sqoop.apache.org/ >> >>> On 22 Mar 2018, at 16:33, Sivaprasanna <sivaprasanna...@

Re: Stackoverflow question: Moving data from one RDB to another through NiFi

Could Sqoop [1] be an option? [1]: http://sqoop.apache.org/ > On 22 Mar 2018, at 16:33, Sivaprasanna wrote: > > I had a chance to attempt a question raised on stackoverflow regarding > moving data from SQL Server to MySQL using NiFi. The user is using >

Re: [DISCUSS] Apache NiFi distribution has grown too large

> On 18 Jan 2018, at 03:07, Matt Burgess wrote: > > BTW, talking about mixin inheritance, shared dependencies, improved > classloading, and module repositories, I feel like OSGi is the > elephant in the room. I can see perfectly good reasons NOT to move to > an

Re: [DISCUSS] Apache NiFi distribution has grown too large

> On 18 Jan 2018, at 03:05, Matt Burgess wrote: > > - Some NiFi installs will be located on systems that cannot contact an > outside (or any external) repository. When we consider NAR > repositories, we should consider providing a repo-to-go or something > of that sort.

Re: Building unit tests

Hi Mikhail, I’m surprised using asLong works. What you need to do is int maxRowSize = ctx.getProperty(MAX_ROW_SIZE) .asDataSize(DataUnit.B) .intValue(); You may specify other units. If the property is not set the asDataSize call returns null, if this possible likely check

Re: [DISCUSS] Apache NiFi distribution has grown too large

Why are core modules not listing everything as provided? IDE’s solve this problem with the use of dependency libraries. As an example NetBeans nbm’s have a single purpose, you must export the packages to be exposed. We do the same with confluence modules using felix. Why is NiFi doing things

Re: clear all flowfiles in all queues upon NiFi restart

Is there a plugin hook that can be written to allow one to see the current state of all flows? One thought I have is that this process could look for a configured property, like volatile.delete.strategy that a hook on startup could delete this based on given factors. One use case would be

Re: DBCP connection pool problem - connection already closed

Hi Ben. It’s often recommended to use the jTDS driver [1] as the MS provided driver is considered buggy. I don’t make this claim, however; I’ve always used this driver for ms sql server and never encountered issues. [1]: http://jtds.sourceforge.net/faq.html > On 11 Jan 2018, at 18:56, 尹文才

Re: FlowFile Routing to Failure

> On 10 Jan 2018, at 04:29, Jean De Leon wrote: > > What are reasons why a FlowFile will route to failure without prompting an > error message? If there was no exception reported or any error logs emitted by the processor then I think nothing will appear in the

Re: Enrichment plugin for adding attributes from SQL

uld be worth contributing I'll invest further time in neatening these up and provide full test cases, atm these are just a PoC that's been proven. > On 9 Jan 2018, at 18:50, Brett Ryan <brett.r...@gmail.com> wrote: > > >> On 7 Jan 2018, at 22:23, Brett Ryan <bret

Re: Enrichment plugin for adding attributes from SQL

> On 7 Jan 2018, at 22:23, Brett Ryan <brett.r...@gmail.com> wrote: > > (1) Write a processor that updates an attribute on a FF from a Record based > LookupService. Say I call it UpdateAttributeFromRecordLookup Ok, I've implemented this now as a P

Re: NiFi data HA in cluster mode

I had someone from Hortonworks suggest to me that I should also set any PutSQL processors to only execute on primary. The reasoning was due to flooding of the JDBC pool. > On 9 Jan 2018, at 17:25, Joe Witt wrote: > > I'd avoid setting any processor to primary node only

Re: Getting lost a bit

and any from the git repo) that were made > since the commit hash from which you were making your changes. > > - Jeff > >> On Mon, Jan 8, 2018 at 1:58 AM Brett Ryan <brett.r...@gmail.com> wrote: >> >> I must be an odd one, I’m more comfortable resolving confl

Re: Contributing to NiFi

I am not a NiFi contributor myself, however; I have just started writing a plugin for publication. My advice would be to use the NiFi developer guide [1] and publish on github under the apache 2 licence. Having done this and you think this is a plugin that could be beneficial for the NiFi user

Re: Status from the Jar

Hi Anakit. From a processor you can control whatever your relationships are for down stream processors. This can be achieved by a few steps. Create a static relationship reference within your processor. For example, if you want to handle success and failure routing behaviour static final

Re: Getting lost a bit

I must be an odd one, I’m more comfortable resolving conflicts from the cli ;) The simplest way is the CSV resolution strategy. Edit each file looking for <<< —- >>> blocks where conflicts are presented. Keep the section you want deleting the other side, then git add the file. Do for all

Re: Enrichment plugin for adding attributes from SQL

or names, it would be better for a label name to be used. > [1] > https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#adding-components-to-the-canvas > [2] https://issues.apache.org/jira/browse/NIFI-3338 > [3] https://issues.apache.org/jira/browse/NIFI-4249 > > -Drew >

Re: Enrichment plugin for adding attributes from SQL

sen <mikerthom...@gmail.com> wrote: > > Take a look at the mongo lookup service. I think it could serve as a good > example here. > On Fri, Jan 5, 2018 at 10:49 PM Brett Ryan <brett.r...@gmail.com > <mailto:brett.r...@gmail.com>> wrote: > >> Looking at using

Re: Enrichment plugin for adding attributes from SQL

gt; makes your flow work is fine. Just explaining why you likely won’t see a > solution like that in the NiFi bundled components. > > > Andy LoPresto > alopre...@apache.org <mailto:alopre...@apache.org> > alopresto.apa...@gmail.com <mailto:alopresto.apa...@gmail.com>

Re: Enrichment plugin for adding attributes from SQL

.org > alopresto.apa...@gmail.com > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > >> On Jan 4, 2018, at 4:07 PM, Brett Ryan <brett.r...@gmail.com> wrote: >> >> Thanks Andy, how would update attribute be able to get the value from sql? >

Re: Enrichment plugin for adding attributes from SQL

ike that in the NiFi bundled components. > > > Andy LoPresto > alopre...@apache.org > alopresto.apa...@gmail.com > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > >> On Jan 4, 2018, at 4:07 PM, Brett Ryan <brett.r...@gmail.com> wrote: >> >

Re: Enrichment plugin for adding attributes from SQL

ex.html >> >> Andy LoPresto >> alopre...@apache.org >> alopresto.apa...@gmail.com >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >> >>> On Jan 4, 2018, at 7:03 AM, Brett Ryan <brett.r...@gmail.com> wrote: >>>

Re: Enrichment plugin for adding attributes from SQL

>> >> [1] >> https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-update-attribute-nar/1.4.0/org.apache.nifi.processors.attributes.UpdateAttribute/index.html >> >> Andy LoPresto >> alopre...@apache.org >> alopresto.apa...@gmail.com >> PGP Fingerprint: 70EC B3E5 98A6

Enrichment plugin for adding attributes from SQL

Hi all, having used NiFi for a couple days I wanted to add some attributes to a FlowFile while not altering the contents of that FlowFile. I had suggestions to use a script processor but that just sounded like a hack which could become a nuisance to replicate. Anyway, I figured I'd write a