Re: Calling getLogger() from @OnScheduled, @OnStopped, etc.

2018-04-12 Thread Russell Bateman
Yes, this is what I assumed, but I was hoping that someone had developed a technique for reaching the log in some (twisted) way perhaps that I hadn't figured out yet. It would really help me visualize the order in which my code's called and help me feel better about what I've written. Thanks,

Re: Restarting NiFi causing SiteToSiteBulletinReportingTask to fail

2018-04-12 Thread Pierre Villard
Hi Chad, I believe this could have been fixed recently but I've very limited access right now (and for the next few days) and can't be sure... I will check next week if no one gave you feedbacks before. Pierre 2018-04-12 19:57 GMT+02:00 Woodhead, Chad : > I am running

Re: Calling getLogger() from @OnScheduled, @OnStopped, etc.

2018-04-12 Thread Bryan Bende
The example processor you showed won’t work because you are calling getLogger() inline as part of the variable declaration. The logger is given to the processor in an init method which hasn’t been called yet at that point, so that is assigning null to the variable. Generally you should just call

Re: Calling getLogger() from @OnScheduled, @OnStopped, etc.

2018-04-12 Thread Matt Burgess
I’m AFK right now but I’m thinking there is a point in the lifecycle where you can expect a logger, and before that (like when the processor is instantiated as you have it coded), you may not be able to assume there is a logger (either in real NiFi or the mock-verse :) My guess is the rule of

Re: Calling getLogger() from @OnScheduled, @OnStopped, etc.

2018-04-12 Thread Russell Bateman
Thanks for responding, Andy. I am able to use it, like you, in onTrigger(). Where I haven't been able to use it is from annotated methods (in the sense that onTrigger()isn't annotated except by @Overridewhich is not relevant in this question). Imagine: public class Fun extends

Re: Calling getLogger() from @OnScheduled, @OnStopped, etc.

2018-04-12 Thread Andy LoPresto
Hi Russ, Are you saying the code that breaks is having “getLogger()” executed inside one of the processor lifecycle methods (i.e. GetFile#onTrigger()) or in your test code (i.e. GetFileTest#testOnTriggerShouldReadFile())? I’m not aware of anything with the JUnit runner that would cause issues

Re: Ongoing support for a contribution

2018-04-12 Thread Aldrin Piri
Hi Anthony, >From a general ASF standpoint, when that code is contributed to/accepted by the project it is taken on by the community and its contributors. Of course you and any others involved in the contribution process would be encouraged to help out as much as you care to, but there is no

Re: Ongoing support for a contribution

2018-04-12 Thread Mike Thomsen
Anthony, I think monitoring those lists would be a great idea. In the interim, if you could get a Docker image up on Docker Hub that allows members of the community to casually spin up a working MarkLogic instance suitable for kicking the tires with your processor(s) that would be a big help.

Ongoing support for a contribution

2018-04-12 Thread Anthony Roach
Hi, I am very new to the open source world. My organization is preparing to submit a NiFi processor to load data into our commercial NoSQL database. Once the review and commit are completed, what are our responsibilities for managing issues? Do we monitor the issues distribution list?

Restarting NiFi causing SiteToSiteBulletinReportingTask to fail

2018-04-12 Thread Woodhead, Chad
I am running HDF 3.0.1.1 which comes with NiFi 1.2.0.3.0.1.1-5. We are using SiteToSiteBulletinReportingTask to monitor bulletins (for things like Disk Usage and Memory Usage). When we restart NiFi via Ambari (either with a Restart or Stop and then Start), when NiFi comes back up the

Re: Data migration in sequence

2018-04-12 Thread Mike Thomsen
> Hi Joe, The systems are the same. A and B. Just that i will be pulling > different data set and that has to be done in sequence If your input is JSON that is being sent to the originating API to get different data sets, what you can do is put all of the GenerateFlowFile instances' content into

Re: Data migration in sequence

2018-04-12 Thread Anil Rai
Thanks Mike and Joe for the quick response. Hi Mike, Nifi is making api calls to source -> pull data -> Maps -> Post data to target via API. Generate flow file triggers the flow. Right now i manually run 1 process group after the previous one is complete. Hi Joe, The systems are the same. A and B.

Re: [EXT] Suggestion: Apache NiFi component enhancement

2018-04-12 Thread Sivaprasanna
No my suggestion was a simpler approach. It just affects only the UI aspect as my intention is to just override how the 'type' gets rendered in the UI. For example, a processor's type is set to its canonical class name ( DtoFactory.java#createProcessorDto

Re: Data migration in sequence

2018-04-12 Thread Joe Witt
Anil What is a little hard to tell here is do you mean: 1) I have to transfer data from system A to system B where there are 10 steps I want to execute in NiFi between A and B. -or- 2) I have to transfer data from systems A,B,C,D,E,F,G,H,I,J to system Z where first I want to load data from

Re: Data migration in sequence

2018-04-12 Thread Mike Thomsen
It'd be very helpful to know what the input and output processors are for each of the flows because they have different capabilities for triggering them. On Thu, Apr 12, 2018 at 12:03 PM Anil Rai wrote: > sorry, sent email in between. > This can be achieved by putting few

Data migration in sequence

2018-04-12 Thread Anil Rai
Experts, Below is my requirenment. I have to do a 1 time data load from system a to b. I have to transfer 10 different data sets. These transfers have to happen in sequence. 1 followed by 2 followed by 3 so on till 10. We have created seperate process groups for each of the data set migration.

Calling getLogger() from @OnScheduled, @OnStopped, etc.

2018-04-12 Thread Russell Bateman
I seem to crash NiFi JUnit test runner when I have code that calls getLogger()or attempts to make use of the product of calling getLogger()in situations where some context (probably) is missing like methods annotated for call at "special" times. This makes sense, but is there a technique I can

Re: Nifi Parallel Execution

2018-04-12 Thread Eric Ulicny
We have attempted to use the distributed map cache with the Detect Duplicate processor as recommended to no avail. The first time two identical flowfiles are sent simultaneously they both make it through to the the non-duplicate relationship. After that point they will be appropriately detected.

Re: Nifi UI improvement

2018-04-12 Thread Scott Aslan
Hello Jorge, There are many efforts ongoing toward the improvement of the NiFi UI and you can see a list of them here: https://issues.apache.org/jira/browse/NIFI-5066?jql=project%20%3D%20NIFI%20AND%20component%20%3D%20%22Core%20UI%22. Please feel free to continue this discussion to determine the

RE: [EXT] Suggestion: Apache NiFi component enhancement

2018-04-12 Thread Peter Wicks (pwicks)
I think this is a good idea. But based on your example I think you would want to provide a primary Type along with a list of Alias types. If NiFi starts and it can no longer find a processor by the Type name it had in the flow.xml it can check he annotations/aliases to see if it's been renamed.

RE: (NiFi 1.6) Funnels with no outgoing relationship filling my app log

2018-04-12 Thread Peter Wicks (pwicks)
https://issues.apache.org/jira/browse/NIFI-5075 From: Peter Wicks (pwicks) Sent: Thursday, April 12, 2018 14:30 To: 'dev@nifi.apache.org' Subject: (NiFi 1.6) Funnels with no outgoing relationship filling my app log I just upgraded one of my servers to NiFi 1.6.0. I have a

Nifi UI improvement

2018-04-12 Thread Jorge Machado
Hi guys, Is there any effort to improve the UI ? One nice feature to have would be: Have the possibility to limit searches on a canvas. Let’s say we are inside a processor group and we only want to search from there on. Regards Jorge Machado

(NiFi 1.6) Funnels with no outgoing relationship filling my app log

2018-04-12 Thread Peter Wicks (pwicks)
I just upgraded one of my servers to NiFi 1.6.0. I have a couple of funnels that just dead end, Flow File's come in but never go anywhere after that. Mostly I use this for troubleshooting/validation. The inbound relationships all have expiration times, and it's a quick way for me to inspect the