State Management - Set initial State or update State through GUI or API

I'm using the QueryDatabaseTable processor. Frequently I would like to set an initial state value for my columns max value, or I'd like to change the value stored in State Management for that column, this comes up in operational scenarios ("reload all data after this exact timestamp, that's

Externally update processor State

I'm looking for a way to externally update a processors state, preferably through an API call and then at some point the existing State viewer with the clear button. From digging through the code base I couldn't find an exposed API that would allow for state modification; only a state clearing.

RE: Externally update processor State

in the process flow going on - like a workflow manager. Can you describe the details of your use case/ideal user experience a bit further? Thanks Joe On Mon, Aug 15, 2016 at 12:22 PM, Peter Wicks (pwicks) <pwi...@micron.com> wrote: > I'm looking for a way to externally update a processors state, p

RE: Externally update processor State

this. And there is already a clear and well managed model and REST API for operating on those. Does that sound like a fair alternative? Thanks JOe On Mon, Aug 15, 2016 at 1:03 PM, Peter Wicks (pwicks) <pwi...@micron.com> wrote: > Joe, > > The specific use cases at the moment both surround QueryDataba

RE: Discuss: Process Group as a Process / "Single Threaded" Process Group

as asked is a bit outside the spirit of flow based > programming fundamentals that NiFi is built on so I'm hopeful we can > find another path that gets you the result you need. > > Thanks > Joe > > On Wed, Feb 15, 2017 at 11:18 PM, Peter Wicks (pwicks) > <pwi...@micron

RE: Discuss: Process Group as a Process / "Single Threaded" Process Group

it of flow based programming fundamentals that NiFi is built on so I'm hopeful we can find another path that gets you the result you need. Thanks Joe On Wed, Feb 15, 2017 at 11:18 PM, Peter Wicks (pwicks) <pwi...@micron.com> wrote: > I've been throwing around some ideas for how t

Discuss: Process Group as a Process / "Single Threaded" Process Group

I've been throwing around some ideas for how to migrate processes that would work great in NiFi if you only processed one FlowFile at a time, but if there are multiple Flow Files in the Flow processing through various Processes it would cause issues. One of the concepts I've been playing with

RE: Creating a custom DatabaseAdapter

Matt/Stanislav, As part of NIFI-3093 (HIVE support for QueryDatabaseTable) I had planned to move DatabaseAdapter to the DBCP Service Controller project; whether or not that makes sense I have no idea, and I haven't done it at this time. But I wanted to be able to rehome the new HIVE

RE: [VOTE] Establish Registry, a sub-project of Apache NiFi

+1 (non-binding) -Original Message- From: Bryan Bende [mailto:bbe...@gmail.com] Sent: Friday, February 10, 2017 9:41 AM To: dev@nifi.apache.org Subject: [VOTE] Establish Registry, a sub-project of Apache NiFi All, Following a solid discussion for the past few days [1] regarding the

How to get a ProcessorGroup's Version#

Running NiFi 1.1.0. I'm trying to use the nifi-api to call Delete on a Process Group; URL format looks like this: /nifi-api/process-groups/{ID}?version={#}={clientid}. I have the Process Group's ID, and ClientID is not really an issue, but I can't figure out how to get the current Version #

RE: 1.0 Run Status Colors

. On Wed, Aug 17, 2016 at 2:33 PM, Peter Wicks (pwicks) <pwi...@micron.com> wrote: > We've been playing with NiFi 1.0 and of course the new theme/layout is a big > change. > One thing we don't particularly care for is the bland gray > Running/Stopped/Disabled colors. I would be wi

Data Formats in PutSQL

As part of NIFI-2591 and https://github.com/apache/nifi/pull/883 I've been looking at adding Binary data support to PutSQL, which until now has not supported binary data types at all. After submitting the PR a discussion arose around supporting more than one style of binary encoding.

1.0 Run Status Colors

We've been playing with NiFi 1.0 and of course the new theme/layout is a big change. One thing we don't particularly care for is the bland gray Running/Stopped/Disabled colors. I would be willing to have them bland in the header, but on the Processor/Process Group level I feel they really need

RE: Looking for feedback on my WIP Design

`RecordSetConverter` that can convert between arbitrary types, and `RecordSetToSQL` etc... Could be really cool. --Peter -Original Message- From: Peter Wicks (pwicks) Sent: Monday, August 22, 2016 8:57 PM To: 'dev@nifi.apache.org' <dev@nifi.apache.org> Subject: RE: Looking for fe

Working with FlowFiles and Sessions

I have a couple of innards questions: 1. I'm familiar with how to write flow files, but what if I want to write to multiple FlowFiles in parallel, rather than serially? The use case I have in mind involves reading through an input stream and generating multiple derivative copies in

RE: ExecuteSql output formats.

Toivo, My initial design was also much simpler, and more like what you described. But Matt made some suggestions towards a bigger picture allowing for a wider audience and more use cases and I've been building off of those guidelines. I've got a lot more code done than what is checked into

RE: Looking for feedback on my WIP Design

re we want to run some filtering logic. We may want to get in a record, check if it passes the filter, and if so, pass on the original object to the serializer. Thanks -Mark > On Sep 6, 2016, at 1:38 PM, Peter Wicks (pwicks) <pwi...@micron.com> wrote: > > Mark, > > Thank

RE: Looking for feedback on my WIP Design

eric and return an object of type T? I know this is quite a bit of feedback. Hopefully not too overwhelming :) There's a whole lot of good stuff here, and I think this is going to turn out to be an awesome addition to NiFi. Very excited about it! -Mark > On Aug 23, 2016, at 1:00

Back Pressure Indicators?

One of the things I've been noticing that doesn't seem to have any status indicator for is back pressure. When back pressure kicks in I usually notice because I start thinking that NiFi is being buggy or is broken, "Why aren't these Flow Files moving through the queue?". Then I'll notice the

RE: Re: PutSQL error sql.arg3.value cannot convert to timestamp

Bryan/Matt, Wouldn't a JDBC truncation blacklist be easiest? If the destination type is timestamp/any other applicables, then don't take the substring? --Peter -Original Message- From: Bryan Bende [mailto:bbe...@gmail.com] Sent: Saturday, September 10, 2016 1:00 PM To:

RE: NIFI java.lang.UnsupportedOperationException

We are using NiFi to connect to MS SQL Server and have had success with both Sqljdbc41.jar and Sqljdbc42.jar. -Original Message- From: Bryan Bende [mailto:bbe...@gmail.com] Sent: Monday, September 19, 2016 9:12 AM To: dev@nifi.apache.org Subject: Re: NIFI

Looking for feedback on my WIP Design

I'm working on a change to QueryDatabaseTable (and eventually would apply to ExecuteSQL) to allow users to choose the output format, so something besides just Avro. I'm planning to put in a ticket soon for my work. Goals: - If this update goes out no user should be affected, as

RE: Looking for feedback on my WIP Design

re in good shape for world domination :) Not being sarcastic, I'm saying let's make it happen! Regards, Matt On Mon, Aug 22, 2016 at 9:32 PM, Peter Wicks (pwicks) <pwi...@micron.com> wrote: > I'm working on a change to QueryDatabaseTable (and eventually would apply to > ExecuteSQL) to

RE: 1.0 Run Status Colors

d like to read > some document supporting/justifying the UI changes, which is not to > say that I presume to condemn them, just that I'd like to understand the > reasoning. > The existing NiFi UI was already very good. > > NiFi rocks; I hope it will keep rocking! > > On Thu

RE: NiFi 1.0.0-RC3 REST API to start a processgroup

Hi, I have created a custom wizard using ASP.Net/C# that uses only the 1.0 API to create data flows in NiFI. The code may not be 100% translatable, but here is how I do it in C#: public ProcessorGroup ChangeProcessGroupRunningState(string groupId, bool running) { using

RE: Questions about heterogeneous cluster and queue problem/bug/oddity in 1.0.0

Joe, I didn’t get the images (might just be my exchange server). How many files are in the queue? (exact count please) --Peter From: Joe Skora [mailto:jsk...@gmail.com] Sent: Monday, September 26, 2016 8:20 PM To: dev@nifi.apache.org Subject: Questions about heterogeneous cluster and queue

Get PR reviewed for 1.1? (PR-1001)

I've had a PR open since 9/9 that I'm hoping to have reviewed and included in 1.1. https://github.com/apache/nifi/pull/1001 I made one small change, as requested in the comments, and there are no open issues that I'm aware of. If it can get reviewed, I'd appreciate it. Thanks, Peter

Extra Pair of Eyes

I'm working on a ticket to add HIVE support to QueryDatabaseTable (NIFI-3093). I'm doing this by adding a HIVE DatabaseAdapter and everytime I find something that doesn't work in HIVE I add a "getXXXSupported" property to DatabaseAdapter. Initially I had my HiveDatabaseAdapter in Standard

RE: Extra Pair of Eyes

: org.apache.nifi nifi-standard-services-api-nar nar This will will cause the parent of the Hive NAR to be the standard-services-api NAR, so that it can properly link the interface & implementation together. Thanks -Mark > On Nov 23, 2016, at 1:02 PM, Peter Wicks (pwicks) <pwi...@micron.

RE: Extra Pair of Eyes

ter Wicks (pwicks) <pwi...@micron.com> wrote: > > Mark, I added the dependency but it appears to have had no effect after a > rebuild and redeploy. > I checked the change into the same branch. > > Thoughts? > > > -Original Message- > From: Peter Wic

RE: Extra Pair of Eyes

Mark, I added the dependency but it appears to have had no effect after a rebuild and redeploy. I checked the change into the same branch. Thoughts? -Original Message- From: Peter Wicks (pwicks) Sent: Wednesday, November 23, 2016 11:27 AM To: dev@nifi.apache.org Subject: RE: Extra

NiFi Processors show 30 Second Execution time, 0 executions

I just noticed that on the latest version (which is now working for me) that all processors that have not recently run show 0 Tasks and 30 Seconds of Time. Is this a bug, or something that just isn't obvious? I previously executed ExecuteSQL and GenerateTableFetch in this view, but it was over

RE: NiFi Processors show 30 Second Execution time, 0 executions

8-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile Repository with 4 records in 50 milliseconds On Fri, Mar 31, 2017 at 4:25 AM, Peter Wicks (pwicks) <pwi...@micron.com> wrote: > I just noticed that on the latest version (which is now working for > me) that al

Error during startup on latest code

I updated my fork to the latest this morning to do some testing and have been getting the below error on startup. My first assumption was a build issue on my end so I cleaned out the NiFi MVN cache on my box and built again, but I'm still getting this error. I've only tested this on my Windows

Increase Concurrent Tasks While Running

Apart from the feature not being there, is there any real technical hurdle from allowing users to increase the Concurrent Tasks count while a processor is already running? I found myself waiting for a very large file to process and a second file came in on a different relationship. I couldn't

RE: NiFi Processors show 30 Second Execution time, 0 executions

7 VM anywhere, I know Witt was using Win10 and didnt see it... I find it odd that ALL your processors have a 30 second number not just the UpdateAttribute. Anything else about your environment you can share that may be unique? On Fri, Mar 31, 2017 at 10:15 AM, Peter Wicks (pwicks) <

RE: NiFi Processors show 30 Second Execution time, 0 executions

gt; > On Mar 31, 2017, at 6:58 AM, Peter Wicks (pwicks) > > <pwi...@micron.com> > wrote: > > > > channel. > -- Joseph

RE: [EXT] Re: Improved Color Scheme for NiFi UI

Another option that I had been playing around with was using the Chrome plugin Stylish to create a NiFi color scheme that you can turn on/off on the fly. It's very much a work in progress, as I've only been working on it for myself so far and didn't make it available

RE: [EXT] Re: NiFi 1.2.0 Record processors question

I appreciate the clarification as well. I was really confused why my Avro files weren't converting, and this explains it; though I have to say the error messages you run into in this scenario are not clear. -Original Message- From: Koji Kawamura [mailto:ijokaruma...@gmail.com] Sent:

RE: NiFi Processors show 30 Second Execution time, 0 executions

iday, March 31, 2017 10:54 PM To: dev@nifi.apache.org Subject: Re: NiFi Processors show 30 Second Execution time, 0 executions What version of Java are you running on ? Major_minor? On Fri, Mar 31, 2017 at 10:49 AM, Peter Wicks (pwicks) <pwi...@micron.com<mailto:pwi...@micron.com>&g

RE: [EXT] Re: NiFi Processors show 30 Second Execution time, 0 executions

est to go ahead and file a JIRA. In it you can reliably > post the attachments. There was a potentially related timezone > handling issue as I recall in this past release so perhaps there is > some relationship. > > Thanks > Joe > > On Sun, May 28, 2017 at 10:04 PM, Peter Wick

Lifecycle of a Processor instance

There had been a lot of internal discussions at work about the actual life cycle of the class instance instantiated by NiFi; and the variables scoped at the class level. When do Processors "reset", are new instances created for each run, or are instances recycled? What about concurrent threads

Please refresh my memory on NAR dependencies

For NIFI-4465 I want the nifi-poi-bundle to include a Maven dependency on nifi-record-serialization-services. So I start by adding the dependency to the pom.xml. org.apache.nifi nifi-record-serialization-services I've tried several variations on this, with version numbers,

RE: [EXT] Re: Please refresh my memory on NAR dependencies

(though I'm guessing about the latter). Regards, Matt On Sun, Oct 15, 2017 at 10:27 PM, Peter Wicks (pwicks) <pwi...@micron.com> wrote: > For NIFI-4465 I want the nifi-poi-bundle to include a Maven dependency on > nifi-record-serialization-services. So I start by adding th

RE: [EXT] Re: Funnel Queue Slowness

think? Thanks On Mon, Oct 9, 2017 at 9:01 PM, Peter Wicks (pwicks) <pwi...@micron.com> wrote: > Bryan, > > Yes, it was the penalty causing the issue. This feels like weird behavior for > Funnel’s, and I’m not sure if it makes sense for penalties to work this way. > >

RE: [EXT] Re: Funnel Queue Slowness

05 GMT+02:00 Peter Wicks (pwicks) <pwi...@micron.com>: > Joe, > > It really is about just forgetting that penalization is a thing. > Penalized files are fairly well marked when you do a List Queue. > > I think Funnel's need an overall re-examination. I noticed another >

RE: [EXT] Re: Please refresh my memory on NAR dependencies

is the API JAR I think. If you need the implementations behind > it, you will probably need to declare that as a parent (not a > dependency) and perhaps still use the API JAR (though I'm guessing about the > latter). > > Regards, > Matt > > > On Sun, Oct 15, 2017 at 10:

RE: [EXT] Re: JAVA_HOME trouble in nifi.sh

n, JDK versions, etc)? That locateJava blurb is something we used from another ASF project and use heavily throughout our executables, so definitely need to track it down. Did you happen to see this substitution in any other files? On Thu, Oct 12, 2017 at 10:51 PM, Peter Wicks (pwicks) <pwi...@m

Architecting the MS SQL CDC Processor

I've been working on a new processor that does Change Data Capture with Microsoft SQL Server. I followed Microsoft's documentation on how CDC works, and I've got some code that gets me the changes and is testing well. Right now, I don't actually have a processor, but a number of scripts that

RE: [EXT] ConvertCSVToAvro vs CSVReader - Value Delimiter

Arun, I'm also using Ctrl+A as a delimiter and had the same problem. I haven't had time to write up a PR but it looked like a pretty easy fix to me too. I can't merge the change if you submit it, but I'd be happy to review it. --Peter -Original Message- From: Arun Manivannan

RE: [EXT] Re: [VOTE] Release Apache NiFi 1.4.0 (RC2)

+1 (non-binding). Upgraded to 1.4.0, openJDK 1.8.0_102, RHEL. No issues. -Original Message- From: Joey Frazee [mailto:joey.fra...@icloud.com] Sent: Monday, October 02, 2017 3:50 AM To: dev@nifi.apache.org Subject: [EXT] Re: [VOTE] Release Apache NiFi 1.4.0 (RC2) +1 (non-binding) -

RE: [EXT] Re: Funnel Queue Slowness

, is it possible all the flow files going to failure are being penalized which would cause them to not be processed immediately? -Bryan On Oct 8, 2017, at 10:49 PM, Peter Wicks (pwicks) <pwi...@micron.com<mailto:pwi...@micron.com>> wrote: I’ve been running into an issue on 1.

Funnel Queue Slowness

I've been running into an issue on 1.4.0 where my Funnel sometimes runs slow. I haven't been able to create a nice reproducible test case to pass on. What I'm seeing is that my failure queue on the right will start to fill up, even though there is plenty of room for them in the next queue. You

CIFS/SMB Shares from Linux

We have a number of CIFS networks shares that I've been denied permission to mount on my NiFi Linux server. In the past I've accessed similar shares by running NiFi on a Windows server and then pushing the files using Site-to-Site, but I'd rather avoid that if I can. In the past those Windows

Contrib Check Build - RAT and Unit Test Failure

I'm trying to successfully run Contrib Check on my dev box. Windows 10 x64 jdk1.8.0_91 MVN 3.3.9 I'm using IntelliJ, so my Run looks like: Command line: -T2 -Drat.skip=true clean install Profile: contrib-check If I don't disable RAT (-Drat.skip=true) then I get too many RAT failures for the

RE: [EXT] Re: Contrib Check Build - RAT and Unit Test Failure

and Unit Test Failure Peter I dont' believe RAT works in parallel builds (-T2 for example). If I use RAT it is during a full clean build and activated via the 'contrib-check' profile. 'mvn clean install -Pcontrib-check' Thanks On Wed, Nov 22, 2017 at 9:19 PM, Peter Wicks (pwicks) <pwi...@micron.

RE: [EXT] Re: Contrib Check Build - RAT and Unit Test Failure

be an actual windows problem and the test is possibly flagging it. Thanks On Wed, Nov 22, 2017 at 9:34 PM, Peter Wicks (pwicks) <pwi...@micron.com> wrote: > Joe, > > This did resolve the RAT issue, but the unit test I mentioned still fails > with the same error. > > -

Automated Testing of Flows

I would like to find a way to automatically run integration/upgrade tests on flows, rather than just one or two processors like the mock framework supports. Preferably something Template driven, where I can provide a template xml file, and an accompanying config file to setup the test, then run

RE: [EXT] Re: Automated Testing of Flows

, Nov 15, 2017 at 8:07 PM, Peter Wicks (pwicks) <pwi...@micron.com> wrote: > I would like to find a way to automatically run integration/upgrade > tests on flows, rather than just one or two processors like the mock > framework supports. Preferably something Template driven, where I

RE: [EXT] Re: [DISCUSS] CI / Travis / Jenkins

Best news I've heard in a long time :) Thanks for all the hard work! -Original Message- From: af...@fucs.org [mailto:af...@fucs.org] On Behalf Of Andre Sent: Thursday, December 07, 2017 12:15 To: dev@nifi.apache.org Subject: [EXT] Re: [DISCUSS] CI / Travis / Jenkins Joe, Thanks for

Adding EL support to UpdateAttribute for the Attribute Name

I've run into a use case for adding EL support into the Attribute Name itself in UpdateAttribute. Looking for thoughts on other approaches, pros/cons of doing this. I'm generically extracting data from a database. Right now I have ~30 tables, but that number could be anything, just think big

Initial Admin Identity can't query provenance?

I just setup a new NiFi instance and setup myself as the Initial Admin Identity, I'm currently the only user on the box and I've made no security changes. I noticed today that I couldn't query provenance. I had to go in and create a new policy and add myself to it. This seems odd for an

RE: [EXT] Re: ReplaceText Flow File Processing Count

pressure threshold to 1 on the queue > leading into ReplaceText. > > On Fri, May 4, 2018 at 3:50 AM, Peter Wicks (pwicks) <pwi...@micron.com> > wrote: >> Had a user notice today that a ReplaceText processor, scheduled to run every >> 20 minutes, had process

Copying small portions of code form other Apache Projects

I'm working on some changes and found that the exact code I need, along with a lot of code I can make derivatives from, are buried in the Apache Ant project. I'm not going to include Ant as a dependency, just copy some code from it. Should there be some annotation on the code/license

Primary Only Content Migration

I'm sure many of you have the same situation, a flow that runs on a cluster, and at some point merges back down to a primary only processor; your files sit there in the queue with nowhere to go... We've used the work around of having a remote processor group that loops the data back to the

RE: [EXT] Re: Primary Only Content Migration

primary node will continue to be worked off by the downstream processors on the old node until they are all processed. -Bryan On Thu, Jun 7, 2018 at 1:55 PM, Peter Wicks (pwicks) wrote: > I'm sure many of you have the same situation, a flow that runs on a cluster, > and at some point

RE: [EXT] Re: Primary Only Content Migration

. It needs a lot of thought/design/testing/etc.. I was just having a conversation about this yesterday. It is definitely a thing and will be a major effort. Will make a JIRA for this soon. Thanks On Thu, Jun 7, 2018 at 5:21 PM, Peter Wicks (pwicks) wrote: > Bryan, > > We see this with la

RE: [EXT] [ANNOUNCE] New Apache NiFi Committer Sivaprasanna Sethuraman

Congratulations Sivaprasanna! -Original Message- From: Tony Kurc [mailto:tk...@apache.org] Sent: Tuesday, June 05, 2018 08:09 To: dev@nifi.apache.org Subject: [EXT] [ANNOUNCE] New Apache NiFi Committer Sivaprasanna Sethuraman On behalf of the Apache NiFI PMC, I am very pleased to

Preliminary work on SCP and SSH processors

I’ve been working on NIFI-539 and NIFI-3698, adding support for SSH based processors like SCP and an Execute SSH. I’ve submitted a PR, but it definitely needs some work still. Would appreciate some feedback on the approaches. Unlike most of my code changes, this one should be easy for most

RE: [EXT] RE: JIRA #NIFI-5327

Prashanth, Woops, I had a fix for that test error months ago, but dropped it while pushing a PR and never put in a ticket for it... I've submitted a PR with the fix. https://github.com/apache/nifi/pull/2819 --Peter

ReplaceText Flow File Processing Count

Had a user notice today that a ReplaceText processor, scheduled to run every 20 minutes, had processed all 14 files in queue at once. I looked at the code and see that ReplaceText does not do a standard session.get, but instead calls: final List flowFiles =

RE: [EXT] Re: Architecting the MS SQL CDC Processor

essor were the first to support this, it could be the exemplar for past and future CDC processors :) Regards, Matt [1] https://issues.apache.org/jira/browse/NIFI-4491 On Mon, Oct 16, 2017 at 10:36 PM, Peter Wicks (pwicks) <pwi...@micron.com> wrote: > I've been working on a new processor th

RE: [EXT] Re: Moving UI objects on a parent you don't have access to

ow this due to time constraints. I do not object to >> your proposal to change this (maybe Matt Gilman feels differently?). >> If you open a Jira, it should be doable. >> >> Andy LoPresto >> alopre...@apache.org >> *alopresto.apa...@gmail.com * PGP >> Fing

Moving UI objects on a parent you don't have access to

While experimenting with permissions, I found that if I have no permissions to a process group, but do have permissions to a child that lives in that group, I can move that child around on the UI. I know that in the object model the x,y position values are part of the child, which I have

Looking for reviewer for Unit Test only PR (NiFi-5381)

I try really hard not to ask for PR reviews... but I have a PR that only adds unit tests for GetSFTP and PutSFTP that's been open for about a month. I have a sizeable number of other code changes ready that affect these processors and wanted to get better unit tests in place before moving

RE: [EXT] Re: Dark mode

I did something similar back in 1.2.0 using CSS overrides and a chrome plugin : https://userstyles.org/styles/142978/dark-nifi-1-2-0. -Original Message- From: Brandon DeVries [mailto:b...@jhu.edu] Sent: Friday, July 13, 2018 9:42 AM To: dev@nifi.apache.org Subject: [EXT] Re: Dark mode

RE: [EXT] Re: Embedded Nifi

Vincent, Embedded NiFi still has a long ways to go to be really useful, in my opinion; and I don't know if anyone is actively working on those improvements. The PR Andy mentioned simply allows you to startup NiFi inside your process instead of running it directly from a startup script, but

RE: [EXT] Suggestion: Apache NiFi component enhancement

I think this is a good idea. But based on your example I think you would want to provide a primary Type along with a list of Alias types. If NiFi starts and it can no longer find a processor by the Type name it had in the flow.xml it can check he annotations/aliases to see if it's been renamed.

RE: (NiFi 1.6) Funnels with no outgoing relationship filling my app log

https://issues.apache.org/jira/browse/NIFI-5075 From: Peter Wicks (pwicks) Sent: Thursday, April 12, 2018 14:30 To: 'dev@nifi.apache.org' <dev@nifi.apache.org> Subject: (NiFi 1.6) Funnels with no outgoing relationship filling my app log I just upgraded one of my servers to NiFi 1.6.0.

(NiFi 1.6) Funnels with no outgoing relationship filling my app log

I just upgraded one of my servers to NiFi 1.6.0. I have a couple of funnels that just dead end, Flow File's come in but never go anywhere after that. Mostly I use this for troubleshooting/validation. The inbound relationships all have expiration times, and it's a quick way for me to inspect the

Should I postpone submitting PR's on seriously breaking changes until prep for 2.0?

I have a couple ideas for changes that would cause wide spread breakage (like moving the DatabaseAdapter selection into the DBCP Service). >From what I've seen, this level of non-backwards compatibility should be >postponed until a major version change? Thanks, Peter

RE: [EXT] Re: NiFi Versioned Process Group Status Icons

cess Group is not configured to have any encapsulated components, we still render the counts as zero. Additionally, the presence of the icons helps drive awareness of the feature. Rob or Drew may have some additional insight to add. Hope this helps Matt On Fri, Mar 23, 2018 at 12:10 AM, Peter Wicks (p

Help with frontend-maven-plugin?

The version of "frontend-maven-plugin" used in NiFi is out of date, and the versions of NPM and Node referenced by it are also a bit old. I wouldn't bring this up, except there is a bug in this version of NPM that can cause issues when trying to download NPM packages via proxy server. This is

RE: [EXT] Re: Help with frontend-maven-plugin?

tomatically use the proxy for downloading node and npm, as well as passing the proxy to npm commands <https://docs.npmjs.com/misc/config#proxy>. -Scott On Tue, Mar 20, 2018 at 5:02 AM, Peter Wicks (pwicks) <pwi...@micron.com<mailto:pwi...@micron.com>> wrote: > The versi

NiFi Versioned Process Group Status Icons

Why does NiFi show status icons for Versioned Process Group's on servers that are not configured to connect to a NiFi Registry? Thanks, Peter

License of NLKBufferedReader

While digging into ReplaceText today, I noticed that org.apache.nifi.processors.standard.util.NLKBufferedReader is actually a direct copy of the JDK version of BufferedReader, with a few minor code edits made on top of it so that new line characters are kept. This code was in the original code

RE: [EXT] Calcite version used in Nifi 1.7+

In NiFi 1.7.0 it was Calcite v1.12.0. In NiFi 1.8 it will be Calcite v1.17.0 -Original Message- From: ruurd.schoonh...@dbiq.nl [mailto:ruurd.schoonh...@dbiq.nl] Sent: Tuesday, October 16, 2018 6:58 AM To: dev@nifi.apache.org Subject: [EXT] Calcite version used in Nifi 1.7+ Hi guys,

New Standard Pattern - Put Exception that caused failure in an attribute

When a FlowFile is routed to failure, frequently there is no clear reason without looking into the actual error message. Some processors work around this by creating many different relationships, but even then frequently the generic Failure relationship also provides little guidance. I've seen

RE: [EXT] Re: New Standard Pattern - Put Exception that caused failure in an attribute

the errors out to >>> the original failure relationship, that could be quite confusing to >>> the user, especially as time goes on (how to suppress the "new" >>> errors, e.g.). >>> >>> IMHO I think we're between a rock and a hard place her

RE: [EXT] Re: New Standard Pattern - Put Exception that caused failure in an attribute

to a race hazard in an external system) but I don't imagine we'd route on invokehttp.java.exception.* since (as others have mentioned) it looks pretty fragile. -- James On Tue, 30 Oct 2018 at 16:44, Peter Wicks (pwicks) wrote: > > Sorry for the delayed response, I've been traveling. > &

RE: [EXT] Re: New Standard Pattern - Put Exception that caused failure in an attribute

/interpreting of the message with a bunch of routing > processors, which seems more brittle than just having the appropriate > relationships. > On Thu, Oct 25, 2018 at 1:36 AM Peter Wicks (pwicks) > wrote: >> >> When a FlowFile is routed to failure, frequently there is

RE: [EXT] Re: New Standard Pattern - Put Exception that caused failure in an attribute

ation from in NiFi 2.0. Regards, Matt On Thu, Oct 25, 2018 at 10:12 PM Peter Wicks (pwicks) wrote: > > Mark, > > I agree with you that this is the best option in general terms. After > thinking about it some more I think the biggest use case is for > troubleshooting. If a file

RE: [EXT] Re: New Standard Pattern - Put Exception that caused failure in an attribute

'there is nothing new to do now' then right? Do you agree? Thanks On Fri, Nov 9, 2018 at 3:59 PM Peter Wicks (pwicks) wrote: > > Joe, > > Several different opinions have been expressed by Matt Burgess, Mark Payne > and Bryan Bende about whether we should be storing exception information i

RE: [EXT] Re: New Standard Pattern - Put Exception that caused failure in an attribute

. So, with the above said can you clarify what you are precisely requesting and for 'who' - who is the actor. Thanks On Fri, Nov 9, 2018 at 2:12 PM Peter Wicks (pwicks) wrote: > > A one week bump on this thread. --Peter > > -Original Message----- > From: Peter Wicks (pwicks)

RE: [EXT] Re: New Standard Pattern - Put Exception that caused failure in an attribute

A one week bump on this thread. --Peter -Original Message- From: Peter Wicks (pwicks) Sent: Friday, November 2, 2018 11:54 AM To: dev@nifi.apache.org Subject: RE: [EXT] Re: New Standard Pattern - Put Exception that caused failure in an attribute Dev Team, I don’t think we've reached

RE: [EXT] XKCD use case for NiFi

I don't think NiFi will run on his phone, MiNiFi maybe? -Original Message- From: Joe Gresock [mailto:jgres...@gmail.com] Sent: Wednesday, October 3, 2018 12:08 PM To: dev@nifi.apache.org Subject: [EXT] XKCD use case for NiFi https://xkcd.com/2054/ He needs NiFi!

Future of HDF at Hortonworks/Cloudera

Normally, I wouldn’t ask a question like this on the NiFi Dev group. But I did a census and out of the top 10 committers to NiFi in the last year, 8 definitely work for Hortonworks (couldn’t figure out where MikeThomsen works). In the subsequent top 10 committers, there are at least four more

RE: [EXT] Re: [DISCUSS] Early, voluntary relocation to GitBox

Does closing a PR through GitHub close the JIRA ticket correctly? I'm assuming sign-off is automated this way, but wasn't sure about this step. Thanks, Peter -Original Message- From: Aldrin Piri [mailto:aldrinp...@gmail.com] Sent: Friday, December 14, 2018 6:23 AM To: dev Subject:

Backwards Compatibility for Property with Default Value

I'm working on a ticket that tweaks the way the Minimum File Age property works in GetFile and ListFile (NIFI-5897). Right now users see "0 sec" and assumes that means the minimum age check does not happen, when in fact it does; and in some timezone scenarios you have future dated files, and

RE: [EXT] Re: [DISCUSS] Early, voluntary relocation to GitBox

://blogs.apache.org/infra/entry/improved_integration_between_apache_and On Fri, Dec 14, 2018 at 3:34 PM Peter Wicks (pwicks) wrote: > Does closing a PR through GitHub close the JIRA ticket correctly? I'm > assuming sign-off is automated this way, but wasn't sure about this step. > > Thank

  1   2   >