Re: [discuss] NiFi support for Hadoop ecosystem components

2023-03-24 Thread James Srinivasan
I'm a Hadoop and Nifi user without vendor support so unsurprisingly aren't keen on #1, but then relying on community support and development is always going to be a risk for us. If it came to it, we'd probably stop using Nifi rather than pay a vendor which would be a real shame. Are certain

Re: Missing maven dependencies when building nifi

2022-04-05 Thread James Srinivasan
Possibly not your problem, but did you follow the instructions for building on Windows here: https://nifi.apache.org/quickstart.html On Tue, 5 Apr 2022, 12:42 Phil H, wrote: > Okay, I started over with a completely new local repo. This is the exact > sequence of commands I ran: > > $ git

Re: SSLPeerUnverifiedException following upgrade from 1.6.0 to 1.13.2

2021-06-11 Thread James Srinivasan
This is my usual go-to reference for getting SANs working with openssl CSRs: https://geekflare.com/san-ssl-certificate/ Newer openssl versions apparently allow it on the command line:

Re: [discuss] we need to enable secure by default...

2021-02-10 Thread James Srinivasan
Would a suitably large warning on the http ui be a good starting point? Browsers are getting increasingly wary of self signed certs and we probably don't want to be encouraging people to ignore them. What about easier acme+certbot support? (notwithstanding all the non public deployments) On

Re: Dependency and skipping tests

2020-09-19 Thread James Srinivasan
tation explicitly says compilation of tests is skipped. > > Hopefully, fresh eyes after a weekend will reveal the issue. Any additional > suggestions are still welcome! > > Thanks. > > On Fri, Sep 18, 2020 at 4:01 PM James Srinivasan < > james.sriniva...@gmail.com> >

Re: Dependency and skipping tests

2020-09-18 Thread James Srinivasan
-DskipTests will compile but not run tests You probably want something like -Dmaven.test.skip See http://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html#skip On Fri, 18 Sep 2020 at 20:58, Mark Bean wrote: > > I had an interesting problem building Apache NiFi (1.11.4) using Java

Re: Nifi Over HTTPS

2020-05-29 Thread James Srinivasan
Have you found the walkthroughs? https://nifi.apache.org/docs/nifi-docs/html/walkthroughs.html#securing-nifi-with-tls On Fri, 29 May 2020, 19:41 MYERS, KYLE, wrote: > Hello, > > I am trying to set up Nifi over SSL/TLS connections. I am following the > guide >

Re: Airflow and NiFi

2019-08-15 Thread James Srinivasan
We use NiFi for all our ETL, and have done so very happily for several years (including persuading partner organisations to adopt it); we have recently started using Airflow for scheduling periodic machine learning algorithm execution and retraining. My £0.02 - we probably could do everything we

Re: Lowering the barrier of entry

2019-01-28 Thread James Srinivasan
w in a monkey wrench, but does it really make sense to split > User and Developer? In all years I've never seen a user who wasn't a > developer. > > Maybe call it a User and Extending NiFi sections? > > On Mon, Jan 28, 2019, 9:18 AM James Srinivasan > wrote: > > > Rathe

Re: Lowering the barrier of entry

2019-01-28 Thread James Srinivasan
eneral and Developer, so were you thinking of > splitting General into User and Admin? > > On Mon, Jan 28, 2019 at 11:34 AM James Srinivasan < > james.sriniva...@gmail.com> wrote: > > > How about separating out User/Developer/Admin into separate docs? >

Re: Lowering the barrier of entry

2019-01-28 Thread James Srinivasan
the canvas > > > > > > I imagine this being written more conversationally/blog-like than most of > > > our current reference documentation to be used as a split-screen > > > walkthrough. Each section could certainly link to the existing detailed > > >

Re: Lowering the barrier of entry

2019-01-26 Thread James Srinivasan
y link to the existing detailed documentation for > various topics, like the processor lifecycle, etc. > > Does this sounds like something that would have helped you? > > Andy LoPresto > alopre...@apache.org > alopresto.apa...@gmail.com > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3

Re: Lowering the barrier of entry

2019-01-25 Thread James Srinivasan
9) Oh, and the wiki is a little hard to navigate and the contents rather patchy On Fri, 25 Jan 2019 at 21:57, James Srinivasan wrote: > > As someone relatively new to NiFi dev, here's my £0.02. (Yes, I > realise I could and possibly should submit PRs :) > > 1) I'm used to Java and

Re: Lowering the barrier of entry

2019-01-25 Thread James Srinivasan
As someone relatively new to NiFi dev, here's my £0.02. (Yes, I realise I could and possibly should submit PRs :) 1) I'm used to Java and Maven, so used the archetype. It worked fine, it would have been nice it if set up unit tests for me. 2) The User and Developer documentation is great and

Re: Processor forwards/backwards compatibility

2018-11-16 Thread James Srinivasan
of the > nifi-api that you use > in 1.y.0 and we can guarantee that will not be removed in 1.z.0 but it did > not exist in 1.x.0. > > Thanks > -Mark > > > > > On Nov 16, 2018, at 2:35 PM, James Srinivasan > > wrote: > > > > I've read > &

Processor forwards/backwards compatibility

2018-11-16 Thread James Srinivasan
I've read https://cwiki.apache.org/confluence/display/NIFI/Version+Scheme+and+API+Compatibility but I'm still not quite clear: I develop my own custom Processor against NiFi v. 1.y.0 Should that processor work with older NiFi releases 1.x.0 (x

Re: [EXT] Re: New Standard Pattern - Put Exception that caused failure in an attribute

2018-10-30 Thread James Srinivasan
Apologies if I've missed this in the discussion so far - we use the InvokeHTTP processor a lot, and the invokehttp.java.exception.message attribute is really handy diving into why things have failed without having to match up logs with flow files (from a system with hundreds of processors making

Code coverage for NiFi processor

2018-09-21 Thread James Srinivasan
Has anyone got JaCoCo (or similar) working with a custom processor created from the archetype? Many thanks, James

Licenses for test dependencies

2018-08-30 Thread James Srinivasan
I am not a lawyer (and you probably aren't too), but what should I do to document the licenses of dependent libraries of the processor I want to contribute that are only used with a Maven 'test' scope i.e. not bundled in the nar? Also, same question for plugins that run in the test scope. Many

Re: Error handling in @OnScheduled

2018-08-26 Thread James Srinivasan
> > Thanks! > -Mark > > > > > On Aug 24, 2018, at 11:34 AM, James Srinivasan > > wrote: > > > > Mark, > > > > Thanks very much for the detailed answer. In my particular case, I > > have a parameter corresponding to a schema file on disk and

Re: Error handling in @OnScheduled

2018-08-24 Thread James Srinivasan
If you wait until the user tries to start the Processor and throw an > Exception, it will be less obvious that there's a configuration problem > and the error message that they receive is likely not to be as clear. > > Thanks > -Mark > > > On Aug 23, 2018, at 5:25 PM,

Re: Error handling in @OnScheduled

2018-08-23 Thread James Srinivasan
hod directly from your unit test and catch the Exception > > there. The TestRunner > > expects to run the full lifecycle of the Processor. > > > > Thanks > > -Mark > > > > > > > On Aug 23, 2018, at 10:49 AM, James Srinivasan < > > j

Re: Error handling in @OnScheduled

2018-08-23 Thread James Srinivasan
True(error.cause instanceof SomeException) > assertTrue(error.cause.message.contains(msg)) > } > > Obviously play around with the finally block, but I've had success with > that pattern. > > On Thu, Aug 23, 2018 at 10:19 AM James Srinivasan < > james.sriniva...@gmail.c

Error handling in @OnScheduled

2018-08-23 Thread James Srinivasan
What is the best way to handle exceptions which might be thrown in my @OnScheduled method? Right now, I'm logging and propagating the exception which has the desired behaviour in NiFi (bulletin in GUI and processor cannot be started) but when trying to add a unit test, the (expected) exception is