[VOTE] Release Apache NiFi 1.13.0 (rc4)

2021-02-10 Thread Joe Witt
Hello, I am pleased to be calling this vote for the source release of Apache NiFi 1.13.0. The source zip, including signatures, digests, etc. can be found at: https://repository.apache.org/content/repositories/orgapachenifi-1178 The source being voted upon and the convenience binaries can be

Re: Analytic Framework Metrics / Logging

2021-02-10 Thread Yolanda Davis
Hello Aliesha, Thank you for reaching out! What specific analytics settings did you change in nifi.properties? For example by default the *nifi.analytics.predict.enabled* property is set to false. Enabling that to be true by itself should turn on predictions (after a restart of nifi). Also if

Analytic Framework Metrics / Logging

2021-02-10 Thread Aliesha Alli
Hello NiFi Team, I have downloaded both NiFi Linux and Windows on separate systems and configured "analytics.predict" properties, created a FlowFile, and changed the processors to Debug. 100% of the threshold object is met, but I don't see any events for predictions or R-Square score in the

Re: [discuss] we need to enable secure by default...

2021-02-10 Thread M Tien
Hi, Agreed that NiFi should be secure by default, or at the very least not completely open for direct abuse. Although I understand the hesitation of requiring a secure startup can be overwhelming for new users or inconvenient for those who just want to “test” out NiFi - given the powerful

Re: [discuss] we need to enable secure by default...

2021-02-10 Thread Joe Witt
Joey We need to move to a model where NiFi comes with *nothing* by default and all items are sourced as needed when someone tries to put them in a flow or wants to create a deployable bundle. That gives them better control and us a much better community management experience. Smaller and more

Re: [discuss] we need to enable secure by default...

2021-02-10 Thread Joey Frazee
This is probably a very important and overdue change. I think it’s important to recognize that there’s a lot of different ways to unintentionally end up with a publicly accessible application and it can’t always be pinned to one person or role. Routing, firewall rules, OS admin, NiFi admin

Re: [discuss] we need to enable secure by default...

2021-02-10 Thread Joe Witt
Nathan just commented on the Jira about an obvious first step being to restrict HTTP to localhost only by default. This is an easy and immediately doable first step. I am planning to wait for the RC4 creation for that PR to land and get merged. Thanks On Wed, Feb 10, 2021 at 10:24 AM Nathan

Re: [discuss] we need to enable secure by default...

2021-02-10 Thread Nathan Gough
I 100% agree that something needs to be done. We cannot allow NiFi to build a reputation that it is 'insecure' by allowing its default installation to start up without any security. Especially considering how much work we put in to make sure it IS a secure product that integrates with many

Re: [discuss] we need to enable secure by default...

2021-02-10 Thread David Handermann
Integrating an option to use Let's Encrypt would be a great improvement over self-signed certificates, but it is difficult to support that for instances of NiFi running on servers without Internet access. Even when running NiFi for local development purposes, the development system is not likely

Re: [discuss] we need to enable secure by default...

2021-02-10 Thread Joe Witt
Otto Installers like you mention are inherently brutal for portability so very difficult for us in the community. From the vendor world we of course do things like that. I think in apache nifi land we need a default 'even for devs' which is not wide open. James I do think adding such a

Re: [discuss] we need to enable secure by default...

2021-02-10 Thread James Srinivasan
Would a suitably large warning on the http ui be a good starting point? Browsers are getting increasingly wary of self signed certs and we probably don't want to be encouraging people to ignore them. What about easier acme+certbot support? (notwithstanding all the non public deployments) On

Re: Stardog Data Flow Automation with NiFi

2021-02-10 Thread Pierre Villard
Hi Chandan, Note that your picture didn't come through your email. Please add it to an external location and give a link to it. Also, this processor is not a processor that is distributed via the Apache NiFi project, I'd recommend reaching out to the authors/creators of this processor. Thanks,

Stardog Data Flow Automation with NiFi

2021-02-10 Thread Kumar, Chandan W.
Hi Team, I have a query related to the Stardog processor in Nifi. I am using the stardog processor in Nifi to connect the stardog instance in my organization and my organization, all authentication is done using Kerberos. So here the problem comes, these stardog processors seem like doesn't

Re: [discuss] we need to enable secure by default...

2021-02-10 Thread Otto Fowler
Aren’t most of these applications installed by an installer? Maybe we can have one or more installers that setup a secure instance, and those installers could be the *production* nifi, and keep the zip distribution open for developers? > On Feb 10, 2021, at 10:04, David Handermann > wrote: >

Re: [discuss] we need to enable secure by default...

2021-02-10 Thread David Handermann
I agree that a generated password is the way to go, and the challenge is making it available to the user. Depending on how it is stored for the identity provider, having a command line tool to read and print it could be a reasonable option. Although this particular thread referenced a specific

Re: [discuss] we need to enable secure by default...

2021-02-10 Thread Joe Witt
I dont believe automated deployments would be impacted. They can use any of the existing security mechanisms. We would not be doing 'http basic' for anything. The username and password would not be static. These would be auto generated and made available to the user in some way that isn't by

Re: [discuss] we need to enable secure by default...

2021-02-10 Thread Kevin Doran
I am in favor of requiring some authentication by default. I’m in favor of an admin username and generated password. (It sounds li9ke most of us are on the same page, but I don’t think a static, default password buys us much against the types of abuse shown on the twitter thread Joe shared.)

Re: [discuss] we need to enable secure by default...

2021-02-10 Thread John McGinn
I'm on the opposite side of this discussion. I disagree that NiFi has to implement something immediately. Now, I admit that I am old school, and feel that if you aren't capable of doing what is necessary to secure your own things, it's your own fault. But, I also realize, everyone on the

Re: [discuss] we need to enable secure by default...

2021-02-10 Thread David Handermann
Mark, Thanks for clarifying, that makes sense. Regards, David Handermann On Wed, Feb 10, 2021 at 8:41 AM Mark Payne wrote: > David, > > My concern was purely around generating client certs and using mutual TLS. > I definitely think we should have a server cert if using username & > password.

Re: [discuss] we need to enable secure by default...

2021-02-10 Thread Mark Payne
David, My concern was purely around generating client certs and using mutual TLS. I definitely think we should have a server cert if using username & password. Thanks -Mark > On Feb 10, 2021, at 9:37 AM, David Handermann > wrote: > > Mark, > > Thanks for your perspective on certificate

Re: [discuss] we need to enable secure by default...

2021-02-10 Thread David Handermann
Mark, Thanks for your perspective on certificate generation, I agree that troubleshooting certificates can be confusing due to unclear error messaging. Generating self-signed certificates for one-way TLS still requires the user to accept the certificate presented, but at least that is more

Re: [discuss] we need to enable secure by default...

2021-02-10 Thread Bryan Bende
I think the way Joe broke it down makes sense in that there are technically two different parts... For https, I don't see how we could do that without generating a default self-signed server cert where you'd get a warning in your browser to proceed. I suppose this could be confusing for users,

Re: [discuss] we need to enable secure by default...

2021-02-10 Thread Joe Witt
Created https://issues.apache.org/jira/browse/NIFI-8220 I think we need to do this or some variation of it in 1.14 (said 1.15 before but I meant 1.14). Thanks On Wed, Feb 10, 2021 at 7:23 AM Mark Payne wrote: > I would be in favor of this as well. I agree that there is no need to wait > for a

Re: [discuss] we need to enable secure by default...

2021-02-10 Thread Mark Payne
I would be in favor of this as well. I agree that there is no need to wait for a 2.0 version - there is no inherit backward compatibility challenge here. Requiring that new application configs be set, etc. I think is completely fair game between minor versions. Personally, though, I would very

Re: [discuss] we need to enable secure by default...

2021-02-10 Thread Joe Witt
There are a range of things to consider. And yes whatever we do will involve writing code. We also have to find the right place for the bar here. 1. Disable HTTP by default. And if they want to enable HTTP they should also have to make a config change indicating they are fully willing to

Re: [discuss] we need to enable secure by default...

2021-02-10 Thread David Handermann
Having NiFi enforce authentication by default seems like the right way to go, especially given the capabilities of the system. Bryan raises a good point about storage of account information, so weighing the positives and negatives of various identity providers should be considered. Following on

Re: [discuss] we need to enable secure by default...

2021-02-10 Thread Bryan Bende
Just to clarify, I was not suggesting that we make a default secure setup that requires LDAP. I was just saying that in order to provide a default secure setup, we'd have to provide a login identity provider implementation that is backed by a file or database table, which in the past we decided

Re: [discuss] we need to enable secure by default...

2021-02-10 Thread Russell Bateman
I second the concerns expressed, but second especially Bryan's pointing out that requiring LDAP/AD to be set up in order even to begin to use our framework would be a bit onerous for developers just interested in getting work done and a barrier to considering the framework should it be erected

Re: [discuss] we need to enable secure by default...

2021-02-10 Thread Bryan Bende
I agree with the overall idea, although I would think it requires a major release to make this kind of change to the default behavior. Also, we have always avoided NiFi being a store of usernames and passwords, so we don't have a login provider that uses a local file or a database, we've always