This is probably a very important and overdue change. I think it’s important to recognize that there’s a lot of different ways to unintentionally end up with a publicly accessible application and it can’t always be pinned to one person or role. Routing, firewall rules, OS admin, NiFi admin aren’t always the same people.
I like the idea of the localhost change. I’d also be in favor of a making build profiles for opting in to some of the more dangerous restricted processors, or delivering the binaries for those separately. This exploit allows people to run a reverse shell and it probably doesn’t stop on the NiFi host, so good Internet citizenship sort of says something must be done. -joey > On Feb 10, 2021, at 10:14 AM, Joe Witt <[email protected]> wrote: > > Nathan just commented on the Jira about an obvious first step being to > restrict HTTP to localhost only by default. This is an easy and > immediately doable first step. I am planning to wait for the RC4 creation > for that PR to land and get merged. > > Thanks > >> On Wed, Feb 10, 2021 at 10:24 AM Nathan Gough <[email protected]> wrote: >> >> I 100% agree that something needs to be done. We cannot allow NiFi to build >> a reputation that it is 'insecure' by allowing its default installation to >> start up without any security. Especially considering how much work we put >> in to make sure it IS a secure product that integrates with many >> applications in a secure way. Security reputation is very important for >> software. If some major exploitation of NiFi were to happen in the future, >> we should be able to confidently say that we did our absolute best to >> create a secure application. We shouldn't point at new users and say 'they >> didn't configure it right'. >> >> Personally, I am in favor of providing automatically generated certificates >> and requiring the user to insert the client certificate in their browser, >> and providing instructions and perhaps a YouTube video on how to do that. >> Yes, X509 certificate errors are confusing and can be difficult for >> beginners to troubleshoot. But those beginner users will also be the most >> likely to use NiFi insecurely, connect it to the internet, and become part >> of a user base who got burnt by NiFi being 'insecure'. I acknowledge this >> is increasing the barrier for entry. If we intend to use a >> username/password + server cert for HTTPs but no client cert, as stated >> above we could automatically generate the password and provide this to the >> user in a log or file. >> >> >> On Wed, Feb 10, 2021 at 12:21 PM David Handermann < >> [email protected]> wrote: >> >>> Integrating an option to use Let's Encrypt would be a great improvement >>> over self-signed certificates, but it is difficult to support that for >>> instances of NiFi running on servers without Internet access. Even when >>> running NiFi for local development purposes, the development system is >> not >>> likely to have a publicly addressable DNS name, so Let's Encrypt is not >> an >>> option in those cases. >>> >>> Regards, >>> David Handermann >>> >>>> On Wed, Feb 10, 2021 at 11:09 AM Joe Witt <[email protected]> wrote: >>> >>>> Otto >>>> >>>> Installers like you mention are inherently brutal for portability so >> very >>>> difficult for us in the community. From the vendor world we of course >> do >>>> things like that. I think in apache nifi land we need a default 'even >>> for >>>> devs' which is not wide open. >>>> >>>> James >>>> >>>> I do think adding such a warning is good. But it doesn't help avoid >>> these >>>> wildly abusive cases. We need a secure by default path. We can >> probably >>>> do better than the self signed path if we add a 'before running' step >> in >>>> the CLI for the user. Not sure. >>>> >>>> Thanks >>>> >>>> On Wed, Feb 10, 2021 at 10:05 AM James Srinivasan < >>>> [email protected]> wrote: >>>> >>>>> Would a suitably large warning on the http ui be a good starting >> point? >>>>> >>>>> Browsers are getting increasingly wary of self signed certs and we >>>> probably >>>>> don't want to be encouraging people to ignore them. >>>>> >>>>> What about easier acme+certbot support? (notwithstanding all the non >>>> public >>>>> deployments) >>>>> >>>>> On Wed, 10 Feb 2021, 15:25 Otto Fowler, <[email protected]> >>> wrote: >>>>> >>>>>> Aren’t most of these applications installed by an installer? >>>>>> Maybe we can have one or more installers that setup a secure >>> instance, >>>>> and >>>>>> those installers >>>>>> could be the *production* nifi, and keep the zip distribution open >>> for >>>>>> developers? >>>>>> >>>>>> >>>>>>> On Feb 10, 2021, at 10:04, David Handermann < >>>>> [email protected]> >>>>>> wrote: >>>>>>> >>>>>>> I agree that a generated password is the way to go, and the >>> challenge >>>>> is >>>>>>> making it available to the user. Depending on how it is stored >> for >>>> the >>>>>>> identity provider, having a command line tool to read and print >> it >>>>> could >>>>>> be >>>>>>> a reasonable option. >>>>>>> >>>>>>> Although this particular thread referenced a specific Twitter >> post, >>>>> this >>>>>>> general discussion is more of a reflection of the need to make >>> things >>>>>> more >>>>>>> secure by default as other products have followed similar >>> approaches. >>>>>>> >>>>>>> Regards, >>>>>>> David Handermann >>>>>>> >>>>>>> On Wed, Feb 10, 2021 at 8:53 AM Kevin Doran <[email protected]> >>>> wrote: >>>>>>> >>>>>>>> I am in favor of requiring some authentication by default. >>>>>>>> >>>>>>>> I’m in favor of an admin username and generated password. (It >>> sounds >>>>>> li9ke >>>>>>>> most of us are on the same page, but I don’t think a static, >>> default >>>>>>>> password buys us much against the types of abuse shown on the >>>> twitter >>>>>>>> thread Joe shared.) >>>>>>>> >>>>>>>> We need some way of making the generated password discoverable… >>>> Print >>>>> to >>>>>>>> the logs on first boot? Not great but are there other >> mechanisms? >>> A >>>>>> setup >>>>>>>> CLI utility? >>>>>>>> >>>>>>>> To help not impede automated deployments, maybe we should change >>> the >>>>>>>> startup flow such that there is a way to provide this password. >>> That >>>>>> would >>>>>>>> be better than people just disabling whatever default >>> authentication >>>>> we >>>>>> set. >>>>>>>> >>>>>>>> >>>>>>>>> On Feb 10, 2021, at 09:45, David Handermann < >>>>>> [email protected]> >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> Mark, >>>>>>>>> >>>>>>>>> Thanks for clarifying, that makes sense. >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> David Handermann >>>>>>>>> >>>>>>>>> On Wed, Feb 10, 2021 at 8:41 AM Mark Payne < >> [email protected] >>>> >>>>>> wrote: >>>>>>>>> >>>>>>>>>> David, >>>>>>>>>> >>>>>>>>>> My concern was purely around generating client certs and using >>>>> mutual >>>>>>>> TLS. >>>>>>>>>> I definitely think we should have a server cert if using >>> username >>>> & >>>>>>>>>> password. >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> -Mark >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> On Feb 10, 2021, at 9:37 AM, David Handermann < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>> Mark, >>>>>>>>>>> >>>>>>>>>>> Thanks for your perspective on certificate generation, I >> agree >>>> that >>>>>>>>>>> troubleshooting certificates can be confusing due to unclear >>>> error >>>>>>>>>>> messaging. Generating self-signed certificates for one-way >> TLS >>>>> still >>>>>>>>>>> requires the user to accept the certificate presented, but at >>>> least >>>>>>>> that >>>>>>>>>> is >>>>>>>>>>> more common in various products. Are you concerned about >> that >>>>>>>> situation, >>>>>>>>>>> or are you more concerned about the additional challenges of >>>> mutual >>>>>> TLS >>>>>>>>>>> authentication? >>>>>>>>>>> >>>>>>>>>>> Generating a random password in absence of any other >>>> configuration >>>>>>>> would >>>>>>>>>>> certainly be easier from a new user perspective. Some of the >>>>>> security >>>>>>>>>>> concerns with password authentication can be mitigated with >>>> one-way >>>>>>>> TLS, >>>>>>>>>> so >>>>>>>>>>> a blending of these approaches, as Joe describes in >> NIFI-8220, >>>>> seems >>>>>>>>>> like a >>>>>>>>>>> good way to go. >>>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> David Handermann >>>>>>>>>>> >>>>>>>>>>> On Wed, Feb 10, 2021 at 8:23 AM Mark Payne < >>> [email protected] >>>>> >>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> I would be in favor of this as well. I agree that there is >> no >>>> need >>>>>> to >>>>>>>>>> wait >>>>>>>>>>>> for a 2.0 version - there is no inherit backward >> compatibility >>>>>>>> challenge >>>>>>>>>>>> here. >>>>>>>>>>>> Requiring that new application configs be set, etc. I think >> is >>>>>>>>>> completely >>>>>>>>>>>> fair game between minor versions. >>>>>>>>>>>> >>>>>>>>>>>> Personally, though, I would very much stray away from >>>>>> auto-generating >>>>>>>>>>>> certificates. For those of us who have dealt with >> certificates >>>>>>>>>> significantly >>>>>>>>>>>> In the past, minor configuration issues are easy to address. >>> But >>>>> for >>>>>>>>>>>> someone who hasn’t spent much time dealing with >> certificates, >>>> the >>>>>>>> error >>>>>>>>>>>> messages >>>>>>>>>>>> that are often intentionally vague can quickly result in >> users >>>>> being >>>>>>>>>>>> overwhelmed. This is especially true if browser >> configurations >>>> are >>>>>>>>>> already >>>>>>>>>>>> setup to >>>>>>>>>>>> automatically offer certificates that area already >> installed - >>>>> this >>>>>>>> can >>>>>>>>>>>> result in weird error messages about untrusted certificates >>> when >>>>> the >>>>>>>>>> user >>>>>>>>>>>> thinks >>>>>>>>>>>> they haven’t even provided a certificate, etc. It gets >> really >>>>> hairy. >>>>>>>>>>>> >>>>>>>>>>>> I am more in favor of a default username/password >> personally. >>> It >>>>>> would >>>>>>>>>>>> require implementing a new auth provider. And it’s one that >>>>>>>>>> historically we >>>>>>>>>>>> have >>>>>>>>>>>> avoided implementing because a basic auth type of approach >> is >>>> less >>>>>>>>>> secure >>>>>>>>>>>> than mutual auth, ldap, etc. But it’s more secure than >>> nothing. >>>>>>>>>>>> >>>>>>>>>>>> Thanks >>>>>>>>>>>> -Mark >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> On Feb 10, 2021, at 9:13 AM, Joe Witt <[email protected]> >>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> There are a range of things to consider. And yes whatever >> we >>>> do >>>>>> will >>>>>>>>>>>>> involve writing code. We also have to find the right place >>> for >>>>> the >>>>>>>> bar >>>>>>>>>>>>> here. >>>>>>>>>>>>> >>>>>>>>>>>>> 1. Disable HTTP by default. And if they want to enable >> HTTP >>>> they >>>>>>>>>> should >>>>>>>>>>>>> also have to make a config change indicating they are fully >>>>> willing >>>>>>>> to >>>>>>>>>>>>> accept that it is an entirely non secure configuration as >> far >>>> as >>>>>> the >>>>>>>>>>>>> application is concerned. >>>>>>>>>>>>> 2. Provide a default username/password provider out of the >>> box >>>>>>>>>> configured >>>>>>>>>>>>> by default and an auto generate self-signed server cert. >>> This >>>>>> means >>>>>>>>>> the >>>>>>>>>>>>> NiFi server will provide the client browser a cert. It >> won't >>>> be >>>>>>>>>>>>> known/trusted so the browser will advise of this. And on >>>> startup >>>>>>>> NiFi >>>>>>>>>>>> can >>>>>>>>>>>>> auto generate and log this username and password. It would >>>> truly >>>>>> be >>>>>>>> a >>>>>>>>>>>>> single username/password and not some store for various >>> users. >>>>> We >>>>>>>>>>>>> disallow access to all restricted components by default. >>>>>>>>>>>>> >>>>>>>>>>>>> This default configuration at least means someone cannot >>> start >>>>> NiFi >>>>>>>> and >>>>>>>>>>>> it >>>>>>>>>>>>> is totally exposed by default while also preserving a >> pretty >>>>> simple >>>>>>>>>> 'get >>>>>>>>>>>>> started' model for the user. They'd have to take action to >>>> make >>>>> it >>>>>>>>>> less >>>>>>>>>>>>> secure. >>>>>>>>>>>>> >>>>>>>>>>>>> The option DavidH mentions of mutual auth could work well >>> also >>>>> and >>>>>> as >>>>>>>>>> he >>>>>>>>>>>>> mentioned avoids the need for anything special in terms of >>> auth >>>>>>>>>> provider >>>>>>>>>>>>> which is compelling for us but I do worry about the user >>>>> experience >>>>>>>> on >>>>>>>>>>>> that. >>>>>>>>>>>>> >>>>>>>>>>>>> I'll add to this that I think we cannot afford to wait for >> a >>>> NiFi >>>>>> 2.0 >>>>>>>>>>>>> release to take action here. Or that we should expedite a >>> NiFi >>>>> 2.0 >>>>>>>>>>>> release >>>>>>>>>>>>> to take action and that we should just not make the bar for >>>> what >>>>>> 2.0 >>>>>>>> is >>>>>>>>>>>> so >>>>>>>>>>>>> high that we never get this done. But frankly I think we >>> could >>>>>> make >>>>>>>>>> this >>>>>>>>>>>>> change in NiFi 1.15 and document what is happening. For >>>> existing >>>>>>>>>>>>> installs/configs we'd not be changing anything except maybe >>>> when >>>>>>>>>> they're >>>>>>>>>>>>> using HTTP and don't set the 'no seriously i want this >> thing >>>> wide >>>>>>>> open >>>>>>>>>>>>> option'. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks >>>>>>>>>>>>> >>>>>>>>>>>>> On Wed, Feb 10, 2021 at 6:58 AM David Handermann < >>>>>>>>>>>> [email protected]> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Having NiFi enforce authentication by default seems like >> the >>>>> right >>>>>>>> way >>>>>>>>>>>> to >>>>>>>>>>>>>> go, especially given the capabilities of the system. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Bryan raises a good point about storage of account >>>> information, >>>>> so >>>>>>>>>>>> weighing >>>>>>>>>>>>>> the positives and negatives of various identity providers >>>> should >>>>>> be >>>>>>>>>>>>>> considered. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Following on Joe's point about disabling plain HTTP, one >>>> option >>>>>>>> could >>>>>>>>>> be >>>>>>>>>>>>>> generating both client and server certificates and using >>>> Mutual >>>>>> TLS >>>>>>>>>> for >>>>>>>>>>>>>> authentication. This would obviously require installing >> the >>>>>> client >>>>>>>>>>>>>> certificate in a browser, which is not trivial, but could >> be >>>>> part >>>>>> of >>>>>>>>>> an >>>>>>>>>>>>>> installation guide. This approach definitely provides a >>> high >>>>>>>> barrier >>>>>>>>>> of >>>>>>>>>>>>>> entry of new users, but provides a strong level of >> security >>>>> while >>>>>>>>>>>> avoiding >>>>>>>>>>>>>> the need for some other identity provider service to be >>>>>> configured. >>>>>>>>>> As >>>>>>>>>>>>>> others have mentioned, this seems necessary to address the >>>>>> situation >>>>>>>>>> of >>>>>>>>>>>>>> someone installing NiFi without a full understanding of >> the >>>>>>>>>>>> configuration >>>>>>>>>>>>>> required, so it is important to keep the audience in mind >>> when >>>>>>>>>>>> evaluating a >>>>>>>>>>>>>> solution. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>> David Handermann >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Wed, Feb 10, 2021 at 7:39 AM Bryan Bende < >>> [email protected] >>>>> >>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Just to clarify, I was not suggesting that we make a >>> default >>>>>> secure >>>>>>>>>>>>>>> setup that requires LDAP. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I was just saying that in order to provide a default >> secure >>>>>> setup, >>>>>>>>>>>>>>> we'd have to provide a login identity provider >>> implementation >>>>>> that >>>>>>>> is >>>>>>>>>>>>>>> backed by a file or database table, which in the past we >>>>> decided >>>>>>>>>>>>>>> against. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Wed, Feb 10, 2021 at 8:28 AM Russell Bateman < >>>>>>>>>> [email protected] >>>>>>>>>>>>> >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I second the concerns expressed, but second especially >>>> Bryan's >>>>>>>>>>>> pointing >>>>>>>>>>>>>>>> out that requiring LDAP/AD to be set up in order even to >>>> begin >>>>>> to >>>>>>>>>> use >>>>>>>>>>>>>>>> our framework would be a bit onerous for developers just >>>>>>>> interested >>>>>>>>>> in >>>>>>>>>>>>>>>> getting work done and a barrier to considering the >>> framework >>>>>>>> should >>>>>>>>>> it >>>>>>>>>>>>>>>> be erected a little too high. Should we at least glance >> at >>>> how >>>>>>>> this >>>>>>>>>> is >>>>>>>>>>>>>>>> solved by the likes of other projects, Kafka and >> Cassandra >>>>> come >>>>>> to >>>>>>>>>>>>>> mind, >>>>>>>>>>>>>>>> even if it means resorting to a store of a name or two? >> I >>>>> didn't >>>>>>>>>> find >>>>>>>>>>>>>>>> getting into developing with them a pain, but making me >>> jump >>>>>>>> through >>>>>>>>>>>>>> the >>>>>>>>>>>>>>>> hoop of setting up LDAP may very well have changed that. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> These unsecure instances of NiFi out there are not our >>>>>> community's >>>>>>>>>>>>>>>> fault. I suppose we're worried about getting splattered >> by >>>> bad >>>>>>>>>> press? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 2/10/21 5:47 AM, Bryan Bende wrote: >>>>>>>>>>>>>>>>> I agree with the overall idea, although I would think >> it >>>>>>>> requires a >>>>>>>>>>>>>>>>> major release to make this kind of change to the >> default >>>>>>>> behavior. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Also, we have always avoided NiFi being a store of >>>> usernames >>>>>> and >>>>>>>>>>>>>>>>> passwords, so we don't have a login provider that uses >> a >>>>> local >>>>>>>> file >>>>>>>>>>>>>> or >>>>>>>>>>>>>>>>> a database, we've always said you connect to LDAP/AD >> for >>>>> that. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Obviously it can be implemented, but just pointing out >>> that >>>>>> we'd >>>>>>>>>> have >>>>>>>>>>>>>>>>> to change our stance here if we want to provide a >> default >>>>>>>> username >>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>> password to authenticate with. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On Tue, Feb 9, 2021 at 11:25 PM Andrew Grande < >>>>>>>> [email protected]> >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>> Mysql has been generating an admin password on default >>>>>> installs >>>>>>>>>> for, >>>>>>>>>>>>>>> like, >>>>>>>>>>>>>>>>>> forever. This workflow should be familiar for many >>> users. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I'd suggest taking the automation tooling into account >>> and >>>>>> how a >>>>>>>>>>>>>>> production >>>>>>>>>>>>>>>>>> rollout (user-provided password) would fit into the >>>>> workflow. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Andrew >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Tue, Feb 9, 2021, 8:15 PM Tony Kurc < >>> [email protected]> >>>>>>>> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Joe, >>>>>>>>>>>>>>>>>>> In addition to your suggestions, were you thinking of >>>>> making >>>>>>>> this >>>>>>>>>>>>>>> processor >>>>>>>>>>>>>>>>>>> disabled by default as well? >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Tony >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On Tue, Feb 9, 2021, 11:04 PM Joe Witt < >>>> [email protected] >>>>>> >>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Team >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> While secure by default may not be practical perhaps >>>> ‘not >>>>>>>>>>>>>> blatantly >>>>>>>>>>>>>>> wide >>>>>>>>>>>>>>>>>>>> open’ by default should be adopted. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I think we should consider killing support for http >>>>> entirely >>>>>>>> and >>>>>>>>>>>>>>> support >>>>>>>>>>>>>>>>>>>> only https. We should consider auto generating a >> user >>>> and >>>>>>>>>>>>>> password >>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>>> possibly server cert if nothing is configured and >> log >>>> the >>>>>>>>>>>>>> generated >>>>>>>>>>>>>>> user >>>>>>>>>>>>>>>>>>>> and password. Sure it could still be configured to >>> be >>>>> non >>>>>>>>>> secure >>>>>>>>>>>>>>> but >>>>>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>>>>>> would truly be an admins fault. Now its just ‘on’ >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> This tweet is a great example of why >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>> https://twitter.com/_escctrl_/status/1359280656174510081?s=21 >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Who agrees? Who disagrees? Please share ideas. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >>
