Jim, You're welcome! Thanks for following up and confirming the solution, great collaborative effort!
Regard, David Handermann On Tue, Nov 8, 2022, 7:25 PM James McMahon <[email protected]> wrote: > That was it. Adding the port to the docker run command proxy got me to the > promised land. I was then able to use the userid and password from the > docker log to access nifi on my ec2 instance. > > David, Dmitry - thank you so much. This was a huge help to me, and I hope > it will help others trying the same approach in the future. > Jim > > On Tue, Nov 8, 2022 at 8:13 PM David Handermann < > [email protected]> wrote: > >> It may also be necessary to include the port in the host variable: >> >> docker run --name nifi -p 8443:8443 -e NIFI_WEB_PROXY_HOST= >> ec2-3-238-27-220.compute-1.amazonaws.com:8443 -d apache/nifi:latest >> >> It is possible to access the configuration and logs files using an >> interactive shell with the following Docker command: >> >> docker exec -it nifi /bin/bash >> >> Regards, >> David Handermann >> >> On Tue, Nov 8, 2022 at 7:09 PM Dmitry Stepanov <[email protected]> >> wrote: >> >>> Make sure you use your full domain name >>> ec2-3-238-27-220.compute-1.amazonaws.com >>> David shorten it in his code >>> >>> On November 8, 2022 5:57:26 p.m. James McMahon <[email protected]> >>> wrote: >>> >>>> Thank you, David. I’ve made that change, adding the proxy host >>>> specification on the docker command line. I continue to get the same error >>>> message. Is it possible I need to indicate my key on the docker command >>>> line too? >>>> >>>> Related, how can one access nifi.properties and the usual nifi config >>>> files, as well as the family of nifi-app.log files and bootstrap.conf, when >>>> nifi is running inside a docker container? >>>> >>>> Thanks again for sticking with this. I feel like we’re getting closer. >>>> Jim >>>> >>>> On Tue, Nov 8, 2022 at 7:31 PM David Handermann < >>>> [email protected]> wrote: >>>> >>>>> Hi Jim, >>>>> >>>>> Good adjustment on the security group inbound rules. >>>>> >>>>> The error page is the result of NiFi receiving an unexpected HTTP Host >>>>> header, not matching one of the expected values. >>>>> >>>>> For this to work, it is possible to pass the external DNS name as the >>>>> value of the NIFI_WEB_PROXY_HOST environment variable. This can be >>>>> specified in the docker run command as follows: >>>>> >>>>> docker run --name nifi -p 8443:8443 -e NIFI_WEB_PROXY_HOST=ec2... >>>>> amazonaws.com -d apache/nifi:latest >>>>> >>>>> That will allow NiFi to accept the Host header from the browser, and >>>>> then present the login screen. >>>>> >>>>> Regards, >>>>> David Handermann >>>>> >>>>> On Tue, Nov 8, 2022 at 6:06 PM James McMahon <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi David. This is very helpful, thank you. I feel like I am close, >>>>>> but I get an error. My Inbound Rules for my security group now include: >>>>>> 8443 TCP (MyIP)/32 >>>>>> 443 TCP (MyIP)/32 >>>>>> 22 TCP (MyIP)/32 >>>>>> >>>>>> In my browser - I tried both Edge and Chrome - I use this >>>>>> URL: >>>>>> https://ec2-3-238-27-230.compute-1.amazonaws.com:8443 >>>>>> I have also tried with /nifi at the tail end. >>>>>> >>>>>> I get this error: >>>>>> >>>>>> *System Error* >>>>>> >>>>>> *The request contained an invalid host header >>>>>> [ec2-3-238-27-220.compute-1.amazonaws.com:8443 >>>>>> <http://ec2-3-238-27-220.compute-1.amazonaws.com:8443/>] in the request >>>>>> [/]. Check for request manipulation or third-party intercept.* >>>>>> >>>>>> *Valid host headers are [empty] or:* >>>>>> >>>>>> - *127.0.0.1* >>>>>> - *127.0.0.1:8443 <http://127.0.0.1:8443/>* >>>>>> - *localhost* >>>>>> - *localhost:8443* >>>>>> - *[::1]* >>>>>> - *[::1]:8443* >>>>>> - *7f661ae687d7* >>>>>> - *7f661ae687d7:8443* >>>>>> - *172.17.0.2* >>>>>> - *172.17.0.2:8443 <http://172.17.0.2:8443/>* >>>>>> >>>>>> >>>>>> Does this mean I have formed the URL incorrectly? >>>>>> >>>>>> I also see that I had to add an exception to permit https. When I >>>>>> created the instance, I created my own pem key pair. It is not signed by >>>>>> any CA. For a self-signed key pair like this, do I need to install a key >>>>>> in >>>>>> my browser security store to avoid adding that exception? >>>>>> >>>>>> Thank you for helping me get that much closer. >>>>>> Jim >>>>>> >>>>>> On Tue, Nov 8, 2022 at 5:13 PM David Handermann < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hi Jim, >>>>>>> >>>>>>> Thanks for the reply and additional background. >>>>>>> >>>>>>> The instructions are dated March 2021, which is prior to the release >>>>>>> of NiFi 1.14.0. In particular, the run command is no longer accurate >>>>>>> with >>>>>>> the default NiFi container image. >>>>>>> >>>>>>> The current Docker Hub instructions [1] show the basic command needed >>>>>>> >>>>>>> docker run --name nifi -p 8443:8443 -d apache/nifi:latest >>>>>>> >>>>>>> In addition, any references to port 8080 in the AWS Security Group >>>>>>> rules should be changed to 8443. The security group rules for port 80 >>>>>>> and >>>>>>> 18080 should be removed. >>>>>>> >>>>>>> The instructions that allow plain HTTP access to NiFi on port 8080 >>>>>>> should NEVER be followed, as this exposes unfiltered and unauthenticated >>>>>>> access. >>>>>>> >>>>>>> Following those changes, it should be possible to access the NiFi UI >>>>>>> using the AWS URL: >>>>>>> >>>>>>> https://ec2...amazonaws.com:8443 >>>>>>> >>>>>>> The default installation will generate a username and password, >>>>>>> which can be found in the container logs: >>>>>>> >>>>>>> docker logs nifi | grep Generated >>>>>>> >>>>>>> Regards, >>>>>>> David Handermann >>>>>>> >>>>>>> [1] https://hub.docker.com/r/apache/nifi >>>>>>> >>>>>>> On Tue, Nov 8, 2022 at 4:00 PM James McMahon <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi and thank you, David and Dmitry. In my case I was following this >>>>>>>> example, >>>>>>>> >>>>>>>> https://joeygoksu.com/software/apache-nifi-on-aws/ >>>>>>>> >>>>>>>> which results in NiFi installed within a container. So to answer >>>>>>>> one of your questions, I don’t yet know how or where to find >>>>>>>> nifi.properties in the container framework. I don’t seem to have the >>>>>>>> usual >>>>>>>> /opt/nifi/….. directories on my ec2 instance. Any idea where I need to >>>>>>>> look >>>>>>>> for that? >>>>>>>> >>>>>>>> These ports are open by my security group Inbound Rules: 22 to >>>>>>>> MyIP, 80, 8080, and 18080 (per the link) to 0.0.0.0/0, 443 to MyIP. >>>>>>>> >>>>>>>> I am able to Putty into my instance as ec2-user with my ppk file, >>>>>>>> which I created using putty tools from the original pem key pair. When >>>>>>>> I do >>>>>>>> putty in, under /opt I find three subdirectories: aws, containerd, and >>>>>>>> rh. >>>>>>>> Nothing nifi under any of the three that I can see so far. >>>>>>>> >>>>>>>> I start my docker instance with this command: >>>>>>>> docker run —name nifi -p 18080:8080 -d apache/nifi:latest >>>>>>>> >>>>>>>> I can do a ps -ef and see running nifi processes. But I don’t yet >>>>>>>> know how to get to the nifi logs or properties file. >>>>>>>> >>>>>>>> You mentioned using using localhost to get to the canvas UI. This >>>>>>>> confuses me. Nifi is running on my EC2 instance - a linux host without >>>>>>>> a >>>>>>>> browser. I’m in a browser on my laptop. How would localhost in my >>>>>>>> browser >>>>>>>> get me to my EC2 instance running nifi? >>>>>>>> >>>>>>>> This is the URL I’m using in my browser: >>>>>>>> http://ec2-3-238-27-220.compute-1.amazonaws.com >>>>>>>> (that url changes with each Stop/Start of my instance. I’ve yet to >>>>>>>> investigate how to get AWS to stop changing that IP, but I know it can >>>>>>>> be >>>>>>>> done). >>>>>>>> >>>>>>>> The browser replies with: ec2…….amazonaws refused to connect. >>>>>>>> >>>>>>>> I can ping my laptop IP address from the putty terminal where I am >>>>>>>> logged in to my instance. I cannot ping the Public DNS of my instance >>>>>>>> from >>>>>>>> Powershell on my laptop. Again, that Public DNS is >>>>>>>> ec2-3-238-27-220.compute-1.amazonaws.com >>>>>>>> >>>>>>>> Any help is much appreciated. >>>>>>>> Jim >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Nov 8, 2022 at 3:03 PM David Handermann < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hi Jim, >>>>>>>>> >>>>>>>>> NiFi 1.14.0 and following default to HTTPS on port 8443, listening >>>>>>>>> on the localhost address. The nifi.web.https.host can be changed to >>>>>>>>> blank >>>>>>>>> in order to listen on all interfaces, but the default HTTPS setting >>>>>>>>> with >>>>>>>>> authenticated required should be retained. >>>>>>>>> >>>>>>>>> Can you provide the version of NiFi and some additional details on >>>>>>>>> the nifi.web values from nifi.properties? >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> David Handermann >>>>>>>>> >>>>>>>>> On Tue, Nov 8, 2022 at 1:54 PM James McMahon <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Has anyone successfully configured NiFi on AWS, and accessed it >>>>>>>>>> from a browser on a Windows desktop? I’ve tried following a few >>>>>>>>>> links to do >>>>>>>>>> this. I’ve verified that my instance security group allows access to >>>>>>>>>> 8080 >>>>>>>>>> via its inbound rules. I’ve putty’ed into the instance via ssh port >>>>>>>>>> 22 to >>>>>>>>>> verify that there are no firewall restrictions. But still I get a >>>>>>>>>> message >>>>>>>>>> to the effect that the server rejected the connection request. Can >>>>>>>>>> anyone >>>>>>>>>> recommend a link that describes a success path for this? >>>>>>>>>> Thanks in advance for your help. >>>>>>>>>> Jim >>>>>>>>>> >>>>>>>>> >>>
