Re: [Dorset] Permissions Quandary with nginx
On Saturday, 6 February 2021 16:05:19 GMT Ralph Corderoy wrote: > The web version has a chapter on deploying to Ubuntu Linux, which is > similar to Debian and thus Raspbian, and then touches on the Pi and its > Raspbian at the end. > https://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-xvii-deplo > yment-on-linux I'll look into that. > I would expect these would be readable by ‘other’ because Python is > system wide and the page you're following uses ‘sudo pip3 install flask > uwsgi’ which again looks like it is available to all users by being run > as root. Yes. However, when I was developing the Flask App and testing it with the Flask Development Server, I wasn't installing Python Modules using pip3 sudo. > You're configuring the server and thus your Python code to run as user > www-data and group www-data. Consider if a process with that user and > group can access your ‘html’ directory and read files within it. > If not, does ‘other’ need to be given r-x on the directories and r-- on > the files? You can test this a bit with > > sudo su -g www-data www-data -c 'wc foo.py' > > which will try and read foo.py as that user and group. > > It depends how locked down you're trying to make access to the Python > source. I suspect not very, so www-data being classed as ‘other’ for > searching and reading could be fine. > https://en.wikipedia.org/wiki/File-system_permissions#Permissions Thanks. I'll look into this too. -- Terry Coles -- Next meeting: Online, Jitsi, Tuesday, 2021-03-02 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Permissions Quandary with nginx
Hi Terry, > I used sudo chown -hR www-data:www-data: /home/pi/html/ So unless ‘other’ has permission to write to directories, etc., user pi will find it awkward to update that area, as you found. > > Understand the aim: your web-server Python code will be running as > > user and group www-data so the user, group, and permissions of what > > it needs to access have to be compatible. > > I think I understood that; see above. The question I had was how do > I get the needed python modules accessible to the running code? First determine if they're not accessible... > When you say scrapping it all; do you mean starting from scratch and > NOT following the instructions in the link I gave. The problem is > that I would still need some guidance from somewhere on how to deploy > my Flask App under nginx. No, sorry, I mean stick with that for installing nginx, etc., and pointing it at a /home/pi directory which holds your server's code. Just don't mess with the default permissions on the new directory it creates as it doesn't explain why and isn't clear on what it's doing. > Various text books and web tutorials talk about this, but they tend to > be specific to the things they are interested in (Miguel Grinberg's > book majors on deploying into the cloud and in particular AWS). The web version has a chapter on deploying to Ubuntu Linux, which is similar to Debian and thus Raspbian, and then touches on the Pi and its Raspbian at the end. https://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-xvii-deployment-on-linux > > Your Python source code needs to be readable to nginx and its > > children so it can be opened and read to be interpreted by Python > > and run. What other I/O to local disk does your program do? If you > > don't change anything from the defaults, what doesn't work? > > Apart from importing a number of flask and python modules I would expect these would be readable by ‘other’ because Python is system wide and the page you're following uses ‘sudo pip3 install flask uwsgi’ which again looks like it is available to all users by being run as root. > the main external I/O is to write to a logs directory which is outside > the html directory at /home/ pi/logs. I'll be looking into that. You're configuring the server and thus your Python code to run as user www-data and group www-data. Consider if a process with that user and group can access your ‘html’ directory and read files within it. If not, does ‘other’ need to be given r-x on the directories and r-- on the files? You can test this a bit with sudo su -g www-data www-data -c 'wc foo.py' which will try and read foo.py as that user and group. It depends how locked down you're trying to make access to the Python source. I suspect not very, so www-data being classed as ‘other’ for searching and reading could be fine. https://en.wikipedia.org/wiki/File-system_permissions#Permissions -- Cheers, Ralph. -- Next meeting: Online, Jitsi, Tuesday, 2021-03-02 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Permissions Quandary with nginx
Hi Terry, > I've always thought of remotes as being something like GitLab. No, it just means repositories which aren't this one which have branches you want to track. A remote can be on the same machine in a nearby directory. -- Cheers, Ralph. -- Next meeting: Online, Jitsi, Tuesday, 2021-03-02 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Permissions Quandary with nginx
On Saturday, 6 February 2021 14:38:56 GMT Ralph Corderoy wrote: > Why not git-push(1) on the PC to Pi. Or git-pull(1) on the Pi from the > PC. GitLab need not be the only remote repository. > https://git-scm.com/book/en/v2/Git-Basics-Working-with-Remotes I'll look into that. I've always thought of remotes as being something like GitLab. -- Terry Coles -- Next meeting: Online, Jitsi, Tuesday, 2021-03-02 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Permissions Quandary with nginx
Hi Hamish, Hamish wrote: > Maybe you could set up a VM for testing locally, and then deploy > production versions via gitlab? > > Or use a testing branch and the merge to master once all is done and > tested. In reply to Terry: > > The problem with that is that I would need to push intermediate > > versions from my desktop to GitLab or do or my development directly > > on the Pi. Why not git-push(1) on the PC to Pi. Or git-pull(1) on the Pi from the PC. GitLab need not be the only remote repository. https://git-scm.com/book/en/v2/Git-Basics-Working-with-Remotes -- Cheers, Ralph. -- Next meeting: Online, Jitsi, Tuesday, 2021-03-02 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Permissions Quandary with nginx
On Saturday, 6 February 2021 13:45:35 GMT Ralph Corderoy wrote: > That's confused. It claims to be setting ‘its group owner’ but uses > chown(1) which changes the owner. chgrp(1) would change the group. > There is no such thing as a ‘group owner’ so what was intended and why? If you don't know, I certainly don't :-) > How? Knowing the command would help spot problems. I used sudo chown -hR www-data:www-data: /home/pi/html/ This moved the directory and all its contents into the www-data group and owned by www-data. > That suggests the permissions like read, write and execute, don't marry > with the user and group of the process which is making the accesses, > i.e. nginx and its offspring. The group, for example, can't just be > changed without considering those as they work in concert. Well I assumed that the reason for the group and ownership change was related to allowing nginx proper access. I also assumed that the problems came about because the python modules that the Flask App was trying to load were inaccessible to code running under the ownership of the www-data group. > Understand the aim: your web-server Python code will be running as user > and group www-data so the user, group, and permissions of what it needs > to access have to be compatible. I think I understood that; see above. The question I had was how do I get the needed python modules accessible to the running code? > You seem to be piling wrong things up instead of scrapping it all and > working out what's required, if anything, different from user pi's > default permissions based on its umask. When you say scrapping it all; do you mean starting from scratch and NOT following the instructions in the link I gave. The problem is that I would still need some guidance from somewhere on how to deploy my Flask App under nginx. Various text books and web tutorials talk about this, but they tend to be specific to the things they are interested in (Miguel Grinberg's book majors on deploying into the cloud and in particular AWS). The Raspberry Pi tutorial looked ideal, but I now realise that it is extremely limited because his testsite App doesn't actually do anything. In particular it doesn't import any python modules apart from flask itself. > Your Python source code needs to be readable to nginx and its children > so it can be opened and read to be interpreted by Python and run. What > other I/O to local disk does your program do? If you don't change > anything from the defaults, what doesn't work? Apart from importing a number of flask and python modules the main external I/O is to write to a logs directory which is outside the html directory at /home/ pi/logs. I'll be looking into that. -- Terry Coles -- Next meeting: Online, Jitsi, Tuesday, 2021-03-02 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Permissions Quandary with nginx
Hmm, Maybe you could set up a VM for testing locally, and then deploy production versions via gitlab? Or use a testing branch and the merge to master once all is done and tested. Hamish On 6 Feb 2021, at 13:19, Terry Coles mailto:d-...@hadrian-way.co.uk>> wrote: On Saturday, 6 February 2021 13:02:14 GMT Hamish MB wrote: Best to set it up so you can just use git pull I imagine, if possible The problem with that is that I would need to push intermediate versions from my desktop to GitLab or do or my development directly on the Pi. I prefer to do the initial work on the code using Thonny on my desktop because I find that it gives me better visibility of the errors that can be detected before deployment. (That's not to say that I can't get the same information from the shell, it's just that I find it easier to interpret in Thonny.) Once I have a version that passes those tests, I scp it to the Pi and only push the code when I reach a point where the code is running without error (maybe with missing functionality but no bugs). BTW. I have realised that if I scp the file(s) to /home/pi/ and then sudo cp them to /home/pi/html, they take on the www-data ownership and I don't need to do the extra chown. I can live with that. -- Next meeting: Online, Jitsi, Tuesday, 2021-03-02 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Permissions Quandary with nginx
Hi Terry, > > Can anyone suggest a way forward with this issue. My Flask code has > > been working well while I've been testing it using the Flask > > Development Server, so I decided to deploy it on nginx. To do this > > I followed the Tutorial at: > > > > https://www.raspberrypi-spy.co.uk/2018/12/running-flask-under-nginx-raspberry-pi/ That's not a tutorial. It's a ‘paste this’. :-) > > In Step 4 of this, the user is instructed to change the group owner > > of the Flask App to www-data. Not quite. Create an example Flask app by first creating a folder called “flasktest” : mkdir flasktest and setting its group owner to “www-data” : sudo chown www-data /home/pi/flasktest That's confused. It claims to be setting ‘its group owner’ but uses chown(1) which changes the owner. chgrp(1) would change the group. There is no such thing as a ‘group owner’ so what was intended and why? > > My main programs are installed in the directory /home/ pi/ and the > > Flask App is installed into the directory /home/pi/html so I > > recursively changed everything in html (including html itself) to be > > in the www-data group. How? Knowing the command would help spot problems. > > This caused all sorts of problems, including not being able to > > import some of the modules used by the Flask App. That suggests the permissions like read, write and execute, don't marry with the user and group of the process which is making the accesses, i.e. nginx and its offspring. The group, for example, can't just be changed without considering those as they work in concert. > > What do I need to do to get this to work? Understand the aim: your web-server Python code will be running as user and group www-data so the user, group, and permissions of what it needs to access have to be compatible. Which does not necessarily mean everything should be www-data:www-data rwxrwxrwx. > I may have partially solved this. Since in the Tutorial the > application directory is created before the App is download directly > to it using wget, I assume that the contents of the directory will > inherit the group ownership That's an incorrect assumption. New directory entries only take on the group of the directory rather than the program's effective group if the ‘set-gid’ bit of the directory is set. $ mkdir d $ ls -ld d drwxr-xr-x 2 ralph ralph 4096 Feb 6 13:39 d $ chmod g+s d $ ls -ld d drwxrwsr-x 2 ralph ralph 4096 Feb 6 13:39 d $ > so I have used chown -hR to change both the owner and group name from > pi. It appears to work. > > The question I now have, is that what is intended? Copying updates to > the html directory becomes a bit problematic, because of permissions > errors. I presume that I will have to use sudo to copy updated files > there and chown to change the ownership from root to www-data. > > Or am I totally up the creek? You seem to be piling wrong things up instead of scrapping it all and working out what's required, if anything, different from user pi's default permissions based on its umask. Your Python source code needs to be readable to nginx and its children so it can be opened and read to be interpreted by Python and run. What other I/O to local disk does your program do? If you don't change anything from the defaults, what doesn't work? -- Cheers, Ralph. -- Next meeting: Online, Jitsi, Tuesday, 2021-03-02 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Permissions Quandary with nginx
On Saturday, 6 February 2021 13:02:14 GMT Hamish MB wrote: > Best to set it up so you can just use git pull I imagine, if possible The problem with that is that I would need to push intermediate versions from my desktop to GitLab or do or my development directly on the Pi. I prefer to do the initial work on the code using Thonny on my desktop because I find that it gives me better visibility of the errors that can be detected before deployment. (That's not to say that I can't get the same information from the shell, it's just that I find it easier to interpret in Thonny.) Once I have a version that passes those tests, I scp it to the Pi and only push the code when I reach a point where the code is running without error (maybe with missing functionality but no bugs). BTW. I have realised that if I scp the file(s) to /home/pi/ and then sudo cp them to /home/pi/html, they take on the www-data ownership and I don't need to do the extra chown. I can live with that. -- Terry Coles -- Next meeting: Online, Jitsi, Tuesday, 2021-03-02 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Permissions Quandary with nginx
Best to set it up so you can just use git pull I imagine, if possible On 6 Feb 2021, at 11:40, Terry Coles mailto:d-...@hadrian-way.co.uk>> wrote: On Saturday, 6 February 2021 10:55:23 GMT Terry Coles wrote: What do I need to do to get this to work? I may have partially solved this. Since in the Tutorial the application directory is created before the App is download directly to it using wget, I assume that the contents of the directory will inherit the group ownership, so I have used chown -hR to change both the owner and group name from pi. It appears to work. The question I now have, is that what is intended? Copying updates to the html directory becomes a bit problematic, because of permissions errors. I presume that I will have to use sudo to copy updated files there and chown to change the ownership from root to www-data. Or am I totally up the creek? -- Next meeting: Online, Jitsi, Tuesday, 2021-03-02 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Permissions Quandary with nginx
On Saturday, 6 February 2021 10:55:23 GMT Terry Coles wrote: > What do I need to do to get this to work? I may have partially solved this. Since in the Tutorial the application directory is created before the App is download directly to it using wget, I assume that the contents of the directory will inherit the group ownership, so I have used chown -hR to change both the owner and group name from pi. It appears to work. The question I now have, is that what is intended? Copying updates to the html directory becomes a bit problematic, because of permissions errors. I presume that I will have to use sudo to copy updated files there and chown to change the ownership from root to www-data. Or am I totally up the creek? -- Terry Coles -- Next meeting: Online, Jitsi, Tuesday, 2021-03-02 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
