Re: [Pulp-dev] Using Gunicorn and Static Files
This came up again in the channel today, and we remembered we wanted to serve static content natively. I filed this work at the link below to add it to the next sprint. https://pulp.plan.io/issues/4180 On Mon, Oct 15, 2018 at 1:08 PM Brian Bouterse wrote: > tl;dr: let's adopt Whitenoise as the static solution for pulp3. If anyone > is -1 on this, please reply. > > Really this is a choice between (a) having all users struggle through a > STATIC_FILES config or (b) having users who plan to use Nginx or Apache > struggle through it. The less struggle the better I think, so let's adopt > Whitenoise as the default. It would be a nice batteries included default to > provide which I think is worth the downside of picking up Whitenoise as a > Pulp3 dependency. Note most users will still want to use Nginx or Apache > because Whitenoise can only serve files that exist when it is started. What > you've shown me though is that there is a group for (a) also. > > @ehelms would you be willing to make a PR for this change, a corresponding > issue, and also include the docs updates. I can provide review and testing. > > If anyone is -1 on this please reply. > > On Wed, Oct 10, 2018 at 3:48 PM Eric Helms wrote: > >> I tested this method out and it did work with gunicorn. The only ugly >> part is having to re-define the MIDDLEWARE entirely. I couldn't find a way >> to simply just add the middleware I wanted in the spot I needed it. >> >> Brian any thoughts on supporting this out of the box? I'm happy to do the >> work but I'd like to know it's desired. >> >> On Tue, Oct 9, 2018 at 9:11 AM Patrick Creech wrote: >> >>> On Wed, 2018-10-03 at 16:28 -0400, Eric Helms wrote: >>> > Howdy, >>> > >>> > When switching a deployment over to use gunicorn, DEBUG = TRUE for >>> serving static files stopped working. I endeavored to follow the production >>> install method using collectstatic. This required >>> > setting STATIC_ROOT which appeared to not be set by default. >>> > >>> > 1) Is there a default value for STATIC_ROOT I can set for >>> collectstatic? >>> > 2) Can gunicorn serve these static files for me? >>> > >>> > I realize the documentation calls for serving static files via a >>> webserver like nginx or Apache. However, that becomes a bit overkill in >>> something like a container deployment. If I run a separate >>> > webserver, then I would run it as a separate container and have to >>> mount the static files volume. This felt like overkill to me to have a >>> persistent volume to store and mount static files instead of >>> > the application server providing them for the application. >>> > >>> > Any thoughts and help are appreciated. >>> > >>> > Eric >>> >>> Eric, >>> >>> I got curious and took a look into your situation. >>> >>> I came across this project, suggested to be used by heroku, to achieve >>> the goal you are working towards: http://whitenoise.evans.io/en/stable/ >>> >>> It comes with django middleware support out of the box, so all you would >>> need to do is to 'pip install whietnoise' and edit the MIDDLEWARE >>> collection in the settings.py to include it, You want it to >>> load AFTER SecurityMiddleware, but BEFORE everything else, so make it >>> the second line, here: >>> https://github.com/pulp/pulp/blob/master/pulpcore/pulpcore/app/settings.py#L91 >>> >>> For your deployment model, it shouldn't be hard to apply a patch to the >>> code at a stage in your container process to inject this middleware into >>> the settings.py >>> >>> Otherwise, it's the standard 'staticfiles' process for django, They >>> have it simplified in step 1 here: >>> http://whitenoise.evans.io/en/stable/django.html >>> >> ___ >> Pulp-dev mailing list >> Pulp-dev@redhat.com >> https://www.redhat.com/mailman/listinfo/pulp-dev >> > ___ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev
Re: [Pulp-dev] Using Gunicorn and Static Files
tl;dr: let's adopt Whitenoise as the static solution for pulp3. If anyone is -1 on this, please reply. Really this is a choice between (a) having all users struggle through a STATIC_FILES config or (b) having users who plan to use Nginx or Apache struggle through it. The less struggle the better I think, so let's adopt Whitenoise as the default. It would be a nice batteries included default to provide which I think is worth the downside of picking up Whitenoise as a Pulp3 dependency. Note most users will still want to use Nginx or Apache because Whitenoise can only serve files that exist when it is started. What you've shown me though is that there is a group for (a) also. @ehelms would you be willing to make a PR for this change, a corresponding issue, and also include the docs updates. I can provide review and testing. If anyone is -1 on this please reply. On Wed, Oct 10, 2018 at 3:48 PM Eric Helms wrote: > I tested this method out and it did work with gunicorn. The only ugly part > is having to re-define the MIDDLEWARE entirely. I couldn't find a way to > simply just add the middleware I wanted in the spot I needed it. > > Brian any thoughts on supporting this out of the box? I'm happy to do the > work but I'd like to know it's desired. > > On Tue, Oct 9, 2018 at 9:11 AM Patrick Creech wrote: > >> On Wed, 2018-10-03 at 16:28 -0400, Eric Helms wrote: >> > Howdy, >> > >> > When switching a deployment over to use gunicorn, DEBUG = TRUE for >> serving static files stopped working. I endeavored to follow the production >> install method using collectstatic. This required >> > setting STATIC_ROOT which appeared to not be set by default. >> > >> > 1) Is there a default value for STATIC_ROOT I can set for >> collectstatic? >> > 2) Can gunicorn serve these static files for me? >> > >> > I realize the documentation calls for serving static files via a >> webserver like nginx or Apache. However, that becomes a bit overkill in >> something like a container deployment. If I run a separate >> > webserver, then I would run it as a separate container and have to >> mount the static files volume. This felt like overkill to me to have a >> persistent volume to store and mount static files instead of >> > the application server providing them for the application. >> > >> > Any thoughts and help are appreciated. >> > >> > Eric >> >> Eric, >> >> I got curious and took a look into your situation. >> >> I came across this project, suggested to be used by heroku, to achieve >> the goal you are working towards: http://whitenoise.evans.io/en/stable/ >> >> It comes with django middleware support out of the box, so all you would >> need to do is to 'pip install whietnoise' and edit the MIDDLEWARE >> collection in the settings.py to include it, You want it to >> load AFTER SecurityMiddleware, but BEFORE everything else, so make it the >> second line, here: >> https://github.com/pulp/pulp/blob/master/pulpcore/pulpcore/app/settings.py#L91 >> >> For your deployment model, it shouldn't be hard to apply a patch to the >> code at a stage in your container process to inject this middleware into >> the settings.py >> >> Otherwise, it's the standard 'staticfiles' process for django, They have >> it simplified in step 1 here: >> http://whitenoise.evans.io/en/stable/django.html >> > ___ > Pulp-dev mailing list > Pulp-dev@redhat.com > https://www.redhat.com/mailman/listinfo/pulp-dev > ___ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev
Re: [Pulp-dev] Using Gunicorn and Static Files
I tested this method out and it did work with gunicorn. The only ugly part is having to re-define the MIDDLEWARE entirely. I couldn't find a way to simply just add the middleware I wanted in the spot I needed it. Brian any thoughts on supporting this out of the box? I'm happy to do the work but I'd like to know it's desired. On Tue, Oct 9, 2018 at 9:11 AM Patrick Creech wrote: > On Wed, 2018-10-03 at 16:28 -0400, Eric Helms wrote: > > Howdy, > > > > When switching a deployment over to use gunicorn, DEBUG = TRUE for > serving static files stopped working. I endeavored to follow the production > install method using collectstatic. This required > > setting STATIC_ROOT which appeared to not be set by default. > > > > 1) Is there a default value for STATIC_ROOT I can set for collectstatic? > > 2) Can gunicorn serve these static files for me? > > > > I realize the documentation calls for serving static files via a > webserver like nginx or Apache. However, that becomes a bit overkill in > something like a container deployment. If I run a separate > > webserver, then I would run it as a separate container and have to mount > the static files volume. This felt like overkill to me to have a persistent > volume to store and mount static files instead of > > the application server providing them for the application. > > > > Any thoughts and help are appreciated. > > > > Eric > > Eric, > > I got curious and took a look into your situation. > > I came across this project, suggested to be used by heroku, to achieve the > goal you are working towards: http://whitenoise.evans.io/en/stable/ > > It comes with django middleware support out of the box, so all you would > need to do is to 'pip install whietnoise' and edit the MIDDLEWARE > collection in the settings.py to include it, You want it to > load AFTER SecurityMiddleware, but BEFORE everything else, so make it the > second line, here: > https://github.com/pulp/pulp/blob/master/pulpcore/pulpcore/app/settings.py#L91 > > For your deployment model, it shouldn't be hard to apply a patch to the > code at a stage in your container process to inject this middleware into > the settings.py > > Otherwise, it's the standard 'staticfiles' process for django, They have > it simplified in step 1 here: > http://whitenoise.evans.io/en/stable/django.html > ___ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev
Re: [Pulp-dev] Using Gunicorn and Static Files
On Wed, 2018-10-03 at 16:28 -0400, Eric Helms wrote: > Howdy, > > When switching a deployment over to use gunicorn, DEBUG = TRUE for serving > static files stopped working. I endeavored to follow the production install > method using collectstatic. This required > setting STATIC_ROOT which appeared to not be set by default. > > 1) Is there a default value for STATIC_ROOT I can set for collectstatic? > 2) Can gunicorn serve these static files for me? > > I realize the documentation calls for serving static files via a webserver > like nginx or Apache. However, that becomes a bit overkill in something like > a container deployment. If I run a separate > webserver, then I would run it as a separate container and have to mount the > static files volume. This felt like overkill to me to have a persistent > volume to store and mount static files instead of > the application server providing them for the application. > > Any thoughts and help are appreciated. > > Eric Eric, I got curious and took a look into your situation. I came across this project, suggested to be used by heroku, to achieve the goal you are working towards: http://whitenoise.evans.io/en/stable/ It comes with django middleware support out of the box, so all you would need to do is to 'pip install whietnoise' and edit the MIDDLEWARE collection in the settings.py to include it, You want it to load AFTER SecurityMiddleware, but BEFORE everything else, so make it the second line, here: https://github.com/pulp/pulp/blob/master/pulpcore/pulpcore/app/settings.py#L91 For your deployment model, it shouldn't be hard to apply a patch to the code at a stage in your container process to inject this middleware into the settings.py Otherwise, it's the standard 'staticfiles' process for django, They have it simplified in step 1 here: http://whitenoise.evans.io/en/stable/django.html signature.asc Description: This is a digitally signed message part ___ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev
Re: [Pulp-dev] Using Gunicorn and Static Files
On Fri, Oct 5, 2018 at 4:45 PM Brian Bouterse wrote: > Hi Eric, Thanks for writing, see inline. > > On Wed, Oct 3, 2018 at 4:28 PM Eric Helms wrote: > >> Howdy, >> >> When switching a deployment over to use gunicorn, DEBUG = TRUE for >> serving static files stopped working. I endeavored to follow the production >> install method using collectstatic. This required setting STATIC_ROOT which >> appeared to not be set by default. >> >> 1) Is there a default value for STATIC_ROOT I can set for collectstatic? >> > I think we should set one, and more and more we're having everything live > in /var/lib/pulp// so it could have a default of > /var/lib/pulp/static_media/. Do you have any interest in sending us a PR > and opening an issue? > Sure. > > 2) Can gunicorn serve these static files for me? >> > I looked into doing this a while back and I reached the conclusion that we > could but we shouldn't. It is convenient to not have to do anything extra > to deploy it, but the Django docs say in several places that we should > expressly not do this. Convenience is more than a nicety, the longer Pulp > takes to deploy the fewer users we will have. I think the Ansible Installer > and templates on openshift should let us do it the recommended way (with > apache or nginx, etc) and have it be just as easy for the user. > Unfortunately no one has done that ... yet! > > Reach out to me on irc if you want any help with any of those things. > Can you clarify why the deployment would take longer if the application itself is serving up the static files? In a containerized setup, this current model would require a persistent volume store dedicated just to static files (seems like overkill) and running a separate webserver just to serve those files. I will agree the likelyhood of running Apache or Nginx is high to handle SSL and routing for the user in a deployment, however, I still find the static files part a bit odd to be required to run the application. > > >> I realize the documentation calls for serving static files via a >> webserver like nginx or Apache. However, that becomes a bit overkill in >> something like a container deployment. If I run a separate webserver, then >> I would run it as a separate container and have to mount the static files >> volume. This felt like overkill to me to have a persistent volume to store >> and mount static files instead of the application server providing them for >> the application. >> >> Any thoughts and help are appreciated. >> >> Eric >> ___ >> Pulp-dev mailing list >> Pulp-dev@redhat.com >> https://www.redhat.com/mailman/listinfo/pulp-dev >> > ___ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev
Re: [Pulp-dev] Using Gunicorn and Static Files
Hi Eric, Thanks for writing, see inline. On Wed, Oct 3, 2018 at 4:28 PM Eric Helms wrote: > Howdy, > > When switching a deployment over to use gunicorn, DEBUG = TRUE for serving > static files stopped working. I endeavored to follow the production install > method using collectstatic. This required setting STATIC_ROOT which > appeared to not be set by default. > > 1) Is there a default value for STATIC_ROOT I can set for collectstatic? > I think we should set one, and more and more we're having everything live in /var/lib/pulp// so it could have a default of /var/lib/pulp/static_media/. Do you have any interest in sending us a PR and opening an issue? 2) Can gunicorn serve these static files for me? > I looked into doing this a while back and I reached the conclusion that we could but we shouldn't. It is convenient to not have to do anything extra to deploy it, but the Django docs say in several places that we should expressly not do this. Convenience is more than a nicety, the longer Pulp takes to deploy the fewer users we will have. I think the Ansible Installer and templates on openshift should let us do it the recommended way (with apache or nginx, etc) and have it be just as easy for the user. Unfortunately no one has done that ... yet! Reach out to me on irc if you want any help with any of those things. > I realize the documentation calls for serving static files via a webserver > like nginx or Apache. However, that becomes a bit overkill in something > like a container deployment. If I run a separate webserver, then I would > run it as a separate container and have to mount the static files volume. > This felt like overkill to me to have a persistent volume to store and > mount static files instead of the application server providing them for the > application. > > Any thoughts and help are appreciated. > > Eric > ___ > Pulp-dev mailing list > Pulp-dev@redhat.com > https://www.redhat.com/mailman/listinfo/pulp-dev > ___ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev
[Pulp-dev] Using Gunicorn and Static Files
Howdy, When switching a deployment over to use gunicorn, DEBUG = TRUE for serving static files stopped working. I endeavored to follow the production install method using collectstatic. This required setting STATIC_ROOT which appeared to not be set by default. 1) Is there a default value for STATIC_ROOT I can set for collectstatic? 2) Can gunicorn serve these static files for me? I realize the documentation calls for serving static files via a webserver like nginx or Apache. However, that becomes a bit overkill in something like a container deployment. If I run a separate webserver, then I would run it as a separate container and have to mount the static files volume. This felt like overkill to me to have a persistent volume to store and mount static files instead of the application server providing them for the application. Any thoughts and help are appreciated. Eric ___ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev
