Re: [openstack-dev] [devstack] apache wsgi application support
On 06/24/2015 07:57 AM, Chris Dent wrote: On Wed, 24 Jun 2015, Sean Dague wrote: So on https://review.openstack.org/#/c/194442 ... I was kind of thinking we'd be able to get keystone-wsgi-public into a bin directory. I put up a half baked sketch of this idea in https://review.openstack.org/#/c/195044. Would be curious if some version of that could be made to work here. I remain somewhat confused on why you want this? It's a fairly common convention for the module which has the wsgi 'application' global to live somewhere in the web tree (/var/www/ etc) or just some random location that is referenced from the hosting server. Putting in */bin doesn't has some risk of confusion because you shouldn't really run the wsgi script in any normal sense of the word (from the command line, from cron, whatever). If you've set your wsgi app up so that is possible then you're conflating purposes and that's bad mojo. Your patch has exceptions to watch out for that, but it seems...crufty? If the primary reason is so that we can rely on the console_scripts entry point to handle getting the application somewhere useful then that too feels a bit crufty, in the sense of that's a hack. That may be perfectly reasonable stuff to do given the constraints and lack of other options but I thought I would check in. (I happen to be working on making this a reality for gnocchi's devstack plugin at the moment and your message popped up at exactly the moment when I needed a reference to in progress reviews on this topic. Thanks.) The reason I want this is so that the upgrade process for keystone is: pip install ./keystone And not have to also have knowledge about the contents of the keystone source directory. Today a lot of installation and upgrade logic for packages is left as an exercise for the reader, which means devstack, ansible, rpms, debs all end up doing a bunch of work beyond pip install. Minimizing that by bringing more of this back into what pip install does will make for a more common base moving forward for everyone. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [devstack] apache wsgi application support
On 06/23/2015 04:07 PM, Brant Knudson wrote: I've got a few related changes proposed to keystone and devstack: https://review.openstack.org/#/c/193891/ - Changes Apache Httpd config so that /identity is the keystone public (aka main) handler and /identity_admin is the keystone admin handler. Httpd can have multiple aliases for the same wsgi handler so :5000 and :35357 still work. The follow-on patch at https://review.openstack.org/193894 shows some further work to change config so that the new endpoints are used by the tests. There are a lot of devstack variables that aren't going to apply or are going to be reinterpreted if we switch to this so I'll have to think about how that's going to work. https://review.openstack.org/#/c/194442/ - Creates files keystone/httpd/admin.py and public.py , in addition to the old httpd/keystone.py that you had to copy and rename or symlink. https://review.openstack.org/#/c/194729/ - Changes devstack to use keystone/httpd/admin.py and public.py rather than copying httpd/keystone.py. Grenade is failing so I'll need to figure that out. Great, thanks much! Reviewing now. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [devstack] apache wsgi application support
On Wed, 24 Jun 2015, Sean Dague wrote: On 06/24/2015 07:57 AM, Chris Dent wrote: If the primary reason is so that we can rely on the console_scripts entry point to handle getting the application somewhere useful then that too feels a bit crufty, in the sense of that's a hack. [snip] The reason I want this is so that the upgrade process for keystone is: pip install ./keystone And not have to also have knowledge about the contents of the keystone source directory. Today a lot of installation and upgrade logic for packages is left as an exercise for the reader, which means devstack, ansible, rpms, debs all end up doing a bunch of work beyond pip install. Minimizing that by bringing more of this back into what pip install does will make for a more common base moving forward for everyone. Makes sense. My only concern would be the somewhat of a mismatch in semantics between console_scripts and what is not a console script. Does anyone who is more familiar with entry points know of an alternative that will accomplish the same sort of thing? If there's not, then awesome, let's do it. -- Chris Dent tw:@anticdent freenode:cdent https://tank.peermore.com/tanks/cdent __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [devstack] apache wsgi application support
On 06/24/2015 07:19 AM, Sean Dague wrote: On 06/23/2015 04:07 PM, Brant Knudson wrote: I've got a few related changes proposed to keystone and devstack: https://review.openstack.org/#/c/193891/ - Changes Apache Httpd config so that /identity is the keystone public (aka main) handler and /identity_admin is the keystone admin handler. Httpd can have multiple aliases for the same wsgi handler so :5000 and :35357 still work. The follow-on patch at https://review.openstack.org/193894 shows some further work to change config so that the new endpoints are used by the tests. There are a lot of devstack variables that aren't going to apply or are going to be reinterpreted if we switch to this so I'll have to think about how that's going to work. https://review.openstack.org/#/c/194442/ - Creates files keystone/httpd/admin.py and public.py , in addition to the old httpd/keystone.py that you had to copy and rename or symlink. https://review.openstack.org/#/c/194729/ - Changes devstack to use keystone/httpd/admin.py and public.py rather than copying httpd/keystone.py. Grenade is failing so I'll need to figure that out. Great, thanks much! Reviewing now. -Sean So on https://review.openstack.org/#/c/194442 ... I was kind of thinking we'd be able to get keystone-wsgi-public into a bin directory. I put up a half baked sketch of this idea in https://review.openstack.org/#/c/195044. Would be curious if some version of that could be made to work here. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [devstack] apache wsgi application support
On Wed, 24 Jun 2015, Sean Dague wrote: So on https://review.openstack.org/#/c/194442 ... I was kind of thinking we'd be able to get keystone-wsgi-public into a bin directory. I put up a half baked sketch of this idea in https://review.openstack.org/#/c/195044. Would be curious if some version of that could be made to work here. I remain somewhat confused on why you want this? It's a fairly common convention for the module which has the wsgi 'application' global to live somewhere in the web tree (/var/www/ etc) or just some random location that is referenced from the hosting server. Putting in */bin doesn't has some risk of confusion because you shouldn't really run the wsgi script in any normal sense of the word (from the command line, from cron, whatever). If you've set your wsgi app up so that is possible then you're conflating purposes and that's bad mojo. Your patch has exceptions to watch out for that, but it seems...crufty? If the primary reason is so that we can rely on the console_scripts entry point to handle getting the application somewhere useful then that too feels a bit crufty, in the sense of that's a hack. That may be perfectly reasonable stuff to do given the constraints and lack of other options but I thought I would check in. (I happen to be working on making this a reality for gnocchi's devstack plugin at the moment and your message popped up at exactly the moment when I needed a reference to in progress reviews on this topic. Thanks.) -- Chris Dent tw:@anticdent freenode:cdent https://tank.peermore.com/tanks/cdent __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [devstack] apache wsgi application support
Hi guys, I worked on running nova-api on http://localhost/compute. I prepared raw reviews: nova review: https://review.openstack.org/#/c/195303/1 devstack review: https://review.openstack.org/#/c/195300/ We need to determine proper URLs for ec2-api(used 8773 port) and metadata(used 8775 port) services and structure of nova URLs. There are options: 1. For each nova service we can use separate URL node under the root: http://localhost/compute (8774) http://localhost/ec2-api (8773) http://localhost/metadata (8775) 2. We can have general nova URL like /compute and services will use sub URLs: http://localhost/compute/osapi (8774) http://localhost/compute/ec2-api (8773) http://localhost/compute/metadata (8775) First option looks better for me. I noticed that there are many templates in devstack/files with almost the same content. Instead of using templates - apache-ceilometer.template - apache-ironic.template - apache-keystone.template - apache-nova-api.template - apache-nova-ec2-api.template - ... we can use generic template file with additional variables. Marian __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [devstack] apache wsgi application support
On 06/24/2015 01:51 PM, Brant Knudson wrote: On Wed, Jun 24, 2015 at 7:27 AM, Chris Dent chd...@redhat.com mailto:chd...@redhat.com wrote: On Wed, 24 Jun 2015, Sean Dague wrote: On 06/24/2015 07:57 AM, Chris Dent wrote: If the primary reason is so that we can rely on the console_scripts entry point to handle getting the application somewhere useful then that too feels a bit crufty, in the sense of that's a hack. [snip] The reason I want this is so that the upgrade process for keystone is: pip install ./keystone And not have to also have knowledge about the contents of the keystone source directory. Today a lot of installation and upgrade logic for packages is left as an exercise for the reader, which means devstack, ansible, rpms, debs all end up doing a bunch of work beyond pip install. Minimizing that by bringing more of this back into what pip install does will make for a more common base moving forward for everyone. Makes sense. My only concern would be the somewhat of a mismatch in semantics between console_scripts and what is not a console script. Does anyone who is more familiar with entry points know of an alternative that will accomplish the same sort of thing? If there's not, then awesome, let's do it. -- Chris Dent tw:@anticdent freenode:cdent https://tank.peermore.com/tanks/cdent I tried console_scripts and there is a mismatch between what a console script is and a wsgi script. A wsgi script must export an application symbol and the generated console script doesn't. Maybe setuptools will be enhanced someday to also support wsgi scripts but until then I think we're stuck. One thing I should have looked at earlier is how devstack handles horizon, since it's already running in Apache Httpd. The devstack horizon setup uses the wsgi script out of the devstack source directories: WSGIScriptAlias / /opt/stack/horizon/openstack_dashboard/wsgi/django.wsgi Docs: [1] https://pythonhosted.org/setuptools/setuptools.html#automatic-script-creation -- console_scripts and gui_scripts [2] docs.openstack.org/developer/pbr/#files http://docs.openstack.org/developer/pbr/#files -- I couldn't get this to work for some reason Hmmm... bummer. Oh well. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [devstack] apache wsgi application support
On Wed, Jun 24, 2015 at 7:27 AM, Chris Dent chd...@redhat.com wrote: On Wed, 24 Jun 2015, Sean Dague wrote: On 06/24/2015 07:57 AM, Chris Dent wrote: If the primary reason is so that we can rely on the console_scripts entry point to handle getting the application somewhere useful then that too feels a bit crufty, in the sense of that's a hack. [snip] The reason I want this is so that the upgrade process for keystone is: pip install ./keystone And not have to also have knowledge about the contents of the keystone source directory. Today a lot of installation and upgrade logic for packages is left as an exercise for the reader, which means devstack, ansible, rpms, debs all end up doing a bunch of work beyond pip install. Minimizing that by bringing more of this back into what pip install does will make for a more common base moving forward for everyone. Makes sense. My only concern would be the somewhat of a mismatch in semantics between console_scripts and what is not a console script. Does anyone who is more familiar with entry points know of an alternative that will accomplish the same sort of thing? If there's not, then awesome, let's do it. -- Chris Dent tw:@anticdent freenode:cdent https://tank.peermore.com/tanks/cdent I tried console_scripts and there is a mismatch between what a console script is and a wsgi script. A wsgi script must export an application symbol and the generated console script doesn't. Maybe setuptools will be enhanced someday to also support wsgi scripts but until then I think we're stuck. One thing I should have looked at earlier is how devstack handles horizon, since it's already running in Apache Httpd. The devstack horizon setup uses the wsgi script out of the devstack source directories: WSGIScriptAlias / /opt/stack/horizon/openstack_dashboard/wsgi/django.wsgi Docs: [1] https://pythonhosted.org/setuptools/setuptools.html#automatic-script-creation -- console_scripts and gui_scripts [2] docs.openstack.org/developer/pbr/#files -- I couldn't get this to work for some reason -Brant __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [devstack] apache wsgi application support
On 06/24/2015 02:05 PM, Sean Dague wrote: On 06/24/2015 01:51 PM, Brant Knudson wrote: On Wed, Jun 24, 2015 at 7:27 AM, Chris Dent chd...@redhat.com mailto:chd...@redhat.com wrote: On Wed, 24 Jun 2015, Sean Dague wrote: On 06/24/2015 07:57 AM, Chris Dent wrote: If the primary reason is so that we can rely on the console_scripts entry point to handle getting the application somewhere useful then that too feels a bit crufty, in the sense of that's a hack. [snip] The reason I want this is so that the upgrade process for keystone is: pip install ./keystone And not have to also have knowledge about the contents of the keystone source directory. Today a lot of installation and upgrade logic for packages is left as an exercise for the reader, which means devstack, ansible, rpms, debs all end up doing a bunch of work beyond pip install. Minimizing that by bringing more of this back into what pip install does will make for a more common base moving forward for everyone. Makes sense. My only concern would be the somewhat of a mismatch in semantics between console_scripts and what is not a console script. Does anyone who is more familiar with entry points know of an alternative that will accomplish the same sort of thing? If there's not, then awesome, let's do it. -- Chris Dent tw:@anticdent freenode:cdent https://tank.peermore.com/tanks/cdent I tried console_scripts and there is a mismatch between what a console script is and a wsgi script. A wsgi script must export an application symbol and the generated console script doesn't. Maybe setuptools will be enhanced someday to also support wsgi scripts but until then I think we're stuck. One thing I should have looked at earlier is how devstack handles horizon, since it's already running in Apache Httpd. The devstack horizon setup uses the wsgi script out of the devstack source directories: WSGIScriptAlias / /opt/stack/horizon/openstack_dashboard/wsgi/django.wsgi Docs: [1] https://pythonhosted.org/setuptools/setuptools.html#automatic-script-creation -- console_scripts and gui_scripts [2] docs.openstack.org/developer/pbr/#files http://docs.openstack.org/developer/pbr/#files -- I couldn't get this to work for some reason Hmmm... bummer. Oh well. pbr is actually the one that creates console_scripts for us, as the ones generated by setuptools are unusable for us (they use entrypoints which is the reason rootwrap was super slow 8 months ago) As we have all the bits to produce such scripts in the way we want and already do so - adding declarative wsgi scripts is likely about 5 lines of code and I think would be super useful. (I said most of this to sdague in IRC) __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [devstack] apache wsgi application support
Brant,
We likely need to back port a simplified version of the wsgi files and/or make
the Juno (and kilo) versions of dev stack use the same simplified / split
files. Grenade doesn't re-run stack - so new files that are outside pip's
purview won't be used afaik.
--Morgab
Sent via mobile
On Jun 23, 2015, at 13:07, Brant Knudson b...@acm.org wrote:
On Wed, Jun 17, 2015 at 1:21 PM, Sean Dague s...@dague.net wrote:
On 06/16/2015 05:25 PM, Chris Dent wrote:
On Tue, 16 Jun 2015, Sean Dague wrote:
I was just looking at the patches that put Nova under apache wsgi for
the API, and there are a few things that I think are going in the wrong
direction. Largely I think because they were copied from the
lib/keystone code, which we've learned is kind of the wrong direction.
Yes, that's certainly what I've done the few times I've done it.
devstack is deeply encouraging of cargo culting for reasons that are
not entirely clear.
Yeh, hence why I decided to put the brakes on a little here and get this
on the list.
The first is the fact that a big reason for putting {SERVICES} under
apache wsgi is we aren't running on a ton of weird unregistered ports.
We're running on 80 and 443 (when appropriate). In order to do this we
really need to namespace the API urls. Which means that service catalog
needs to be updated appropriately.
So:
a) I'm very glad to hear of this. I've been bristling about the weird
ports thing for the last year.
b) You make it sound like there's been a plan in place to not use
those ports for quite some time and we'd get to that when we all
had some spare time. Where do I go to keep abreast of such plans?
Unfortunately, this is one of those in the ether kinds of plans. It's
been talked about for so long, but it never really got written down.
Hopefully this can be driven into the service catalog standardization
spec (or tag along somewhere close).
Or if nothing else, we're documenting it now on the mailing list as
permanent storage.
I also think this -
https://github.com/openstack-dev/devstack/blob/master/lib/nova#L266-L268
is completely wrong.
The Apache configs should instead specify access rules such that the
installed console entry point of nova-api can be used in place as the
WSGIScript.
I'm not able to parse this paragraph in any actionable way. The lines
you reference are one of several ways of telling mod wsgi where the
virtualenv is, which has to happen in some fashion if you are using
a virtualenv.
This doesn't appear to have anything to do with locating the module
that contains the WSGI app, so I'm missing the connection. Can you
explain please?
(Basically I'm keen on getting gnocchi and ceilometer wsgi servers
in devstack aligned with whatever the end game is, so knowing the plan
makes it a bit easier.)
Gah, the problem of linking to 'master' with line numbers. The three
lines I cared about were:
# copy proxy vhost and wsgi helper files
sudo cp $NOVA_DIR/nova/wsgi/nova-api.py $NOVA_WSGI_DIR/nova-api
sudo cp $NOVA_DIR/nova/wsgi/nova-ec2-api.py $NOVA_WSGI_DIR/nova-ec2-api
I don't think that we should be copying py files around to other
directories outside of normal pip install process. We should just have
mod_wsgi reference a thing that is installed in /usr/{local}/bin or
/usr/share via the python install process.
This should also make lines like -
https://github.com/openstack-dev/devstack/blob/master/lib/nova#L272 and
L274 uneeded. (The WSGI Script will be in a known place). It will also
make upgrades much more friendly.
It sounds like maybe you are saying that the api console script and
the module containing the wsgi 'application' variable ought to be the
same thing. I don't reckon that's a great idea as the api console
scripts will want to import a bunch of stuff that the wsgi application
will not.
Or I may be completely misreading you. It's been a long day, etc.
They don't need to be actually the same thing. They could be different
scripts, but they should be scripts that install via the normal pip
install process to a place, and we reference them by known name.
I think that we need to get these things sorted before any further
progression here. Volunteers welcomed to help get us there.
Find me, happy to help. The sooner we can kill wacky port weirdness
the better.
Agreed.
-Sean
--
Sean Dague
http://dague.net
I've got a few related changes proposed to keystone and devstack:
https://review.openstack.org/#/c/193891/ - Changes Apache Httpd config so
that /identity is the keystone public (aka main) handler and /identity_admin
is the keystone admin handler. Httpd can have multiple aliases for the same
wsgi handler so :5000 and :35357 still work. The follow-on patch at
https://review.openstack.org/193894 shows some further work to change config
so that the new endpoints
Re: [openstack-dev] [devstack] apache wsgi application support
On Wed, Jun 17, 2015 at 1:21 PM, Sean Dague s...@dague.net wrote:
On 06/16/2015 05:25 PM, Chris Dent wrote:
On Tue, 16 Jun 2015, Sean Dague wrote:
I was just looking at the patches that put Nova under apache wsgi for
the API, and there are a few things that I think are going in the wrong
direction. Largely I think because they were copied from the
lib/keystone code, which we've learned is kind of the wrong direction.
Yes, that's certainly what I've done the few times I've done it.
devstack is deeply encouraging of cargo culting for reasons that are
not entirely clear.
Yeh, hence why I decided to put the brakes on a little here and get this
on the list.
The first is the fact that a big reason for putting {SERVICES} under
apache wsgi is we aren't running on a ton of weird unregistered ports.
We're running on 80 and 443 (when appropriate). In order to do this we
really need to namespace the API urls. Which means that service catalog
needs to be updated appropriately.
So:
a) I'm very glad to hear of this. I've been bristling about the weird
ports thing for the last year.
b) You make it sound like there's been a plan in place to not use
those ports for quite some time and we'd get to that when we all
had some spare time. Where do I go to keep abreast of such plans?
Unfortunately, this is one of those in the ether kinds of plans. It's
been talked about for so long, but it never really got written down.
Hopefully this can be driven into the service catalog standardization
spec (or tag along somewhere close).
Or if nothing else, we're documenting it now on the mailing list as
permanent storage.
I also think this -
https://github.com/openstack-dev/devstack/blob/master/lib/nova#L266-L268
is completely wrong.
The Apache configs should instead specify access rules such that the
installed console entry point of nova-api can be used in place as the
WSGIScript.
I'm not able to parse this paragraph in any actionable way. The lines
you reference are one of several ways of telling mod wsgi where the
virtualenv is, which has to happen in some fashion if you are using
a virtualenv.
This doesn't appear to have anything to do with locating the module
that contains the WSGI app, so I'm missing the connection. Can you
explain please?
(Basically I'm keen on getting gnocchi and ceilometer wsgi servers
in devstack aligned with whatever the end game is, so knowing the plan
makes it a bit easier.)
Gah, the problem of linking to 'master' with line numbers. The three
lines I cared about were:
# copy proxy vhost and wsgi helper files
sudo cp $NOVA_DIR/nova/wsgi/nova-api.py $NOVA_WSGI_DIR/nova-api
sudo cp $NOVA_DIR/nova/wsgi/nova-ec2-api.py $NOVA_WSGI_DIR/nova-ec2-api
I don't think that we should be copying py files around to other
directories outside of normal pip install process. We should just have
mod_wsgi reference a thing that is installed in /usr/{local}/bin or
/usr/share via the python install process.
This should also make lines like -
https://github.com/openstack-dev/devstack/blob/master/lib/nova#L272 and
L274 uneeded. (The WSGI Script will be in a known place). It will also
make upgrades much more friendly.
It sounds like maybe you are saying that the api console script and
the module containing the wsgi 'application' variable ought to be the
same thing. I don't reckon that's a great idea as the api console
scripts will want to import a bunch of stuff that the wsgi application
will not.
Or I may be completely misreading you. It's been a long day, etc.
They don't need to be actually the same thing. They could be different
scripts, but they should be scripts that install via the normal pip
install process to a place, and we reference them by known name.
I think that we need to get these things sorted before any further
progression here. Volunteers welcomed to help get us there.
Find me, happy to help. The sooner we can kill wacky port weirdness
the better.
Agreed.
-Sean
--
Sean Dague
http://dague.net
I've got a few related changes proposed to keystone and devstack:
https://review.openstack.org/#/c/193891/ - Changes Apache Httpd config so
that /identity is the keystone public (aka main) handler and
/identity_admin is the keystone admin handler. Httpd can have multiple
aliases for the same wsgi handler so :5000 and :35357 still work. The
follow-on patch at https://review.openstack.org/193894 shows some further
work to change config so that the new endpoints are used by the tests.
There are a lot of devstack variables that aren't going to apply or are
going to be reinterpreted if we switch to this so I'll have to think about
how that's going to work.
https://review.openstack.org/#/c/194442/ - Creates files
keystone/httpd/admin.py and public.py , in addition to the old
httpd/keystone.py that you had to copy and rename or symlink.
Re: [openstack-dev] [devstack] apache wsgi application support
On Tue, Jun 23, 2015 at 4:08 PM, Morgan Fainberg morgan.fainb...@gmail.com wrote: We likely need to back port a simplified version of the wsgi files and/or make the Juno (and kilo) versions of dev stack use the same simplified / split files. Grenade doesn't re-run stack - so new files that are outside pip's purview won't be used afaik. You should only need to go back to kilo, the juno-kilo should continue to work the old way, the kilo-master run should start the new way. dt -- Dean Troyer dtro...@gmail.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [devstack] apache wsgi application support
Dean, If we change how Kilo works, then we'll just move the error state from Kilo - Master to Juno - Kilo since the same issue will now occur in upgrading from Juno in grenade. It's unfortunate. --Morgan On Tue, Jun 23, 2015 at 2:22 PM, Dean Troyer dtro...@gmail.com wrote: On Tue, Jun 23, 2015 at 4:08 PM, Morgan Fainberg morgan.fainb...@gmail.com wrote: We likely need to back port a simplified version of the wsgi files and/or make the Juno (and kilo) versions of dev stack use the same simplified / split files. Grenade doesn't re-run stack - so new files that are outside pip's purview won't be used afaik. You should only need to go back to kilo, the juno-kilo should continue to work the old way, the kilo-master run should start the new way. dt -- Dean Troyer dtro...@gmail.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [devstack] apache wsgi application support
Aha! Right, Thanks Sean. I'll just go back under my rock (too many things going on to remember them all) and see what we can do about getting Kilo - Master fixed up. On Tue, Jun 23, 2015 at 4:53 PM, Sean Dague s...@dague.net wrote: On 06/23/2015 07:49 PM, Morgan Fainberg wrote: Dean, If we change how Kilo works, then we'll just move the error state from Kilo - Master to Juno - Kilo since the same issue will now occur in upgrading from Juno in grenade. It's unfortunate. Actually Juno - Kilo is back on eventlet because of the other bug with wsgi stuff and keystone. https://github.com/openstack-infra/devstack-gate/blob/master/devstack-vm-gate.sh#L341 So a backport to Kilo is all that's required. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [devstack] apache wsgi application support
On 06/23/2015 07:49 PM, Morgan Fainberg wrote: Dean, If we change how Kilo works, then we'll just move the error state from Kilo - Master to Juno - Kilo since the same issue will now occur in upgrading from Juno in grenade. It's unfortunate. Actually Juno - Kilo is back on eventlet because of the other bug with wsgi stuff and keystone. https://github.com/openstack-infra/devstack-gate/blob/master/devstack-vm-gate.sh#L341 So a backport to Kilo is all that's required. -Sean -- Sean Dague http://dague.net signature.asc Description: OpenPGP digital signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [devstack] apache wsgi application support
Adam pointed to this url as a proposal for the namespaces:
https://wiki.openstack.org/wiki/URLs
How about this gets turned into a cross project spec or part of a larger one
with the stuff in this ML thread? Then we can get the projects aware and
buying into this little slice of sanity.
--Rocky
-Original Message-
From: Sean Dague [mailto:s...@dague.net]
Sent: Wednesday, June 17, 2015 11:22
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [devstack] apache wsgi application support
On 06/16/2015 05:25 PM, Chris Dent wrote:
On Tue, 16 Jun 2015, Sean Dague wrote:
I was just looking at the patches that put Nova under apache wsgi for
the API, and there are a few things that I think are going in the wrong
direction. Largely I think because they were copied from the
lib/keystone code, which we've learned is kind of the wrong direction.
Yes, that's certainly what I've done the few times I've done it.
devstack is deeply encouraging of cargo culting for reasons that are
not entirely clear.
Yeh, hence why I decided to put the brakes on a little here and get this
on the list.
The first is the fact that a big reason for putting {SERVICES} under
apache wsgi is we aren't running on a ton of weird unregistered ports.
We're running on 80 and 443 (when appropriate). In order to do this we
really need to namespace the API urls. Which means that service catalog
needs to be updated appropriately.
So:
a) I'm very glad to hear of this. I've been bristling about the weird
ports thing for the last year.
b) You make it sound like there's been a plan in place to not use
those ports for quite some time and we'd get to that when we all
had some spare time. Where do I go to keep abreast of such plans?
Unfortunately, this is one of those in the ether kinds of plans. It's
been talked about for so long, but it never really got written down.
Hopefully this can be driven into the service catalog standardization
spec (or tag along somewhere close).
Or if nothing else, we're documenting it now on the mailing list as
permanent storage.
I also think this -
https://github.com/openstack-dev/devstack/blob/master/lib/nova#L266-L268
is completely wrong.
The Apache configs should instead specify access rules such that the
installed console entry point of nova-api can be used in place as the
WSGIScript.
I'm not able to parse this paragraph in any actionable way. The lines
you reference are one of several ways of telling mod wsgi where the
virtualenv is, which has to happen in some fashion if you are using
a virtualenv.
This doesn't appear to have anything to do with locating the module
that contains the WSGI app, so I'm missing the connection. Can you
explain please?
(Basically I'm keen on getting gnocchi and ceilometer wsgi servers
in devstack aligned with whatever the end game is, so knowing the plan
makes it a bit easier.)
Gah, the problem of linking to 'master' with line numbers. The three
lines I cared about were:
# copy proxy vhost and wsgi helper files
sudo cp $NOVA_DIR/nova/wsgi/nova-api.py $NOVA_WSGI_DIR/nova-api
sudo cp $NOVA_DIR/nova/wsgi/nova-ec2-api.py $NOVA_WSGI_DIR/nova-ec2-api
I don't think that we should be copying py files around to other
directories outside of normal pip install process. We should just have
mod_wsgi reference a thing that is installed in /usr/{local}/bin or
/usr/share via the python install process.
This should also make lines like -
https://github.com/openstack-dev/devstack/blob/master/lib/nova#L272 and
L274 uneeded. (The WSGI Script will be in a known place). It will also
make upgrades much more friendly.
It sounds like maybe you are saying that the api console script and
the module containing the wsgi 'application' variable ought to be the
same thing. I don't reckon that's a great idea as the api console
scripts will want to import a bunch of stuff that the wsgi application
will not.
Or I may be completely misreading you. It's been a long day, etc.
They don't need to be actually the same thing. They could be different
scripts, but they should be scripts that install via the normal pip
install process to a place, and we reference them by known name.
I think that we need to get these things sorted before any further
progression here. Volunteers welcomed to help get us there.
Find me, happy to help. The sooner we can kill wacky port weirdness
the better.
Agreed.
-Sean
--
Sean Dague
http://dague.net
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ
Re: [openstack-dev] [devstack] apache wsgi application support
On 06/16/2015 05:25 PM, Chris Dent wrote:
On Tue, 16 Jun 2015, Sean Dague wrote:
I was just looking at the patches that put Nova under apache wsgi for
the API, and there are a few things that I think are going in the wrong
direction. Largely I think because they were copied from the
lib/keystone code, which we've learned is kind of the wrong direction.
Yes, that's certainly what I've done the few times I've done it.
devstack is deeply encouraging of cargo culting for reasons that are
not entirely clear.
Yeh, hence why I decided to put the brakes on a little here and get this
on the list.
The first is the fact that a big reason for putting {SERVICES} under
apache wsgi is we aren't running on a ton of weird unregistered ports.
We're running on 80 and 443 (when appropriate). In order to do this we
really need to namespace the API urls. Which means that service catalog
needs to be updated appropriately.
So:
a) I'm very glad to hear of this. I've been bristling about the weird
ports thing for the last year.
b) You make it sound like there's been a plan in place to not use
those ports for quite some time and we'd get to that when we all
had some spare time. Where do I go to keep abreast of such plans?
Unfortunately, this is one of those in the ether kinds of plans. It's
been talked about for so long, but it never really got written down.
Hopefully this can be driven into the service catalog standardization
spec (or tag along somewhere close).
Or if nothing else, we're documenting it now on the mailing list as
permanent storage.
I also think this -
https://github.com/openstack-dev/devstack/blob/master/lib/nova#L266-L268
is completely wrong.
The Apache configs should instead specify access rules such that the
installed console entry point of nova-api can be used in place as the
WSGIScript.
I'm not able to parse this paragraph in any actionable way. The lines
you reference are one of several ways of telling mod wsgi where the
virtualenv is, which has to happen in some fashion if you are using
a virtualenv.
This doesn't appear to have anything to do with locating the module
that contains the WSGI app, so I'm missing the connection. Can you
explain please?
(Basically I'm keen on getting gnocchi and ceilometer wsgi servers
in devstack aligned with whatever the end game is, so knowing the plan
makes it a bit easier.)
Gah, the problem of linking to 'master' with line numbers. The three
lines I cared about were:
# copy proxy vhost and wsgi helper files
sudo cp $NOVA_DIR/nova/wsgi/nova-api.py $NOVA_WSGI_DIR/nova-api
sudo cp $NOVA_DIR/nova/wsgi/nova-ec2-api.py $NOVA_WSGI_DIR/nova-ec2-api
I don't think that we should be copying py files around to other
directories outside of normal pip install process. We should just have
mod_wsgi reference a thing that is installed in /usr/{local}/bin or
/usr/share via the python install process.
This should also make lines like -
https://github.com/openstack-dev/devstack/blob/master/lib/nova#L272 and
L274 uneeded. (The WSGI Script will be in a known place). It will also
make upgrades much more friendly.
It sounds like maybe you are saying that the api console script and
the module containing the wsgi 'application' variable ought to be the
same thing. I don't reckon that's a great idea as the api console
scripts will want to import a bunch of stuff that the wsgi application
will not.
Or I may be completely misreading you. It's been a long day, etc.
They don't need to be actually the same thing. They could be different
scripts, but they should be scripts that install via the normal pip
install process to a place, and we reference them by known name.
I think that we need to get these things sorted before any further
progression here. Volunteers welcomed to help get us there.
Find me, happy to help. The sooner we can kill wacky port weirdness
the better.
Agreed.
-Sean
--
Sean Dague
http://dague.net
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [devstack] apache wsgi application support
On Tue, 16 Jun 2015, Sean Dague wrote: I'd expect nova to be running on http://localhost/compute not http://localhost:8774 when running under wsgi. That's going to probably interestingly break a lot of weird assumptions by different projects, but that's part of the reason for doing this exercise. Things should be using the service catalog, and when they aren't, we need to figure it out. Absolutely agree. I think that we need to get these things sorted before any further progression here. Volunteers welcomed to help get us there. You can count on me. Marian __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [devstack] apache wsgi application support
I was just looking at the patches that put Nova under apache wsgi for
the API, and there are a few things that I think are going in the wrong
direction. Largely I think because they were copied from the
lib/keystone code, which we've learned is kind of the wrong direction.
The first is the fact that a big reason for putting {SERVICES} under
apache wsgi is we aren't running on a ton of weird unregistered ports.
We're running on 80 and 443 (when appropriate). In order to do this we
really need to namespace the API urls. Which means that service catalog
needs to be updated appropriately.
I'd expect nova to be running on http://localhost/compute not
http://localhost:8774 when running under wsgi. That's going to probably
interestingly break a lot of weird assumptions by different projects,
but that's part of the reason for doing this exercise. Things should be
using the service catalog, and when they aren't, we need to figure it out.
(Exceptions can be made for third party APIs that don't work this way,
like the metadata server).
I also think this -
https://github.com/openstack-dev/devstack/blob/master/lib/nova#L266-L268
is completely wrong.
The Apache configs should instead specify access rules such that the
installed console entry point of nova-api can be used in place as the
WSGIScript.
This should also make lines like -
https://github.com/openstack-dev/devstack/blob/master/lib/nova#L272 and
L274 uneeded. (The WSGI Script will be in a known place). It will also
make upgrades much more friendly.
I think that we need to get these things sorted before any further
progression here. Volunteers welcomed to help get us there.
-Sean
--
Sean Dague
http://dague.net
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [devstack] apache wsgi application support
Long term we want to see Keystone move to http://host/identity. However the
reason for choosing 5000/35357 for ports was compatibility and avoiding
breaking horizon. At the time we did the initial change over, sharing the root
80/443 ports with horizon was more than challenging since horizon needed to
be based at /.
If that issue/assumption for horizon is no longer present, moving keystone to
be on port 80/443 would be doable. The last factor is that keystone was an a
priori knowledge for discovering other services. As long as we update docs
(possibly 302? For a cycle in devstack from the alternate ports) I think we're
good to make the change.
--Morgan
Sent via mobile
On Jun 16, 2015, at 09:25, Sean Dague s...@dague.net wrote:
I was just looking at the patches that put Nova under apache wsgi for
the API, and there are a few things that I think are going in the wrong
direction. Largely I think because they were copied from the
lib/keystone code, which we've learned is kind of the wrong direction.
The first is the fact that a big reason for putting {SERVICES} under
apache wsgi is we aren't running on a ton of weird unregistered ports.
We're running on 80 and 443 (when appropriate). In order to do this we
really need to namespace the API urls. Which means that service catalog
needs to be updated appropriately.
I'd expect nova to be running on http://localhost/compute not
http://localhost:8774 when running under wsgi. That's going to probably
interestingly break a lot of weird assumptions by different projects,
but that's part of the reason for doing this exercise. Things should be
using the service catalog, and when they aren't, we need to figure it out.
(Exceptions can be made for third party APIs that don't work this way,
like the metadata server).
I also think this -
https://github.com/openstack-dev/devstack/blob/master/lib/nova#L266-L268
is completely wrong.
The Apache configs should instead specify access rules such that the
installed console entry point of nova-api can be used in place as the
WSGIScript.
This should also make lines like -
https://github.com/openstack-dev/devstack/blob/master/lib/nova#L272 and
L274 uneeded. (The WSGI Script will be in a known place). It will also
make upgrades much more friendly.
I think that we need to get these things sorted before any further
progression here. Volunteers welcomed to help get us there.
-Sean
--
Sean Dague
http://dague.net
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [devstack] apache wsgi application support
On Tue, 16 Jun 2015, Sean Dague wrote:
I was just looking at the patches that put Nova under apache wsgi for
the API, and there are a few things that I think are going in the wrong
direction. Largely I think because they were copied from the
lib/keystone code, which we've learned is kind of the wrong direction.
Yes, that's certainly what I've done the few times I've done it.
devstack is deeply encouraging of cargo culting for reasons that are
not entirely clear.
The first is the fact that a big reason for putting {SERVICES} under
apache wsgi is we aren't running on a ton of weird unregistered ports.
We're running on 80 and 443 (when appropriate). In order to do this we
really need to namespace the API urls. Which means that service catalog
needs to be updated appropriately.
So:
a) I'm very glad to hear of this. I've been bristling about the weird
ports thing for the last year.
b) You make it sound like there's been a plan in place to not use
those ports for quite some time and we'd get to that when we all
had some spare time. Where do I go to keep abreast of such plans?
I also think this -
https://github.com/openstack-dev/devstack/blob/master/lib/nova#L266-L268
is completely wrong.
The Apache configs should instead specify access rules such that the
installed console entry point of nova-api can be used in place as the
WSGIScript.
I'm not able to parse this paragraph in any actionable way. The lines
you reference are one of several ways of telling mod wsgi where the
virtualenv is, which has to happen in some fashion if you are using
a virtualenv.
This doesn't appear to have anything to do with locating the module
that contains the WSGI app, so I'm missing the connection. Can you
explain please?
(Basically I'm keen on getting gnocchi and ceilometer wsgi servers
in devstack aligned with whatever the end game is, so knowing the plan
makes it a bit easier.)
This should also make lines like -
https://github.com/openstack-dev/devstack/blob/master/lib/nova#L272 and
L274 uneeded. (The WSGI Script will be in a known place). It will also
make upgrades much more friendly.
It sounds like maybe you are saying that the api console script and
the module containing the wsgi 'application' variable ought to be the
same thing. I don't reckon that's a great idea as the api console
scripts will want to import a bunch of stuff that the wsgi application
will not.
Or I may be completely misreading you. It's been a long day, etc.
I think that we need to get these things sorted before any further
progression here. Volunteers welcomed to help get us there.
Find me, happy to help. The sooner we can kill wacky port weirdness
the better.
--
Chris Dent tw:@anticdent freenode:cdent
https://tank.peermore.com/tanks/cdent
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [devstack] apache wsgi application support
On 06/16/2015 12:25 PM, Sean Dague wrote:
I was just looking at the patches that put Nova under apache wsgi for
the API, and there are a few things that I think are going in the wrong
direction. Largely I think because they were copied from the
lib/keystone code, which we've learned is kind of the wrong direction.
The first is the fact that a big reason for putting {SERVICES} under
apache wsgi is we aren't running on a ton of weird unregistered ports.
We're running on 80 and 443 (when appropriate). In order to do this we
really need to namespace the API urls. Which means that service catalog
needs to be updated appropriately.
I'd expect nova to be running on http://localhost/compute not
YES!
I had writtten this up for just this reason:
https://wiki.openstack.org/URLs
Lets make that the cannonical list.
Keystone suffers from the fact that the AUTH_URL is composed lots of
places, and people hard coded port 5000 in...I would like that to die.
http://localhost:8774 when running under wsgi. That's going to probably
interestingly break a lot of weird assumptions by different projects,
but that's part of the reason for doing this exercise. Things should be
using the service catalog, and when they aren't, we need to figure it out.
Amen!
(Exceptions can be made for third party APIs that don't work this way,
like the metadata server).
I also think this -
https://github.com/openstack-dev/devstack/blob/master/lib/nova#L266-L268
is completely wrong.
The Apache configs should instead specify access rules such that the
installed console entry point of nova-api can be used in place as the
WSGIScript.
This should also make lines like -
https://github.com/openstack-dev/devstack/blob/master/lib/nova#L272 and
L274 uneeded. (The WSGI Script will be in a known place). It will also
make upgrades much more friendly.
I think that we need to get these things sorted before any further
progression here. Volunteers welcomed to help get us there.
-Sean
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [devstack] apache wsgi application support
On 06/16/2015 12:48 PM, Morgan Fainberg wrote:
Long term we want to see Keystone move to http://host/identity. However the reason for choosing
5000/35357 for ports was compatibility and avoiding breaking horizon. At the time we did the initial
change over, sharing the root 80/443 ports with horizon was more than challenging since
horizon needed to be based at /.
If that issue/assumption for horizon is no longer present, moving keystone to
be on port 80/443 would be doable. The last factor is that keystone was an a
priori knowledge for discovering other services. As long as we update docs
(possibly 302? For a cycle in devstack from the alternate ports) I think we're
good to make the change.
The change to do this made its way into Horizon (courtesy of Matt Runge)
and is in devstack as well, I think. You need to specify WEBROOT for
the Horizon install.
--Morgan
Sent via mobile
On Jun 16, 2015, at 09:25, Sean Dague s...@dague.net wrote:
I was just looking at the patches that put Nova under apache wsgi for
the API, and there are a few things that I think are going in the wrong
direction. Largely I think because they were copied from the
lib/keystone code, which we've learned is kind of the wrong direction.
The first is the fact that a big reason for putting {SERVICES} under
apache wsgi is we aren't running on a ton of weird unregistered ports.
We're running on 80 and 443 (when appropriate). In order to do this we
really need to namespace the API urls. Which means that service catalog
needs to be updated appropriately.
I'd expect nova to be running on http://localhost/compute not
http://localhost:8774 when running under wsgi. That's going to probably
interestingly break a lot of weird assumptions by different projects,
but that's part of the reason for doing this exercise. Things should be
using the service catalog, and when they aren't, we need to figure it out.
(Exceptions can be made for third party APIs that don't work this way,
like the metadata server).
I also think this -
https://github.com/openstack-dev/devstack/blob/master/lib/nova#L266-L268
is completely wrong.
The Apache configs should instead specify access rules such that the
installed console entry point of nova-api can be used in place as the
WSGIScript.
This should also make lines like -
https://github.com/openstack-dev/devstack/blob/master/lib/nova#L272 and
L274 uneeded. (The WSGI Script will be in a known place). It will also
make upgrades much more friendly.
I think that we need to get these things sorted before any further
progression here. Volunteers welcomed to help get us there.
-Sean
--
Sean Dague
http://dague.net
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
