Re: [openstack-dev] [all][zaqar][cloudkitty] Default ports list

[Brant Knudson]

On Thu, Mar 10, 2016 at 10:43 AM, Brant Knudson  wrote:

>
>
> On Thu, Mar 10, 2016 at 8:10 AM, Thomas Herve  wrote:
>
>> On Thu, Mar 10, 2016 at 2:55 PM, Sean Dague  wrote:
>> > On 03/10/2016 08:40 AM, Thomas Herve wrote:
>> >> On Thu, Mar 10, 2016 at 2:28 PM, Chris Dent 
>> wrote:
>> >>> +many. It would be great if we just got rid of the runnable web
>> >>> servers in the projects and just expose wsgi apps (the tools like
>> >>> devstack etc mounted under whatever the available server is).
>> >>
>> >> Isn't devstack meant for development? Running the APIs in a WSGI
>> >> container like Apache or uwsgi makes for a terrible debugging
>> >> experience. Just this morning I had to prevent aodh from running in
>> >> Apache to be able to run it standalone.
>> >>
>> >> Also, those apps that use WSGI still bind a different port. The fact
>> >> that it runs in Apache doesn't really solve the URLs problem.
>> >
>> > But they shouldn't. I do realize it's taking a while to get there, but
>> > this is the push to get rid of all these weird ports. The point is to
>> > get them all on 80 in a subpath or a separate domain.
>> >
>> >
>> > If projects use the pbr wsgi_script directive the wsgi script will run
>> > on wsgi ref on the commandline if you need that for debug -
>> >
>> https://github.com/openstack/keystone/blob/4db54810e58ad86edb92869a608fb5630a6b99e5/setup.cfg#L75
>> > that was built to make this simpler.
>>
>> Right, but if we move to everything in WSGI besides a subpath, you
>> won't be able to switch to the script easily. Unless you actually
>> implement that using a reverse proxy.
>>
>> I totally understand the will to remove those ports from the final
>> product. I question whether it's the right choice for devstack. It
>> would seem that at least keeping those 'weird' ports for internal
>> consumption would be useful. It's not like we're close to use the 65K
>> available.
>>
>> --
>> Thomas
>>
>>
> For some time, devstack has been running with keystone accepting on both
> it's legacy ports (:5000 and :35357), and on subpaths (:80/identity and
> /identity_admin). I tried to change devstack to have keystone only
> listening on subpaths but this is failing in tempest-lib -- see the errors
> in https://review.openstack.org/#/c/193894/. At first I thought it would
> be best to get tempest-lib doing version discovery but this turned into a
> lot of code since version discovery requires quite a bit of parsing and
> searching through dicts which always requires lots of error handling and
> test code (see reviews ending at https://review.openstack.org/#/c/263452/).
> (You might wonder why I didn't use the code that's already in
> keystoneclient/keystoneauth for version discovery -- it's because tempest
> doesn't use the client libs.)
>
> Anyways I think the alternative that's going to be backwards compatible is
> to have the user be able to pass in the identity endpoints for v2 and v3
> and have tempest use those if provided. I haven't had time to propose this
> yet.
>
> My preference would be to have the keystone processes running separately
> under uwsgi or gunicorn, then httpd proxies to that from /identity and
> /identity_admin (and :5000 and :35357 for legacy). Hopefully this would be
> over a unix socket talking uwsgi protocol or whatever the process and httpd
> support. Note that devstack already has a TLS-proxy deployment option
> that's similar. I've been hoping to have time to propose the keystone
> devstack deployment use this but I'm easily distracted.
>
>
Here's the proposed change to devstack to allow keystone running under
uwsgi while apache accepts http / https (using mod_uwsgi_proxy) if you want
to take a look: https://review.openstack.org/#/c/291817/

We've got a few too many deployment options for keystone devstack now. I'd
like to get this down to just mod_wsgi and mod_uwsgi_proxy. The uwsgi
deploy we should be able to get rid of quickly, just add a non-voting job
in keystone for mod_uwsgi_proxy and get rid of the uwsgi job. We should be
able to get rid of eventlet right away in N since it's not supported to run
eventlet keystone anymore.

Next, we should be able to simplify the tls-proxy deployment since keystone
is always running in apache.

Then simplify further: Get rid of the tls-proxy option and instead have
keystone apache listen on https + http by default. Then eventually stop
listening on http and run https all the time.

- Brant
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][zaqar][cloudkitty] Default ports list

[Adam Young]

On 03/09/2016 04:35 PM, Fei Long Wang wrote:

Hi all,

Yesterday I just found cloudkitty is using the same default port 
() which is used by Zaqar now. So I'm wondering if there is any 
rule/policy for those new services need to be aware. I googled but 
can't find anything about this. The only link I can find is 
http://docs.openstack.org/liberty/config-reference/content/firewalls-default-ports.html. 
So my question is should we document the default ports list on an 
official place given the big tent mode? Thanks.



Run it on port 443.  In Apache HTTPD. Avoid the madness.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][zaqar][cloudkitty] Default ports list

[Morgan Fainberg]

On Thu, Mar 10, 2016 at 1:54 PM, Xav Paice  wrote:

>
>
> On 11 March 2016 at 10:45, Morgan Fainberg 
> wrote:
>
>>
>>
>> On Thu, Mar 10, 2016 at 1:29 PM, Xav Paice  wrote:
>>
>>>
>>>
>>> A simple list is probably enough for a quick ref - it's not a massive
>>> blocker if two projects slip up and get the same port number, and yes if
>>> they're doing subpaths and not ports then great.  Doesn't need to be a gate
>>> item.  But it helps communications if we have a list, even if that's
>>> temporary.
>>>
>>>
>> I really disagree that it doesn't need to be a gate item. It should
>> absolutely be gated on that services can run as a subpath.
>>
>
> Ah - yes I didn't make that very clear at all, did I  I don't think
> that selecting port numbers that don't conflict with other projects should
> be a gate item, simply because it's easy to change, and we're trying to
> steer away from that model.  I'm OK with that.  But yes, subpaths being
> unique should be gated otherwise we'll have all sorts of pain.
>

Fantastic! We're on the same page then. :)
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][zaqar][cloudkitty] Default ports list

[Xav Paice]

On 11 March 2016 at 10:45, Morgan Fainberg 
wrote:

>
>
> On Thu, Mar 10, 2016 at 1:29 PM, Xav Paice  wrote:
>
>>
>>
>> A simple list is probably enough for a quick ref - it's not a massive
>> blocker if two projects slip up and get the same port number, and yes if
>> they're doing subpaths and not ports then great.  Doesn't need to be a gate
>> item.  But it helps communications if we have a list, even if that's
>> temporary.
>>
>>
> I really disagree that it doesn't need to be a gate item. It should
> absolutely be gated on that services can run as a subpath.
>

Ah - yes I didn't make that very clear at all, did I  I don't think
that selecting port numbers that don't conflict with other projects should
be a gate item, simply because it's easy to change, and we're trying to
steer away from that model.  I'm OK with that.  But yes, subpaths being
unique should be gated otherwise we'll have all sorts of pain.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][zaqar][cloudkitty] Default ports list

[Morgan Fainberg]

On Thu, Mar 10, 2016 at 1:29 PM, Xav Paice  wrote:

> Remember that we're talking here about all the projects, not just
> keystone.  I can't see that we'll move everything to subpaths at any time
> soon, and until that point we still need to at least make an informal
> agreement as to which 'default' port to expect services to live on.  Even
> if that's just for devstack until we get to the subpaths nirvana.
>
> It's great that services are looking to the catalog for the locations of
> endpoints - but unless we're comfortable that every cloud is going to
> select a bunch of (different) random ports for each service until such time
> as subpaths are a reality, then we need to communicate in some way.
>
> I don't see the need for a full web service environment in devstack - all
> that would really do is limit the choices that ops can make about the best
> web server/wsgi service.  If people concentrate efforts on apache/mod_wsgi,
> those wanting to use uwsgi, nginx, gunicorn, etc are going to have a hard
> road.  There's valid choices for using other web services (in fact, there's
> some massive arguments against mod_wsgi).
>
> A simple list is probably enough for a quick ref - it's not a massive
> blocker if two projects slip up and get the same port number, and yes if
> they're doing subpaths and not ports then great.  Doesn't need to be a gate
> item.  But it helps communications if we have a list, even if that's
> temporary.
>
>
I really disagree that it doesn't need to be a gate item. It should
absolutely be gated on that services can run as a subpath.


> How about a 'default settings' list for a 'standard' reference
> environment?  When ops deviate from the list (and we will) that's a
> concious decision we make.  Should we say that the ports used in devstack
> are the default list, and if a new project wants to get into devstack then
> they need to choose a port not in use (or subpaths)?
>
> On 11 March 2016 at 05:43, Brant Knudson  wrote:
>
>>
>>
>> On Thu, Mar 10, 2016 at 8:10 AM, Thomas Herve  wrote:
>>
>>> On Thu, Mar 10, 2016 at 2:55 PM, Sean Dague  wrote:
>>> > On 03/10/2016 08:40 AM, Thomas Herve wrote:
>>> >> On Thu, Mar 10, 2016 at 2:28 PM, Chris Dent 
>>> wrote:
>>> >>> +many. It would be great if we just got rid of the runnable web
>>> >>> servers in the projects and just expose wsgi apps (the tools like
>>> >>> devstack etc mounted under whatever the available server is).
>>> >>
>>> >> Isn't devstack meant for development? Running the APIs in a WSGI
>>> >> container like Apache or uwsgi makes for a terrible debugging
>>> >> experience. Just this morning I had to prevent aodh from running in
>>> >> Apache to be able to run it standalone.
>>> >>
>>> >> Also, those apps that use WSGI still bind a different port. The fact
>>> >> that it runs in Apache doesn't really solve the URLs problem.
>>> >
>>> > But they shouldn't. I do realize it's taking a while to get there, but
>>> > this is the push to get rid of all these weird ports. The point is to
>>> > get them all on 80 in a subpath or a separate domain.
>>> >
>>> >
>>> > If projects use the pbr wsgi_script directive the wsgi script will run
>>> > on wsgi ref on the commandline if you need that for debug -
>>> >
>>> https://github.com/openstack/keystone/blob/4db54810e58ad86edb92869a608fb5630a6b99e5/setup.cfg#L75
>>> > that was built to make this simpler.
>>>
>>> Right, but if we move to everything in WSGI besides a subpath, you
>>> won't be able to switch to the script easily. Unless you actually
>>> implement that using a reverse proxy.
>>>
>>> I totally understand the will to remove those ports from the final
>>> product. I question whether it's the right choice for devstack. It
>>> would seem that at least keeping those 'weird' ports for internal
>>> consumption would be useful. It's not like we're close to use the 65K
>>> available.
>>>
>>> --
>>> Thomas
>>>
>>>
>> For some time, devstack has been running with keystone accepting on both
>> it's legacy ports (:5000 and :35357), and on subpaths (:80/identity and
>> /identity_admin). I tried to change devstack to have keystone only
>> listening on subpaths but this is failing in tempest-lib -- see the errors
>> in https://review.openstack.org/#/c/193894/. At first I thought it would
>> be best to get tempest-lib doing version discovery but this turned into a
>> lot of code since version discovery requires quite a bit of parsing and
>> searching through dicts which always requires lots of error handling and
>> test code (see reviews ending at https://review.openstack.org/#/c/263452/).
>> (You might wonder why I didn't use the code that's already in
>> keystoneclient/keystoneauth for version discovery -- it's because tempest
>> doesn't use the client libs.)
>>
>> Anyways I think the alternative that's going to be backwards compatible
>> is to have the user be able to pass in the identity endpoints for 

Re: [openstack-dev] [all][zaqar][cloudkitty] Default ports list

[Matt Fischer]

On Thu, Mar 10, 2016 at 2:29 PM, Xav Paice  wrote:

> Remember that we're talking here about all the projects, not just
> keystone.  I can't see that we'll move everything to subpaths at any time
> soon, and until that point we still need to at least make an informal
> agreement as to which 'default' port to expect services to live on.  Even
> if that's just for devstack until we get to the subpaths nirvana.
>
> It's great that services are looking to the catalog for the locations of
> endpoints - but unless we're comfortable that every cloud is going to
> select a bunch of (different) random ports for each service until such time
> as subpaths are a reality, then we need to communicate in some way.
>
> I don't see the need for a full web service environment in devstack - all
> that would really do is limit the choices that ops can make about the best
> web server/wsgi service.  If people concentrate efforts on apache/mod_wsgi,
> those wanting to use uwsgi, nginx, gunicorn, etc are going to have a hard
> road.  There's valid choices for using other web services (in fact, there's
> some massive arguments against mod_wsgi).
>
> A simple list is probably enough for a quick ref - it's not a massive
> blocker if two projects slip up and get the same port number, and yes if
> they're doing subpaths and not ports then great.  Doesn't need to be a gate
> item.  But it helps communications if we have a list, even if that's
> temporary.
>
> How about a 'default settings' list for a 'standard' reference
> environment?  When ops deviate from the list (and we will) that's a
> concious decision we make.  Should we say that the ports used in devstack
> are the default list, and if a new project wants to get into devstack then
> they need to choose a port not in use (or subpaths)?
>
>
+1. This is how we would set the default values in things like the puppet
modules. It's a not a good experience if two puppet modules out of the box
collide with each other. As you said operators can make whatever choices
they want later, but lets make it a conscious decision to deviate from the
standard list rather than presenting someone standing up OpenStack a port
collision and leaving them to search for something open to use.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][zaqar][cloudkitty] Default ports list

[Morgan Fainberg]

On Thu, Mar 10, 2016 at 4:43 AM, Sean Dague <s...@dague.net> wrote:

> On 03/10/2016 07:11 AM, Tim Bell wrote:
> >
> >
> > From: Sylvain Bauza <sba...@redhat.com <mailto:sba...@redhat.com>>
> > Reply-To: "OpenStack Development Mailing List (not for usage questions)"
> > <openstack-dev@lists.openstack.org
> > <mailto:openstack-dev@lists.openstack.org>>
> > Date: Thursday 10 March 2016 at 10:04
> > To: "OpenStack Development Mailing List (not for usage questions)"
> > <openstack-dev@lists.openstack.org
> > <mailto:openstack-dev@lists.openstack.org>>
> > Subject: Re: [openstack-dev] [all][zaqar][cloudkitty] Default ports list
> >
> >
> >
> > Le 09/03/2016 23:41, Matt Fischer a écrit :
> >> This is not the first time. Monasca and Murano had a collision
> >> too[1]. When this happens the changes trickle down into automation
> >> tools also and complicates things.
> >>
> >> [1] https://bugs.launchpad.net/murano/+bug/1505785
> >
> >
> > IMHO, all that info has to be standardized in the Service Catalog.
> > That's where endpoint informations can be found for a specific
> > service type and that's the basement for cross-project communication.
> >
> > FWIW, there is one cross-project spec trying to clean-up the
> > per-project bits that are not common
> >
> https://github.com/openstack/openstack-specs/blob/master/specs/service-catalog.rst
> >
> > I'm torn between 2 opinions :
> >  - either we consider that all those endpoints are (or should be -
> > for those which aren't) manageable thru config options, and thus
> > that's not a problem we should solve. Any operator can then modify
> > the ports to make sure that two conflicting big-tent projects can
> > work together.
> >  - or, we say that it can be a concern for interoperability, and
> > then we should somehow ensure that all projects can work together.
> > Then, a documentation link isn't enough IMHO, we should rather test
> > that.
> >
> >
> > If we can make it so that there are reasonable port commonalities
> > between OpenStack clouds, this would be good. Clearly, the service
> > catalog is the master so I don’t think there is an interoperability
> > concern but having each of the projects using different ports would
> > simplify some of the smaller configurations with multiple services on a
> > single box.
> >
> > This does assume that there are less big tent projects than available
> > TCP/IP ports :-)
>
> These are HTTP services. They really shoudn't be claiming new ports,
> they should be running on a real webserver on 80 or 443.
>
>
100% Correct. Services should be looking to exist on the proper HTTP ports
instead of custom ports. The "custom" ports used in the past are legacy and
we should encourage moving away from that. The only IANA assigned port is
the keystone 35357, and that comes with a host of other issues (Linux
Ephemeral Port range); we are actively working to get keystone moved to
port 80 or 443 as the default/recommended method. There are a number of
deployers/cloud operators that already run all services on 443 (using TLS
and apache/nginx to front the service).


> There is some legacy there with the original services that we are
> churning through. It would be nice if new services *started* with
> staying on wsgi only, and stopped grabbing random ports.


++
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][zaqar][cloudkitty] Default ports list

[Xav Paice]

On 10 March 2016 at 10:35, Fei Long Wang  wrote:

> The only link I can find is
> http://docs.openstack.org/liberty/config-reference/content/firewalls-default-ports.html.
> So my question is should we document the default ports list on an official
> place given the big tent mode? Thanks.
>


Maybe what we should do is just keep the documentation up to date by adding
more projects to that list?
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][zaqar][cloudkitty] Default ports list

[Xav Paice]

Remember that we're talking here about all the projects, not just
keystone.  I can't see that we'll move everything to subpaths at any time
soon, and until that point we still need to at least make an informal
agreement as to which 'default' port to expect services to live on.  Even
if that's just for devstack until we get to the subpaths nirvana.

It's great that services are looking to the catalog for the locations of
endpoints - but unless we're comfortable that every cloud is going to
select a bunch of (different) random ports for each service until such time
as subpaths are a reality, then we need to communicate in some way.

I don't see the need for a full web service environment in devstack - all
that would really do is limit the choices that ops can make about the best
web server/wsgi service.  If people concentrate efforts on apache/mod_wsgi,
those wanting to use uwsgi, nginx, gunicorn, etc are going to have a hard
road.  There's valid choices for using other web services (in fact, there's
some massive arguments against mod_wsgi).

A simple list is probably enough for a quick ref - it's not a massive
blocker if two projects slip up and get the same port number, and yes if
they're doing subpaths and not ports then great.  Doesn't need to be a gate
item.  But it helps communications if we have a list, even if that's
temporary.

How about a 'default settings' list for a 'standard' reference
environment?  When ops deviate from the list (and we will) that's a
concious decision we make.  Should we say that the ports used in devstack
are the default list, and if a new project wants to get into devstack then
they need to choose a port not in use (or subpaths)?

On 11 March 2016 at 05:43, Brant Knudson  wrote:

>
>
> On Thu, Mar 10, 2016 at 8:10 AM, Thomas Herve  wrote:
>
>> On Thu, Mar 10, 2016 at 2:55 PM, Sean Dague  wrote:
>> > On 03/10/2016 08:40 AM, Thomas Herve wrote:
>> >> On Thu, Mar 10, 2016 at 2:28 PM, Chris Dent 
>> wrote:
>> >>> +many. It would be great if we just got rid of the runnable web
>> >>> servers in the projects and just expose wsgi apps (the tools like
>> >>> devstack etc mounted under whatever the available server is).
>> >>
>> >> Isn't devstack meant for development? Running the APIs in a WSGI
>> >> container like Apache or uwsgi makes for a terrible debugging
>> >> experience. Just this morning I had to prevent aodh from running in
>> >> Apache to be able to run it standalone.
>> >>
>> >> Also, those apps that use WSGI still bind a different port. The fact
>> >> that it runs in Apache doesn't really solve the URLs problem.
>> >
>> > But they shouldn't. I do realize it's taking a while to get there, but
>> > this is the push to get rid of all these weird ports. The point is to
>> > get them all on 80 in a subpath or a separate domain.
>> >
>> >
>> > If projects use the pbr wsgi_script directive the wsgi script will run
>> > on wsgi ref on the commandline if you need that for debug -
>> >
>> https://github.com/openstack/keystone/blob/4db54810e58ad86edb92869a608fb5630a6b99e5/setup.cfg#L75
>> > that was built to make this simpler.
>>
>> Right, but if we move to everything in WSGI besides a subpath, you
>> won't be able to switch to the script easily. Unless you actually
>> implement that using a reverse proxy.
>>
>> I totally understand the will to remove those ports from the final
>> product. I question whether it's the right choice for devstack. It
>> would seem that at least keeping those 'weird' ports for internal
>> consumption would be useful. It's not like we're close to use the 65K
>> available.
>>
>> --
>> Thomas
>>
>>
> For some time, devstack has been running with keystone accepting on both
> it's legacy ports (:5000 and :35357), and on subpaths (:80/identity and
> /identity_admin). I tried to change devstack to have keystone only
> listening on subpaths but this is failing in tempest-lib -- see the errors
> in https://review.openstack.org/#/c/193894/. At first I thought it would
> be best to get tempest-lib doing version discovery but this turned into a
> lot of code since version discovery requires quite a bit of parsing and
> searching through dicts which always requires lots of error handling and
> test code (see reviews ending at https://review.openstack.org/#/c/263452/).
> (You might wonder why I didn't use the code that's already in
> keystoneclient/keystoneauth for version discovery -- it's because tempest
> doesn't use the client libs.)
>
> Anyways I think the alternative that's going to be backwards compatible is
> to have the user be able to pass in the identity endpoints for v2 and v3
> and have tempest use those if provided. I haven't had time to propose this
> yet.
>
> My preference would be to have the keystone processes running separately
> under uwsgi or gunicorn, then httpd proxies to that from /identity and
> /identity_admin (and :5000 and :35357 for legacy). Hopefully this would be

Re: [openstack-dev] [all][zaqar][cloudkitty] Default ports list

[Brant Knudson]

On Thu, Mar 10, 2016 at 8:10 AM, Thomas Herve  wrote:

> On Thu, Mar 10, 2016 at 2:55 PM, Sean Dague  wrote:
> > On 03/10/2016 08:40 AM, Thomas Herve wrote:
> >> On Thu, Mar 10, 2016 at 2:28 PM, Chris Dent 
> wrote:
> >>> +many. It would be great if we just got rid of the runnable web
> >>> servers in the projects and just expose wsgi apps (the tools like
> >>> devstack etc mounted under whatever the available server is).
> >>
> >> Isn't devstack meant for development? Running the APIs in a WSGI
> >> container like Apache or uwsgi makes for a terrible debugging
> >> experience. Just this morning I had to prevent aodh from running in
> >> Apache to be able to run it standalone.
> >>
> >> Also, those apps that use WSGI still bind a different port. The fact
> >> that it runs in Apache doesn't really solve the URLs problem.
> >
> > But they shouldn't. I do realize it's taking a while to get there, but
> > this is the push to get rid of all these weird ports. The point is to
> > get them all on 80 in a subpath or a separate domain.
> >
> >
> > If projects use the pbr wsgi_script directive the wsgi script will run
> > on wsgi ref on the commandline if you need that for debug -
> >
> https://github.com/openstack/keystone/blob/4db54810e58ad86edb92869a608fb5630a6b99e5/setup.cfg#L75
> > that was built to make this simpler.
>
> Right, but if we move to everything in WSGI besides a subpath, you
> won't be able to switch to the script easily. Unless you actually
> implement that using a reverse proxy.
>
> I totally understand the will to remove those ports from the final
> product. I question whether it's the right choice for devstack. It
> would seem that at least keeping those 'weird' ports for internal
> consumption would be useful. It's not like we're close to use the 65K
> available.
>
> --
> Thomas
>
>
For some time, devstack has been running with keystone accepting on both
it's legacy ports (:5000 and :35357), and on subpaths (:80/identity and
/identity_admin). I tried to change devstack to have keystone only
listening on subpaths but this is failing in tempest-lib -- see the errors
in https://review.openstack.org/#/c/193894/. At first I thought it would be
best to get tempest-lib doing version discovery but this turned into a lot
of code since version discovery requires quite a bit of parsing and
searching through dicts which always requires lots of error handling and
test code (see reviews ending at https://review.openstack.org/#/c/263452/).
(You might wonder why I didn't use the code that's already in
keystoneclient/keystoneauth for version discovery -- it's because tempest
doesn't use the client libs.)

Anyways I think the alternative that's going to be backwards compatible is
to have the user be able to pass in the identity endpoints for v2 and v3
and have tempest use those if provided. I haven't had time to propose this
yet.

My preference would be to have the keystone processes running separately
under uwsgi or gunicorn, then httpd proxies to that from /identity and
/identity_admin (and :5000 and :35357 for legacy). Hopefully this would be
over a unix socket talking uwsgi protocol or whatever the process and httpd
support. Note that devstack already has a TLS-proxy deployment option
that's similar. I've been hoping to have time to propose the keystone
devstack deployment use this but I'm easily distracted.

- Brant
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][zaqar][cloudkitty] Default ports list

[Thomas Herve]

On Thu, Mar 10, 2016 at 2:55 PM, Sean Dague  wrote:
> On 03/10/2016 08:40 AM, Thomas Herve wrote:
>> On Thu, Mar 10, 2016 at 2:28 PM, Chris Dent  wrote:
>>> +many. It would be great if we just got rid of the runnable web
>>> servers in the projects and just expose wsgi apps (the tools like
>>> devstack etc mounted under whatever the available server is).
>>
>> Isn't devstack meant for development? Running the APIs in a WSGI
>> container like Apache or uwsgi makes for a terrible debugging
>> experience. Just this morning I had to prevent aodh from running in
>> Apache to be able to run it standalone.
>>
>> Also, those apps that use WSGI still bind a different port. The fact
>> that it runs in Apache doesn't really solve the URLs problem.
>
> But they shouldn't. I do realize it's taking a while to get there, but
> this is the push to get rid of all these weird ports. The point is to
> get them all on 80 in a subpath or a separate domain.
>
>
> If projects use the pbr wsgi_script directive the wsgi script will run
> on wsgi ref on the commandline if you need that for debug -
> https://github.com/openstack/keystone/blob/4db54810e58ad86edb92869a608fb5630a6b99e5/setup.cfg#L75
> that was built to make this simpler.

Right, but if we move to everything in WSGI besides a subpath, you
won't be able to switch to the script easily. Unless you actually
implement that using a reverse proxy.

I totally understand the will to remove those ports from the final
product. I question whether it's the right choice for devstack. It
would seem that at least keeping those 'weird' ports for internal
consumption would be useful. It's not like we're close to use the 65K
available.

-- 
Thomas

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][zaqar][cloudkitty] Default ports list

[Julien Danjou]

On Thu, Mar 10 2016, Thomas Herve wrote:

> Isn't devstack meant for development? Running the APIs in a WSGI
> container like Apache or uwsgi makes for a terrible debugging
> experience. Just this morning I had to prevent aodh from running in
> Apache to be able to run it standalone.

That shouldn't be the case, debugging using Apache shouldn't be much
harder to debug. Could you elaborate on the issues you encountered so we
can possibly improve the situation?

> Also, those apps that use WSGI still bind a different port. The fact
> that it runs in Apache doesn't really solve the URLs problem.

They should not, I know some projects (Keystone at least since I fixed
it, and Gnocchi) are able to run under a prefix on standard ports.

Cheers,
-- 
Julien Danjou
;; Free Software hacker
;; https://julien.danjou.info


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][zaqar][cloudkitty] Default ports list

[Sean Dague]

On 03/10/2016 08:40 AM, Thomas Herve wrote:
> On Thu, Mar 10, 2016 at 2:28 PM, Chris Dent  wrote:
>> On Thu, 10 Mar 2016, Sean Dague wrote:
>>
>>> These are HTTP services. They really shoudn't be claiming new ports,
>>> they should be running on a real webserver on 80 or 443.
>>>
>>> There is some legacy there with the original services that we are
>>> churning through. It would be nice if new services *started* with
>>> staying on wsgi only, and stopped grabbing random ports.
>>
>>
>> +many. It would be great if we just got rid of the runnable web
>> servers in the projects and just expose wsgi apps (the tools like
>> devstack etc mounted under whatever the available server is).
> 
> Isn't devstack meant for development? Running the APIs in a WSGI
> container like Apache or uwsgi makes for a terrible debugging
> experience. Just this morning I had to prevent aodh from running in
> Apache to be able to run it standalone.
> 
> Also, those apps that use WSGI still bind a different port. The fact
> that it runs in Apache doesn't really solve the URLs problem.

But they shouldn't. I do realize it's taking a while to get there, but
this is the push to get rid of all these weird ports. The point is to
get them all on 80 in a subpath or a separate domain.

If projects use the pbr wsgi_script directive the wsgi script will run
on wsgi ref on the commandline if you need that for debug -
https://github.com/openstack/keystone/blob/4db54810e58ad86edb92869a608fb5630a6b99e5/setup.cfg#L75
that was built to make this simpler.

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][zaqar][cloudkitty] Default ports list

[Thomas Herve]

On Thu, Mar 10, 2016 at 2:28 PM, Chris Dent  wrote:
> On Thu, 10 Mar 2016, Sean Dague wrote:
>
>> These are HTTP services. They really shoudn't be claiming new ports,
>> they should be running on a real webserver on 80 or 443.
>>
>> There is some legacy there with the original services that we are
>> churning through. It would be nice if new services *started* with
>> staying on wsgi only, and stopped grabbing random ports.
>
>
> +many. It would be great if we just got rid of the runnable web
> servers in the projects and just expose wsgi apps (the tools like
> devstack etc mounted under whatever the available server is).

Isn't devstack meant for development? Running the APIs in a WSGI
container like Apache or uwsgi makes for a terrible debugging
experience. Just this morning I had to prevent aodh from running in
Apache to be able to run it standalone.

Also, those apps that use WSGI still bind a different port. The fact
that it runs in Apache doesn't really solve the URLs problem.

-- 
Thomas

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][zaqar][cloudkitty] Default ports list

[Flavio Percoco]

On 10/03/16 13:28 +, Chris Dent wrote:

On Thu, 10 Mar 2016, Sean Dague wrote:


These are HTTP services. They really shoudn't be claiming new ports,
they should be running on a real webserver on 80 or 443.

There is some legacy there with the original services that we are
churning through. It would be nice if new services *started* with
staying on wsgi only, and stopped grabbing random ports.


+many. It would be great if we just got rid of the runnable web
servers in the projects and just expose wsgi apps (the tools like
devstack etc mounted under whatever the available server is).


FWIW, Zaqar doesn't ship a wsgi container. It's a pure wsgi app. I think the
issue Fei Long is raising here is w.r.t the gate. From a Zaqar perspective, I
wouldn't be too worried and as far as the gate goes, I think it'd be more than
fine to just switch from using wsgiref in the gate to using httpd.

This is to say, I agree with Chris's and Sean's statements.


Also, as already stated sort of vaguely earlier in the thread, isn't
it the case that if you aren't using the service catalog to find an
endpoint, you're doing it wrong?


++

Flavio

--
@flaper87
Flavio Percoco


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][zaqar][cloudkitty] Default ports list

[Chris Dent]

On Thu, 10 Mar 2016, Sean Dague wrote:


These are HTTP services. They really shoudn't be claiming new ports,
they should be running on a real webserver on 80 or 443.

There is some legacy there with the original services that we are
churning through. It would be nice if new services *started* with
staying on wsgi only, and stopped grabbing random ports.


+many. It would be great if we just got rid of the runnable web
servers in the projects and just expose wsgi apps (the tools like
devstack etc mounted under whatever the available server is).

Also, as already stated sort of vaguely earlier in the thread, isn't
it the case that if you aren't using the service catalog to find an
endpoint, you're doing it wrong?

--
Chris Dent   (�s°□°)�s�喋擤ォ�http://anticdent.org/
freenode: cdent tw: @anticdent__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][zaqar][cloudkitty] Default ports list

[Sean Dague]

On 03/10/2016 07:11 AM, Tim Bell wrote:
> 
> 
> From: Sylvain Bauza <sba...@redhat.com <mailto:sba...@redhat.com>>
> Reply-To: "OpenStack Development Mailing List (not for usage questions)"
> <openstack-dev@lists.openstack.org
> <mailto:openstack-dev@lists.openstack.org>>
> Date: Thursday 10 March 2016 at 10:04
> To: "OpenStack Development Mailing List (not for usage questions)"
> <openstack-dev@lists.openstack.org
> <mailto:openstack-dev@lists.openstack.org>>
> Subject: Re: [openstack-dev] [all][zaqar][cloudkitty] Default ports list
> 
> 
> 
> Le 09/03/2016 23:41, Matt Fischer a écrit :
>> This is not the first time. Monasca and Murano had a collision
>> too[1]. When this happens the changes trickle down into automation
>> tools also and complicates things.
>>
>> [1] https://bugs.launchpad.net/murano/+bug/1505785
> 
> 
> IMHO, all that info has to be standardized in the Service Catalog.
> That's where endpoint informations can be found for a specific
> service type and that's the basement for cross-project communication.
> 
> FWIW, there is one cross-project spec trying to clean-up the
> per-project bits that are not common
> 
> https://github.com/openstack/openstack-specs/blob/master/specs/service-catalog.rst
> 
> I'm torn between 2 opinions :
>  - either we consider that all those endpoints are (or should be -
> for those which aren't) manageable thru config options, and thus
> that's not a problem we should solve. Any operator can then modify
> the ports to make sure that two conflicting big-tent projects can
> work together.
>  - or, we say that it can be a concern for interoperability, and
> then we should somehow ensure that all projects can work together.
> Then, a documentation link isn't enough IMHO, we should rather test
> that.
> 
> 
> If we can make it so that there are reasonable port commonalities
> between OpenStack clouds, this would be good. Clearly, the service
> catalog is the master so I don’t think there is an interoperability
> concern but having each of the projects using different ports would
> simplify some of the smaller configurations with multiple services on a
> single box. 
> 
> This does assume that there are less big tent projects than available
> TCP/IP ports :-)

These are HTTP services. They really shoudn't be claiming new ports,
they should be running on a real webserver on 80 or 443.

There is some legacy there with the original services that we are
churning through. It would be nice if new services *started* with
staying on wsgi only, and stopped grabbing random ports.

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][zaqar][cloudkitty] Default ports list

[Tim Bell]

From: Sylvain Bauza <sba...@redhat.com<mailto:sba...@redhat.com>>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>>
Date: Thursday 10 March 2016 at 10:04
To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>>
Subject: Re: [openstack-dev] [all][zaqar][cloudkitty] Default ports list



Le 09/03/2016 23:41, Matt Fischer a écrit :
This is not the first time. Monasca and Murano had a collision too[1]. When 
this happens the changes trickle down into automation tools also and 
complicates things.

[1] https://bugs.launchpad.net/murano/+bug/1505785


IMHO, all that info has to be standardized in the Service Catalog. That's where 
endpoint informations can be found for a specific service type and that's the 
basement for cross-project communication.

FWIW, there is one cross-project spec trying to clean-up the per-project bits 
that are not common 
https://github.com/openstack/openstack-specs/blob/master/specs/service-catalog.rst

I'm torn between 2 opinions :
 - either we consider that all those endpoints are (or should be - for those 
which aren't) manageable thru config options, and thus that's not a problem we 
should solve. Any operator can then modify the ports to make sure that two 
conflicting big-tent projects can work together.
 - or, we say that it can be a concern for interoperability, and then we should 
somehow ensure that all projects can work together. Then, a documentation link 
isn't enough IMHO, we should rather test that.


If we can make it so that there are reasonable port commonalities between 
OpenStack clouds, this would be good. Clearly, the service catalog is the 
master so I don’t think there is an interoperability concern but having each of 
the projects using different ports would simplify some of the smaller 
configurations with multiple services on a single box.

This does assume that there are less big tent projects than available TCP/IP 
ports :-)

Tim





On Wed, Mar 9, 2016 at 3:30 PM, Xav Paice 
<xavpa...@gmail.com<mailto:xavpa...@gmail.com>> wrote:
From an ops point of view, this would be extremely helpful information to share 
with various teams around an organization.  Even a simple wiki page would be 
great.

On 10 March 2016 at 10:35, Fei Long Wang 
<feil...@catalyst.net.nz<mailto:feil...@catalyst.net.nz>> wrote:
Hi all,

Yesterday I just found cloudkitty is using the same default port () which 
is used by Zaqar now. So I'm wondering if there is any rule/policy for those 
new services need to be aware. I googled but can't find anything about this. 
The only link I can find is 
http://docs.openstack.org/liberty/config-reference/content/firewalls-default-ports.html.
 So my question is should we document the default ports list on an official 
place given the big tent mode? Thanks.

--
Cheers & Best regards,
Fei Long Wang (王飞龙)
--
Senior Cloud Software Engineer
Tel: +64-48032246<tel:%2B64-48032246>
Email: flw...@catalyst.net.nz<mailto:flw...@catalyst.net.nz>
Catalyst IT Limited
Level 6, Catalyst House, 150 Willis Street, Wellington
--


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev





__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe<mailto:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][zaqar][cloudkitty] Default ports list

[Sylvain Bauza]

Le 09/03/2016 23:41, Matt Fischer a écrit :
This is not the first time. Monasca and Murano had a collision too[1]. 
When this happens the changes trickle down into automation tools also 
and complicates things.


[1] https://bugs.launchpad.net/murano/+bug/1505785



IMHO, all that info has to be standardized in the Service Catalog. 
That's where endpoint informations can be found for a specific service 
type and that's the basement for cross-project communication.


FWIW, there is one cross-project spec trying to clean-up the per-project 
bits that are not common 
https://github.com/openstack/openstack-specs/blob/master/specs/service-catalog.rst 



I'm torn between 2 opinions :
 - either we consider that all those endpoints are (or should be - for 
those which aren't) manageable thru config options, and thus that's not 
a problem we should solve. Any operator can then modify the ports to 
make sure that two conflicting big-tent projects can work together.
 - or, we say that it can be a concern for interoperability, and then 
we should somehow ensure that all projects can work together. Then, a 
documentation link isn't enough IMHO, we should rather test that.







On Wed, Mar 9, 2016 at 3:30 PM, Xav Paice > wrote:


From an ops point of view, this would be extremely helpful
information to share with various teams around an organization. 
Even a simple wiki page would be great.


On 10 March 2016 at 10:35, Fei Long Wang > wrote:

Hi all,

Yesterday I just found cloudkitty is using the same default
port () which is used by Zaqar now. So I'm wondering if
there is any rule/policy for those new services need to be
aware. I googled but can't find anything about this. The only
link I can find is

http://docs.openstack.org/liberty/config-reference/content/firewalls-default-ports.html.
So my question is should we document the default ports list on
an official place given the big tent mode? Thanks.

-- 
Cheers & Best regards,

Fei Long Wang (王飞龙)

--
Senior Cloud Software Engineer
Tel: +64-48032246 
Email: flw...@catalyst.net.nz 
Catalyst IT Limited
Level 6, Catalyst House, 150 Willis Street, Wellington

--



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][zaqar][cloudkitty] Default ports list

[Matt Fischer]

This is not the first time. Monasca and Murano had a collision too[1]. When
this happens the changes trickle down into automation tools also and
complicates things.

[1] https://bugs.launchpad.net/murano/+bug/1505785

On Wed, Mar 9, 2016 at 3:30 PM, Xav Paice  wrote:

> From an ops point of view, this would be extremely helpful information to
> share with various teams around an organization.  Even a simple wiki page
> would be great.
>
> On 10 March 2016 at 10:35, Fei Long Wang  wrote:
>
>> Hi all,
>>
>> Yesterday I just found cloudkitty is using the same default port ()
>> which is used by Zaqar now. So I'm wondering if there is any rule/policy
>> for those new services need to be aware. I googled but can't find anything
>> about this. The only link I can find is
>> http://docs.openstack.org/liberty/config-reference/content/firewalls-default-ports.html.
>> So my question is should we document the default ports list on an official
>> place given the big tent mode? Thanks.
>>
>> --
>> Cheers & Best regards,
>> Fei Long Wang (王飞龙)
>> --
>> Senior Cloud Software Engineer
>> Tel: +64-48032246
>> Email: flw...@catalyst.net.nz
>> Catalyst IT Limited
>> Level 6, Catalyst House, 150 Willis Street, Wellington
>> --
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][zaqar][cloudkitty] Default ports list

[Xav Paice]

>From an ops point of view, this would be extremely helpful information to
share with various teams around an organization.  Even a simple wiki page
would be great.

On 10 March 2016 at 10:35, Fei Long Wang  wrote:

> Hi all,
>
> Yesterday I just found cloudkitty is using the same default port ()
> which is used by Zaqar now. So I'm wondering if there is any rule/policy
> for those new services need to be aware. I googled but can't find anything
> about this. The only link I can find is
> http://docs.openstack.org/liberty/config-reference/content/firewalls-default-ports.html.
> So my question is should we document the default ports list on an official
> place given the big tent mode? Thanks.
>
> --
> Cheers & Best regards,
> Fei Long Wang (王飞龙)
> --
> Senior Cloud Software Engineer
> Tel: +64-48032246
> Email: flw...@catalyst.net.nz
> Catalyst IT Limited
> Level 6, Catalyst House, 150 Willis Street, Wellington
> --
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev