Re: [openstack-dev] [all][tc][ptls][glance] final stages of python 3 transition

2018-05-20 Thread Thomas Goirand 
On 05/14/2018 06:42 PM, Victoria Martínez de la Cruz wrote:
> Hi,
> 
> Jumping in now as I'm helping with py3 support efforts in the manila side.
> 
> In manila we have both support for Apache WSGI and the built-in server
> (which depends in eventlet). Would it be a possible workaround to rely
> on the Apache WSGI server while we wait for evenlet issues to be sorted
> out? Is there any chance the upper constraints will be updated soon-ish
> and this can be fixed in a newer eventlet version?

Probably we can update the upper-constraints file, though the newer
eventlet doesn't have a fix. It's been more than 2 years there's the
issue, and nobody seems to work on it.

> This is the only change it's preventing us to be fully py3 compatible,
> hence it's a big deal for us.

I don't think Eventlet is a blocker, as long as you're supporting uwsgi
and Apache2. The case of Glance not supporting Apache2 is a real issue
though, for Ubuntu at least, since they don't want uwsgi to be promoted
to main (ie: it's in Universe, and they don't support it for security).

As for Debian, since it is looking like I managed to find a solution for
running everything in uwsgi, then I probably will do that. I have btw
recently joined the team maintaining uwsgi in Debian, and managed to fix
all RC bugs on it.

Like Matt, I do prefer to be able to restart only *one* daemon at a
time, which is why I don't really like setting-up everything with
mod_wsgi. Though in such setup, I wonder what the point is to still use
Apache for proxying the requests. Is there any added value doing that?

Also, does anyone know if uwsgi uses the Python subinterpreter thing,
which is the reason why mod_wsgi is outperforming everything else? If
I'm not mistaking, the API is described at:
https://www.python.org/dev/peps/pep-0554/ (ie: PEP554), and that, if I
understand correctly, works around the global interpreter lock issue. As
much as I could see, uwsgi doesn't use that, so Apache should still be
outperforming uwsgi.

Cheers,

Thomas Goirand (zigo)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc][ptls][glance] final stages of python 3 transition

2018-05-20 Thread Thomas Goirand 
On 05/08/2018 09:16 PM, Matthew Treinish wrote:
> Although, I don't think glance uses oslo.service even in the case where it's
> using the standalone eventlet server. It looks like it launches eventlet.wsgi
> directly:
> 
> https://github.com/openstack/glance/blob/master/glance/common/wsgi.py

I can confirm this through my last week (bad) experience with Eventlet
over SSL.

Cheers,

Thomas Goirand (zigo)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc][ptls][glance] final stages of python 3 transition

2018-05-19 Thread Matthew Treinish 
On Sat, May 19, 2018 at 07:21:22PM +0200, Thomas Goirand wrote:
> On 05/08/2018 07:55 PM, Matthew Treinish wrote:
> > I wrote up a doc about running under
> > apache when I added the uwsgi chunked transfer encoding support to glance 
> > about
> > running glance under apache here:
> > 
> > https://docs.openstack.org/glance/latest/admin/apache-httpd.html
> > 
> > Which includes how you have to configure things to get it working and a 
> > section
> > on why mod_wsgi doesn't work.
> 
> Thanks for that. Could you also push a uWSGI .ini configuration example
> file, as well as the mod_proxy example? There's so many options in uwsgi
> that I don't want to risk doing something wrong. I've pasted my config
> at the end of this message. Do you think it's also OK to use SSL
> directly with uwsgi, using the --https option? What about the 104 error
> that I've been experiencing? Is it because I'm not using mod_proxy?

There already are example configs in the glance repo. I pushed them up
when I added the documentation:

https://github.com/openstack/glance/tree/master/httpd

Those configs are are more or less just a mirror of what I setup for the
gate (and my personal cloud):

http://logs.openstack.org/47/566747/1/gate/tempest-full-py3/c7f3b2e/controller/logs/etc/glance/glance-uwsgi.ini.gz
http://logs.openstack.org/47/566747/1/gate/tempest-full-py3/c7f3b2e/controller/logs/apache_config/glance-wsgi-api_conf.txt.gz

The way I normally configure things is to do the ssl termination with
apache and then just limit the uwsgi socket on localhost. I haven't
tried setting up the ssl in uwsgi directly, since the idea was to
share a single web server with different endpoints off of it for
each api service. 

As for the 104 error there are several probable causes, without seeing
the full configuration and looking at the traffic it's hard to say
where the connections are getting reset. I would try getting your
config to mirror what we know to be working and then go from there.

> 
> BTW, there's no need to manually do the symlink, you can use instead:
> a2ensite uwsgi-glance-api.conf

Feel free to push a patch to update the docs.

> 
> Cheers,
> 
> Thomas Goirand (zigo)
> 
> 
> [uwsgi]
> 
> ### Generic UWSGI config ###
> 
> 
> # Override the default size for headers from the 4k default.
> buffer-size = 65535
> 
> # This avoids error 104: "Connection reset by peer"
> rem-header = Content-Lenght
> 
> # This is running standalone
> master = true
> 
> # Threads and processes
> enable-threads = true
> 
> processes = 4
> 
> # uwsgi recommends this to prevent thundering herd on accept.
> thunder-lock = true
> 
> plugins = python3
> 
> # This ensures that file descriptors aren't shared between the WSGI
> application processes.
> lazy-apps = true
> 
> # Log from the wsgi application: needs python3-pastescript as runtime
> depends.
> paste-logger = true
> 
> # automatically kill workers if master dies
> no-orphans = true
> 
> # exit instead of brutal reload on SIGTERM
> die-on-term = true
> 
> ##
> ### OpenStack service specific ###
> ##
> 
> # This is the standard port for the WSGI application, listening on all
> available IPs
> http-socket = :9292
> logto = /var/log/glance/glance-api.log
> name = glance-api
> uid = glance
> gid = glance
> chdir = /var/lib/glance
> wsgi-file = /usr/bin/glance-wsgi-api
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc][ptls][glance] final stages of python 3 transition

2018-05-19 Thread Thomas Goirand 
On 05/08/2018 07:55 PM, Matthew Treinish wrote:
> I wrote up a doc about running under
> apache when I added the uwsgi chunked transfer encoding support to glance 
> about
> running glance under apache here:
> 
> https://docs.openstack.org/glance/latest/admin/apache-httpd.html
> 
> Which includes how you have to configure things to get it working and a 
> section
> on why mod_wsgi doesn't work.

Thanks for that. Could you also push a uWSGI .ini configuration example
file, as well as the mod_proxy example? There's so many options in uwsgi
that I don't want to risk doing something wrong. I've pasted my config
at the end of this message. Do you think it's also OK to use SSL
directly with uwsgi, using the --https option? What about the 104 error
that I've been experiencing? Is it because I'm not using mod_proxy?

BTW, there's no need to manually do the symlink, you can use instead:
a2ensite uwsgi-glance-api.conf

Cheers,

Thomas Goirand (zigo)


[uwsgi]

### Generic UWSGI config ###


# Override the default size for headers from the 4k default.
buffer-size = 65535

# This avoids error 104: "Connection reset by peer"
rem-header = Content-Lenght

# This is running standalone
master = true

# Threads and processes
enable-threads = true

processes = 4

# uwsgi recommends this to prevent thundering herd on accept.
thunder-lock = true

plugins = python3

# This ensures that file descriptors aren't shared between the WSGI
application processes.
lazy-apps = true

# Log from the wsgi application: needs python3-pastescript as runtime
depends.
paste-logger = true

# automatically kill workers if master dies
no-orphans = true

# exit instead of brutal reload on SIGTERM
die-on-term = true

##
### OpenStack service specific ###
##

# This is the standard port for the WSGI application, listening on all
available IPs
http-socket = :9292
logto = /var/log/glance/glance-api.log
name = glance-api
uid = glance
gid = glance
chdir = /var/lib/glance
wsgi-file = /usr/bin/glance-wsgi-api

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc][ptls][glance] final stages of python 3 transition

2018-05-14 Thread Victoria Martínez de la Cruz 
2018-05-08 16:31 GMT-03:00 Zane Bitter :

> On 08/05/18 15:16, Matthew Treinish wrote:
>
>> Although, I don't think glance uses oslo.service even in the case where
>> it's
>> using the standalone eventlet server. It looks like it launches
>> eventlet.wsgi
>> directly:
>>
>> https://github.com/openstack/glance/blob/master/glance/common/wsgi.py
>>
>> and I don't see oslo.service in the requirements file either:
>>
>> https://github.com/openstack/glance/blob/master/requirements.txt
>>
>
> It would probably independently suffer from https://bugs.launchpad.net/man
> ila/+bug/1482633 in Python 3 then. IIUC the code started in oslo
> incubator but projects like neutron and manila converted to use the
> oslo.service version. There may be other copies of it still floating
> around...
>
> - ZB
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

Hi,

Jumping in now as I'm helping with py3 support efforts in the manila side.

In manila we have both support for Apache WSGI and the built-in server
(which depends in eventlet). Would it be a possible workaround to rely on
the Apache WSGI server while we wait for evenlet issues to be sorted out?
Is there any chance the upper constraints will be updated soon-ish and this
can be fixed in a newer eventlet version?

This is the only change it's preventing us to be fully py3 compatible,
hence it's a big deal for us.

Thanks,

Victoria
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc][ptls][glance] final stages of python 3 transition

2018-05-08 Thread Zane Bitter 

On 08/05/18 15:16, Matthew Treinish wrote:

Although, I don't think glance uses oslo.service even in the case where it's
using the standalone eventlet server. It looks like it launches eventlet.wsgi
directly:

https://github.com/openstack/glance/blob/master/glance/common/wsgi.py

and I don't see oslo.service in the requirements file either:

https://github.com/openstack/glance/blob/master/requirements.txt


It would probably independently suffer from 
https://bugs.launchpad.net/manila/+bug/1482633 in Python 3 then. IIUC 
the code started in oslo incubator but projects like neutron and manila 
converted to use the oslo.service version. There may be other copies of 
it still floating around...


- ZB

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc][ptls][glance] final stages of python 3 transition

2018-05-08 Thread Matthew Treinish 
On Tue, May 08, 2018 at 03:02:05PM -0400, Doug Hellmann wrote:
> Excerpts from Matthew Treinish's message of 2018-05-08 13:55:43 -0400:
> > On Tue, May 08, 2018 at 01:34:11PM -0400, Doug Hellmann wrote:
> > > 
> > > (added [glance] subject tag)
> > > 
> > > Excerpts from Matthew Treinish's message of 2018-05-08 12:22:56 -0400:
> > > > On Tue, May 08, 2018 at 05:01:36PM +0100, Graham Hayes wrote:
> > > > > On 08/05/18 16:53, Doug Hellmann wrote:
> > > > > > Excerpts from Graham Hayes's message of 2018-05-08 16:28:46 +0100:
> 
> [snip]
> 
> > > > > Glance - Has issues with image upload + uwsgi + eventlet [1]
> > > > 
> > > > This actually is a bit misleading. Glance works fine with image upload 
> > > > and uwsgi.
> > > > That's the only configuration of glance in a wsgi app that works because
> > > > of chunked transfer encoding not being in the WSGI protocol. [2] uwsgi 
> > > > provides
> > > > an alternate interface to read chunked requests which enables this to 
> > > > work.
> > > > If you look at the bugs linked off that release note about image upload
> > > > you'll see they're all fixed.
> > > 
> > > Is this documented somewhere?
> > 
> > The wsgi limitation or the glance usage? I wrote up a doc about running 
> > under
> > apache when I added the uwsgi chunked transfer encoding support to glance 
> > about
> > running glance under apache here:
> > 
> > https://docs.openstack.org/glance/latest/admin/apache-httpd.html
> > 
> > Which includes how you have to configure things to get it working and a 
> > section
> > on why mod_wsgi doesn't work.
> 
> I meant the glance usage so it sounds like you've covered the docs
> for that. Thanks!
> 
> > > > The issues glance has with running in a wsgi app are related to it's 
> > > > use of
> > > > async tasks via taskflow. (which includes the tasks api and image 
> > > > import stuff)
> > > > This shouldn't be hard to fix, and I've had patches up to address these 
> > > > for
> > > > months:
> > > > 
> > > > https://review.openstack.org/#/c/531498/
> > > > https://review.openstack.org/#/c/549743/
> > > > 
> > > > Part of the issue is that there is no api driven testing for these 
> > > > async api
> > > > functions or any documented way to test them. Which is why I marked the 
> > > > 2nd
> > > > one WIP, since I have no method to test it and after asking several 
> > > > times
> > > > for a test case or some other method to validate these APIs without an 
> > > > answer.
> > > 
> > > It would be helpful if some of this detail made its way into the glance
> > > section of 
> > > https://wiki.openstack.org/wiki/Python3#Python_3_Status_of_OpenStack_projects
> > 
> > It really doesn't have anything to do with Python 3 though since the bug 
> > with
> > glance's taskflow usage is on both py2 and py3. In fact we're already 
> > running
> > glance under uwsgi in the gate with python 3 today for the dsvm py3 jobs. 
> > The
> > reason these bugs haven't come up there is because there is no test coverage
> > for any of these async APIs. But I can add it to the wiki later today.
> 
> Will it block us from moving glance to python 3 if we drop the WSGI
> code from oslo.service so that the only way to deploy is behind
> some other WSGI server?
> 

It shouldn't be a blocker, the wsgi entrypoint just uses paste to expose the
wsgi app directly:

https://github.com/openstack/glance/blob/master/glance/common/wsgi_app.py#L59-L67

oslo.service doesn't come into play in that code path. So it won't block
the deploying with uwsgi model. The bugs addressed by the 2 patches I referenced
above will still be present though.

Although, I don't think glance uses oslo.service even in the case where it's
using the standalone eventlet server. It looks like it launches eventlet.wsgi
directly:

https://github.com/openstack/glance/blob/master/glance/common/wsgi.py

and I don't see oslo.service in the requirements file either:

https://github.com/openstack/glance/blob/master/requirements.txt

-Matt Treinish


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc][ptls][glance] final stages of python 3 transition

2018-05-08 Thread Doug Hellmann 
Excerpts from Matthew Treinish's message of 2018-05-08 13:55:43 -0400:
> On Tue, May 08, 2018 at 01:34:11PM -0400, Doug Hellmann wrote:
> > 
> > (added [glance] subject tag)
> > 
> > Excerpts from Matthew Treinish's message of 2018-05-08 12:22:56 -0400:
> > > On Tue, May 08, 2018 at 05:01:36PM +0100, Graham Hayes wrote:
> > > > On 08/05/18 16:53, Doug Hellmann wrote:
> > > > > Excerpts from Graham Hayes's message of 2018-05-08 16:28:46 +0100:

[snip]

> > > > Glance - Has issues with image upload + uwsgi + eventlet [1]
> > > 
> > > This actually is a bit misleading. Glance works fine with image upload 
> > > and uwsgi.
> > > That's the only configuration of glance in a wsgi app that works because
> > > of chunked transfer encoding not being in the WSGI protocol. [2] uwsgi 
> > > provides
> > > an alternate interface to read chunked requests which enables this to 
> > > work.
> > > If you look at the bugs linked off that release note about image upload
> > > you'll see they're all fixed.
> > 
> > Is this documented somewhere?
> 
> The wsgi limitation or the glance usage? I wrote up a doc about running under
> apache when I added the uwsgi chunked transfer encoding support to glance 
> about
> running glance under apache here:
> 
> https://docs.openstack.org/glance/latest/admin/apache-httpd.html
> 
> Which includes how you have to configure things to get it working and a 
> section
> on why mod_wsgi doesn't work.

I meant the glance usage so it sounds like you've covered the docs
for that. Thanks!

> > > The issues glance has with running in a wsgi app are related to it's use 
> > > of
> > > async tasks via taskflow. (which includes the tasks api and image import 
> > > stuff)
> > > This shouldn't be hard to fix, and I've had patches up to address these 
> > > for
> > > months:
> > > 
> > > https://review.openstack.org/#/c/531498/
> > > https://review.openstack.org/#/c/549743/
> > > 
> > > Part of the issue is that there is no api driven testing for these async 
> > > api
> > > functions or any documented way to test them. Which is why I marked the 
> > > 2nd
> > > one WIP, since I have no method to test it and after asking several times
> > > for a test case or some other method to validate these APIs without an 
> > > answer.
> > 
> > It would be helpful if some of this detail made its way into the glance
> > section of 
> > https://wiki.openstack.org/wiki/Python3#Python_3_Status_of_OpenStack_projects
> 
> It really doesn't have anything to do with Python 3 though since the bug with
> glance's taskflow usage is on both py2 and py3. In fact we're already running
> glance under uwsgi in the gate with python 3 today for the dsvm py3 jobs. The
> reason these bugs haven't come up there is because there is no test coverage
> for any of these async APIs. But I can add it to the wiki later today.

Will it block us from moving glance to python 3 if we drop the WSGI
code from oslo.service so that the only way to deploy is behind
some other WSGI server?

Doug

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc][ptls][glance] final stages of python 3 transition

2018-05-08 Thread Matthew Treinish 
On Tue, May 08, 2018 at 01:34:11PM -0400, Doug Hellmann wrote:
> 
> (added [glance] subject tag)
> 
> Excerpts from Matthew Treinish's message of 2018-05-08 12:22:56 -0400:
> > On Tue, May 08, 2018 at 05:01:36PM +0100, Graham Hayes wrote:
> > > On 08/05/18 16:53, Doug Hellmann wrote:
> > > > Excerpts from Graham Hayes's message of 2018-05-08 16:28:46 +0100:
> > > >> On 08/05/18 16:09, Zane Bitter wrote:
> > > >>> On 30/04/18 17:16, Ben Nemec wrote:
> > > > Excerpts from Doug Hellmann's message of 2018-04-25 16:54:46 -0400:
> > > >> 1. Fix oslo.service functional tests -- the Oslo team needs help
> > > >>     maintaining this library. Alternatively, we could move all
> > > >>     services to use cotyledon 
> > > >> (https://pypi.org/project/cotyledon/).
> > > >>>
> > > >>> I submitted a patch that fixes the py35 gate (which was broken due to
> > > >>> changes between CPython 3.4 and 3.5), so once that merges we can flip
> > > >>> the gate back to voting:
> > > >>>
> > > >>> https://review.openstack.org/566714
> > > >>>
> > >  For everyone's awareness, we discussed this in the Oslo meeting today
> > >  and our first step is to see how many, if any, services are actually
> > >  relying on the oslo.service functionality that doesn't work in Python
> > >  3 today.  From there we will come up with a plan for how to move 
> > >  forward.
> > > 
> > >  https://bugs.launchpad.net/manila/+bug/1482633 is the original bug.
> > > >>>
> > > >>> These tests are currently skipped in both oslo_service and nova.
> > > >>> (Equivalent tests were removed from Neutron and Manila on the 
> > > >>> principle
> > > >>> that they're now oslo_service's responsibility.)
> > > >>>
> > > >>> This appears to be a series of long-standing bugs in eventlet:
> > > >>>
> > > >>> Python 3.5 failure mode:
> > > >>> https://github.com/eventlet/eventlet/issues/308
> > > >>> https://github.com/eventlet/eventlet/issues/189
> > > >>>
> > > >>> Python 3.4 failure mode:
> > > >>> https://github.com/eventlet/eventlet/issues/476
> > > >>> https://github.com/eventlet/eventlet/issues/145
> > > >>>
> > > >>> There are also more problems coming down the pipeline in Python 3.6:
> > > >>>
> > > >>> https://github.com/eventlet/eventlet/issues/371
> > > >>>
> > > >>> That one is resolved in eventlet 0.21, but we have that blocked by
> > > >>> upper-constraints:
> > > >>> http://git.openstack.org/cgit/openstack/requirements/tree/upper-constraints.txt#n135
> > > >>>
> > > >>>
> > > >>> Given that the code in question relates solely to standalone WSGI
> > > >>> servers with SSL and everything should have already migrated to 
> > > >>> Apache,
> > > >>> and that the upstream is clearly overworked and unlikely to merge 
> > > >>> fixes
> > > >>> any time soon (plus we would have to deal with the fallout of moving 
> > > >>> the
> > > >>> upper constraint), I agree that it would be preferable if we could 
> > > >>> just
> > > >>> ditch this functionality.
> > > >>
> > > >> There are a few projects that have not migrated, and some that have
> > > >> issues running in non standalone WSGI mode (due, ironically to 
> > > >> eventlet)
> > > >>
> > > >> We should probably get people to run these projects behind an reverse
> > > >> proxy, and terminate SSL there, but right now we don't have that
> > > >> documented.
> > > > 
> > > > Do you know which projects?
> > > 
> > > I know of 2:
> > > 
> > > Designate - mainly due to the major lack of resources available during
> > > the uwsgi goal period, and the level of work needed to unravel our
> > > tooling to support it.
> > > 
> > > Glance - Has issues with image upload + uwsgi + eventlet [1]
> > 
> > This actually is a bit misleading. Glance works fine with image upload and 
> > uwsgi.
> > That's the only configuration of glance in a wsgi app that works because
> > of chunked transfer encoding not being in the WSGI protocol. [2] uwsgi 
> > provides
> > an alternate interface to read chunked requests which enables this to work.
> > If you look at the bugs linked off that release note about image upload
> > you'll see they're all fixed.
> 
> Is this documented somewhere?

The wsgi limitation or the glance usage? I wrote up a doc about running under
apache when I added the uwsgi chunked transfer encoding support to glance about
running glance under apache here:

https://docs.openstack.org/glance/latest/admin/apache-httpd.html

Which includes how you have to configure things to get it working and a section
on why mod_wsgi doesn't work.

> 
> > 
> > The issues glance has with running in a wsgi app are related to it's use of
> > async tasks via taskflow. (which includes the tasks api and image import 
> > stuff)
> > This shouldn't be hard to fix, and I've had patches up to address these for
> > months:
> > 
> > https://review.openstack.org/#/c/531498/
> > https://review.openstack.org/#/c/549743/
> > 
> > Part of the issue is that there is no api driven testing for these async api

Re: [openstack-dev] [all][tc][ptls][glance] final stages of python 3 transition

2018-05-08 Thread Doug Hellmann 

(added [glance] subject tag)

Excerpts from Matthew Treinish's message of 2018-05-08 12:22:56 -0400:
> On Tue, May 08, 2018 at 05:01:36PM +0100, Graham Hayes wrote:
> > On 08/05/18 16:53, Doug Hellmann wrote:
> > > Excerpts from Graham Hayes's message of 2018-05-08 16:28:46 +0100:
> > >> On 08/05/18 16:09, Zane Bitter wrote:
> > >>> On 30/04/18 17:16, Ben Nemec wrote:
> > > Excerpts from Doug Hellmann's message of 2018-04-25 16:54:46 -0400:
> > >> 1. Fix oslo.service functional tests -- the Oslo team needs help
> > >>     maintaining this library. Alternatively, we could move all
> > >>     services to use cotyledon (https://pypi.org/project/cotyledon/).
> > >>>
> > >>> I submitted a patch that fixes the py35 gate (which was broken due to
> > >>> changes between CPython 3.4 and 3.5), so once that merges we can flip
> > >>> the gate back to voting:
> > >>>
> > >>> https://review.openstack.org/566714
> > >>>
> >  For everyone's awareness, we discussed this in the Oslo meeting today
> >  and our first step is to see how many, if any, services are actually
> >  relying on the oslo.service functionality that doesn't work in Python
> >  3 today.  From there we will come up with a plan for how to move 
> >  forward.
> > 
> >  https://bugs.launchpad.net/manila/+bug/1482633 is the original bug.
> > >>>
> > >>> These tests are currently skipped in both oslo_service and nova.
> > >>> (Equivalent tests were removed from Neutron and Manila on the principle
> > >>> that they're now oslo_service's responsibility.)
> > >>>
> > >>> This appears to be a series of long-standing bugs in eventlet:
> > >>>
> > >>> Python 3.5 failure mode:
> > >>> https://github.com/eventlet/eventlet/issues/308
> > >>> https://github.com/eventlet/eventlet/issues/189
> > >>>
> > >>> Python 3.4 failure mode:
> > >>> https://github.com/eventlet/eventlet/issues/476
> > >>> https://github.com/eventlet/eventlet/issues/145
> > >>>
> > >>> There are also more problems coming down the pipeline in Python 3.6:
> > >>>
> > >>> https://github.com/eventlet/eventlet/issues/371
> > >>>
> > >>> That one is resolved in eventlet 0.21, but we have that blocked by
> > >>> upper-constraints:
> > >>> http://git.openstack.org/cgit/openstack/requirements/tree/upper-constraints.txt#n135
> > >>>
> > >>>
> > >>> Given that the code in question relates solely to standalone WSGI
> > >>> servers with SSL and everything should have already migrated to Apache,
> > >>> and that the upstream is clearly overworked and unlikely to merge fixes
> > >>> any time soon (plus we would have to deal with the fallout of moving the
> > >>> upper constraint), I agree that it would be preferable if we could just
> > >>> ditch this functionality.
> > >>
> > >> There are a few projects that have not migrated, and some that have
> > >> issues running in non standalone WSGI mode (due, ironically to eventlet)
> > >>
> > >> We should probably get people to run these projects behind an reverse
> > >> proxy, and terminate SSL there, but right now we don't have that
> > >> documented.
> > > 
> > > Do you know which projects?
> > 
> > I know of 2:
> > 
> > Designate - mainly due to the major lack of resources available during
> > the uwsgi goal period, and the level of work needed to unravel our
> > tooling to support it.
> > 
> > Glance - Has issues with image upload + uwsgi + eventlet [1]
> 
> This actually is a bit misleading. Glance works fine with image upload and 
> uwsgi.
> That's the only configuration of glance in a wsgi app that works because
> of chunked transfer encoding not being in the WSGI protocol. [2] uwsgi 
> provides
> an alternate interface to read chunked requests which enables this to work.
> If you look at the bugs linked off that release note about image upload
> you'll see they're all fixed.

Is this documented somewhere?

> 
> The issues glance has with running in a wsgi app are related to it's use of
> async tasks via taskflow. (which includes the tasks api and image import 
> stuff)
> This shouldn't be hard to fix, and I've had patches up to address these for
> months:
> 
> https://review.openstack.org/#/c/531498/
> https://review.openstack.org/#/c/549743/
> 
> Part of the issue is that there is no api driven testing for these async api
> functions or any documented way to test them. Which is why I marked the 2nd
> one WIP, since I have no method to test it and after asking several times
> for a test case or some other method to validate these APIs without an answer.

It would be helpful if some of this detail made its way into the glance
section of 
https://wiki.openstack.org/wiki/Python3#Python_3_Status_of_OpenStack_projects

> 
> In fact people are running glance under uwsgi in production already because 
> it 
> makes a lot of things easier and the current issues don't effect most users.

That's good to know!

> 
> -Matt Treinish
> 
> > 
> > I am sure there are probably others, but I know of these 2.
> > 
> > [1]