On 10/06/2014 08:30 AM, Ihar Hrachyshka wrote:
Hash: SHA512

Hi all,

tl;dr we should enforce utf8 on server and client sides of db
connection, and this requires changes to docs and oslo.db. The latter
may require raising effective libpq version dep to 9.1+.

Recently I was working on making sure we always run using utf8 on both
database server and client side, and that we do it with all the proper
connection options set (f.e. mysqldb requires use_unicode=0 to avoid
performance drop - a thing that is not set anywhere in gate as of now,
and not documented).

For server side, it's a matter of how we create databases, and in
devstack, it's usually utf8 (though, for unclear reason, latin1 is
still enforced for nova for initial creation due to Folsom (sic!) bug;
I've reverted to utf8 with [1]).

For client side, it's not that clear. Though devstack enforces utf8 in
SQL connection strings, we don't ask to do so in official docs [2].
There is a bug [3] in LP that seems to be relevant.

I think the proper way is to enforce utf8 everywhere. The first step
would be to make sure we suggest adding charset=utf8 to SQL connection
strings, and configure SQL servers to use utf8. That is something to
be covered by docs team.

But we can do better. We should also enforce utf8 on client side, so
that there is no way to run with a different encoding, and so that we
may get rid of additional options in sql connection strings. I've sent
a patch for oslo.db [4] to do just that.

So, the subject of this email is a bit misleading :) You are suggesting a change to the UTF-8 charset enforcement here, and below you are suggesting removing libpq<9.1 for related reasons.

I suppose setting of utf8 on the client side, and supporting utf8 on the server side, but I do *not* recommend using a blanket policy of charset=utf8 on every text column on the database side. It really is wasteful for certain columns like uuids that are used in joins all over the schema...


Though as you can see, the patch fails on py26 postgresql unit tests.
This is because I set charset_encoding for PostgreSQL via libpq
connection option that is available since PostgreSQL 9.1 only. The
failure is because we run py26 test suite on CentOS 6.5 that still
uses PostgreSQL 8.4.*, meaning the option is not available for us.

Now, there are several other ways available for us to enforce encoding
without using the option. First, we may set encoding thru
PGCLIENTENCODING environment variable. Second, we may execute "SET
CLIENT_ENCODING TO utf8" after connecting to psql server.

Now, the question is whether it's worth reimplementing the feature
using some of the available hacks that will work with PSQL < 9.1.
We're about to drop Python 2.6 support in Kilo, and that probably
means that CentOS 6.5 will be dropped from gate too. Do we have any
other distribution of interest that uses both python 2.7 and psql <
9.1, and is going to run Kilo pieces?

On Red Hat side, we're not interested in running Juno on EL6 that uses
old psql. As for EL7, it has a psql version that supports the
connection option. On Ubuntu side, the only LTS release that is still
supported and uses psql < 9.1 is Lucid, but it's based on py26, so it
should not be an issue. Any other consumers that could be affected by
effectively raising minimum version bar for psql to 9.1 in Kilo?

Though we've (almost) decided to drop py26 support in Kilo, we didn't
have similar discussions for other components that we use. So I'd like
to hear what's our take on that. Do we really drop *py26* support, or
maybe what we really do is we drop *CentOS 6.5* support?..

On a more general note, do we track those kind of external
dependencies anywhere? There are lots of them: neutron depends on
specific versions of openvswitch; oslo.db depends on specific versions
of underlying backend libraries, etc.


[1]: https://review.openstack.org/126267
[3]: https://bugs.launchpad.net/glance/+bug/1279000
[4]: https://review.openstack.org/111236

Version: GnuPG/MacGPG2 v2.0.22 (Darwin)


OpenStack-dev mailing list

OpenStack-dev mailing list

Reply via email to