-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi all,
tl;dr we should enforce utf8 on server and client sides of db connection, and this requires changes to docs and oslo.db. The latter may require raising effective libpq version dep to 9.1+. Recently I was working on making sure we always run using utf8 on both database server and client side, and that we do it with all the proper connection options set (f.e. mysqldb requires use_unicode=0 to avoid performance drop - a thing that is not set anywhere in gate as of now, and not documented). For server side, it's a matter of how we create databases, and in devstack, it's usually utf8 (though, for unclear reason, latin1 is still enforced for nova for initial creation due to Folsom (sic!) bug; I've reverted to utf8 with [1]). For client side, it's not that clear. Though devstack enforces utf8 in SQL connection strings, we don't ask to do so in official docs [2]. There is a bug [3] in LP that seems to be relevant. I think the proper way is to enforce utf8 everywhere. The first step would be to make sure we suggest adding charset=utf8 to SQL connection strings, and configure SQL servers to use utf8. That is something to be covered by docs team. But we can do better. We should also enforce utf8 on client side, so that there is no way to run with a different encoding, and so that we may get rid of additional options in sql connection strings. I've sent a patch for oslo.db [4] to do just that. Though as you can see, the patch fails on py26 postgresql unit tests. This is because I set charset_encoding for PostgreSQL via libpq connection option that is available since PostgreSQL 9.1 only. The failure is because we run py26 test suite on CentOS 6.5 that still uses PostgreSQL 8.4.*, meaning the option is not available for us. Now, there are several other ways available for us to enforce encoding without using the option. First, we may set encoding thru PGCLIENTENCODING environment variable. Second, we may execute "SET CLIENT_ENCODING TO utf8" after connecting to psql server. Now, the question is whether it's worth reimplementing the feature using some of the available hacks that will work with PSQL < 9.1. We're about to drop Python 2.6 support in Kilo, and that probably means that CentOS 6.5 will be dropped from gate too. Do we have any other distribution of interest that uses both python 2.7 and psql < 9.1, and is going to run Kilo pieces? On Red Hat side, we're not interested in running Juno on EL6 that uses old psql. As for EL7, it has a psql version that supports the connection option. On Ubuntu side, the only LTS release that is still supported and uses psql < 9.1 is Lucid, but it's based on py26, so it should not be an issue. Any other consumers that could be affected by effectively raising minimum version bar for psql to 9.1 in Kilo? Though we've (almost) decided to drop py26 support in Kilo, we didn't have similar discussions for other components that we use. So I'd like to hear what's our take on that. Do we really drop *py26* support, or maybe what we really do is we drop *CentOS 6.5* support?.. On a more general note, do we track those kind of external dependencies anywhere? There are lots of them: neutron depends on specific versions of openvswitch; oslo.db depends on specific versions of underlying backend libraries, etc. Comments? [1]: https://review.openstack.org/126267 [2]: http://docs.openstack.org/icehouse/install-guide/install/yum/content/nova-controller.html [3]: https://bugs.launchpad.net/glance/+bug/1279000 [4]: https://review.openstack.org/111236 /Ihar -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.22 (Darwin) iQEcBAEBCgAGBQJUMotwAAoJEC5aWaUY1u57IQQIAN2yHfxbj0DYDMHE40OGbEwU 8fis/KOAxSyUEhHbbECS8ODGjHNK9SeuddUplzL5sKVbYIMZgt3wKXGT/rT5ex0C iMmc7/mb6JGGTUYvR1sF2EXxioJ3kWJSt5oQqBhVT14y2g8rwqN6tQaTusfDVZWN yZ7KCmQDc+3HMyY3EmYCp6V1ndubLbx8Oh56/jtdSGdNtc7V4IwYIF3F4WQaKBcB 54tZkAO0wvlbbRrXqEiyf99/dVLDiMquM3LNKEbYg4zjDUEX74fGJ3mscQtWdDLv 0TcqOvQ2pwLvpksf30kvBBHGcNCF9x8Ov8H4vg5wRTD4bkpjEDwmXzQXTk7xfHM= =aYRM -----END PGP SIGNATURE----- _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
