Re: mysql lookup table and utf8

wietse: > I'd appreciate it if you could do some tests with the postmap command > in postfix-3.2-20160925, which I just rolled out to ftp/www.porcupine.org. John Fawcett: > I've downloaded the postfix-3.2-20160925 release and done the following > tests with postmap and get the desired results. >

Re: mysql lookup table and utf8

On 09/25/2016 08:04 PM, John Fawcett wrote: >> I'll update their documentation with a >> >> suggestion to specify "option_group = client". An incompatible >> >> code change would be forbidden. > > I'd appreciate it if you could do some tests with the postmap command > > in postfix-3.2-20160925,

Re: mysql lookup table and utf8

On 09/25/2016 06:36 PM, Wietse Venema wrote: > Wietse Venema: >> For Postfix 3.2 I'll update code and documentation, such that the >> default option_group value becomes "client". This causes the "client" >> option group to be read twice instead of never (reading it once >> is not an option). To

Re: mysql lookup table and utf8

Wietse Venema: > For Postfix 3.2 I'll update code and documentation, such that the > default option_group value becomes "client". This causes the "client" > option group to be read twice instead of never (reading it once > is not an option). To get backwards compatibility, specify an empty >

Re: mysql lookup table and utf8

For Postfix 3.2 I'll update code and documentation, such that the default option_group value becomes "client". This causes the "client" option group to be read twice instead of never (reading it once is not an option). To get backwards compatibility, specify an empty option_group value. For the

Re: mysql lookup table and utf8

On 09/25/2016 02:21 AM, Wietse Venema wrote: > Wietse: > Summary > === > I think I found a libmysqlclient bug. According to documentation: > > The [client] option group is read by all client programs (but > not by mysqld). This enables you to specify options that apply > to all

Re: mysql lookup table and utf8

Wietse: > Before I forget, did you verify that: > > - The 'old' code reproduces the problem (postmap -q fails to look > up a database key that contains UTF8) and the 'new' code fixes it. John Fawcett: > The fix is not specifically about utf8 lookups, but for the issue > that Postfix was not

Re: mysql lookup table and utf8

On 09/24/2016 05:18 PM, John Fawcett wrote: > I have also looked at the latest mysql > 5.7.15 source and it does the same thing. > see mysql_read_default_options in mysql-5.7.15/sql-common/client.c > groups[0]= (char*) "client"; groups[1]= (char*) group; groups[2]=0; I opened a bug report

Re: mysql lookup table and utf8

On 09/24/2016 03:52 PM, Wietse Venema wrote: > Wietse Venema: >> Looks good (I'll fix typos). It works with the system by using what oops, thanks for picking that up. > Before I forget, did you verify that: > > - The 'old' code reproduces the problem (postmap -q fails to look > up a database key

Re: mysql lookup table and utf8

Wietse Venema: > John Fawcett: > > Looking into it further, most of the code was already there and the > > fix is a simple one liner + documentation updates. > > > In this fix I have not used a hard coded group of [postfix] as I had > > imagined above, but followed the way that dovecot is doing

Re: mysql lookup table and utf8

John Fawcett: > Looking into it further, most of the code was already there and the > fix is a simple one liner + documentation updates. > In this fix I have not used a hard coded group of [postfix] as I had > imagined above, but followed the way that dovecot is doing this. > The default group is

Re: mysql lookup table and utf8

On 09/19/2016 12:50 AM, Wietse Venema wrote: > John Fawcett: >>> Care to post a patch for src/global/dict_mysql.c? Do you >>> think that the "prog_name" should be configurable? Perhaps >>> allowing different Postfix instances to run with different >>> settings? Or would a fixed name like

Re: mysql lookup table and utf8

John Fawcett: > > Care to post a patch for src/global/dict_mysql.c? Do you > > think that the "prog_name" should be configurable? Perhaps > > allowing different Postfix instances to run with different > > settings? Or would a fixed name like "postfix" be better? > > > ok I can do that. I would

Re: mysql lookup table and utf8

On 09/19/2016 12:13 AM, Viktor Dukhovni wrote: >> On Sep 18, 2016, at 6:03 PM, John Fawcett wrote: >> >> In order to get libmysqlclient to read the standard options file >> you have to specifically define the group name to read from that file >> for example "your_prog_name".

Re: mysql lookup table and utf8

> On Sep 18, 2016, at 6:03 PM, John Fawcett wrote: > > In order to get libmysqlclient to read the standard options file > you have to specifically define the group name to read from that file > for example "your_prog_name". > >

Re: mysql lookup table and utf8

On 09/15/2016 05:35 PM, Phil Stracchino wrote: > On 09/15/16 06:49, Wietse Venema wrote: >> Phil Stracchino: >>> Well, it's supposed to Just Work if they're using libmysqlclient. I'm >>> not sure where to get the information of NOT using libmysqlclient, other >>> than just searching likely

Re: mysql lookup table and utf8

On 15/09/2016 16:47, Viktor Dukhovni wrote: On Thu, Sep 15, 2016 at 09:51:11AM -0400, Wietse Venema wrote: Stephan Seitz: On Thu, Sep 15, 2016 at 06:49:08AM -0400, Wietse Venema wrote: FYI, Postfix uses libmysqlclient. So what's up with not reading the default config file? Can it be the

Re: mysql lookup table and utf8

On 09/15/16 06:49, Wietse Venema wrote: > Phil Stracchino: >> Well, it's supposed to Just Work if they're using libmysqlclient. I'm >> not sure where to get the information of NOT using libmysqlclient, other >> than just searching likely locations. > > FYI, Postfix uses libmysqlclient. So what's

Re: mysql lookup table and utf8

On Thu, Sep 15, 2016 at 09:51:11AM -0400, Wietse Venema wrote: > Stephan Seitz: > > On Thu, Sep 15, 2016 at 06:49:08AM -0400, Wietse Venema wrote: > > >FYI, Postfix uses libmysqlclient. So what's up with not reading > > >the default config file? > > > > Can it be the usual chroot problem? > >

Re: mysql lookup table and utf8

Stephan Seitz: > On Thu, Sep 15, 2016 at 06:49:08AM -0400, Wietse Venema wrote: > >FYI, Postfix uses libmysqlclient. So what's up with not reading > >the default config file? > > Can it be the usual chroot problem? Postfix normally opens tables before chroot() and before dropping root

Re: mysql lookup table and utf8

On Thu, Sep 15, 2016 at 06:49:08AM -0400, Wietse Venema wrote: FYI, Postfix uses libmysqlclient. So what's up with not reading the default config file? Can it be the usual chroot problem? Stephan -- | Stephan Seitz E-Mail: s...@fsing.rootsland.net | | Public Keys:

Re: mysql lookup table and utf8

Phil Stracchino: [ Charset windows-1252 converted... ] > On 09/14/16 19:29, Wietse Venema wrote: > Perhaps Postfix needs to try to explicitly load the default config > location when no override is set. > >>> > >>> How do we do that, without hard-coding '/etc/my.cnf' into Postfix? > >>>

Re: mysql lookup table and utf8

On 09/14/16 19:29, Wietse Venema wrote: Perhaps Postfix needs to try to explicitly load the default config location when no override is set. >>> >>> How do we do that, without hard-coding '/etc/my.cnf' into Postfix? >>> That pathname is almost certainly incorrect on systems like *BSD >>>

Re: mysql lookup table and utf8

> >> Perhaps Postfix needs to try to explicitly load the default config > >> location when no override is set. > > > > How do we do that, without hard-coding '/etc/my.cnf' into Postfix? > > That pathname is almost certainly incorrect on systems like *BSD > > that install ports configuration files

Re: mysql lookup table and utf8

On 14/09/2016 22:07, Phil Stracchino wrote: On 09/14/16 13:44, Wietse Venema wrote: Viktor Dukhovni: On Wed, Sep 14, 2016 at 06:07:22PM +0200, Antoine Nguyen wrote: Looks like I've found the issue. Actually, the mysql connector does not read the default /etc/my.cnf file (or at least the

Re: mysql lookup table and utf8

On 09/14/16 13:44, Wietse Venema wrote: > Viktor Dukhovni: >> On Wed, Sep 14, 2016 at 06:07:22PM +0200, Antoine Nguyen wrote: >> >>> Looks like I've found the issue. Actually, the mysql connector does not read >>> the default /etc/my.cnf file (or at least the package I installed for CentOS >>> 7).

Re: mysql lookup table and utf8

On Wed, Sep 14, 2016 at 04:14:57PM +, Viktor Dukhovni wrote: > > @Viktor Do you confirm it is not currently possible to host UTF8 domains > > with a postgres backend ? > I don't believe it is. We probably need to make that "LATIN1" > configurable via the ".cf" file. I'm curious what just

Re: mysql lookup table and utf8

Viktor Dukhovni: > On Wed, Sep 14, 2016 at 06:07:22PM +0200, Antoine Nguyen wrote: > > > Looks like I've found the issue. Actually, the mysql connector does not read > > the default /etc/my.cnf file (or at least the package I installed for CentOS > > 7). > > Perhaps Postfix needs to try to

Re: mysql lookup table and utf8

On Wed, Sep 14, 2016 at 06:07:22PM +0200, Antoine Nguyen wrote: > Looks like I've found the issue. Actually, the mysql connector does not read > the default /etc/my.cnf file (or at least the package I installed for CentOS > 7). Perhaps Postfix needs to try to explicitly load the default config

Re: mysql lookup table and utf8

On Wed, Sep 14, 2016 at 05:39:50PM +0200, Antoine Nguyen wrote: > >Yes, the Postgres driver is unconditionally LATIN1 at present. > > And this limitation does not exist with the mysql one ? No. Nothing in Postfix explicitly sets the MySQL client encoding. Perhaps that's needed for utf-8 (i.e.

Re: mysql lookup table and utf8

On 14/09/2016 14:53, Viktor Dukhovni wrote: Please look for any client-side configuration files that might be loaded by the MySQL interactive shell that may affect connection settings, such as perhaps the client character set. Looks like I've found the issue. Actually, the mysql connector

Re: mysql lookup table and utf8

On 14/09/2016 17:06, Viktor Dukhovni wrote: On Wed, Sep 14, 2016 at 04:14:32PM +0200, Antoine Nguyen wrote: I haven't found how to change default client encoding yet... Does postfix set it ? Sorry to reply to myself but looks like the answer is here:

Re: mysql lookup table and utf8

On Wed, Sep 14, 2016 at 04:14:32PM +0200, Antoine Nguyen wrote: > >I haven't found how to change default client encoding yet... Does postfix > >set it ? > > Sorry to reply to myself but looks like the answer is here: > >

Re: mysql lookup table and utf8

On 14/09/2016 16:09, Antoine Nguyen wrote: As I said, I made some tests with postgres and I think I found an interesting hint inside logs: LOG: statement: set client_encoding to 'LATIN1' LOG: statement: SELECT name FROM admin_domain WHERE name='raté.com' AND type='domain' AND enabled

Re: mysql lookup table and utf8

On 14/09/2016 14:53, Viktor Dukhovni wrote: Please post a single message with two complete traces in which: * The Unix login users executing "postmap -q" and using the MySQL shell are the same. (That is the same unix shell spawns both "postmap -q" and the MySQL interactive

Re: mysql lookup table and utf8

On Wed, Sep 14, 2016 at 02:33:47PM +0200, Antoine Nguyen wrote: > >>> SELECT name FROM admin_domain WHERE name='' AND > >>> type='domain' AND enabled=1 > >>> > >>I've enabled queries logging. Here is the result : > >> > >>160914 8:24:14 57 QuerySELECT name FROM admin_domain WHERE

Re: mysql lookup table and utf8

On 14/09/2016 14:17, Viktor Dukhovni wrote: On Wed, Sep 14, 2016 at 10:27:32AM +0200, Antoine Nguyen wrote: Do make sure that your tests with the sql CLI really make the same query against the same database: SELECT name FROM admin_domain WHERE name='' AND type='domain' AND enabled=1

Re: mysql lookup table and utf8

On Wed, Sep 14, 2016 at 10:27:32AM +0200, Antoine Nguyen wrote: > >Do make sure that your tests with the sql CLI really make the same > >query against the same database: > > > > SELECT name FROM admin_domain WHERE name='' AND > > type='domain' AND enabled=1 > > > I've enabled queries

Re: mysql lookup table and utf8

On 14/09/2016 12:53, Wietse Venema wrote: Antoine Nguyen: [ Charset windows-1252 converted... ] On 14/09/2016 10:27, Antoine Nguyen wrote: The first query was executed using mysql shell and it returns 1 row. The second query was executed by postmap and returns nothing whereas we can see they

Re: mysql lookup table and utf8

Antoine Nguyen: [ Charset windows-1252 converted... ] > On 14/09/2016 10:27, Antoine Nguyen wrote: > > The first query was executed using mysql shell and it returns 1 row. > > The second query was executed by postmap and returns nothing whereas > > we can see they are the same... > > > I've just

Re: mysql lookup table and utf8

On 14/09/2016 10:27, Antoine Nguyen wrote: The first query was executed using mysql shell and it returns 1 row. The second query was executed by postmap and returns nothing whereas we can see they are the same... I've just tried with postgresql and the result is the same... Antoine

Re: mysql lookup table and utf8

On 14/09/2016 04:01, Viktor Dukhovni wrote: On Tue, Sep 13, 2016 at 09:52:16PM +0200, Antoine Nguyen wrote: query = SELECT name FROM admin_domain WHERE name='%s' AND type='domain' AND enabled=1 What is not clear is how MySQL's quoting behaves given the input UTF-8 string. postmap:

Re: mysql lookup table and utf8

On Tue, Sep 13, 2016 at 09:52:16PM +0200, Antoine Nguyen wrote: > query = SELECT name FROM admin_domain WHERE name='%s' AND type='domain' AND > enabled=1 What is not clear is how MySQL's quoting behaves given the input UTF-8 string. > postmap: cfg_get_str: /etc/postfix/sql-domains.cf: query =

Re: mysql lookup table and utf8

On 13/09/2016 18:08, Viktor Dukhovni wrote: Make sure your shell or terminal emulator is not mangling utf-8 input or command-line arguments. * Post your map.cf file content, perhaps that map file restricts which lookup keys are passed on to the underlying database. * Place the lookup

Re: mysql lookup table and utf8

On Tue, Sep 13, 2016 at 05:30:40PM +0200, Antoine Nguyen wrote: > Thank you for this link but, since it is a new setup, I have no data. I've > already checked mysql configuration : default character set is utf8, default > collation is utf8_general_ci and database and tables are using those

Re: mysql lookup table and utf8

On 13/09/2016 16:10, Admin Beckspaced wrote: hello ;) not sure if this will help? but had a similar issue with mysql queries a while back. it actually had nothing to do with postfix and sql lookup tables ... but ran into trouble with different mysql character sets ... causing queries to

Re: mysql lookup table and utf8

Am 13.09.2016 um 14:22 schrieb Antoine Nguyen: On 13/09/2016 12:43, Wietse Venema wrote: Antoine Nguyen: On 12/09/2016 17:04, Wietse Venema wrote: Just for test, what happens when you do: (this is what Postfix daemons use by default) LC_ALL=C postmap -q rat?.com

Re: mysql lookup table and utf8

On 13/09/2016 12:43, Wietse Venema wrote: Antoine Nguyen: On 12/09/2016 17:04, Wietse Venema wrote: Just for test, what happens when you do: (this is what Postfix daemons use by default) LC_ALL=C postmap -q rat?.com mysql:/etc/postfix/sql-domains.cf (this is to test what happens with

Re: mysql lookup table and utf8

Antoine Nguyen: > On 12/09/2016 17:04, Wietse Venema wrote: > > > > Just for test, what happens when you do: > > > > (this is what Postfix daemons use by default) > > LC_ALL=C postmap -q rat?.com mysql:/etc/postfix/sql-domains.cf > > > > (this is to test what happens with an UTF-8 locale) > >

Re: mysql lookup table and utf8

On 12/09/2016 17:04, Wietse Venema wrote: Just for test, what happens when you do: (this is what Postfix daemons use by default) LC_ALL=C postmap -q rat?.com mysql:/etc/postfix/sql-domains.cf (this is to test what happens with an UTF-8 locale) LC_ALL=en_US.UTF-8 postmap -q rat?.com

Re: mysql lookup table and utf8

On 12/09/2016 17:04, Wietse Venema wrote: Antoine Nguyen: On 12/09/2016 12:42, Wietse Venema wrote: Antoine Nguyen: Dear list, sorry for my previous incomplete message. I'm trying to setup a postfix 3.0 with a mysql backend to host utf8 domains. The setup is almost ready but when I make

Re: mysql lookup table and utf8

Antoine Nguyen: > On 12/09/2016 12:42, Wietse Venema wrote: > > Antoine Nguyen: > >> Dear list, > >> > >> sorry for my previous incomplete message. > >> > >> I'm trying to setup a postfix 3.0 with a mysql backend to host utf8 > >> domains. The setup is almost ready but when I make some tests, utf8

Re: mysql lookup table and utf8

On 12/09/2016 12:42, Wietse Venema wrote: Antoine Nguyen: Dear list, sorry for my previous incomplete message. I'm trying to setup a postfix 3.0 with a mysql backend to host utf8 domains. The setup is almost ready but when I make some tests, utf8 domains are not found (I see "relay access

Re: mysql lookup table and utf8

Antoine Nguyen: > Dear list, > > sorry for my previous incomplete message. > > I'm trying to setup a postfix 3.0 with a mysql backend to host utf8 > domains. The setup is almost ready but when I make some tests, utf8 > domains are not found (I see "relay access denied" errors in the logs) >