Re: [HACKERS] Initdb-cs_CZ.WIN-1250 buildfarm failures

2014-12-31 Thread Tomas Vondra
On 28.12.2014 00:46, Noah Misch wrote:
 On Tue, Dec 23, 2014 at 03:32:59PM +0100, Tomas Vondra wrote:
 On 23.12.2014 15:21, Andrew Dunstan wrote:

 No, config_opts is what's passed to configure. Try something like:

 if ($branch eq 'REL9_0_STABLE')
 {
 $skip_steps{'pl-install-check'} = 1;
 }

 Applied to all three animals.
 
 As of the time of this change, the animals ceased to report in.

The strange thing is that while the first run worked fine (as
illustrated by the results submitted to pgbuildfarm.org), all the
following runs fail like this:

Global symbol %skip_steps requires explicit package name at
build-farm.conf line 308.
Compilation failed in require at ./run_branches.pl line 55.

Apparently, something is broken, but my Perl-fu is limited so I have no
idea what/why :-(

regards
Tomas



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Initdb-cs_CZ.WIN-1250 buildfarm failures

2014-12-31 Thread Andrew Dunstan


On 12/31/2014 07:49 AM, Tomas Vondra wrote:

On 28.12.2014 00:46, Noah Misch wrote:

On Tue, Dec 23, 2014 at 03:32:59PM +0100, Tomas Vondra wrote:

On 23.12.2014 15:21, Andrew Dunstan wrote:

No, config_opts is what's passed to configure. Try something like:

 if ($branch eq 'REL9_0_STABLE')
 {
 $skip_steps{'pl-install-check'} = 1;
 }

Applied to all three animals.

As of the time of this change, the animals ceased to report in.

The strange thing is that while the first run worked fine (as
illustrated by the results submitted to pgbuildfarm.org), all the
following runs fail like this:

Global symbol %skip_steps requires explicit package name at
build-farm.conf line 308.
Compilation failed in require at ./run_branches.pl line 55.

Apparently, something is broken, but my Perl-fu is limited so I have no
idea what/why :-(





Sorry, I should have tested it. This seems to work:

   if ($branch eq 'REL9_0_STABLE')
   {
$PGBuild::Options::skip_steps .= ' pl-install-check';
$main::skip_steps{'pl-install-check'} = 1;
   }

cheers

andrew




--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Initdb-cs_CZ.WIN-1250 buildfarm failures

2014-12-31 Thread Tomas Vondra
On 31.12.2014 17:29, Andrew Dunstan wrote:
 
 Sorry, I should have tested it. This seems to work:
 
if ($branch eq 'REL9_0_STABLE')
{
 $PGBuild::Options::skip_steps .= ' pl-install-check';
 $main::skip_steps{'pl-install-check'} = 1;
}
 
 cheers

Meh, no problem. We've fixed it in 2014, so it's OK. To quote one of my
colleagues I haven't tested it, I believe it works correctly. ;-)

Any ideas why it worked for the first run?

regards
Tomas


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Initdb-cs_CZ.WIN-1250 buildfarm failures

2014-12-31 Thread Andrew Dunstan


On 12/31/2014 12:26 PM, Tomas Vondra wrote:

On 31.12.2014 17:29, Andrew Dunstan wrote:

Sorry, I should have tested it. This seems to work:

if ($branch eq 'REL9_0_STABLE')
{
 $PGBuild::Options::skip_steps .= ' pl-install-check';
 $main::skip_steps{'pl-install-check'} = 1;
}

cheers

Meh, no problem. We've fixed it in 2014, so it's OK. To quote one of my
colleagues I haven't tested it, I believe it works correctly. ;-)

Any ideas why it worked for the first run?



No. It failed for me right off the bat.

cheers

andrew


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Initdb-cs_CZ.WIN-1250 buildfarm failures

2014-12-27 Thread Noah Misch
On Tue, Dec 23, 2014 at 03:32:59PM +0100, Tomas Vondra wrote:
 On 23.12.2014 15:21, Andrew Dunstan wrote:
  
  No, config_opts is what's passed to configure. Try something like:
  
  if ($branch eq 'REL9_0_STABLE')
  {
  $skip_steps{'pl-install-check'} = 1;
  }
 
 Applied to all three animals.

As of the time of this change, the animals ceased to report in.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Initdb-cs_CZ.WIN-1250 buildfarm failures

2014-12-23 Thread Noah Misch
On Sat, Dec 20, 2014 at 07:28:33PM +0100, Tomas Vondra wrote:
 On 20.12.2014 19:05, Tom Lane wrote:
  Locale cs_CZ.WIN-1250 is evidently marked with a codeset property of
  ANSI_X3.4-1968 (which means old-school US-ASCII).  That's certainly
  wrong.  I believe the correct thing would be CP1250.
 
 Yes. I fixed the locales and added the locales back to the client
 configuration.

Thanks.  These animals are now OK except on REL9_0_STABLE.  The log message of
commit 2dfa15d explains their ongoing 9.0 failures.  I recommend adding
--skip-steps=pl-install-check to their 9.0 invocations.  Dropping the affected
locales is another option, but we benefit from the rare encoding coverage more
than we benefit from the src/bin/pl coverage.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Initdb-cs_CZ.WIN-1250 buildfarm failures

2014-12-23 Thread Tomas Vondra
On 23.12.2014 09:19, Noah Misch wrote:
 On Sat, Dec 20, 2014 at 07:28:33PM +0100, Tomas Vondra wrote:
 On 20.12.2014 19:05, Tom Lane wrote:
 Locale cs_CZ.WIN-1250 is evidently marked with a codeset property of
 ANSI_X3.4-1968 (which means old-school US-ASCII).  That's certainly
 wrong.  I believe the correct thing would be CP1250.

 Yes. I fixed the locales and added the locales back to the client
 configuration.
 
 Thanks.  These animals are now OK except on REL9_0_STABLE.  The log message of
 commit 2dfa15d explains their ongoing 9.0 failures.  I recommend adding
 --skip-steps=pl-install-check to their 9.0 invocations.  Dropping the affected
 locales is another option, but we benefit from the rare encoding coverage more
 than we benefit from the src/bin/pl coverage.

OK, can do. Something like this in build-farm.conf should work, right?

if ($branch eq 'REL9_0_STABLE')
{
push(@{$conf{config_opts}},--skip-steps=pl-install-check);
}


regards
Tomas


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Initdb-cs_CZ.WIN-1250 buildfarm failures

2014-12-23 Thread Andrew Dunstan


On 12/23/2014 07:14 AM, Tomas Vondra wrote:

On 23.12.2014 09:19, Noah Misch wrote:

On Sat, Dec 20, 2014 at 07:28:33PM +0100, Tomas Vondra wrote:

On 20.12.2014 19:05, Tom Lane wrote:

Locale cs_CZ.WIN-1250 is evidently marked with a codeset property of
ANSI_X3.4-1968 (which means old-school US-ASCII).  That's certainly
wrong.  I believe the correct thing would be CP1250.

Yes. I fixed the locales and added the locales back to the client
configuration.

Thanks.  These animals are now OK except on REL9_0_STABLE.  The log message of
commit 2dfa15d explains their ongoing 9.0 failures.  I recommend adding
--skip-steps=pl-install-check to their 9.0 invocations.  Dropping the affected
locales is another option, but we benefit from the rare encoding coverage more
than we benefit from the src/bin/pl coverage.

OK, can do. Something like this in build-farm.conf should work, right?

if ($branch eq 'REL9_0_STABLE')
{
 push(@{$conf{config_opts}},--skip-steps=pl-install-check);
}




No, config_opts is what's passed to configure. Try something like:

if ($branch eq 'REL9_0_STABLE')
{
$skip_steps{'pl-install-check'} = 1;
}

cheers

andrew


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Initdb-cs_CZ.WIN-1250 buildfarm failures

2014-12-23 Thread Tomas Vondra
On 23.12.2014 15:21, Andrew Dunstan wrote:
 
 No, config_opts is what's passed to configure. Try something like:
 
 if ($branch eq 'REL9_0_STABLE')
 {
 $skip_steps{'pl-install-check'} = 1;
 }

Applied to all three animals.

Tomas


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Initdb-cs_CZ.WIN-1250 buildfarm failures

2014-12-20 Thread Noah Misch
On Sat, Dec 20, 2014 at 05:14:03PM +0100, CSPUG wrote:
 On 20.12.2014 07:39, Noah Misch wrote:
  Buildfarm members magpie, treepie and fulmar went absent on
  2014-10-29. Since returning on 2014-11-16, they have consistently
  failed with 'initdb: invalid locale name cs_CZ.WIN-1250'. No
  commits in that period readily explain a regression in this area. Did
  the underlying system configurations change?
 
 I'm pretty sure it's because of broken locales at the system level. It was
 working fine, and I haven't done any substantial changes to the system
 except for occasional yum update, so I'm unaware of what went wong :(
 
 The issue looks like this:
 
 # locale -a | grep en_US
 en_US
 en_US.iso88591
 en_US.iso885915
 en_US.utf8
 
 # locale en_US
 locale: unknown name en_US

The NAME argument to the locale tool is something like LC_PAPER, not a
locale name.  Use LANG=en_US locale LC_NUMERIC to test locale loading.

 The only reasons I can think of is that some of the updates required
 a reboot, and I haven't done that because that would kill all the VMs
 running on that HW, including the one with CLOBBER_CACHE_RECURSIVE
 tests. And that'd throw away tests running for ~3 months.
 
 I've disabled the three animals (magpie, fulmar, treepie) for now, because
 there's no point in running the tests until the locale issues are fixed. If
 anyone has an idea of what might be wrong, let me know.

Those animals have been successfully completing initdb for several locales,
including en_US, before failing at cs_CZ.WIN-1250.  You could disable just the
cs_CZ.WIN-1250 steps.  A CentOS 6.6 system here also lacks such a locale:

$ LANG=cs_CZ.WIN-1250 locale LC_NUMERIC
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
.

-1
46
0
ANSI_X3.4-1968
$ locale -a|grep cs_
cs_CZ
cs_CZ.iso88592
cs_CZ.utf8

Perhaps an update made the system stricter about rejecting unknown locales.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Initdb-cs_CZ.WIN-1250 buildfarm failures

2014-12-20 Thread Tom Lane
Noah Misch n...@leadboat.com writes:
 On Sat, Dec 20, 2014 at 05:14:03PM +0100, CSPUG wrote:
 On 20.12.2014 07:39, Noah Misch wrote:
 Buildfarm members magpie, treepie and fulmar went absent on
 2014-10-29. Since returning on 2014-11-16, they have consistently
 failed with 'initdb: invalid locale name cs_CZ.WIN-1250'. No
 commits in that period readily explain a regression in this area. Did
 the underlying system configurations change?

That's what I'd been assuming, but I had not got round to inquiring.

 I'm pretty sure it's because of broken locales at the system level. It was
 working fine, and I haven't done any substantial changes to the system
 except for occasional yum update, so I'm unaware of what went wong :(
 
 The only reasons I can think of is that some of the updates required
 a reboot, and I haven't done that because that would kill all the VMs
 running on that HW, including the one with CLOBBER_CACHE_RECURSIVE
 tests. And that'd throw away tests running for ~3 months.
 
 I've disabled the three animals (magpie, fulmar, treepie) for now, because
 there's no point in running the tests until the locale issues are fixed. If
 anyone has an idea of what might be wrong, let me know.

 Those animals have been successfully completing initdb for several locales,
 including en_US, before failing at cs_CZ.WIN-1250.  You could disable just the
 cs_CZ.WIN-1250 steps.  A CentOS 6.6 system here also lacks such a locale:

 $ LANG=cs_CZ.WIN-1250 locale LC_NUMERIC
 locale: Cannot set LC_CTYPE to default locale: No such file or directory
 locale: Cannot set LC_MESSAGES to default locale: No such file or directory
 locale: Cannot set LC_ALL to default locale: No such file or directory

FWIW, an actual RHEL 6.6 system doesn't like that either; at least not
with a fairly minimal set of locales installed.

 Perhaps an update made the system stricter about rejecting unknown locales.

Red Hat released RHEL 6.6 on Oct 14.  I am not sure how long it took the
CentOS crew to follow suit, but it's well within the realm of plausibility
that when these buildfarm critters went offline it was associated with a
system update from 6.5 to 6.6 ... which was a *major* update.  (On my
machine, something like 480 out of 1500 packages were updated.)  It's easy
to believe that either the glibc locale code is now stricter, or that Red
Hat reshuffled the packaging a bit and the missing locale is now in a
new package that didn't get installed.

Personally I'd not have tried to run such a system without rebooting ...
it's unlikely that not rebooting has anything directly to do with this
problem, but I would not be surprised by issues elsewhere.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Initdb-cs_CZ.WIN-1250 buildfarm failures

2014-12-20 Thread Pavel Stehule
2014-12-20 17:48 GMT+01:00 Noah Misch n...@leadboat.com:

 On Sat, Dec 20, 2014 at 05:14:03PM +0100, CSPUG wrote:
  On 20.12.2014 07:39, Noah Misch wrote:
   Buildfarm members magpie, treepie and fulmar went absent on
   2014-10-29. Since returning on 2014-11-16, they have consistently
   failed with 'initdb: invalid locale name cs_CZ.WIN-1250'. No
   commits in that period readily explain a regression in this area. Did
   the underlying system configurations change?
 
  I'm pretty sure it's because of broken locales at the system level. It
 was
  working fine, and I haven't done any substantial changes to the system
  except for occasional yum update, so I'm unaware of what went wong :(
 
  The issue looks like this:
 
  # locale -a | grep en_US
  en_US
  en_US.iso88591
  en_US.iso885915
  en_US.utf8
 
  # locale en_US
  locale: unknown name en_US

 The NAME argument to the locale tool is something like LC_PAPER, not a
 locale name.  Use LANG=en_US locale LC_NUMERIC to test locale loading.

  The only reasons I can think of is that some of the updates required
  a reboot, and I haven't done that because that would kill all the VMs
  running on that HW, including the one with CLOBBER_CACHE_RECURSIVE
  tests. And that'd throw away tests running for ~3 months.
 
  I've disabled the three animals (magpie, fulmar, treepie) for now,
 because
  there's no point in running the tests until the locale issues are fixed.
 If
  anyone has an idea of what might be wrong, let me know.

 Those animals have been successfully completing initdb for several locales,
 including en_US, before failing at cs_CZ.WIN-1250.  You could disable just
 the
 cs_CZ.WIN-1250 steps.  A CentOS 6.6 system here also lacks such a locale:

 $ LANG=cs_CZ.WIN-1250 locale LC_NUMERIC


It is Microsoft encoding, - it is not available on Linux

Regards

Pavel


 locale: Cannot set LC_CTYPE to default locale: No such file or directory
 locale: Cannot set LC_MESSAGES to default locale: No such file or directory
 locale: Cannot set LC_ALL to default locale: No such file or directory
 .

 -1
 46
 0
 ANSI_X3.4-1968
 $ locale -a|grep cs_
 cs_CZ
 cs_CZ.iso88592
 cs_CZ.utf8

 Perhaps an update made the system stricter about rejecting unknown locales.


 --
 Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
 To make changes to your subscription:
 http://www.postgresql.org/mailpref/pgsql-hackers



Re: [HACKERS] Initdb-cs_CZ.WIN-1250 buildfarm failures

2014-12-20 Thread Tomas Vondra
On 20.12.2014 18:13, Pavel Stehule wrote:
 
 2014-12-20 17:48 GMT+01:00 Noah Misch n...@leadboat.com

 $ LANG=cs_CZ.WIN-1250 locale LC_NUMERIC
 
 
 It is Microsoft encoding, - it is not available on Linux

Not true. It is available on Linux, and the regression tests were
running with it for a long time (essentially from the moment magpie was
added to the buildfarm).

Tomas


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Initdb-cs_CZ.WIN-1250 buildfarm failures

2014-12-20 Thread Pavel Stehule
2014-12-20 18:17 GMT+01:00 Tomas Vondra t...@fuzzy.cz:

 On 20.12.2014 18:13, Pavel Stehule wrote:
 
  2014-12-20 17:48 GMT+01:00 Noah Misch n...@leadboat.com
 
  $ LANG=cs_CZ.WIN-1250 locale LC_NUMERIC
 
 
  It is Microsoft encoding, - it is not available on Linux

 Not true. It is available on Linux, and the regression tests were
 running with it for a long time (essentially from the moment magpie was
 added to the buildfarm).


aha

I am sorry for my mistake

Pavel



 Tomas


 --
 Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
 To make changes to your subscription:
 http://www.postgresql.org/mailpref/pgsql-hackers



Re: [HACKERS] Initdb-cs_CZ.WIN-1250 buildfarm failures

2014-12-20 Thread Tom Lane
Tomas Vondra t...@fuzzy.cz writes:
 On 20.12.2014 18:13, Pavel Stehule wrote:
 It is Microsoft encoding, - it is not available on Linux

 Not true. It is available on Linux, and the regression tests were
 running with it for a long time (essentially from the moment magpie was
 added to the buildfarm).

Well, you had it at one time, but it sure doesn't seem to be there now.

I poked around in the RHEL 6.6 release notes, and found this possibly
relevant tidbit:

 mingw component, BZ#1063396

 Following the deprecation of Matahari packages in Red Hat
 Enterprise Linux 6.3, at which time the mingw packages were noted
 as deprecated, and the subsequent removal of Matahari packages
 from Red Hat Enterprise Linux 6.4, the mingw packages are now
 being removed from Red Hat Enterprise Linux 6.6.

 The mingw packages will no longer be shipped in future Red Hat
 Enterprise Linux 6 minor releases, nor will they receive
 security-related updates. Consequently, users are advised to
 uninstall any earlier releases of the mingw packages from their
 Red Hat Enterprise Linux 6 systems.

It seems plausible that a WIN-1250 locale would have been something
that would've been supplied by mingw rather than being part of either
glibc-common or any official locale package.  I'm not sure how that
translates to it actively disappearing from your machine --- as the note
says, you're advised to uninstall mingw, but I don't think the 6.6
update would have removed it automatically.  Still, the outlines of a
theory are starting to come into focus.

Immediate recommendation is to restart the buildfarm critters with the
missing locale removed from their configurations.  If you can find the
locale again somewhere, by all means re-enable it, but better less testing
than no testing in the meantime.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Initdb-cs_CZ.WIN-1250 buildfarm failures

2014-12-20 Thread Andrew Dunstan


On 12/20/2014 12:32 PM, Tom Lane wrote:



Immediate recommendation is to restart the buildfarm critters with the
missing locale removed from their configurations.  If you can find the
locale again somewhere, by all means re-enable it, but better less testing
than no testing in the meantime.





Yeah, all they need to do is alter the locales setting in the config 
file. If they want to force new runs, they can use 
https://wiki.postgresql.org/wiki/PostgreSQL_Buildfarm_Howto#Tips_and_Tricks



cheers

andrew


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Initdb-cs_CZ.WIN-1250 buildfarm failures

2014-12-20 Thread Tomas Vondra
On 20.12.2014 17:48, Noah Misch wrote:
 On Sat, Dec 20, 2014 at 05:14:03PM +0100, CSPUG wrote:
 On 20.12.2014 07:39, Noah Misch wrote:
 Buildfarm members magpie, treepie and fulmar went absent on
 2014-10-29. Since returning on 2014-11-16, they have consistently
 failed with 'initdb: invalid locale name cs_CZ.WIN-1250'. No
 commits in that period readily explain a regression in this area. Did
 the underlying system configurations change?

 I'm pretty sure it's because of broken locales at the system level. It was
 working fine, and I haven't done any substantial changes to the system
 except for occasional yum update, so I'm unaware of what went wong :(

 The issue looks like this:

 # locale -a | grep en_US
 en_US
 en_US.iso88591
 en_US.iso885915
 en_US.utf8

 # locale en_US
 locale: unknown name en_US
 
 The NAME argument to the locale tool is something like LC_PAPER, not a
 locale name.  Use LANG=en_US locale LC_NUMERIC to test locale loading.

D'oh! That kinda explains the strange `locale` failures, of course ...

But still, the three animals (magpie, fulmar, treepie) are all running
with these locales:

   C en_US
   cs_CZ.UTF-8 cs_CZ.ISO-8859-2 cs_CZ.WIN-1250
   sk_SK.UTF-8 sk_SK.ISO-8859-2 sk_SK.WIN-1250

and the only one missing (so the 'locale LC_NUMERIC' failed) was
sk_SK.WIN-1250. I've fixed that, and now all the locales work fine.

That however does not explain why the tests are failing with
cs_CZ.WIN-1250 because that locale was working for some time.

 
 The only reasons I can think of is that some of the updates required
 a reboot, and I haven't done that because that would kill all the VMs
 running on that HW, including the one with CLOBBER_CACHE_RECURSIVE
 tests. And that'd throw away tests running for ~3 months.

 I've disabled the three animals (magpie, fulmar, treepie) for now, because
 there's no point in running the tests until the locale issues are fixed. If
 anyone has an idea of what might be wrong, let me know.
 
 Those animals have been successfully completing initdb for several locales,
 including en_US, before failing at cs_CZ.WIN-1250.  You could disable just the
 cs_CZ.WIN-1250 steps.  A CentOS 6.6 system here also lacks such a locale:
 
 $ LANG=cs_CZ.WIN-1250 locale LC_NUMERIC
 locale: Cannot set LC_CTYPE to default locale: No such file or directory
 locale: Cannot set LC_MESSAGES to default locale: No such file or directory
 locale: Cannot set LC_ALL to default locale: No such file or directory
 .
 
 -1
 46
 0
 ANSI_X3.4-1968
 $ locale -a|grep cs_
 cs_CZ
 cs_CZ.iso88592
 cs_CZ.utf8
 
 Perhaps an update made the system stricter about rejecting unknown locales.

I believe the locale system (at the OS level) works just like before. I
remember I had to manually create the locales while initially setting up
the animals. Then, ~2 months ago something happened (I asssume a yum
update) and some of the locales disappeared. But I have recreated them,
except for sk_SK.WIN-1250. But the tests fail because of cs_CZ.WIN-1250
which does exist.

Attached is a log of

for l in cs_CZ.UTF-8 sk_SK.UTF-8 cs_CZ.ISO-8859-2 sk_SK.ISO-8859-2
cs_CZ.WIN-1250 sk_SK.WIN-1250; do
  echo $l; LANG=$l locale LC_NUMERIC;
done;

showing that all the locales exist. However when I tried to initialize a
cluster with cs_CZ.WIN-1250, I got an error like this:

[pgbuild@regular-builds ~]$ locale
LANG=cs_CZ.WIN-1250
LC_CTYPE=cs_CZ.WIN-1250
LC_NUMERIC=cs_CZ.WIN-1250
LC_TIME=cs_CZ.WIN-1250
LC_COLLATE=cs_CZ.WIN-1250
LC_MONETARY=cs_CZ.WIN-1250
LC_MESSAGES=cs_CZ.WIN-1250
LC_PAPER=cs_CZ.WIN-1250
LC_NAME=cs_CZ.WIN-1250
LC_ADDRESS=cs_CZ.WIN-1250
LC_TELEPHONE=cs_CZ.WIN-1250
LC_MEASUREMENT=cs_CZ.WIN-1250
LC_IDENTIFICATION=cs_CZ.WIN-1250
LC_ALL=
[pgbuild@regular-builds ~]$ pg_ctl -D tmp-data init
The files belonging to this database system will be owned by user pgbuild.
This user must also own the server process.

The database cluster will be initialized with locale cs_CZ.WIN-1250.
could not determine encoding for locale cs_CZ.WIN-1250: codeset is
ANSI_X3.4-1968
initdb: could not find suitable encoding for locale cs_CZ.WIN-1250
Rerun initdb with the -E option.
Try initdb --help for more information.
pg_ctl: database system initialization failed

So apparently the locale does exist, but we're unable to cope with it
for some reason ...

Tomas
cs_CZ.UTF-8
,
 
3;3
44
160
UTF-8
sk_SK.UTF-8
,
 
3;3
44
160
UTF-8
cs_CZ.ISO-8859-2
,
�
3;3
44
160
ISO-8859-2
sk_SK.ISO-8859-2
,
�
3;3
44
160
ISO-8859-2
cs_CZ.WIN-1250
,
 
3;3
44
160
ANSI_X3.4-1968
sk_SK.WIN-1250
,
 
3;3
44
160
ANSI_X3.4-1968
-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Initdb-cs_CZ.WIN-1250 buildfarm failures

2014-12-20 Thread Tomas Vondra
On 20.12.2014 18:32, Tom Lane wrote:
 Tomas Vondra t...@fuzzy.cz writes:
 On 20.12.2014 18:13, Pavel Stehule wrote:
 It is Microsoft encoding, - it is not available on Linux
 
 Not true. It is available on Linux, and the regression tests were 
 running with it for a long time (essentially from the moment magpie
 was added to the buildfarm).
 
 Well, you had it at one time, but it sure doesn't seem to be there now.

 I poked around in the RHEL 6.6 release notes, and found this possibly
 relevant tidbit:

Ah, apparently I upgraded this VM to 6.6 - that wasn't exactly planned.

 
mingw component, BZ#1063396
 
Following the deprecation of Matahari packages in Red Hat
Enterprise Linux 6.3, at which time the mingw packages were noted
as deprecated, and the subsequent removal of Matahari packages
from Red Hat Enterprise Linux 6.4, the mingw packages are now
being removed from Red Hat Enterprise Linux 6.6.
 
The mingw packages will no longer be shipped in future Red Hat
Enterprise Linux 6 minor releases, nor will they receive
security-related updates. Consequently, users are advised to
uninstall any earlier releases of the mingw packages from their
Red Hat Enterprise Linux 6 systems.
 
 It seems plausible that a WIN-1250 locale would have been something 
 that would've been supplied by mingw rather than being part of
 either glibc-common or any official locale package. I'm not sure how
 that translates to it actively disappearing from your machine --- as
 the note says, you're advised to uninstall mingw, but I don't think
 the 6.6 update would have removed it automatically. Still, the
 outlines of a theory are starting to come into focus.

I don't think so. As I mentioned in the previous post, I still can
create the locale, but initdb still fails with an error about
ANSI_X3.4-1968 encoding.

But clearly, something changed between RH 6.5 and 6.6, because on 6.5 I
get this:

$ LANG=cs_CZ.WIN-1250 locale LC_NUMERIC
,
�
3;3
44
160
CP1250

while on 6.6 I get this:

$ LANG=cs_CZ.WIN-1250 locale LC_NUMERIC
,

3;3
44
160
ANSI_X3.4-1968

So yeah, the locale definition did change a bit - the  encoding is
different, and we can't cope with it.

 Immediate recommendation is to restart the buildfarm critters with
 the missing locale removed from their configurations. If you can find
 the locale again somewhere, by all means re-enable it, but better
 less testing than no testing in the meantime.

I've removed cs_CZ.WIN-1250 and sk_SK.WIN-1250 from the config. Let's
see how that works.


Tomas


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Initdb-cs_CZ.WIN-1250 buildfarm failures

2014-12-20 Thread Tom Lane
Tomas Vondra t...@fuzzy.cz writes:
 I believe the locale system (at the OS level) works just like before. I
 remember I had to manually create the locales while initially setting up
 the animals. Then, ~2 months ago something happened (I asssume a yum
 update) and some of the locales disappeared. But I have recreated them,
 except for sk_SK.WIN-1250. But the tests fail because of cs_CZ.WIN-1250
 which does exist.

I am betting that you recreated them differently from before.

 However when I tried to initialize a
 cluster with cs_CZ.WIN-1250, I got an error like this:

 [pgbuild@regular-builds ~]$ pg_ctl -D tmp-data init
 The files belonging to this database system will be owned by user pgbuild.
 This user must also own the server process.

 The database cluster will be initialized with locale cs_CZ.WIN-1250.
 could not determine encoding for locale cs_CZ.WIN-1250: codeset is
 ANSI_X3.4-1968
 initdb: could not find suitable encoding for locale cs_CZ.WIN-1250

Locale cs_CZ.WIN-1250 is evidently marked with a codeset property of
ANSI_X3.4-1968 (which means old-school US-ASCII).  That's certainly
wrong.  I believe the correct thing would be CP1250.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Initdb-cs_CZ.WIN-1250 buildfarm failures

2014-12-20 Thread Tom Lane
Tomas Vondra t...@fuzzy.cz writes:
 But clearly, something changed between RH 6.5 and 6.6, because on 6.5 I
 get this:

 $ LANG=cs_CZ.WIN-1250 locale LC_NUMERIC
 ,
 �
 3;3
 44
 160
 CP1250

 while on 6.6 I get this:

 $ LANG=cs_CZ.WIN-1250 locale LC_NUMERIC
 ,

 3;3
 44
 160
 ANSI_X3.4-1968

That's certainly broken.  The entire point of having a cs_CZ.WIN-1250
locale (as opposed to cs_CZ.something-else) would be to specify a
codeset corresponding to WIN-1250.  Our code recognizes the spelling
CP1250 for that.  It's possible there are other spellings we should
recognize, but ANSI_X3.4-1968 is certainly not one.  As I said, that
just means ASCII, so it's completely useless for determining which
ASCII-superset encoding is wanted.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Initdb-cs_CZ.WIN-1250 buildfarm failures

2014-12-20 Thread Tomas Vondra
On 20.12.2014 19:05, Tom Lane wrote:
 Tomas Vondra t...@fuzzy.cz writes:
 I believe the locale system (at the OS level) works just like before. I
 remember I had to manually create the locales while initially setting up
 the animals. Then, ~2 months ago something happened (I asssume a yum
 update) and some of the locales disappeared. But I have recreated them,
 except for sk_SK.WIN-1250. But the tests fail because of cs_CZ.WIN-1250
 which does exist.
 
 I am betting that you recreated them differently from before.

And you're probably right. Apparently, I recreated them like this:

  $ localedef -v -c -i cs_CZ -f WIN-1250 cs_CZ.WIN-1250

but the correct way seems to be this:

  $ localedef -v -c -i cs_CZ -f CP1250 cs_CZ.WIN-1250

 However when I tried to initialize a
 cluster with cs_CZ.WIN-1250, I got an error like this:
 
 [pgbuild@regular-builds ~]$ pg_ctl -D tmp-data init
 The files belonging to this database system will be owned by user pgbuild.
 This user must also own the server process.
 
 The database cluster will be initialized with locale cs_CZ.WIN-1250.
 could not determine encoding for locale cs_CZ.WIN-1250: codeset is
 ANSI_X3.4-1968
 initdb: could not find suitable encoding for locale cs_CZ.WIN-1250
 
 Locale cs_CZ.WIN-1250 is evidently marked with a codeset property of
 ANSI_X3.4-1968 (which means old-school US-ASCII).  That's certainly
 wrong.  I believe the correct thing would be CP1250.

Yes. I fixed the locales and added the locales back to the client
configuration.

regards
Tomas Vondra


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Initdb-cs_CZ.WIN-1250 buildfarm failures

2014-12-20 Thread Tom Lane
Tomas Vondra t...@fuzzy.cz writes:
 On 20.12.2014 19:05, Tom Lane wrote:
 I am betting that you recreated them differently from before.

 And you're probably right. Apparently, I recreated them like this:

   $ localedef -v -c -i cs_CZ -f WIN-1250 cs_CZ.WIN-1250

 but the correct way seems to be this:

   $ localedef -v -c -i cs_CZ -f CP1250 cs_CZ.WIN-1250

Interesting.  Apparently, instead of failing outright on an unrecognized
charmap name, localedef just substituted ASCII and plowed ahead.  Bad dog.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Initdb-cs_CZ.WIN-1250 buildfarm failures

2014-12-20 Thread CSPUG
On 20.12.2014 07:39, Noah Misch wrote:
 Buildfarm members magpie, treepie and fulmar went absent on
 2014-10-29. Since returning on 2014-11-16, they have consistently
 failed with 'initdb: invalid locale name cs_CZ.WIN-1250'. No
 commits in that period readily explain a regression in this area. Did
 the underlying system configurations change?

I'm pretty sure it's because of broken locales at the system level. It was
working fine, and I haven't done any substantial changes to the system
except for occasional yum update, so I'm unaware of what went wong :(

The issue looks like this:

# locale -a | grep en_US
en_US
en_US.iso88591
en_US.iso885915
en_US.utf8

# locale en_US
locale: unknown name en_US

The only reasons I can think of is that some of the updates required
a reboot, and I haven't done that because that would kill all the VMs
running on that HW, including the one with CLOBBER_CACHE_RECURSIVE
tests. And that'd throw away tests running for ~3 months.

I've disabled the three animals (magpie, fulmar, treepie) for now, because
there's no point in running the tests until the locale issues are fixed. If
anyone has an idea of what might be wrong, let me know.

Tomas


Re: [HACKERS] Initdb-cs_CZ.WIN-1250 buildfarm failures

2014-12-20 Thread Tomas Vondra
On 20.12.2014 19:35, Tom Lane wrote:
 Tomas Vondra t...@fuzzy.cz writes:
 On 20.12.2014 19:05, Tom Lane wrote:
 I am betting that you recreated them differently from before.
 
 And you're probably right. Apparently, I recreated them like this:
 
   $ localedef -v -c -i cs_CZ -f WIN-1250 cs_CZ.WIN-1250
 
 but the correct way seems to be this:
 
   $ localedef -v -c -i cs_CZ -f CP1250 cs_CZ.WIN-1250
 
 Interesting.  Apparently, instead of failing outright on an unrecognized
 charmap name, localedef just substituted ASCII and plowed ahead.  Bad dog.

Not really. It's rather about abusive owner of the dog, using '-c' to
force the dog to create the locale even when there are warings:

  # localedef -i cs_CZ -f WIN-1250 cs_CZ.WIN-1250
  character map file `WIN-1250' not found: No such file or directory
  no output file produced because warnings were issued

In my defense, I've been using verbose mode, and that produces a lot of
warnings like 'non-symbolic character value should not be used' (which
gets ignored in non-verbose mode) and thus missed the one important one.

regards
Tomas


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Initdb-cs_CZ.WIN-1250 buildfarm failures

2014-12-19 Thread Noah Misch
Buildfarm members magpie, treepie and fulmar went absent on 2014-10-29.  Since
returning on 2014-11-16, they have consistently failed with 'initdb: invalid
locale name cs_CZ.WIN-1250'.  No commits in that period readily explain a
regression in this area.  Did the underlying system configurations change?

Thanks,
nm


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers