date:20180904

Collation versioning

2018-09-04 Thread Thomas Munro

Hello,

While reviewing the ICU versioning work a while back, I mentioned the
idea of using a user-supplied command to get a collversion string for
the libc collation provider.  I was reminded about that by recent news
about an upcoming glibc/CLDR resync that is likely to affect
PostgreSQL users (though, I guess, probably only when they do a major
OS upgrade).  Here's an experimental patch to try that idea out.  For
example, you might set it like this:

libc_collation_version_command = 'md5
/usr/share/locale/@LC_COLLATE@/LC_COLLATE | sed "s/.* = //"'

... or, on a Debian system using the locales package, like this:

libc_collation_version_command = 'dpkg -s locales | grep Version: |
sed "s/Version: //"'

Using the checksum approach, it works like this:

postgres=# alter collation "xx_XX" refresh version;
NOTICE:  changing version from b88d621596b7e61337e832f7841066a9 to
7b008442fbaf5dfe7a10fb3d82a634ab
ALTER COLLATION
postgres=# select * from pg_collation where collname = 'xx_XX';
-[ RECORD 1 ]-+-
collname  | xx_XX
collnamespace | 2200
collowner | 10
collprovider  | c
collencoding  | 6
collcollate   | en_US.UTF-8
collctype | UTF-8
collversion   | 7b008442fbaf5dfe7a10fb3d82a634ab

When the collation definition changes you get the desired scary
warning on next attempt to use it in a fresh backend:

postgres=# select * from t order by v;
WARNING:  collation "xx_XX" has version mismatch
DETAIL:  The collation in the database was created using version
b88d621596b7e61337e832f7841066a9, but the operating system provides
version 7b008442fbaf5dfe7a10fb3d82a634ab.
HINT:  Rebuild all objects affected by this collation and run ALTER
COLLATION public."xx_XX" REFRESH VERSION, or build PostgreSQL with the
right library version.

The problem is that it isn't in effect at initdb time so if you add
that later it only affects new locales.  You'd need a way to do that
during init to capture the imported system locale versions, and that's
a really ugly string to have to pass into some initdb option.  Ugh.

Another approach would be to decide that we're willing to put
non-portable version extracting magic in pg_locale.c.   On a long
flight I hacked my libc to store a version string (based on CLDR
version or whatever) in its binary locale definitions and provide a
proper interface to ask for it, modelled on querylocale(3):

const char *querylocaleversion(int mask, locale_t locale);

Then the patch for pg_locale.c is trivial, see attached.  While I
could conceivably try to convince my local friendly OS to take such a
patch, the real question is how to deal with glibc.  Does anyone know
of a way to extract a version string from glibc using existing
interfaces?  I heard there was an undocumented way but I haven't been
able to find it -- probably because I was, erm, looking in the
documentation.

Or maybe this isn't worth bothering with, and we should just build out
the ICU support and then make it the default and be done with it.

In passing, here's a patch to add tab completion for ALTER COLLATION
... REFRESH VERSION.

-- 
Thomas Munro
http://www.enterprisedb.com


0001-Auto-complete-ALTER-COLLATION-.-REFRESH-VERSION.patch
Description: Binary data


0001-Add-libc_collation_version_command-GUC.patch
Description: Binary data


0001-Add-libc_collation_version_command-GUC.patch
Description: Binary data

Re: Bug fix for glibc broke freebsd build in REL_11_STABLE

2018-09-04 Thread Andrew Gierth

> "Andres" == Andres Freund  writes:

 >> However, this commit broke float8 test on 32-bit FreeBSD 11 with
 >> clang 3.8.0 compiler. Regressions.diff follows:

 Andres> Does this happen with a newer clang version too?

float8 test (and all other tests) passes for me on clang 3.9.1 on fbsd11
on 32-bit ARM, and on -m32 builds on amd64.

I also confirmed that without #define isinf(x) __builtin_isinf(x), on
both 32bit and 64bit fbsd isinf() compiles as a function call, so the
OP's proposed change would not be desirable.

-- 
Andrew (irc:RhodiumToad)

Re: pointless check in RelationBuildPartitionDesc

2018-09-04 Thread Alvaro Herrera

Proposed patch.  Checking isnull in a elog(ERROR) is important, because
the column is not marked NOT NULL.  This is not true for other columns
where we simply do Assert(!isnull).

-- 
Álvaro Herrerahttps://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
commit dc5f9933595c95471898e506e004a1799ea4eae5[m
Author: Alvaro Herrera 
AuthorDate: Tue Sep 4 13:24:25 2018 -0300
CommitDate: Tue Sep 4 13:27:30 2018 -0300

fix relpartbound accesses

diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index f46af41b56..ffdd14b819 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -773,9 +773,6 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,

  InvalidOid,

  typaddress);
 
-   /* Store inheritance information for new rel. */
-   StoreCatalogInheritance(relationId, inheritOids, stmt->partbound != 
NULL);
-
/*
 * We must bump the command counter to make the newly-created relation
 * tuple visible for opening.
@@ -869,6 +866,9 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
heap_close(parent, NoLock);
}
 
+   /* Store inheritance information for new rel. */
+   StoreCatalogInheritance(relationId, inheritOids, stmt->partbound != 
NULL);
+
/*
 * Process the partitioning specification (if any) and store the 
partition
 * key information into the catalog.
@@ -14705,7 +14705,9 @@ ATExecDetachPartition(Relation rel, RangeVar *name)
 
(void) SysCacheGetAttr(RELOID, tuple, Anum_pg_class_relpartbound,
   );
-   Assert(!isnull);
+   if (isnull)
+   elog(ERROR, "null relpartbound for relation %u",
+RelationGetRelid(partRel));
 
/* Clear relpartbound and reset relispartition */
memset(new_val, 0, sizeof(new_val));
diff --git a/src/backend/partitioning/partbounds.c 
b/src/backend/partitioning/partbounds.c
index 9015a05d32..4ed9920618 100644
--- a/src/backend/partitioning/partbounds.c
+++ b/src/backend/partitioning/partbounds.c
@@ -1641,8 +1641,9 @@ get_qual_for_range(Relation parent, PartitionBoundSpec 
*spec,
datum = SysCacheGetAttr(RELOID, tuple,

Anum_pg_class_relpartbound,

);
+   if (isnull)
+   elog(ERROR, "null relpartbound for relation 
%u", inhrelid);
 
-   Assert(!isnull);
bspec = (PartitionBoundSpec *)
stringToNode(TextDatumGetCString(datum));
if (!IsA(bspec, PartitionBoundSpec))
diff --git a/src/backend/utils/cache/partcache.c 
b/src/backend/utils/cache/partcache.c
index 115a9fe78f..2fc876b7d8 100644
--- a/src/backend/utils/cache/partcache.c
+++ b/src/backend/utils/cache/partcache.c
@@ -302,23 +302,11 @@ RelationBuildPartitionDesc(Relation rel)
if (!HeapTupleIsValid(tuple))
elog(ERROR, "cache lookup failed for relation %u", 
inhrelid);
 
-   /*
-* It is possible that the pg_class tuple of a partition has 
not been
-* updated yet to set its relpartbound field.  The only case 
where
-* this happens is when we open the parent relation to check 
using its
-* partition descriptor that a new partition's bound does not 
overlap
-* some existing partition.
-*/
-   if (!((Form_pg_class) GETSTRUCT(tuple))->relispartition)
-   {
-   ReleaseSysCache(tuple);
-   continue;
-   }
-
datum = SysCacheGetAttr(RELOID, tuple,

Anum_pg_class_relpartbound,
);
-   Assert(!isnull);
+   if (isnull)
+   elog(ERROR, "null relpartbound for partition %u" 
inhrelid);
boundspec = (Node *) stringToNode(TextDatumGetCString(datum));
 
/*
@@ -883,9 +871,8 @@ generate_partition_qual(Relation rel)
boundDatum = SysCacheGetAttr(RELOID, tuple,
 
Anum_pg_class_relpartbound,
 );
-   if (isnull) /* should not happen */
-   elog(ERROR, "relation \"%s\" has relpartbound = null",
-

Re: Collation versioning

2018-09-04 Thread Thomas Munro

On Tue, Sep 4, 2018 at 10:02 AM Thomas Munro
 wrote:
> const char *querylocaleversion(int mask, locale_t locale);
>
> Then the patch for pg_locale.c is trivial, see attached.

Oops, here's that one, FWIW.

-- 
Thomas Munro
http://www.enterprisedb.com


0001-Use-querylocaleversion-3-if-available.patch
Description: Binary data

Re: Add SKIP LOCKED to VACUUM and ANALYZE

2018-09-04 Thread Michael Paquier

On Tue, Sep 04, 2018 at 03:49:09PM +, Bossart, Nathan wrote:
> Yes.  I've started working on this again, but the new patch set is
> probably still a few days out.

Thanks, Nathan.
--
Michael


signature.asc
Description: PGP signature

Re: PostgreSQL logical decoder output plugin - unchanged toast data

2018-09-04 Thread Andres Freund

Hi,

On 2018-09-04 11:45:10 +0200, Georgy Buranov wrote:
> Hi Andres. Thank you very much for your help.
> 
> I tried the following solution and I have some problems.
> 
> * I have 9.6 postgres and I do not have separate field for rd_pkindex
> * All I have is rd_replidindex field. Usually (when REPLICA IDENTITY
> is NOT FULL), it still contains the primary key
> * But in the case of REPLICA IDENTITY FULL - rd_replidindex is NULL
> and rd_pkindex does not exist
> 
> So, is there a way to get the primary key on 9.6 postgres?

You have to to query the syscache yourself, in that case.  I know
nothing about what you're doing in your output plugin.  In the ones I
wrote there's a side-cache for per-relation output plugin specific data,
do you have something like that?  You can probably just crib the code
for the lookup from dblink.c:get_pkey_attnames().  If you have a cache,
itmight be worth caching the results.

Greetings,

Andres Freund

Re: Enable using IS NOT DISTINCT FROM in hash and merge joins

2018-09-04 Thread Alexander Kuzmenkov

Hi,

I heard from my colleagues that we have an extension that does a similar
thing. I'm attaching the source. It creates operator "==" for some data
types, that works like "IS NOT DISTINCT FROM". You can then perform hash
joins on this operator. Apparently the hash join machinery supports
non-strict operators, but I'm not sure about the hash indexes.

I think the question we have to answer is what prevents us from having
hash and merge joins on non-strict operators? Hash joins seem to work
already, so we can just create a custom operator and use it. Merge join
executor can be adapted to work as well, but the planner would require
more complex changes. Just adding a check for DistinctExpr to
check_mergejoinable probably breaks equivalence classes. The problem
with merge join planning is that it has the notion of "mergejoinable
operator", which is a strict btree equality operator, and it uses such
operators both to perform merge joins and to conclude that some two
variables must be equal (that is, create equivalence classes). If we are
going to perform merge joins on some other kinds of operators, we have
to disentangle these two uses. I had to do this to support merge joins
on inequality clauses, you can take a look at this thread if you wish:
https://www.postgresql.org/message-id/flat/b31e1a2d-5ed2-cbca-649e-136f1a7c4c31%40postgrespro.ru

73 matches

Mail list logo