Re: ICU for global collation

2019-08-20 Thread Andrey Borodin
Hi! > 20 авг. 2019 г., в 19:21, Peter Eisentraut > написал(а): > > Here is an initial patch to add the option to use ICU as the global > collation provider, a long-requested feature. > > To activate, use something like > >initdb --collation-provider=icu --locale=... > > A trick here is

Re: range_agg

2019-08-20 Thread Jeff Davis
On Sat, 2019-08-17 at 10:47 -0700, Paul A Jungwirth wrote: > So I'm wondering how seriously I should take this for multiranges? I > guess if a range type did support typmods, it would just delegate to > the underlying element type for their meaning, and so a multirange > should delegate it too? Is

Re: POC: Cleaning up orphaned files using undo logs

2019-08-20 Thread Robert Haas
On Tue, Aug 13, 2019 at 8:11 AM Robert Haas wrote: > > We can probably check the fxid queue and error queue to get that > > value. However, I am not sure if that is sufficient because incase we > > perform the request in the foreground, it won't be present in queues. > > Oh, I forgot about that r

Re: Serialization questions

2019-08-20 Thread Richard Guo
On Wed, Aug 21, 2019 at 9:30 AM Alex wrote: > > first issue "set default_transaction_isolation to 'serializable';" on the > both sessions, then run: > > Session 1: begin; select * from t; (2 rows selected); > Session 2: delete from t; (committed automatically) > Session 1: commit; (com

Re: Add "password_protocol" connection parameter to libpq

2019-08-20 Thread Jeff Davis
On Mon, 2019-08-19 at 14:51 +0900, Michael Paquier wrote: > On Fri, Aug 16, 2019 at 02:11:57PM -0400, Jonathan S. Katz wrote: > > To be pedantic, +1 on the channel_binding param. > > Seems like we are moving in this direction then. I don't object to > the introduction of this parameter. OK, new

Re: Serialization questions

2019-08-20 Thread Tatsuo Ishii
> Before understanding how postgres implements the serializable isolation > level (I have see many paper related to it), I have question about how it > should be. > > > I mainly read the ideas from > https://www.postgresql.org/docs/11/transaction-iso.html. > > > In fact, this isolation level wo

Re: POC: Cleaning up orphaned files using undo logs

2019-08-20 Thread Robert Haas
On Mon, Aug 19, 2019 at 2:04 AM Dilip Kumar wrote: > Currently, In UnpackedUndoRecord we store all members directly which > are set by the caller. We store pointers to some header which are > allocated internally by the undo layer and the caller need not worry > about setting them. So now you ar

Re: Cleanup isolation specs from unused steps

2019-08-20 Thread Michael Paquier
On Tue, Aug 20, 2019 at 09:54:56AM -0400, Alvaro Herrera wrote: > On 2019-Aug-20, Tom Lane wrote: >> If you can warn in both cases, that'd be OK perhaps. But Alvaro's >> description of the intended use of dry-run makes it sound like >> it would be expected for there to be unreferenced steps, since

Re: Serialization questions

2019-08-20 Thread Alex
On Tue, Aug 20, 2019 at 4:47 PM Alex wrote: > Before understanding how postgres implements the serializable isolation > level (I have see many paper related to it), I have question about how it > should be. > > > I mainly read the ideas from > https://www.postgresql.org/docs/11/transaction-iso.ht

Re: [PATCH] Stop ALTER SYSTEM from making bad assumptions

2019-08-20 Thread Ian Barwick
On 8/16/19 12:22 AM, Tom Lane wrote: Stephen Frost writes: * Tom Lane (t...@sss.pgh.pa.us) wrote: In hopes of moving this along, I've pushed Ian's last code change, as there seems to be no real argument about that anymore. As for the doc changes, how about the attached revision of what I wrot

Re: REL_12_STABLE crashing with assertion failure in ExtractReplicaIdentity

2019-08-20 Thread Andres Freund
Hi, On 2019-08-17 01:43:45 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2019-08-16 09:44:15 -0700, Hadi Moshayedi wrote: > >> It seems that sometimes when DELETE cascades to referencing tables we fail > >> to acquire locks on replica identity index. > > > I suspect this "always" has been

Re: Fix typos and inconsistencies for HEAD (take 11)

2019-08-20 Thread Alvaro Herrera
On 2019-Aug-19, Michael Paquier wrote: > On Mon, Aug 19, 2019 at 07:04:04AM +0300, Alexander Lakhin wrote: > > 11.23 TupleLockUpdate -> LockTupleNoKeyExclusive > > Not sure about this one, so discarded for now. Alvaro? Yeah, Alexander proposed change is correct. I just pushed it. > > 11.33 vi

Re: Inadequate executor locking of indexes

2019-08-20 Thread Andres Freund
Hi, On 2018-11-23 17:41:26 +1300, David Rowley wrote: > Ideally, the locking code would realise we already hold a stronger > lock and skip the lock, but I don't see how that's realistically > possible without probing the hash table for all stronger lock types > first, which would likely damage the

Re: Improve default partition

2019-08-20 Thread Alvaro Herrera
On 2019-Aug-20, Rafia Sabih wrote: > This does not sound very convenient. I was thinking of having some > mechanism for such insertions which automatically creates a default > partition and gives a notice for the user to know that it is going to > the default partition. Basically, always having a

Re: errbacktrace

2019-08-20 Thread Peter Eisentraut
On 2019-08-13 15:24, Alvaro Herrera wrote: > On 2019-Aug-13, Peter Eisentraut wrote: > >> I have changed the configuration setting to backtrace_functions plural, >> so that you can debug more than one location at once. I had originally >> wanted to do that but using existing functions like >> Spl

Re: POC: Cleaning up orphaned files using undo logs

2019-08-20 Thread Robert Haas
On Tue, Aug 20, 2019 at 5:02 AM Thomas Munro wrote: > 3. UndoLogDiscard() uses DiscardBuffer() to invalidate any currently > unpinned buffers, and marks as BM_DISCARDED any that happen to be > pinned right now, so they can't be immediately invalidated. Such > buffers are never written back and a

Re: POC: Cleaning up orphaned files using undo logs

2019-08-20 Thread Andres Freund
On 2019-08-20 09:44:23 -0700, Andres Freund wrote: > On 2019-08-20 21:02:18 +1200, Thomas Munro wrote: > > Aside from code changes based on review (and I have more to come of > > those), the attached experimental patchset (also at > > https://github.com/EnterpriseDB/zheap/tree/undo) has a new proto

Re: POC: Cleaning up orphaned files using undo logs

2019-08-20 Thread Andres Freund
Hi, On 2019-08-20 17:11:38 +0530, Dilip Kumar wrote: > On Wed, Aug 14, 2019 at 10:35 PM Andres Freund wrote: > > On 2019-08-14 14:48:07 +0530, Dilip Kumar wrote: > > > On Wed, Aug 14, 2019 at 12:27 PM Andres Freund wrote: > > > I don't think we can normally pin the undo buffers properly at that

Re: Improve default partition

2019-08-20 Thread Dmitry Dolgov
> On Tue, Aug 20, 2019 at 4:45 PM Rafia Sabih wrote: > > This does not sound very convenient. I was thinking of having some > mechanism for such insertions which automatically creates a default > partition and gives a notice for the user to know that it is going to > the default partition. If I r

Re: Global temporary tables

2019-08-20 Thread Pavel Stehule
út 20. 8. 2019 v 18:42 odesílatel Konstantin Knizhnik < k.knizh...@postgrespro.ru> napsal: > > > On 20.08.2019 19:06, Pavel Stehule wrote: > > > > As I wrote at the beginning of this thread, one of the problems with >> temporary table sis that it is not possible to use them at replica. >> Global t

Re: POC: Cleaning up orphaned files using undo logs

2019-08-20 Thread Andres Freund
Hi, On 2019-08-20 21:02:18 +1200, Thomas Munro wrote: > Aside from code changes based on review (and I have more to come of > those), the attached experimental patchset (also at > https://github.com/EnterpriseDB/zheap/tree/undo) has a new protocol > that, I hope, allows for better concurrency, rel

Re: Global temporary tables

2019-08-20 Thread Konstantin Knizhnik
On 20.08.2019 19:06, Pavel Stehule wrote: As I wrote at the beginning of this thread, one of the problems with temporary table sis that it is not possible to use them at replica. Global temp tables allows to share metadata between master and replica. I am not sure if I u

Re: Global temporary tables

2019-08-20 Thread Pavel Stehule
út 20. 8. 2019 v 16:51 odesílatel Konstantin Knizhnik < k.knizh...@postgrespro.ru> napsal: > > > On 19.08.2019 18:53, Pavel Stehule wrote: > > > > >> Certainly, default (small) temp buffer size plays roles. >> But it this IPC host this difference is not so important. >> Result with local temp tabl

Re: POC: Cleaning up orphaned files using undo logs

2019-08-20 Thread Andres Freund
Hi, On 2019-08-20 09:08:29 -0400, Robert Haas wrote: > On Sat, Aug 17, 2019 at 1:28 PM Andres Freund wrote: > > The primary one in the context here is that if we do *not* have to lock > > the buffers all ahead of time, we can simplify the interface. We > > certainly can't lock the buffers over IO

Re: POC: Cleaning up orphaned files using undo logs

2019-08-20 Thread Robert Haas
On Mon, Aug 19, 2019 at 8:22 AM Amit Kapila wrote: > One point to remember in this regard is that we do need to modify the > LSN in undo pages after writing WAL, so all the undo pages need to be > locked by that time or we again need to take the lock on them. Uh, but a big part of the point of se

Re: Change ereport level for QueuePartitionConstraintValidation

2019-08-20 Thread Sergei Kornilov
Hello > Sergei, can we enlist you to submit a patch for this? Namely reduce the > log level to DEBUG1 and add a TAP test in src/test/modules/alter_table/ > that verifies that the message is or isn't emitted, as appropriate. I created this patch. I test message existence. Also I check message "ver

Re: pg_upgrade fails with non-standard ACL

2019-08-20 Thread Stephen Frost
Greetings, * Anastasia Lubennikova (a.lubennik...@postgrespro.ru) wrote: > 14.08.2019 3:28, Stephen Frost wrote: > >* Bruce Momjian (br...@momjian.us) wrote: > >>As much as it would be nice if the release notes covered all that, and > >>we updated pg_upgrade to somehow handle them, it just isn't r

Re: understand the pg locks in in an simple case

2019-08-20 Thread Alex
On Tue, Aug 20, 2019 at 4:59 PM Heikki Linnakangas wrote: > On 20/08/2019 10:23, Alex wrote: > > I have troubles to understand the pg lock in the following simple > > situation. > > > > > > Session 1: > > > > > > begin; update tset a= 1 where a= 10; > > > > > > Session 2: > > > > > > begi

Re: Global temporary tables

2019-08-20 Thread Konstantin Knizhnik
On 19.08.2019 18:53, Pavel Stehule wrote: Certainly, default (small) temp buffer size plays roles. But it this IPC host this difference is not so important. Result with local temp tables and temp_buffers = 1GB: 859k TPS. It is little bit unexpected result.I understand so it par

Re: POC: Cleaning up orphaned files using undo logs

2019-08-20 Thread Robert Haas
On Tue, Aug 20, 2019 at 2:42 AM Amit Kapila wrote: > > Well, my main point, which so far has largely been ignored, was that we > > may not acquire page locks when we still need to search for victim > > buffers later. If we don't need to lock the pages up-front, but only do > > so once we're actual

Improve default partition

2019-08-20 Thread Rafia Sabih
Hello all, I was just playing around with table partitioning and noticed 1. When one inserts into a parent table with no partitions defined yet, it errors out 2. Similarly, if we try to insert into a parent table a value which is not covered in any partition and has no default partition defined, i

Re: POC: Cleaning up orphaned files using undo logs

2019-08-20 Thread Robert Haas
On Mon, Aug 19, 2019 at 5:16 PM Andres Freund wrote: > Well, my main point, which so far has largely been ignored, was that we > may not acquire page locks when we still need to search for victim > buffers later. If we don't need to lock the pages up-front, but only do > so once we're actually cop

ICU for global collation

2019-08-20 Thread Peter Eisentraut
Here is an initial patch to add the option to use ICU as the global collation provider, a long-requested feature. To activate, use something like initdb --collation-provider=icu --locale=... A trick here is that since we need to also still set the normal POSIX locales, the --locale value nee

Re: Cleanup isolation specs from unused steps

2019-08-20 Thread Alvaro Herrera
On 2019-Aug-20, Tom Lane wrote: > If you can warn in both cases, that'd be OK perhaps. But Alvaro's > description of the intended use of dry-run makes it sound like > it would be expected for there to be unreferenced steps, since there'd > be no permutations yet in the input. Well, Heikki/Kevin'

Re: configure still looking for crypt()?

2019-08-20 Thread Tom Lane
Peter Eisentraut writes: > I noticed that configure is still looking for crypt() and crypt.h. > Isn't that long obsolete? > If so, I suggest to remove it with the attached patch. +1 regards, tom lane

Re: Cleanup isolation specs from unused steps

2019-08-20 Thread Tom Lane
Alvaro Herrera writes: > On 2019-Aug-20, Michael Paquier wrote: >> On Mon, Aug 19, 2019 at 10:23:19AM -0700, Melanie Plageman wrote: >>> Could you do the check that all steps have been used in dry_run mode >>> instead of when running the tests for real? >> Sure, I was hesitating to do so. I have

Re: pg_upgrade fails with non-standard ACL

2019-08-20 Thread Anastasia Lubennikova
14.08.2019 3:28, Stephen Frost wrote: * Bruce Momjian (br...@momjian.us) wrote: As much as it would be nice if the release notes covered all that, and we updated pg_upgrade to somehow handle them, it just isn't realistic. As we can see here, the problems often take years to show up, and even th

Re: POC: Cleaning up orphaned files using undo logs

2019-08-20 Thread Robert Haas
On Sat, Aug 17, 2019 at 1:28 PM Andres Freund wrote: > The primary one in the context here is that if we do *not* have to lock > the buffers all ahead of time, we can simplify the interface. We > certainly can't lock the buffers over IO (due to buffer reclaim) as > we're doing right now, so we'd n

mingw32 floating point diff

2019-08-20 Thread Peter Eisentraut
Running the regression tests on mingw32, I get the following diff in circle.out: @@ -111,8 +111,8 @@ WHERE (c1.f1 < c2.f1) AND ((c1.f1 <-> c2.f1) > 0) ORDER BY distance, area(c1.f1), area(c2.f1); five | one | two | distance ---++---

configure still looking for crypt()?

2019-08-20 Thread Peter Eisentraut
I noticed that configure is still looking for crypt() and crypt.h. Isn't that long obsolete? If so, I suggest to remove it with the attached patch. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services From 749310380a47

Re: POC: Cleaning up orphaned files using undo logs

2019-08-20 Thread Dilip Kumar
On Wed, Aug 14, 2019 at 10:35 PM Andres Freund wrote: > > Hi, > > On 2019-08-14 14:48:07 +0530, Dilip Kumar wrote: > > On Wed, Aug 14, 2019 at 12:27 PM Andres Freund wrote: > I don't think we can normally pin the undo buffers properly at that > stage. Without knowing the correct contents of the

Re: Make SQL/JSON error code names match SQL standard

2019-08-20 Thread Alexander Korotkov
On Tue, Aug 20, 2019 at 11:49 AM Peter Eisentraut wrote: > I propose the attached patch to make the new SQL/JSON error code names > match the SQL standard. The existing minor differences don't seem > necessary. Thank you for noticing! +1 for pushing this -- Alexander Korotkov Postgres Profe

Re: Zedstore - compressed in-core columnar storage

2019-08-20 Thread Heikki Linnakangas
On 20/08/2019 05:04, Justin Pryzby wrote: it looks like zedstore with lz4 gets ~4.6x for our largest customer's largest table. zfs using compress=gzip-1 gives 6x compression across all their partitioned tables, and I'm surprised it beats zedstore . I did a quick test, with 10 milli

Serialization questions

2019-08-20 Thread Alex
Before understanding how postgres implements the serializable isolation level (I have see many paper related to it), I have question about how it should be. I mainly read the ideas from https://www.postgresql.org/docs/11/transaction-iso.html. In fact, this isolation level works exactly the same

Re: Make SQL/JSON error code names match SQL standard

2019-08-20 Thread Pavel Stehule
út 20. 8. 2019 v 10:49 odesílatel Peter Eisentraut < peter.eisentr...@2ndquadrant.com> napsal: > I propose the attached patch to make the new SQL/JSON error code names > match the SQL standard. The existing minor differences don't seem > necessary. > +1 Pavel > > -- > Peter Eisentraut

Re: Fixing typos and inconsistencies

2019-08-20 Thread Michael Paquier
On Mon, Aug 19, 2019 at 10:19:19PM +0300, Alexander Lakhin wrote: > Now that the unicums checking is finished, I would like to share the > script I used to find them. > Maybe it can be useful to recheck the source tree from time to time... > I don't think that the check could be fully automated, bu

Re: understand the pg locks in in an simple case

2019-08-20 Thread Heikki Linnakangas
On 20/08/2019 10:23, Alex wrote: I have troubles to understand the pg lock in the following simple situation. Session 1: begin; update tset a= 1 where a= 10; Session 2: begin; update tset a= 2 where a= 10; They update the same row and session 2 is blocked by session 1 wi

Re: understand the pg locks in in an simple case

2019-08-20 Thread Laurenz Albe
Alex wrote: > But when I check the pg_locks: session 1. I can see no tuple lock > there, when I check the session 2, I can see a > tuple(ExclusiveLock) is granted, but it is waiting for a > transactionid. > > since every tuple has txn information, so it is not hard to > implement it this wa

Make SQL/JSON error code names match SQL standard

2019-08-20 Thread Peter Eisentraut
I propose the attached patch to make the new SQL/JSON error code names match the SQL standard. The existing minor differences don't seem necessary. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services From 026299f325e7

Re: [HACKERS] WAL logging problem in 9.4.3?

2019-08-20 Thread Kyotaro Horiguchi
Hello. At Mon, 19 Aug 2019 18:59:59 +0900 (Tokyo Standard Time), Kyotaro Horiguchi wrote in <20190819.185959.118543656.horikyota@gmail.com> > > The comment material being deleted is still correct, so don't delete it. > > Moreover, the code managing rd_firstRelfilenodeSubid has a similar bug

understand the pg locks in in an simple case

2019-08-20 Thread Alex
I have troubles to understand the pg lock in the following simple situation. Session 1: begin; update t set a = 1 where a = 10; Session 2: begin; update t set a = 2 where a = 10; They update the same row and session 2 is blocked by session 1 without surprise. The pretty straight implem

Re: Fixing typos and inconsistencies

2019-08-20 Thread Thomas Munro
On Tue, Aug 20, 2019 at 3:05 PM Alexander Lakhin wrote: > Now that the unicums checking is finished, I would like to share the > script I used to find them. > Maybe it can be useful to recheck the source tree from time to time... > I don't think that the check could be fully automated, but with so