Re: ALTER TABLE ADD COLUMN fast default

2018-02-20 Thread David Rowley
On 21 February 2018 at 00:38, David Rowley wrote: > Using: select sum(c10) from t; > ... > v11 + create.sql: tps = 3330.131437 > v11 + create-alter.sql: tps = 1398.635251 It seems the difference between these two cases is down to slot_getsomeattrs being asked to

Re: Hash Joins vs. Bloom Filters / take 2

2018-02-20 Thread Peter Geoghegan
On Tue, Feb 20, 2018 at 3:48 PM, Claudio Freire wrote: >> Do we need to eliminate 99% of all hash join probes (that find nothing >> to join on) to make this Bloom filter optimization worthwhile? >> Personally, I doubt it. > > Even for 90% it's about 4.6 bits per element.

Re: Duplicate Item Pointers in Gin index

2018-02-20 Thread Peter Geoghegan
On Tue, Feb 20, 2018 at 3:30 PM, R, Siva wrote: > We are currently investigating an issue with a Gin index containing > duplicate item pointers for one of its keys. While we are trying to > reproduce this issue, we wanted to reach out to the community to validate > our

Re: Duplicate Item Pointers in Gin index

2018-02-20 Thread Peter Geoghegan
On Tue, Feb 20, 2018 at 7:19 PM, Masahiko Sawada wrote: > IIUC, ginInsertCleanup() holds ExclusiveLock on metapage during adding > tuples in the pending list to the accumulator. And inserting entries > to the pending list also requires ExclusiveLock on metapage. This >

Re: SHA-2 functions

2018-02-20 Thread Michael Paquier
On Tue, Feb 20, 2018 at 05:09:48PM -0500, Tom Lane wrote: > Peter Eisentraut writes: >> On 2/19/18 21:07, Michael Paquier wrote: >>> varlena.c is already large and messy. I would suggest to split into a >>> new file all the user-facing cryptographic functions,

Re: non-bulk inserts and tuple routing

2018-02-20 Thread Amit Langote
Fujita-san, On 2018/02/20 19:40, Etsuro Fujita wrote: > (2018/02/19 13:19), Amit Langote wrote: >> Attached rebased patch. > > Thanks for the rebased patch! > > One thing I noticed while updating the > tuple-routing-for-foreign-partitions patch on top of this is: we should > switch into the

Re: Hash Joins vs. Bloom Filters / take 2

2018-02-20 Thread Tomas Vondra
On 02/21/2018 12:06 AM, Peter Geoghegan wrote: > On Tue, Feb 20, 2018 at 1:23 PM, Tomas Vondra > wrote: >> In 2015/2016 I've been exploring if we could improve hash joins by >> leveraging bloom filters [1], and I was reminded about this idea in a >> thread about

Re: [HACKERS] path toward faster partition pruning

2018-02-20 Thread David Rowley
v30-0004-Faster-partition-pruning.patch contains: +create table coll_pruning_multi (a text) partition by range (substr(a, 1) collate "en_GB", substr(a, 1) collate "en_US"); This'll likely work okay on Linux. Other collate tests seem to use COLLATE "POSIX or "C" so that work cross-platform. --

Re: Hash Joins vs. Bloom Filters / take 2

2018-02-20 Thread Peter Geoghegan
On Tue, Feb 20, 2018 at 1:23 PM, Tomas Vondra wrote: > In 2015/2016 I've been exploring if we could improve hash joins by > leveraging bloom filters [1], and I was reminded about this idea in a > thread about amcheck [2]. I also see that bloom filters were briefly >

Re: Hash Joins vs. Bloom Filters / take 2

2018-02-20 Thread Claudio Freire
On Tue, Feb 20, 2018 at 8:06 PM, Peter Geoghegan wrote: > You should try to exploit the fact that a Bloom filter can summarize a > large set reasonably well with a very compact, simple representation. > A false positive rate of 10% sounds a lot worse than 1% or 0.1%, but > for cases

Re: Hash Joins vs. Bloom Filters / take 2

2018-02-20 Thread Peter Geoghegan
On Tue, Feb 20, 2018 at 3:17 PM, Claudio Freire wrote: > I've worked a lot with bloom filters, and for large false positive > rates and large sets (multi-million entries), you get bloom filter > sizes of about 10 bits per distinct item. It's generally true that you need

Re: [HACKERS] Runtime Partition Pruning

2018-02-20 Thread David Rowley
On 20 February 2018 at 23:46, Rajkumar Raghuwanshi wrote: > I have applied v10 patch on Amit's v27 over head ad7dbee36. I got "ERROR: > partition missing from Append subplans" with the patch. on head and only > with Amit's patches query is working fine,

Re: Hash Joins vs. Bloom Filters / take 2

2018-02-20 Thread Peter Geoghegan
On Tue, Feb 20, 2018 at 3:54 PM, Tomas Vondra wrote: >> I suspect that it could make sense to use a Bloom filter to >> summarize the entire inner side of the join all at once, even when >> there are multiple batches. I also suspect that this is particularly >>

Re: SHA-2 functions

2018-02-20 Thread Peter Eisentraut
On 2/19/18 21:07, Michael Paquier wrote: > + sha224('abc') > + > \x23097d223405d8228642a477bda255b32aadbce4bda0b3f7e36c9da7 > Some bytea characters from the hash are not able to show up correctly? > This does not result in spaces. U+200B is a zero-width space, used here to hint for

Re: SHA-2 functions

2018-02-20 Thread Tom Lane
Peter Eisentraut writes: > On 2/19/18 21:07, Michael Paquier wrote: >> varlena.c is already large and messy. I would suggest to split into a >> new file all the user-facing cryptographic functions, including md5 and >> hex functions, say in

Re: ALTER TABLE ADD COLUMN fast default

2018-02-20 Thread Tom Lane
David Rowley writes: > It seems the difference between these two cases is down to > slot_getsomeattrs being asked to deform up to attnum 1000 for the > create-alter.sql case, and only up to attnum 10 for the create.sql > case. Both plans are using physical tlists per

Re: Hash Joins vs. Bloom Filters / take 2

2018-02-20 Thread Claudio Freire
On Tue, Feb 20, 2018 at 8:23 PM, Peter Geoghegan wrote: > On Tue, Feb 20, 2018 at 3:17 PM, Claudio Freire > wrote: >> I've worked a lot with bloom filters, and for large false positive >> rates and large sets (multi-million entries), you get bloom filter >>

Re: extern keyword incorrectly used in some function definitions

2018-02-20 Thread David Rowley
On 20 February 2018 at 06:09, Tom Lane wrote: > But the rest > of these look good; pushed. Great. Thanks! -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: Typo in procarray.c

2018-02-20 Thread Masahiko Sawada
On Tue, Feb 20, 2018 at 8:04 PM, Magnus Hagander wrote: > > > On Tue, Feb 20, 2018 at 2:47 AM, Masahiko Sawada > wrote: >> >> Hi, >> >> Attached patch for fixing $subject. >> >> s/replicaton/replication/ > > > Applied, thanks. > Thank you! Regards,

support parameters in CALL

2018-02-20 Thread Peter Eisentraut
The current CALL implementation doesn't support parameters, which was a bad oversight. This patch fixes that. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services From f3e78c7d9ef2fe21102cc8a08d244eda69aaa141 Mon Sep

file cloning in pg_upgrade and CREATE DATABASE

2018-02-20 Thread Peter Eisentraut
Here is another attempt at implementing file cloning for pg_upgrade and CREATE DATABASE. The idea is to take advantage of file systems that can make copy-on-write clones, which would make the copy run much faster. For pg_upgrade, this will give the performance of --link mode without the

Re: Duplicate Item Pointers in Gin index

2018-02-20 Thread Masahiko Sawada
On Wed, Feb 21, 2018 at 8:30 AM, R, Siva wrote: > User backend (let us call this backend 1) has released the lock on the > metapage and is processing the pending list pages, adding the items to the > build accumulator. Let us assume that there are currently 2 pages in the >

Re: pgsql: Avoid valgrind complaint about write() of uninitalized bytes.

2018-02-20 Thread Andres Freund
On 2018-02-06 19:25:04 +, Robert Haas wrote: > Avoid valgrind complaint about write() of uninitalized bytes. > > LogicalTapeFreeze() may write out its first block when it is dirty but > not full, and then immediately read the first block back in from its > BufFile as a BLCKSZ-width block.

RE: Speed up the removal of WAL files

2018-02-20 Thread Tsunakawa, Takayuki
From: Fujii Masao [mailto:masao.fu...@gmail.com] > On Fri, Nov 17, 2017 at 5:20 PM, Tsunakawa, Takayuki > wrote: > > Yes, I noticed it after submitting the patch and was wondering what to > do. Thinking simply, I think it would be just enough to replace >

Re: Speed up the removal of WAL files

2018-02-20 Thread Michael Paquier
On Wed, Feb 21, 2018 at 07:20:00AM +, Tsunakawa, Takayuki wrote: > Right. Then I thought of doing the following to avoid making a new > function only for RemoveNonParentXlogFiles() which is similar to > RemoveXlogFile(). > > * Add an argument "bool durable" to RemoveXlogFile(). Based on

Re: pg_get_functiondef forgets about most GUC_LIST_INPUT GUCs

2018-02-20 Thread Michael Paquier
On Tue, Feb 20, 2018 at 06:46:57PM +0300, Arthur Zakirov wrote: > Just 2 cents from me. It seems that there is a problem with extensions > GUCs. > > [...] > > =# SELECT pg_get_functiondef('func_with_set_params'::regproc); > ERROR: unrecognized configuration parameter "plpgsql.extra_errors" You

Re: Duplicate Item Pointers in Gin index

2018-02-20 Thread Masahiko Sawada
On Wed, Feb 21, 2018 at 12:31 PM, Peter Geoghegan wrote: > On Tue, Feb 20, 2018 at 7:19 PM, Masahiko Sawada > wrote: >> IIUC, ginInsertCleanup() holds ExclusiveLock on metapage during adding >> tuples in the pending list to the accumulator. And inserting

Re: pgsql: Do execGrouping.c via expression eval machinery, take two.

2018-02-20 Thread Andres Freund
Hi, On 2018-02-16 16:03:37 -0800, Andres Freund wrote: > This triggered a failure on rhinoceros, in the sepgsql test: > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=rhinoceros=2018-02-16%2023%3A45%3A02 > > The relevant diff is: > + LOG: SELinux: allowed { execute } >

Re: Hash Joins vs. Bloom Filters / take 2

2018-02-20 Thread Thomas Munro
On Wed, Feb 21, 2018 at 10:23 AM, Tomas Vondra wrote: > In 2015/2016 I've been exploring if we could improve hash joins by > leveraging bloom filters [1], and I was reminded about this idea in a > thread about amcheck [2]. I also see that bloom filters were briefly >

Re: ALTER TABLE ADD COLUMN fast default

2018-02-20 Thread David Rowley
Thanks for making those updates . On 20 February 2018 at 19:33, Andrew Dunstan wrote: > On Mon, Feb 19, 2018 at 1:18 PM, David Rowley > wrote: >> Since the attnum order in the missing values appears to be well >> defined in ascending

Re: ALTER TABLE ADD COLUMN fast default

2018-02-20 Thread David Rowley
On 20 February 2018 at 23:10, David Rowley wrote: > Nevertheless, it would be interesting to see how much a bsearch in > getmissingattr would help Tomas' case. Though, perhaps you're happy > enough with the performance already. I thought I'd give this a quick test

Re: [HACKERS] Runtime Partition Pruning

2018-02-20 Thread Rajkumar Raghuwanshi
On Sat, Feb 17, 2018 at 2:27 PM, David Rowley wrote: > Hi, > > I've attached an updated patch, now at v10. v9 was short lived due to > the evolution of Amit's which which this based on. > > This version is based on Amit's v27 of faster partition pruning [1] > which

Option to ensure monotonic timestamps

2018-02-20 Thread Brent Kerby
Hi, I'm new to Postgres hacking, and I'm interested in the possibility of a new feature to make it possible to ensure that Postgres-generated timestamps never decrease even if the system clock may step backwards. My use case is that I'm implementing a form of temporal tables based on transaction

Re: [HACKERS] why not parallel seq scan for slow functions

2018-02-20 Thread Amit Kapila
On Mon, Feb 19, 2018 at 9:56 AM, Ashutosh Bapat wrote: > On Mon, Feb 19, 2018 at 9:35 AM, Ashutosh Bapat > wrote: >> On Sat, Feb 17, 2018 at 8:17 AM, Amit Kapila wrote: >>> On Fri, Feb 16, 2018 at 9:29

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-02-20 Thread Ashutosh Bapat
On Tue, Feb 13, 2018 at 6:21 PM, Ashutosh Bapat wrote: > > 1. Push down ConvertRowtypeExpr and include it in the pushed down targetlist. > This would solve both the problems described above. Both set_plan_ref() and > get_relation_column_alias_ids() will find

Re: unique indexes on partitioned tables

2018-02-20 Thread Amit Langote
Hi. On 2018/02/20 5:45, Alvaro Herrera wrote: > I pushed this now, with fixes for the last few comments there were. I noticed with the commit that, while ON CONFLICT (conflict_target) DO UPDATE gives a less surprising error message by catching it in the parser, ON CONFLICT (conflict_target) DO

Re: [HACKERS] [PATCH] Vacuum: Update FSM more frequently

2018-02-20 Thread Masahiko Sawada
On Fri, Feb 16, 2018 at 5:00 AM, Claudio Freire wrote: > On Thu, Feb 15, 2018 at 4:47 PM, Claudio Freire > wrote: >> On Wed, Feb 14, 2018 at 3:59 AM, Masahiko Sawada >> wrote: > The final FSM vacuum pass isn't

Re: Typo in procarray.c

2018-02-20 Thread Magnus Hagander
On Tue, Feb 20, 2018 at 2:47 AM, Masahiko Sawada wrote: > Hi, > > Attached patch for fixing $subject. > > s/replicaton/replication/ > Applied, thanks. -- Magnus Hagander Me: https://www.hagander.net/ Work: https://www.redpill-linpro.com/

Re: Contention preventing locking

2018-02-20 Thread Simon Riggs
On 15 February 2018 at 16:00, Konstantin Knizhnik wrote: > So in heap_acquire_tuplock all competing transactions are waiting for TID of > the updated version. When transaction which changed this tuple is committed, > one of the competitors will grant this lock and

Re: [bug fix] Produce a crash dump before main() on Windows

2018-02-20 Thread Craig Ringer
On 20 February 2018 at 21:47, Magnus Hagander wrote: > > > On Fri, Feb 16, 2018 at 8:28 AM, Tsunakawa, Takayuki < > tsunakawa.ta...@jp.fujitsu.com> wrote: > >> Hello, >> >> postgres.exe on Windows doesn't output a crash dump when it crashes >> before main() is called. The

Re: NEXT VALUE FOR sequence

2018-02-20 Thread Laurenz Albe
Tom Lane wrote: > Laurenz Albe writes: > > The SQL standard has the expression "NEXT VALUE FOR asequence" to do > > what we traditionally do with "nextval('asequence')". > > This is an attempt to implement this on top of the recently introduced > > NextValueExpr node. >

Re: heap_lock_updated_tuple_rec can leak a buffer refcount

2018-02-20 Thread Amit Kapila
On Tue, Feb 13, 2018 at 10:11 AM, Amit Kapila wrote: > It seems to me that heap_lock_updated_tuple_rec can lead to a buffer > refcount leak while locking an updated tuple by an aborted > transaction. In commit - 5c609a74, we have added the code to deal > with aborted

committing inside cursor loop

2018-02-20 Thread Peter Eisentraut
Here is a patch that allows COMMIT inside cursor loops in PL/pgSQL. As alluded to in earlier threads, this is done by converting such cursors to holdable automatically. A special flag "auto-held" marks such cursors, so we know to clean them up on exceptions. This is currently only for PL/pgSQL,

Re: Contention preventing locking

2018-02-20 Thread Konstantin Knizhnik
On 20.02.2018 16:42, Simon Riggs wrote: On 20 February 2018 at 13:19, Konstantin Knizhnik wrote: On 20.02.2018 14:26, Simon Riggs wrote: On 15 February 2018 at 16:00, Konstantin Knizhnik wrote: So in heap_acquire_tuplock all

Re: Contention preventing locking

2018-02-20 Thread Konstantin Knizhnik
On 20.02.2018 14:26, Simon Riggs wrote: On 15 February 2018 at 16:00, Konstantin Knizhnik wrote: So in heap_acquire_tuplock all competing transactions are waiting for TID of the updated version. When transaction which changed this tuple is committed, one of the

Re: [bug fix] Produce a crash dump before main() on Windows

2018-02-20 Thread Craig Ringer
On 20 February 2018 at 22:18, Craig Ringer wrote: > So I'm all for just removing that. > ... but just to be clear, about -1000 on backpatching any such thing. At most, a new GUC that defaults to the current behaviour. But I think it's pretty niche really. -- Craig

Re: [PATCH] Add missing type conversion functions for PL/Python

2018-02-20 Thread Haozhou Wang
Thank you very much for your review! I attached a new patch with typo fixed. Regards, Haozhou On Mon, Feb 19, 2018 at 2:37 PM, Anthony Bykov wrote: > On Wed, 31 Jan 2018 11:57:12 +0800 > Haozhou Wang wrote: > > > Hi All, > > > > PL/Python already

Re: committing inside cursor loop

2018-02-20 Thread Simon Riggs
On 20 February 2018 at 14:11, Peter Eisentraut wrote: > Here is a patch that allows COMMIT inside cursor loops in PL/pgSQL. As > alluded to in earlier threads, this is done by converting such cursors > to holdable automatically. A special flag "auto-held" marks

Re: [bug fix] Produce a crash dump before main() on Windows

2018-02-20 Thread Magnus Hagander
On Fri, Feb 16, 2018 at 8:28 AM, Tsunakawa, Takayuki < tsunakawa.ta...@jp.fujitsu.com> wrote: > Hello, > > postgres.exe on Windows doesn't output a crash dump when it crashes before > main() is called. The attached patch fixes this. I'd like this to be > back-patched. I'll add this to the next

[PATCH] Add a few suppression rules for Valgrind

2018-02-20 Thread Aleksander Alekseev
Hello hackers, I decided to run the code from master branch under Valgrind and discovered that it reports some errors. There are multiple reports like this one (seems to be a false alarm): ``` Invalid read of size 16 at 0x605F488: __wcsnlen_sse4_1 (in /usr/lib/libc-2.26.so) by 0x604F5C2:

Re: committing inside cursor loop

2018-02-20 Thread Tom Lane
Peter Eisentraut writes: > Here is a patch that allows COMMIT inside cursor loops in PL/pgSQL. As > alluded to in earlier threads, this is done by converting such cursors > to holdable automatically. A special flag "auto-held" marks such > cursors, so we know

Re: [bug fix] Produce a crash dump before main() on Windows

2018-02-20 Thread Magnus Hagander
On Tue, Feb 20, 2018 at 3:18 PM, Craig Ringer wrote: > On 20 February 2018 at 21:47, Magnus Hagander wrote: > >> >> >> On Fri, Feb 16, 2018 at 8:28 AM, Tsunakawa, Takayuki < >> tsunakawa.ta...@jp.fujitsu.com> wrote: >> >>> Hello, >>> >>> postgres.exe

Re: [PATCH] Add support for ON UPDATE/DELETE actions on ALTER CONSTRAINT

2018-02-20 Thread Matheus de Oliveira
On Tue, Feb 20, 2018 at 12:38 PM, Fabrízio de Royes Mello < fabriziome...@gmail.com> wrote: > > > ... > > I didn't read your patch yet but make sure to register it to the next open > commitfest. > Thanks a lot Fabrízio, I've done that already [1]. Please let me know if I did something wrong,

Re: Contention preventing locking

2018-02-20 Thread Konstantin Knizhnik
On 20.02.2018 14:26, Simon Riggs wrote: Try locking the root tid rather than the TID, that is at least unique per page for a chain of tuples, just harder to locate. As far as I understand, it is necessary to traverse the whole page to locate root tuple, isn't it? If so, then I expect it to

Re: Contention preventing locking

2018-02-20 Thread Konstantin Knizhnik
On 20.02.2018 19:39, Simon Riggs wrote: On 20 February 2018 at 16:07, Konstantin Knizhnik wrote: On 20.02.2018 14:26, Simon Riggs wrote: Try locking the root tid rather than the TID, that is at least unique per page for a chain of tuples, just harder to locate.

Re: [PATCH] Add support for ON UPDATE/DELETE actions on ALTER CONSTRAINT

2018-02-20 Thread Fabrízio de Royes Mello
On Tue, Feb 20, 2018 at 12:10 PM, Matheus de Oliveira < matioli.math...@gmail.com> wrote: > > Hi all. > > I attached a patch to add support for changing ON UPDATE/DELETE actions of a constraint using ALTER TABLE ... ALTER CONSTRAINT. > > Besides that, there is a another change in this patch on

Re: pg_get_functiondef forgets about most GUC_LIST_INPUT GUCs

2018-02-20 Thread Arthur Zakirov
Hello, On Fri, Jan 12, 2018 at 10:24:40AM +0900, Michael Paquier wrote: > OK, I can live with that. What do you think about the attached? I'll be > happy to produce patches for back-branches as necessary. When an option > is not found, I have made the function return 0 as value for the flags, >

Re: pgsql: Allow UNIQUE indexes on partitioned tables

2018-02-20 Thread Alvaro Herrera
Many thanks for reading through it! David G. Johnston wrote: > I found the following change to be confusing. [...] > I was expecting the doc for ADD CONSTRAINT USING INDEX to note the > limitation explicitly - in lieu of the above paragraph. Agreed. I moved the note to ADD CONSTRAINT and added

Re: Contention preventing locking

2018-02-20 Thread Simon Riggs
On 20 February 2018 at 16:07, Konstantin Knizhnik wrote: > > > On 20.02.2018 14:26, Simon Riggs wrote: >> >> Try locking the root tid rather than the TID, that is at least unique >> per page for a chain of tuples, just harder to locate. >> > As far as I understand, it

Re: ALTER TABLE ADD COLUMN fast default

2018-02-20 Thread Tom Lane
... btw, I've not read this patch in any detail, but the recent thread about toast tables for system catalogs prompts me to wonder what happens if a "fast default" value is large enough to require out-of-line toasting. I can easily think of problems that will ensue if we try to support that case,

Re: ALTER TABLE ADD COLUMN fast default

2018-02-20 Thread Tom Lane
Andres Freund writes: > On 2018-02-20 13:07:30 -0500, Tom Lane wrote: >> I can easily think of problems that will ensue if we try to support that >> case, because right now the toast mechanisms assume that OOL toasted >> values can only be referenced from the associated table.

Re: ALTER TABLE ADD COLUMN fast default

2018-02-20 Thread Andres Freund
Hi, On 2018-02-20 13:50:54 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2018-02-20 13:07:30 -0500, Tom Lane wrote: > >> I can easily think of problems that will ensue if we try to support that > >> case, because right now the toast mechanisms assume that OOL toasted

Re: [PATCH] Add a few suppression rules for Valgrind

2018-02-20 Thread Andres Freund
Hi, > I decided to run the code from master branch under Valgrind and > discovered that it reports some errors. > > There are multiple reports like this one (seems to be a false alarm): > > ``` > Invalid read of size 16 >at 0x605F488: __wcsnlen_sse4_1 (in /usr/lib/libc-2.26.so) >by

Re: [PATCH] Add a few suppression rules for Valgrind

2018-02-20 Thread Tom Lane
Andres Freund writes: >> I decided to run the code from master branch under Valgrind and >> discovered that it reports some errors. > On my systems I just include a global valgrind suppression file which > includes libc specific things and then the postgres valgrind.supp. I'm

Re: Option to ensure monotonic timestamps

2018-02-20 Thread Andres Freund
Hi, Leaving Tom's concerns aside: On 2018-02-19 13:42:31 -0700, Brent Kerby wrote: > Hi, I'm new to Postgres hacking, and I'm interested in the possibility of a > new feature to make it possible to ensure that Postgres-generated > timestamps never decrease even if the system clock may step

Re: master check fails on Windows Server 2008

2018-02-20 Thread Tom Lane
Marina Polyakova writes: > On 20-02-2018 3:37, Tom Lane wrote: >> 4. Try to tweak the stats_ext.sql test conditions in some more refined >> way to get the test to pass everywhere. This'd be a lot of work with >> no guarantee of success, so I'm not too excited about

Re: Option to ensure monotonic timestamps

2018-02-20 Thread Tom Lane
Andres Freund writes: > On 2018-02-20 12:32:22 -0500, Tom Lane wrote: >> The "global" variable would actually need to be cluster-wide, ie in shared >> memory, which would imply contention and the need for locks. I think the >> overhead of this would be mighty high, and the

Re: master check fails on Windows Server 2008

2018-02-20 Thread Marina Polyakova
On 20-02-2018 3:37, Tom Lane wrote: Ah-hah. I can reproduce the described failure if I configure with --disable-float8-byval on an otherwise 64-bit machine. It appears that the minimum work_mem setting that will allow this query to use a hashagg plan on such a configuration is about 155kB

Re: Option to ensure monotonic timestamps

2018-02-20 Thread Andres Freund
Hi, On 2018-02-20 12:32:22 -0500, Tom Lane wrote: > The "global" variable would actually need to be cluster-wide, ie in shared > memory, which would imply contention and the need for locks. I think the > overhead of this would be mighty high, and the return pretty low. I think if we wanted to

Re: Option to ensure monotonic timestamps

2018-02-20 Thread Patrick Krecker
On Tue, Feb 20, 2018 at 9:51 AM, Andres Freund wrote: > Hi, > > Leaving Tom's concerns aside: > > On 2018-02-19 13:42:31 -0700, Brent Kerby wrote: >> Hi, I'm new to Postgres hacking, and I'm interested in the possibility of a >> new feature to make it possible to ensure that

Re: ALTER TABLE ADD COLUMN fast default

2018-02-20 Thread Andres Freund
Hi, On 2018-02-20 13:07:30 -0500, Tom Lane wrote: > ... btw, I've not read this patch in any detail, but the recent thread > about toast tables for system catalogs prompts me to wonder what happens > if a "fast default" value is large enough to require out-of-line toasting. Hm, interesting. >

Hash Joins vs. Bloom Filters / take 2

2018-02-20 Thread Tomas Vondra
Hi, In 2015/2016 I've been exploring if we could improve hash joins by leveraging bloom filters [1], and I was reminded about this idea in a thread about amcheck [2]. I also see that bloom filters were briefly mentioned in the thread about parallel hash [3]. So I've decided to revive the old

Re: ALTER TABLE ADD COLUMN fast default

2018-02-20 Thread Petr Jelinek
On 20/02/18 07:42, Andres Freund wrote: > Hi, > > On 2018-02-17 00:23:40 +0100, Tomas Vondra wrote: >> Anyway, I consider the performance to be OK. But perhaps Andres could >> comment on this too, as he requested the benchmarks. > > My performance concerns were less about CREATE TABLE related

Re: Contention preventing locking

2018-02-20 Thread Simon Riggs
On 20 February 2018 at 13:19, Konstantin Knizhnik wrote: > > > On 20.02.2018 14:26, Simon Riggs wrote: >> >> On 15 February 2018 at 16:00, Konstantin Knizhnik >> wrote: >> >>> So in heap_acquire_tuplock all competing transactions are waiting

[PATCH] btree_gin, add support for uuid, bool, name, bpchar and anyrange types

2018-02-20 Thread Matheus de Oliveira
Hi all. Here is a patch to add support for more types on btree_gin. I was missing UUID type, so I added it. Since I was there, I checked all other built-in types with B-tree but not GIN support, and the remaining list was: uuid, bool, name, bpchar and anyrange (at least ones that seem to make

Re: [PATCH] Add a few suppression rules for Valgrind

2018-02-20 Thread Aleksander Alekseev
Hello hackers, > I suggest a patch that adds corresponding suppression rules to the > src/tools/valgrind.supp file. Though I'm having difficulties > understanding why Valgrind complains on wcstombs and thus I can't > explain why in fact everything is OK (or it's actually not?). Hopefully >

Re: pgsql: Allow UNIQUE indexes on partitioned tables

2018-02-20 Thread David G. Johnston
On Tue, Feb 20, 2018 at 8:36 AM, Alvaro Herrera wrote: > Many thanks for reading through it! > > David G. Johnston wrote: > > I found the following change to be confusing. > [...] > > I was expecting the doc for ADD CONSTRAINT USING INDEX to note the > > limitation

Re: [HACKERS] Transactions involving multiple postgres foreign servers

2018-02-20 Thread Robert Haas
On Tue, Feb 13, 2018 at 5:42 AM, Masahiko Sawada wrote: >> The fdw-transactions section of the documentation seems to imply that >> henceforth every FDW must call FdwXactRegisterForeignServer, which I >> think is an unacceptable API break. >> >> It doesn't seem advisable to

Re: ALTER TABLE ADD COLUMN fast default

2018-02-20 Thread Tomas Vondra
On 02/20/2018 06:43 PM, Andres Freund wrote: > > > On February 20, 2018 5:03:58 AM PST, Petr Jelinek > wrote: >> On 20/02/18 07:42, Andres Freund wrote: >>> Hi, >>> >>> On 2018-02-17 00:23:40 +0100, Tomas Vondra wrote: Anyway, I consider the performance to

Re: Option to ensure monotonic timestamps

2018-02-20 Thread Brent Kerby
Right, I'm talking about temporal tables in the sense of the SQL:2011 standard. I know there's a Postgres extension temporal_tables by Vlad Arkhipov (https://github.com/arkhipov/temporal_tables/) that approximates this. There's also a way of doing it using only triggers written in pgplsql, by

Re: Option to ensure monotonic timestamps

2018-02-20 Thread Brent Kerby
The issue is that presence of timestamps is fundamental to the functionality of temporal tables. The users need to.be able to make queries on temporal tables in terms of timestamps; LSNs won't mean anything to them. It would be an option to implement the temporal tables using LSNs under the hood,

Re: ALTER TABLE ADD COLUMN fast default

2018-02-20 Thread Andres Freund
Hi, On 2018-02-20 20:57:36 +0100, Tomas Vondra wrote: > The question is how should the schema for TPC-H look like. Because if > you just do the usual test without any ALTER TABLE ADD COLUMN, then you > really won't get any difference at all. The fast default stuff will be > completely "inactive".

Re: ALTER TABLE ADD COLUMN fast default

2018-02-20 Thread Tom Lane
Andres Freund writes: > On 2018-02-20 20:57:36 +0100, Tomas Vondra wrote: >> The question is how should the schema for TPC-H look like. Because if >> you just do the usual test without any ALTER TABLE ADD COLUMN, then you >> really won't get any difference at all. The fast

Re: ALTER TABLE ADD COLUMN fast default

2018-02-20 Thread Tomas Vondra
On 02/20/2018 09:14 PM, Tom Lane wrote: > Andres Freund writes: >> On 2018-02-20 20:57:36 +0100, Tomas Vondra wrote: >>> The question is how should the schema for TPC-H look like. Because if >>> you just do the usual test without any ALTER TABLE ADD COLUMN, then you >>>

Re: ALTER TABLE ADD COLUMN fast default

2018-02-20 Thread Andres Freund
Hi, On 2018-02-20 21:28:40 +0100, Tomas Vondra wrote: > I don't quite understand why would this case need the TPC-H tests, or > why would TPC-H give us more than the very focused tests we've already > done. Because a more complex query shows the cost of changing cache access costs better than a