Re: Online enabling of checksums

2018-04-06 Thread Magnus Hagander
On Sat, Apr 7, 2018 at 6:26 AM, Andres Freund wrote: > On 2018-04-06 17:59:28 -0700, Andres Freund wrote: > > + /* > > + * Create a database list. We don't need to concern ourselves with > > + * rebuilding this list during runtime since any database created > after > > + * thi

Re: [HACKERS] path toward faster partition pruning

2018-04-06 Thread Amit Langote
On Sat, Apr 7, 2018 at 1:39 PM, Tom Lane wrote: > Amit Langote writes: >> Given that the difference only appeared on animals that David pointed >> out have big-endian architecture, it seems we'd only need two output >> files. > > Dunno, I'm wondering whether 32 vs 64 bit will make a difference.

Re: WIP: a way forward on bootstrap data

2018-04-06 Thread John Naylor
On 4/6/18, Tom Lane wrote: > I don't think there's any great need to incorporate this into your patch > set. As far as I'm concerned, v14 is ready as-is, and I'll just apply > this over the top of it. (Note that I'll probably smash the whole thing > to one commit when the time comes.) Glad to

Re: [HACKERS] Runtime Partition Pruning

2018-04-06 Thread Amit Langote
On Sat, Apr 7, 2018 at 1:58 PM, Amit Langote wrote: > On Sat, Apr 7, 2018 at 1:31 PM, Andres Freund wrote: >> I've not followed this thread/feature at all, but I don't find the >> comments atop partprune.c even remotely sufficient. Unless there's an >> README hidden or such hidden somewhere? > >

Re: PATCH: pgbench - option to build using ppoll() for larger connection counts

2018-04-06 Thread konstantin knizhnik
On Apr 7, 2018, at 12:49 AM, Tom Lane wrote: > Andres Freund writes: >> I'm still not particularly happy with this. > > I'm a bit confused as to what the point is. It seems unlikely that one > pgbench process can effectively drive enough backends for select's > limitations to really be an issu

Re: [HACKERS] Runtime Partition Pruning

2018-04-06 Thread Andres Freund
On 2018-04-07 16:58:01 +1200, David Rowley wrote: > On 7 April 2018 at 16:31, Andres Freund wrote: > > I've not followed this thread/feature at all, but I don't find the > > comments atop partprune.c even remotely sufficient. Unless there's an > > README hidden or such hidden somewhere? > > There

Re: [HACKERS] Runtime Partition Pruning

2018-04-06 Thread Amit Langote
On Sat, Apr 7, 2018 at 1:58 PM, David Rowley wrote: > Probably if we need to explain more there about how pruning works then > it should be a fixup patch to 9fdb675fc, no? Yes, I just replied and working on a patch. Thanks, Amit

Re: [HACKERS] Runtime Partition Pruning

2018-04-06 Thread Amit Langote
On Sat, Apr 7, 2018 at 1:31 PM, Andres Freund wrote: > On 2018-04-07 13:26:51 +0900, Amit Langote wrote: >> On Sat, Apr 7, 2018 at 11:26 AM, David Rowley >> wrote: >> > Everything else looks fine from my point of view. >> >> Me too, although I still think having struct names PartitionPruning >> a

Re: [HACKERS] Runtime Partition Pruning

2018-04-06 Thread David Rowley
On 7 April 2018 at 16:31, Andres Freund wrote: > I've not followed this thread/feature at all, but I don't find the > comments atop partprune.c even remotely sufficient. Unless there's an > README hidden or such hidden somewhere? There's not a README file. The comments for partprune.c, do you wan

Re: [HACKERS] Runtime Partition Pruning

2018-04-06 Thread David Rowley
On 7 April 2018 at 16:26, Amit Langote wrote: > Maybe, PartitionPruneState, because it parallels the > PartitionPruneInfo that comes from the planner for every partitioned > table in the tree. I like that. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development,

Re: [HACKERS] path toward faster partition pruning

2018-04-06 Thread Tom Lane
Amit Langote writes: > Given that the difference only appeared on animals that David pointed > out have big-endian architecture, it seems we'd only need two output > files. Dunno, I'm wondering whether 32 vs 64 bit will make a difference. regards, tom lane

Re: [HACKERS] Runtime Partition Pruning

2018-04-06 Thread Andres Freund
On 2018-04-07 13:26:51 +0900, Amit Langote wrote: > On Sat, Apr 7, 2018 at 11:26 AM, David Rowley > wrote: > > Everything else looks fine from my point of view. > > Me too, although I still think having struct names PartitionPruning > and PartitionRelPruning is going to be a bit confusing. We s

Re: [HACKERS] Runtime Partition Pruning

2018-04-06 Thread Amit Langote
On Sat, Apr 7, 2018 at 11:26 AM, David Rowley wrote: > Everything else looks fine from my point of view. Me too, although I still think having struct names PartitionPruning and PartitionRelPruning is going to be a bit confusing. We should think about naming the latter to something else. I sugg

Re: Online enabling of checksums

2018-04-06 Thread Andres Freund
On 2018-04-06 17:59:28 -0700, Andres Freund wrote: > + /* > + * Create a database list. We don't need to concern ourselves with > + * rebuilding this list during runtime since any database created after > + * this process started will be running with checksums turned on from >

Re: [HACKERS] path toward faster partition pruning

2018-04-06 Thread Amit Langote
On Sat, Apr 7, 2018 at 1:09 PM, Andres Freund wrote: > Hi, > > On 2018-04-07 15:49:54 +1200, David Rowley wrote: >> Right, I suggest we wait and see if all members go green again as a >> result of 40e42e1024c, and if they're happy then we could maybe leave >> it as is with the 2 alternatives outpu

Re: [HACKERS] path toward faster partition pruning

2018-04-06 Thread David Rowley
On 7 April 2018 at 15:18, Andres Freund wrote: > I've pushed the two patches (collapsed). Trying to get the BF green-ish > again... termite has now gone green. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] path toward faster partition pruning

2018-04-06 Thread David Rowley
On 7 April 2018 at 16:09, Andres Freund wrote: > I've also attempted to fix rhinoceros's failure I remarked upon a couple > hours ago in > https://postgr.es/m/20180406210330.wmqw42wqgiick...@alap3.anarazel.de Oh, thanks! I had just been looking at that too... -- David Rowley

Re: [HACKERS] path toward faster partition pruning

2018-04-06 Thread David Rowley
On 7 April 2018 at 09:03, Andres Freund wrote: > There's also > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=rhinoceros&dt=2018-04-06%2020%3A45%3A01 > *** > /opt/src/pgsql-git/build-farm-root/HEAD/pgsql.build/contrib/sepgsql/expected/misc.out >2018-02-20 18:45:02.068665297 -080

Re: [HACKERS] path toward faster partition pruning

2018-04-06 Thread Andres Freund
Hi, On 2018-04-07 15:49:54 +1200, David Rowley wrote: > Right, I suggest we wait and see if all members go green again as a > result of 40e42e1024c, and if they're happy then we could maybe leave > it as is with the 2 alternatives output files. At least the first previously borked animal came bac

Re: [HACKERS] path toward faster partition pruning

2018-04-06 Thread David Rowley
On 7 April 2018 at 15:41, Tom Lane wrote: > David Rowley writes: >> Sounds like you're saying that if we have too many alternative files >> then there's a chance that one could pass by luck. > > Yeah, exactly: it passed, but did it pass for the right reason? > > If there's just two expected-files

Re: [HACKERS] path toward faster partition pruning

2018-04-06 Thread Andres Freund
Hi, On 2018-04-06 23:41:22 -0400, Tom Lane wrote: > David Rowley writes: > > Sounds like you're saying that if we have too many alternative files > > then there's a chance that one could pass by luck. > > Yeah, exactly: it passed, but did it pass for the right reason? > > If there's just two ex

Re: [HACKERS] path toward faster partition pruning

2018-04-06 Thread David Rowley
On 7 April 2018 at 15:41, Tom Lane wrote: > I'm also wondering how come we had hash partitioning before and > did not have this sort of problem. Is it just that we added a > new test that's more sensitive to the details of the hashing > (if so, could it be made less so)? Or is there actually mor

Re: [HACKERS] path toward faster partition pruning

2018-04-06 Thread Tom Lane
David Rowley writes: > Sounds like you're saying that if we have too many alternative files > then there's a chance that one could pass by luck. Yeah, exactly: it passed, but did it pass for the right reason? If there's just two expected-files, it's likely not a big problem, but if you have a bu

Re: [HACKERS] path toward faster partition pruning

2018-04-06 Thread Ashutosh Bapat
On Sat, Apr 7, 2018 at 8:55 AM, David Rowley wrote: > On 7 April 2018 at 15:14, Ashutosh Bapat > wrote: >> On Sat, Apr 7, 2018 at 8:37 AM, David Rowley >>> Why is writing tests that produce the same output required? >>> >>> We have many tests with alternative outputs. Look in >>> src/tests/regres

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

2018-04-06 Thread Andres Freund
Hi Tom, All, On 2018-04-06 14:19:02 +0530, amul sul wrote: > Thanks for the reminder -- fixed in the attached version. Tom, this seems to be the best approach for fixing the visibility issues around this. I've spent a good chunk of time looking at corruption issues like the ones you feared (see [

Re: [HACKERS] path toward faster partition pruning

2018-04-06 Thread David Rowley
On 7 April 2018 at 15:14, Ashutosh Bapat wrote: > On Sat, Apr 7, 2018 at 8:37 AM, David Rowley >> Why is writing tests that produce the same output required? >> >> We have many tests with alternative outputs. Look in >> src/tests/regress/expected for files matching _1.out >> > > That's true, but w

Re: [HACKERS] path toward faster partition pruning

2018-04-06 Thread David Rowley
On 7 April 2018 at 15:18, Andres Freund wrote: > I've pushed the two patches (collapsed). Trying to get the BF green-ish > again... Thanks! -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] path toward faster partition pruning

2018-04-06 Thread Andres Freund
On 2018-04-07 15:04:37 +1200, David Rowley wrote: > On 7 April 2018 at 15:00, Andres Freund wrote: > > On 2018-04-07 14:42:53 +1200, David Rowley wrote: > >> On 7 April 2018 at 13:31, David Rowley > >> wrote: > >> > Maybe the best solution is to pull those tests out of > >> > partition_prune.sql

Re: [HACKERS] pgbench - allow to store select results into variables

2018-04-06 Thread Stephen Frost
Fabien, * Fabien COELHO (fabien.coe...@mines-paristech.fr) wrote: > Patch v16 is a rebase. Here's that review. > diff --git a/doc/src/sgml/ref/pgbench.sgml b/doc/src/sgml/ref/pgbench.sgml > index d52d324..203b6bc 100644 > --- a/doc/src/sgml/ref/pgbench.sgml > +++ b/doc/src/sgml/ref/pgbench.sgml

Re: [HACKERS] path toward faster partition pruning

2018-04-06 Thread Ashutosh Bapat
On Sat, Apr 7, 2018 at 8:37 AM, David Rowley wrote: > On 7 April 2018 at 15:03, Ashutosh Bapat > wrote: >> On Sat, Apr 7, 2018 at 7:25 AM, David Rowley >>> The only alternative would be to change all the hash functions so that >>> they normalise their endianness. It does not sound like something

Re: [HACKERS] path toward faster partition pruning

2018-04-06 Thread David Rowley
On 7 April 2018 at 15:03, Ashutosh Bapat wrote: > On Sat, Apr 7, 2018 at 7:25 AM, David Rowley >> The only alternative would be to change all the hash functions so that >> they normalise their endianness. It does not sound like something that >> will perform very well. Plus it would break everyone

Re: [HACKERS] path toward faster partition pruning

2018-04-06 Thread David Rowley
On 7 April 2018 at 15:00, Andres Freund wrote: > On 2018-04-07 14:42:53 +1200, David Rowley wrote: >> On 7 April 2018 at 13:31, David Rowley wrote: >> > Maybe the best solution is to pull those tests out of >> > partition_prune.sql then create partition_prune_hash and just have an >> > alternativ

Re: [HACKERS] path toward faster partition pruning

2018-04-06 Thread Ashutosh Bapat
On Sat, Apr 7, 2018 at 7:25 AM, David Rowley wrote: > On 7 April 2018 at 13:50, Amit Langote wrote: >> On Sat, Apr 7, 2018 at 10:31 AM, David Rowley >>> I looked at all the regression test diffs for each of the servers you >>> mentioned and I verified that the diffs match on each of the 7 >>> ser

Re: [HACKERS] path toward faster partition pruning

2018-04-06 Thread Andres Freund
On 2018-04-07 14:42:53 +1200, David Rowley wrote: > On 7 April 2018 at 13:31, David Rowley wrote: > > Maybe the best solution is to pull those tests out of > > partition_prune.sql then create partition_prune_hash and just have an > > alternative .out file with the partitions which match on bigendi

Re: [HACKERS] path toward faster partition pruning

2018-04-06 Thread David Rowley
On 7 April 2018 at 13:31, David Rowley wrote: > Maybe the best solution is to pull those tests out of > partition_prune.sql then create partition_prune_hash and just have an > alternative .out file with the partitions which match on bigendian > machines. Here's 1 of 2. I thought it was best to ge

Re: [HACKERS] Runtime Partition Pruning

2018-04-06 Thread David Rowley
On 7 April 2018 at 12:03, David Rowley wrote: > Continuing to read 0003 and 0004 now. 0003: 1. "setup" -> "set" /* If run-time partition pruning is enabled, then setup that up now */ 2. We should be able to get rid of as_noopscan and just have another special negative value for as_whichplan.

Re: PATCH: Configurable file mode mask

2018-04-06 Thread Stephen Frost
David, * David Steele (da...@pgmasters.net) wrote: > On 4/6/18 6:04 PM, David Steele wrote: > >On 4/6/18 3:02 PM, Stephen Frost wrote: > >> > >>- Further discussion in the commit messages > > > >Agreed, these need some more work.  I'm happy to do that but I'll need a > >bit more time.  Have a look

Re: WIP: Covering + unique indexes.

2018-04-06 Thread Peter Geoghegan
On Fri, Apr 6, 2018 at 11:08 AM, Alexander Korotkov wrote: > OK, incorporated into v15. I've also added sentence about pg_upgrade > to the commit message. I will summarize my feelings on this patch. I endorse committing the patch, because I think that the benefits of committing it now noticeably

Re: [HACKERS] path toward faster partition pruning

2018-04-06 Thread David Rowley
On 7 April 2018 at 13:50, Amit Langote wrote: > On Sat, Apr 7, 2018 at 10:31 AM, David Rowley >> I looked at all the regression test diffs for each of the servers you >> mentioned and I verified that the diffs match on each of the 7 >> servers. >> >> Maybe the best solution is to pull those tests

Re: [HACKERS] path toward faster partition pruning

2018-04-06 Thread Amit Langote
On Sat, Apr 7, 2018 at 10:31 AM, David Rowley wrote: > On 7 April 2018 at 12:43, David Rowley wrote: >> On 7 April 2018 at 12:35, Amit Langote wrote: >>> So this same failure occurs on (noting the architecture): >>> >>> Seems to be due to that the hashing function used in partitioning >>> gives

Re: [HACKERS] path toward faster partition pruning

2018-04-06 Thread David Rowley
On 7 April 2018 at 12:43, David Rowley wrote: > On 7 April 2018 at 12:35, Amit Langote wrote: >> So this same failure occurs on (noting the architecture): >> >> Seems to be due to that the hashing function used in partitioning >> gives different answer for a given set of partition key values than

Re: [PATCH] Update README for Resource Owners

2018-04-06 Thread Chapman Flack
On 04/06/18 20:19, Chapman Flack wrote: > On 04/06/18 19:52, Andres Freund wrote: >> Wouldn't it be a better idea not to have a list there, that's guaranteed >> to get out of date? > > That might look like this, then. > > But I'm not sure how bad it is to have a list. How often does ResourceOwner

Re: Online enabling of checksums

2018-04-06 Thread Andres Freund
Here's a pass through the patch: @@ -1033,7 +1034,7 @@ XLogInsertRecord(XLogRecData *rdata, Assert(RedoRecPtr < Insert->RedoRecPtr); RedoRecPtr = Insert->RedoRecPtr; } - doPageWrites = (Insert->fullPageWrites || Insert->forcePageWrites); + doPageW

Re: [HACKERS] path toward faster partition pruning

2018-04-06 Thread David Rowley
On 7 April 2018 at 12:35, Amit Langote wrote: > Thank you Alvaro for rest of the cleanup and committing. +10! > So this same failure occurs on (noting the architecture): > > ppc64: > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=quokka&dt=2018-04-06%2020%3A09%3A52 > > ia64: > https://b

Re: [HACKERS] path toward faster partition pruning

2018-04-06 Thread Amit Langote
Thank you Alvaro for rest of the cleanup and committing. On Sat, Apr 7, 2018 at 5:28 AM, Alvaro Herrera wrote: > So I pushed this 25 minutes ago, and already there's a couple of > buildfarm members complaining: > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=quokka&dt=2018-04-06%2020%3A

Re: [PATCH] Update README for Resource Owners

2018-04-06 Thread Chapman Flack
On 04/06/18 19:52, Andres Freund wrote: > On 2018-04-06 19:46:25 -0400, Chapman Flack wrote: >> Hasn't been updated as built-in support grew for temp files, DSM segments, >> and JIT contexts. >> >> With enough luck, a README update won't break anything. > > Wouldn't it be a better idea not to have

Re: [HACKERS] Runtime Partition Pruning

2018-04-06 Thread David Rowley
On 7 April 2018 at 10:45, David Rowley wrote: > I'm looking over the rebased patches now. I've made a complete read of 0001 and 0002 so far. Your rebase looks fine. After the complete read, I only have the following comments: 0001: 1. missing "the" before "partition key": * Extract Params ma

Re: Vacuum: allow usage of more than 1GB of work mem

2018-04-06 Thread Claudio Freire
On Fri, Apr 6, 2018 at 5:25 PM, Claudio Freire wrote: > On Fri, Apr 6, 2018 at 10:39 AM, Heikki Linnakangas wrote: >> On 06/04/18 01:59, Claudio Freire wrote: >>> >>> The iteration interface, however, seems quite specific for the use >>> case of vacuumlazy, so it's not really a good abstraction.

Re: Online enabling of checksums

2018-04-06 Thread Stephen Frost
Andres, * Andres Freund (and...@anarazel.de) wrote: > On 2018-04-06 19:31:56 -0400, Stephen Frost wrote: > > I'm quite sure that bringing up MERGE in this thread and saying it needs > > to be reverted without even having the committer of that feature on the > > CC list isn't terribly useful and co

Re: [PATCH] Update README for Resource Owners

2018-04-06 Thread Andres Freund
Hi, On 2018-04-06 19:46:25 -0400, Chapman Flack wrote: > Hasn't been updated as built-in support grew for temp files, DSM segments, > and JIT contexts. > > With enough luck, a README update won't break anything. Wouldn't it be a better idea not to have a list there, that's guaranteed to get out

Re: Online enabling of checksums

2018-04-06 Thread Andres Freund
On 2018-04-07 01:27:13 +0200, Daniel Gustafsson wrote: > > On 07 Apr 2018, at 01:13, Andres Freund wrote: > > > > On 2018-04-07 01:04:50 +0200, Daniel Gustafsson wrote: > >>> I'm fairly certain that the bug here is a simple race condition in the > >>> test (not the main code!): > >> > >> I wonde

[PATCH] Update README for Resource Owners

2018-04-06 Thread Chapman Flack
Hasn't been updated as built-in support grew for temp files, DSM segments, and JIT contexts. With enough luck, a README update won't break anything. -Chap >From fb06e17916ad6b445b1cf6361de6a7f2749ea225 Mon Sep 17 00:00:00 2001 From: Chapman Flack Date: Fri, 6 Apr 2018 19:41:57 -0400 Subject: [PA

Re: Online enabling of checksums

2018-04-06 Thread Andres Freund
On 2018-04-06 19:31:56 -0400, Stephen Frost wrote: > Greetings, > > * Robert Haas (robertmh...@gmail.com) wrote: > > On Fri, Apr 6, 2018 at 6:56 PM, Andres Freund wrote: > > > no one can entirely quibble with the rationale that this is ok (I'll > > > post a patch cleaning up the atomics simulatio

Re: Parallel index creation does not properly cleanup after error

2018-04-06 Thread David Rowley
On 7 April 2018 at 11:39, Robert Haas wrote: > Committed. Thanks to David for the report and analysis and to Peter > for the patch and study. Thanks for pushing! -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] path toward faster partition pruning

2018-04-06 Thread Alvaro Herrera
Robert Haas wrote: > On Fri, Apr 6, 2018 at 8:24 AM, Alvaro Herrera > wrote: > > I don't actually like very much the idea of putting all this code in > > optimizer/util. This morning it occurred to me that we should create a new > > src/backend/partitioning/ (and a src/include/partitioning/ to g

Re: Parallel index creation does not properly cleanup after error

2018-04-06 Thread Peter Geoghegan
On Fri, Apr 6, 2018 at 4:39 PM, Robert Haas wrote: > Committed. Thanks to David for the report and analysis and to Peter > for the patch and study. Thanks! -- Peter Geoghegan

Re: Parallel index creation does not properly cleanup after error

2018-04-06 Thread Robert Haas
On Wed, Mar 14, 2018 at 2:14 PM, Peter Geoghegan wrote: > On Wed, Mar 14, 2018 at 5:16 AM, David Rowley > wrote: >> Thanks for putting the patch together. Although in regards to your >> proposed commit message in the patch, I'd disagree with me having just >> reported it. I also investigated it a

Bring atomic flag fallback up to snuff

2018-04-06 Thread Andres Freund
Hi, As Daniel pointed out in: https://postgr.es/m/fb948276-7b32-4b77-83e6-d00167f8e...@yesql.se the pg_atomic_flag fallback implementation is broken. That has gone unnoticed because the fallback implementation wasn't testable until now: - /* --- -* Can't run the test under the semaphore emu

Re: Online enabling of checksums

2018-04-06 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Fri, Apr 6, 2018 at 6:56 PM, Andres Freund wrote: > > no one can entirely quibble with the rationale that this is ok (I'll > > post a patch cleaning up the atomics simulation of flags in a bit), but > > this is certainly not a correct l

Re: Online enabling of checksums

2018-04-06 Thread Daniel Gustafsson
> On 07 Apr 2018, at 01:13, Andres Freund wrote: > > On 2018-04-07 01:04:50 +0200, Daniel Gustafsson wrote: >>> I'm fairly certain that the bug here is a simple race condition in the >>> test (not the main code!): >> >> I wonder if it may perhaps be a case of both? > > See my other message abou

Re: [HACKERS] Add support for tuple routing to foreign partitions

2018-04-06 Thread Robert Haas
On Thu, Apr 5, 2018 at 6:21 AM, Etsuro Fujita wrote: > Attached is an updated version of the patch set plus the patch in [1]. Patch > 0003_foreign-routing-fdwapi-6.patch can be applied on top of patch > 0001_postgres-fdw-refactoring-6.patch and > 0002_copy-from-check-constraint-fix.patch. Committ

Re: Online enabling of checksums

2018-04-06 Thread Andres Freund
On 2018-04-07 01:04:50 +0200, Daniel Gustafsson wrote: > > I'm fairly certain that the bug here is a simple race condition in the > > test (not the main code!): > > I wonder if it may perhaps be a case of both? See my other message about the atomic fallback bit. > > It's > > exceedingly unsurpr

Re: Online enabling of checksums

2018-04-06 Thread Daniel Gustafsson
> On 07 Apr 2018, at 00:23, Andres Freund wrote: > On 2018-04-06 02:28:17 +0200, Daniel Gustafsson wrote: >> Applying this makes the _cancel test pass, moving the failure instead to the >> following _enable test (which matches what coypu and mylodon are seeing). > > FWIW, I'm somewhat annoyed th

Re: PostgreSQL 11 Release Management Team & Feature Freeze

2018-04-06 Thread Robert Haas
On Fri, Apr 6, 2018 at 6:59 PM, Jonathan S. Katz wrote: > Best of luck to everyone over the next 24hrs! I think that's the wrong sentiment, honestly. I think we have too many committers relying way too much on luck already. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterpris

Re: [HACKERS] path toward faster partition pruning

2018-04-06 Thread Robert Haas
On Fri, Apr 6, 2018 at 8:24 AM, Alvaro Herrera wrote: > I don't actually like very much the idea of putting all this code in > optimizer/util. This morning it occurred to me that we should create a new > src/backend/partitioning/ (and a src/include/partitioning/ to go with > it) and drop a bunch

Re: PostgreSQL 11 Release Management Team & Feature Freeze

2018-04-06 Thread Jonathan S. Katz
> On Apr 6, 2018, at 6:57 PM, Robert Haas wrote: > > On Fri, Apr 6, 2018 at 6:54 PM, Jonathan S. Katz wrote: >> Yes. It follows the format from the previous ones, i.e: >> >> https://www.postgresql.org/message-id/ca+tgmoy56w5fozeeo+i48qehl+bsvtwy-q1m0xjuhucwggw...@mail.gmail.com > > I think m

Re: Online enabling of checksums

2018-04-06 Thread Robert Haas
On Fri, Apr 6, 2018 at 6:56 PM, Andres Freund wrote: > no one can entirely quibble with the rationale that this is ok (I'll > post a patch cleaning up the atomics simulation of flags in a bit), but > this is certainly not a correct locking strategy. I think we have enough evidence at this point t

Re: PostgreSQL 11 Release Management Team & Feature Freeze

2018-04-06 Thread Robert Haas
On Fri, Apr 6, 2018 at 6:54 PM, Jonathan S. Katz wrote: > Yes. It follows the format from the previous ones, i.e: > > https://www.postgresql.org/message-id/ca+tgmoy56w5fozeeo+i48qehl+bsvtwy-q1m0xjuhucwggw...@mail.gmail.com I think my confusion resulted from the fact that my 2016 message said "mu

Re: Online enabling of checksums

2018-04-06 Thread Andres Freund
On 2018-04-06 14:33:48 -0700, Andres Freund wrote: > On 2018-04-06 02:28:17 +0200, Daniel Gustafsson wrote: > > Looking into the isolationtester failure on piculet, which builds using > > --disable-atomics, and locust which doesn’t have atomics, the code for > > pg_atomic_test_set_flag seems a bit

Re: PostgreSQL 11 Release Management Team & Feature Freeze

2018-04-06 Thread Jonathan S. Katz
> On Apr 6, 2018, at 6:53 PM, Robert Haas wrote: > > On Tue, Mar 27, 2018 at 9:51 AM, Jonathan S. Katz > wrote: >> The Release Management Team (RMT) for the PostgreSQL 11 release >> has been assembled and has determined that the feature freeze date >> for the PostgreSQL 11 release will be Apri

Re: PostgreSQL 11 Release Management Team & Feature Freeze

2018-04-06 Thread Robert Haas
On Tue, Mar 27, 2018 at 9:51 AM, Jonathan S. Katz wrote: > The Release Management Team (RMT) for the PostgreSQL 11 release > has been assembled and has determined that the feature freeze date > for the PostgreSQL 11 release will be April 7, 2018. This means that any > feature that will be going in

Re: src/backend/partitioning

2018-04-06 Thread Robert Haas
On Fri, Apr 6, 2018 at 12:51 PM, Alvaro Herrera wrote: > My patch is hidden in a very long thread, so I thought letting more > widely known that I'm proposing to add a new subdir "partitioning" under > src/backend and src/include. > TBH I think the new idea is better than our current arrangement w

Re: pgsql: Foreign keys on partitioned tables

2018-04-06 Thread Robert Haas
On Fri, Apr 6, 2018 at 4:49 PM, Alvaro Herrera wrote: > Robert Haas wrote: >> On Wed, Apr 4, 2018 at 1:03 PM, Alvaro Herrera >> wrote: >> > Foreign keys on partitioned tables >> > >> > Author: Álvaro Herrera >> > Discussion: >> > https://postgr.es/m/20171231194359.cvojcour423ulha4@alvherre.pgsq

Re: [HACKERS] Runtime Partition Pruning

2018-04-06 Thread David Rowley
On 7 April 2018 at 09:29, Alvaro Herrera wrote: > Alvaro Herrera wrote: >> I rebased this series on top of the committed version of the other patch. >> Here's v22, with no other changes than rebasing. I did not include >> 0005, though. > > Apologies, I forgot to "git add" one fixup for 0001. 000

Re: PATCH: Configurable file mode mask

2018-04-06 Thread David Steele
On 4/6/18 6:04 PM, David Steele wrote: On 4/6/18 3:02 PM, Stephen Frost wrote: - Further discussion in the commit messages Agreed, these need some more work.  I'm happy to do that but I'll need a bit more time.  Have a look at the new patches and I'll work on some better messages. I'm sur

Re: Online enabling of checksums

2018-04-06 Thread Andres Freund
Hi, On 2018-04-06 02:28:17 +0200, Daniel Gustafsson wrote: > Applying this makes the _cancel test pass, moving the failure instead to the > following _enable test (which matches what coypu and mylodon are seeing). FWIW, I'm somewhat annoyed that I'm now spending time debugging this to get the bui

Re: PATCH: Configurable file mode mask

2018-04-06 Thread David Steele
Hi Stephen, On 4/6/18 3:02 PM, Stephen Frost wrote: Alright, changes I've made, since I got impatient and it didn't seem to make sense to bounce these back to David instead of just making them (I did discuss them with him on the phone today tho, just to be clear). - The PG_FILE_MODE_DEFAULT, P

Re: PATCH: pgbench - option to build using ppoll() for larger connection counts

2018-04-06 Thread Andres Freund
On 2018-04-06 17:49:19 -0400, Tom Lane wrote: > Andres Freund writes: > > I'm still not particularly happy with this. > > I'm a bit confused as to what the point is. It seems unlikely that one > pgbench process can effectively drive enough backends for select's > limitations to really be an issue

Re: The buildfarm is in a pretty bad way, folks

2018-04-06 Thread Tom Lane
Magnus Hagander writes: > On Fri, Apr 6, 2018 at 11:44 PM, Tom Lane wrote: >> (I'm also not real happy about the amount of time the checksum-xxx >> tests consume.) > The isolation tester ones, or the regular ones? Because the regular ones > finish in << 30 seconds here, just wondering if that ac

Re: PATCH: pgbench - option to build using ppoll() for larger connection counts

2018-04-06 Thread Tom Lane
Andres Freund writes: > I'm still not particularly happy with this. I'm a bit confused as to what the point is. It seems unlikely that one pgbench process can effectively drive enough backends for select's limitations to really be an issue. regards, tom lane

Re: The buildfarm is in a pretty bad way, folks

2018-04-06 Thread Magnus Hagander
On Fri, Apr 6, 2018 at 11:44 PM, Tom Lane wrote: > Andres Freund writes: > > On 2018-04-06 16:59:11 -0400, Tom Lane wrote: > >> But in particular, it's clear that partition_prune and > >> isolation/checksum_cancel are showing big problems. > > > While I'm obviously also unhappy about the frantic

Re: The buildfarm is in a pretty bad way, folks

2018-04-06 Thread Tom Lane
Andres Freund writes: > On 2018-04-06 16:59:11 -0400, Tom Lane wrote: >> But in particular, it's clear that partition_prune and >> isolation/checksum_cancel are showing big problems. > While I'm obviously also unhappy about the frantic push to push semi > baked stuff, I'm not sure the two issues

Re: [HACKERS] logical decoding of two-phase transactions

2018-04-06 Thread Andrew Dunstan
On Fri, Apr 6, 2018 at 10:00 PM, Peter Eisentraut wrote: > On 4/3/18 18:05, Andrew Dunstan wrote: >> Currently we seem to have only two machines doing the cross-version >> upgrade checks, which might make it easier to rearrange anything if >> necessary. > > I think we should think about making thi

Re: Online enabling of checksums

2018-04-06 Thread Andres Freund
On 2018-04-06 02:28:17 +0200, Daniel Gustafsson wrote: > Looking into the isolationtester failure on piculet, which builds using > --disable-atomics, and locust which doesn’t have atomics, the code for > pg_atomic_test_set_flag seems a bit odd. > > TAS() is defined to return zero if successful, an

Re: The buildfarm is in a pretty bad way, folks

2018-04-06 Thread Alvaro Herrera
Tom Lane wrote: > It sure looks like there's been a frantic push to commit stuff that > maybe wasn't quite fully baked. I'm not terribly on board with that, > because it's likely to be hard to disentangle who broke what. > But in particular, it's clear that partition_prune and > isolation/checksum

Re: [HACKERS] Runtime Partition Pruning

2018-04-06 Thread Alvaro Herrera
Alvaro Herrera wrote: > I rebased this series on top of the committed version of the other patch. > Here's v22, with no other changes than rebasing. I did not include > 0005, though. Apologies, I forgot to "git add" one fixup for 0001. -- Álvaro Herrerahttps://www.2ndQuadrant.co

Re: The buildfarm is in a pretty bad way, folks

2018-04-06 Thread Magnus Hagander
On Fri, Apr 6, 2018 at 11:19 PM, Andres Freund wrote: > On 2018-04-06 23:12:19 +0200, Magnus Hagander wrote: > > Daniel is working on investigating the isolationtester thing. See a mail > on > > one of the threads where initial indications were the "atomics with no > real > > atomics" (or whateve

Re: [HACKERS] logical decoding of two-phase transactions

2018-04-06 Thread Andrew Dunstan
On Sat, Apr 7, 2018 at 1:50 AM, Andres Freund wrote: > Hi, > > On 2018-04-06 21:30:36 +0930, Andrew Dunstan wrote: >> OK, I think this is now committable. > >> The changes are small, fairly isolated in effect, and I think every >> objection has been met, partly by reducing the scope of the >> chan

Re: The buildfarm is in a pretty bad way, folks

2018-04-06 Thread Andres Freund
On 2018-04-06 23:12:19 +0200, Magnus Hagander wrote: > Daniel is working on investigating the isolationtester thing. See a mail on > one of the threads where initial indications were the "atomics with no real > atomics" (or whatever you'd call it) were to blame. We could redo that > thing without a

Re: PATCH: pgbench - option to build using ppoll() for larger connection counts

2018-04-06 Thread Andres Freund
Hi, I'm still not particularly happy with this. Checking whether I can polish it up. a) the new function names are partially non-descriptive and their meaning is undocumented. As an extreme example: - if (!FD_ISSET(sock, &input_mask)) +

Re: The buildfarm is in a pretty bad way, folks

2018-04-06 Thread Magnus Hagander
On Fri, Apr 6, 2018 at 10:59 PM, Tom Lane wrote: > It sure looks like there's been a frantic push to commit stuff that > maybe wasn't quite fully baked. I'm not terribly on board with that, > because it's likely to be hard to disentangle who broke what. > But in particular, it's clear that parti

Re: [HACKERS] path toward faster partition pruning

2018-04-06 Thread Andres Freund
On 2018-04-06 17:28:00 -0300, Alvaro Herrera wrote: > So I pushed this 25 minutes ago, and already there's a couple of > buildfarm members complaining: > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=quokka&dt=2018-04-06%2020%3A09%3A52 > https://buildfarm.postgresql.org/cgi-bin/show_log.p

Re: The buildfarm is in a pretty bad way, folks

2018-04-06 Thread Andres Freund
Hi, On 2018-04-06 16:59:11 -0400, Tom Lane wrote: > It sure looks like there's been a frantic push to commit stuff that > maybe wasn't quite fully baked. I'm not terribly on board with that, > because it's likely to be hard to disentangle who broke what. > But in particular, it's clear that parti

The buildfarm is in a pretty bad way, folks

2018-04-06 Thread Tom Lane
It sure looks like there's been a frantic push to commit stuff that maybe wasn't quite fully baked. I'm not terribly on board with that, because it's likely to be hard to disentangle who broke what. But in particular, it's clear that partition_prune and isolation/checksum_cancel are showing big pr

Re: [FEATURE PATCH] pg_stat_statements with plans (v02)

2018-04-06 Thread legrand legrand
> I've created a draft patch that provides access to plans in a view > called pg_stat_statements_plans. ++ I like it ! > There is no column that indicates whether the plan is "good" or "bad", > because that is evident from the execution time of both plans and because > that would require s

Re: [HACKERS] pgbench - allow to store select results into variables

2018-04-06 Thread Stephen Frost
Greetings Fabien, * Fabien COELHO (fabien.coe...@mines-paristech.fr) wrote: > >>Here is a v14, after yet another rebase, and some comments added to > >>answer your new comments. > > > >Attached v15 is a simple rebase after Teodor push of new functions & > >operators in pgbench. > > Patch v16 is a

Re: pgsql: Foreign keys on partitioned tables

2018-04-06 Thread Alvaro Herrera
Robert Haas wrote: > On Wed, Apr 4, 2018 at 1:03 PM, Alvaro Herrera > wrote: > > Foreign keys on partitioned tables > > > > Author: Álvaro Herrera > > Discussion: > > https://postgr.es/m/20171231194359.cvojcour423ulha4@alvherre.pgsql > > Reviewed-by: Peter Eisentraut > > The commit message here

Re: [HACKERS] [PATCH] Incremental sort

2018-04-06 Thread Alexander Kuzmenkov
Hi all, This is the other Alexander K. speaking. On 06.04.2018 20:26, Tomas Vondra wrote: I personally am OK with reducing the scope of the patch like this. It's still beneficial for the common ORDER BY + LIMIT case, which is good. I don't think it may negatively affect other cases (at least I

Re: Vacuum: allow usage of more than 1GB of work mem

2018-04-06 Thread Alvaro Herrera
Claudio Freire wrote: > On Fri, Apr 6, 2018 at 11:00 AM, Alvaro Herrera > wrote: > > FWIW I liked the idea of having this abstraction possibly do other > > things -- for instance to vacuum brin indexes you'd like to mark index > > tuples as "containing tuples that were removed" and eventually >

Re: Documentation for bootstrap data conversion

2018-04-06 Thread Tom Lane
Andres Freund writes: > Quick skim only: > "developers" here could possibly be understood to be any sort of > developer, rather than postgres ones. Perhaps just say "But the > structure of the catalogs can change between major versions."? OK. > This sounds like an exhaustive list of what genbk

  1   2   3   >