Re: [HACKERS] Parallel pg_dump's error reporting doesn't work worth squat

2016-05-27 Thread Amit Kapila
On Sat, May 28, 2016 at 5:06 AM, Tom Lane wrote: > > Alvaro Herrera writes: > > Regarding this: > >> + * XXX this certainly looks useless, why not just wait indefinitely? > > > There's another select_loop() in vacuumdb.c suggesting that the timeout > > is used to check for cancel requ

Re: [HACKERS] Hard to maintain duplication in contain_volatile_functions_not_nextval_walker

2016-05-27 Thread Amit Kapila
On Sat, May 28, 2016 at 12:28 AM, Andres Freund wrote: > > Hi, > > contain_volatile_functions_walker is duplicated, near entirely, in > contain_volatile_functions_not_nextval_walker. > Previously, I also had same observation. > Wouldn't it have been better not to duplicate, and keep a flag about

Re: [HACKERS] [GENERAL] Permission Denied Error on pg_xlog/RECOVERYXLOG file

2016-05-27 Thread Andres Freund
On 2016-05-27 20:54:43 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2016-05-26 12:44:51 -0400, Tom Lane wrote: > > 2016-04-27 17:02:06 EDT 572128cd.1811 [7-1] user=,db=,remote= FATAL: 42501: > > could not open file "pg_xlog/RECOVERYXLOG": Permission denied > > > So, what's the permissio

Re: [HACKERS] [GENERAL] Permission Denied Error on pg_xlog/RECOVERYXLOG file

2016-05-27 Thread Tom Lane
Andres Freund writes: > On 2016-05-26 12:44:51 -0400, Tom Lane wrote: > 2016-04-27 17:02:06 EDT 572128cd.1811 [7-1] user=,db=,remote= FATAL: 42501: > could not open file "pg_xlog/RECOVERYXLOG": Permission denied > So, what's the permission of RECOVERYXLOG at that point? It's pretty > weird that

Re: [HACKERS] [GENERAL] Permission Denied Error on pg_xlog/RECOVERYXLOG file

2016-05-27 Thread Andres Freund
On 2016-05-26 12:44:51 -0400, Tom Lane wrote: > > 2016-04-27 17:02:06 EDT 572128cd.1811 [7-1] user=,db=,remote= FATAL: 42501: > > could not open file "pg_xlog/RECOVERYXLOG": Permission denied So, what's the permission of RECOVERYXLOG at that point? It's pretty weird that directly after running r

Re: [HACKERS] Parallel pg_dump's error reporting doesn't work worth squat

2016-05-27 Thread Tom Lane
Alvaro Herrera writes: > Regarding this: >> + * XXX this certainly looks useless, why not just wait >> indefinitely? > There's another select_loop() in vacuumdb.c suggesting that the timeout > is used to check for cancel requests; as I understood while working on > the vacuumdb chang

Re: [HACKERS] Parallel pg_dump's error reporting doesn't work worth squat

2016-05-27 Thread Alvaro Herrera
Regarding this: > *** select_loop(int maxFd, fd_set *workerset > *** 1092,1104 > --- 1150,1160 > fd_set saveSet = *workerset; > > #ifdef WIN32 > for (;;) > { > /* >* sleep a quarter of a second before checking if w

[HACKERS] new PageAddItemFlags()

2016-05-27 Thread Alvaro Herrera
In the course of fixing a BRIN bug, I'm adding a new function to bufpage.c called PageAddItemFlags, which is much like PageAddItem() except that the "ovewrite" and "is_heap" boolean flags are gone and replaced by an integer bitmap and some #defines. If anybody would like to bikeshed, please do so

Re: [HACKERS] [sqlsmith] PANIC: failed to add BRIN tuple

2016-05-27 Thread Alvaro Herrera
Andreas Seltenreich wrote: > I wrote: > > > Re-fuzzing now with your patch applied. > > This so far yielded three BRIN core dumps on different machines with the > same backtraces. Crash recovery fails in these cases. > > I've put the data directory here (before recovery): > > http://ansel.

[HACKERS] copyParamList

2016-05-27 Thread Andrew Gierth
copyParamList does not respect from->paramMask, in what looks to me like an obvious oversight: retval->paramMask = NULL; [...] /* Ignore parameters we don't need, to save cycles and space. */ if (retval->paramMask != NULL && !bms_is_member(i, retval->paramMask)) re

Re: [HACKERS] 9.4 -> 9.5 regression with queries through pgbouncer on RHEL 6

2016-05-27 Thread Andres Freund
Hi, On 2016-05-27 19:57:34 +0300, Vladimir Borodin wrote: > -performance > > Here is how the results look like for 9.4, 9.5 and 9.6. All are built from > > latest commits on yesterday in > > * REL9_4_STABLE (a0cc89a28141595d888d8aba43163d58a1578bfb), > > * REL9_5_STABLE (e504d915bbf352ec

Re: [HACKERS] Parallel pg_dump's error reporting doesn't work worth squat

2016-05-27 Thread Tom Lane
I wrote: >> BTW, after having spent the afternoon staring at it, I will assert with >> confidence that pg_dump/parallel.c is an embarrassment to the project. > I've done a bit of work on a cosmetic cleanup patch, but don't have > anything to show yet. Attached is the threatened cosmetic cleanup o

Re: [HACKERS] [PROPOSAL] Move all am-related reloption code into src/backend/access/[am-name] and get rid of relopt_kind

2016-05-27 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera writes: > > Nikolay Shaplov wrote: > >> Story start from the point that I found out that a.m. can not forbid > >> changing > >> some of it's reloptions with ALTER INDEX command. > > > Hmm, this sounds like a bug to me. In BRIN, if you change the > > pages_per_r

Re: [HACKERS] [PROPOSAL] Move all am-related reloption code into src/backend/access/[am-name] and get rid of relopt_kind

2016-05-27 Thread Tom Lane
Alvaro Herrera writes: > Nikolay Shaplov wrote: >> Story start from the point that I found out that a.m. can not forbid >> changing >> some of it's reloptions with ALTER INDEX command. > Hmm, this sounds like a bug to me. In BRIN, if you change the > pages_per_range option for an existing inde

Re: [HACKERS] Status of 64 bit atomics

2016-05-27 Thread Andres Freund
Hi, On 2016-05-27 11:00:42 -0400, John Gorman wrote: > Someone recently told me that the postgresql atomics library was incomplete > for 64 bit operations such as pg_atomic_fetch_add_u64() and should not be > used. There's currently no fallback for 32bit platforms without 64bit atomics support.

Re: [HACKERS] Status of 64 bit atomics

2016-05-27 Thread Arthur Silva
On May 27, 2016 5:01 PM, "John Gorman" wrote: > > Hi All > > Someone recently told me that the postgresql atomics library was incomplete > for 64 bit operations such as pg_atomic_fetch_add_u64() and should not be used. > > Can someone definitively confirm whether it is okay to rely on the 64 bit a

Re: [HACKERS] [PROPOSAL] Move all am-related reloption code into src/backend/access/[am-name] and get rid of relopt_kind

2016-05-27 Thread Alvaro Herrera
Nikolay Shaplov wrote: > В письме от 27 мая 2016 15:05:58 Вы написали: > > Nikolay Shaplov wrote: > > > Story start from the point that I found out that a.m. can not forbid > > > changing some of it's reloptions with ALTER INDEX command. That was not > > > necessary before, because all reloptions a

Re: [HACKERS] [PROPOSAL] Move all am-related reloption code into src/backend/access/[am-name] and get rid of relopt_kind

2016-05-27 Thread Nikolay Shaplov
В письме от 27 мая 2016 15:05:58 Вы написали: > Nikolay Shaplov wrote: > > Story start from the point that I found out that a.m. can not forbid > > changing some of it's reloptions with ALTER INDEX command. That was not > > necessary before, because all reloptions at that existed at that time can >

Re: [HACKERS] [PROPOSAL] Move all am-related reloption code into src/backend/access/[am-name] and get rid of relopt_kind

2016-05-27 Thread Alvaro Herrera
Nikolay Shaplov wrote: > Story start from the point that I found out that a.m. can not forbid changing > some of it's reloptions with ALTER INDEX command. That was not necessary > before, because all reloptions at that existed at that time can be changed on > fly. But now for bloom index it is

[HACKERS] Hard to maintain duplication in contain_volatile_functions_not_nextval_walker

2016-05-27 Thread Andres Freund
Hi, contain_volatile_functions_walker is duplicated, near entirely, in contain_volatile_functions_not_nextval_walker. Wouldn't it have been better not to duplicate, and keep a flag about ignoring nextval in the context variable? While at it, couldn't we also fold contain_mutable_functions_walker

Re: [HACKERS] Allow COPY to use parameters

2016-05-27 Thread Merlin Moncure
On Fri, May 27, 2016 at 11:36 AM, Corey Huinker wrote: >> >> For the following pretend that "STRING" has the same behavior as the >> "format(...)" function. >> >> EXECUTE STRING('COPY %I TO %L', 'testtable', 'testfile.txt'); > > +1 -1 Why use syntax to do this? If we need a function with specia

Re: [HACKERS] Parallel pg_dump's error reporting doesn't work worth squat

2016-05-27 Thread Tom Lane
Kyotaro HORIGUCHI writes: > By the way, the reason of the "invalid snapshot identifier" is > that some worker threads try to use it after the connection on > the first worker closed. ... BTW, I don't quite see what the issue is there. The snapshot is exported from the master session, so errors i

Re: [HACKERS] 9.4 -> 9.5 regression with queries through pgbouncer on RHEL 6

2016-05-27 Thread Vladimir Borodin
-performance +hackers > 25 мая 2016 г., в 17:33, Vladimir Borodin написал(а): > > Hi all. > > We have found that queries through PgBouncer 1.7.2 (with transaction pooling) > to local PostgreSQL are almost two times slower in 9.5.3 than in 9.4.8 on > RHEL 6 hosts (all packages are updated to l

Re: [HACKERS] Allow COPY to use parameters

2016-05-27 Thread Corey Huinker
> > > For the following pretend that "STRING" has the same behavior as the > "format(...)" function. > > EXECUTE STRING('COPY %I TO %L', 'testtable', 'testfile.txt');​ > +1 We should make string sanitization easy so that people use it by default. In the mean time, if you're just using psql, the n

Re: [HACKERS] PATCH: pg_restore parallel-execution-deadlock issue

2016-05-27 Thread Tom Lane
Michael Paquier writes: > On Fri, May 27, 2016 at 4:07 PM, Amit Kapila wrote: >> On Fri, May 27, 2016 at 3:05 AM, Tom Lane wrote: >>> 2. Armin proposes that WaitForTerminatingWorkers needs to do CloseHandle() >>> on the various thread handles. That sounds plausible but I don't know >>> enough W

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-05-27 Thread Kevin Grittner
On Tue, May 24, 2016 at 4:10 PM, Robert Haas wrote: > On Tue, May 24, 2016 at 3:48 PM, Kevin Grittner wrote: >> On Tue, May 24, 2016 at 12:00 PM, Andres Freund wrote: >>> On 2016-05-24 11:24:44 -0500, Kevin Grittner wrote: On Fri, May 6, 2016 at 8:28 PM, Kevin Grittner wrote: > On Fri,

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-05-27 Thread Kevin Grittner
On Fri, May 27, 2016 at 9:58 AM, Kevin Grittner wrote: > On Tue, May 24, 2016 at 4:56 PM, Andres Freund wrote: >> If an old session with >= repeatable read accesses a clustered >> table (after the cluster committed), they'll now not see any >> errors, because all the LSNs look new. > > Again, it

Re: [HACKERS] Fix a typo in 9.6 release notes

2016-05-27 Thread Tom Lane
=?UTF-8?Q?L=c3=a9onard_Benedetti?= writes: > Patch to fix a typo in 9.6 release notes. It is especially ironic that > this is a mistake of diacritical on my name, on a patch improving the > support of diacritics^^. Pushed, thanks! regards, tom lane -- Sent via pgsql-ha

[HACKERS] Status of 64 bit atomics

2016-05-27 Thread John Gorman
Hi All Someone recently told me that the postgresql atomics library was incomplete for 64 bit operations such as pg_atomic_fetch_add_u64() and should not be used. Can someone definitively confirm whether it is okay to rely on the 64 bit atomics or whether it is better to protect 64 bit operations

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-05-27 Thread Kevin Grittner
On Tue, May 24, 2016 at 4:56 PM, Andres Freund wrote: > The LSNs of the created index pages will be new, and we'll thus > not necessarily error out when requried. It is *old* LSNs that are *safe* -- *new* LSNs are what trigger the "snapshot too old" error. So your observation may be a reason th

Re: [HACKERS] Allow COPY to use parameters

2016-05-27 Thread David G. Johnston
On Fri, May 27, 2016 at 6:02 AM, Craig Ringer wrote: > > COPY FROM $1 TO 'myfilename' > > ​Random thought - how about at least making the following work: For the following pretend that "STRING" has the same behavior as the "format(...)" function. EXECUTE STRING('COPY %I TO %L', 'testtable',

Re: [HACKERS] COMMENT ON, psql and access methods

2016-05-27 Thread Tom Lane
Michael Paquier writes: > As far as I can see, COMMENT ON has no support for access methods. > Wouldn't we want to add it as it is created by a command? On top of > that, perhaps we could have a backslash command in psql to list the > supported access methods, like \dam[S]? The system methods woul

Re: [HACKERS] [PROPOSAL] Move all am-related reloption code into src/backend/access/[am-name] and get rid of relopt_kind

2016-05-27 Thread Nikolay Shaplov
В письме от 24 мая 2016 17:12:16 пользователь Nikolay Shaplov написал: While working on this patch I met some difficulties that makes me to completely rewrite a code that is responsible for interacting reloptions.c with access methods. Story start from the point that I found out that a.m. can n

Re: [HACKERS] PATCH: pg_restore parallel-execution-deadlock issue

2016-05-27 Thread Michael Paquier
On Fri, May 27, 2016 at 4:07 PM, Amit Kapila wrote: > On Fri, May 27, 2016 at 3:05 AM, Tom Lane wrote: >> >> Michael Paquier writes: >> > ea274b2 has changed the way disconnection is done is is now closing >> > both the read and write pipes. So you may want to retry if things get >> > better wit

[HACKERS] Fix a typo in 9.6 release notes

2016-05-27 Thread Léonard Benedetti
Patch to fix a typo in 9.6 release notes. It is especially ironic that this is a mistake of diacritical on my name, on a patch improving the support of diacritics^^. Léonard Benedetti diff --git a/doc/src/sgml/release-9.6.sgml b/doc/src/sgml/release-9.6.sgml index fed1199..38e1967 100644 --- a/do

[HACKERS] COMMENT ON, psql and access methods

2016-05-27 Thread Michael Paquier
Hi all, As far as I can see, COMMENT ON has no support for access methods. Wouldn't we want to add it as it is created by a command? On top of that, perhaps we could have a backslash command in psql to list the supported access methods, like \dam[S]? The system methods would be in this case all th

[HACKERS] Floating point comparison inconsistencies of the geometric types

2016-05-27 Thread Emre Hasegeli
There are those macros defined for the built-in geometric types: > #define EPSILON 1.0E-06 > #define FPzero(A) (fabs(A) <= EPSILON) > #define FPeq(A,B) (fabs((A) - (B)) <= EPSILON) > #define FPne(A,B) (fabs((A) - (B)) > EPSILON) > #define

Re: [HACKERS] Allow COPY to use parameters

2016-05-27 Thread Craig Ringer
On 27 May 2016 at 15:17, Andrew Gierth wrote: > > "Merlin" == Merlin Moncure writes: > > Merlin> Note, the biggest pain point I have with COPY is not being able > Merlin> to parameterize the filename argument. > > Second proof of concept attached. This goes so far as to allow > statements

Re: [HACKERS] Fix a failure of pg_dump with !HAVE_LIBZ

2016-05-27 Thread Kyotaro HORIGUCHI
At Thu, 26 May 2016 11:54:35 -0400, Tom Lane wrote in <7601.1464278...@sss.pgh.pa.us> > Kyotaro HORIGUCHI writes: > > The warning says that it makes uncompressed archive but it really > > doesn't since workers die unexpectedly from the succeeding > > errors. This is because that compressLevel is

Re: [HACKERS] Parallel pg_dump's error reporting doesn't work worth squat

2016-05-27 Thread Kyotaro HORIGUCHI
At Thu, 26 May 2016 10:53:47 -0400, Tom Lane wrote in <5237.1464274...@sss.pgh.pa.us> > Kyotaro HORIGUCHI writes: > >> Next, I got the following behavior for the following command, > >> then freeze. Maybe stopping at the same point with the next > >> paragraph but I'm not sure. The same thing oc

Re: [HACKERS] Parallel pg_dump's error reporting doesn't work worth squat

2016-05-27 Thread Kyotaro HORIGUCHI
At Thu, 26 May 2016 12:15:29 -0400, Tom Lane wrote in <8273.1464279...@sss.pgh.pa.us> > Kyotaro HORIGUCHI writes: > > At Wed, 25 May 2016 10:11:28 -0400, Tom Lane wrote in > > <24577.1464185...@sss.pgh.pa.us> > >> The only case > >> that is certain to work is switches before non-switch argumen

Re: [HACKERS] Allow COPY to use parameters

2016-05-27 Thread Andrew Gierth
> "Merlin" == Merlin Moncure writes: Merlin> Note, the biggest pain point I have with COPY is not being able Merlin> to parameterize the filename argument. Second proof of concept attached. This goes so far as to allow statements like: do $$ declare t text := 'bar'; f text := '/tmp/copy

Re: [HACKERS] PATCH: pg_restore parallel-execution-deadlock issue

2016-05-27 Thread Amit Kapila
On Fri, May 27, 2016 at 3:05 AM, Tom Lane wrote: > > Michael Paquier writes: > > ea274b2 has changed the way disconnection is done is is now closing > > both the read and write pipes. So you may want to retry if things get > > better with the next round of minor releases. > > Hadn't paid attentio