Re: Loaded footgun open_datasync on Windows

2018-09-13 Thread Laurenz Albe
Michael Paquier wrote: > On Thu, Sep 13, 2018 at 03:55:20PM +0200, Laurenz Albe wrote: > > Attached is the new, improved version of the patch. > > Thanks, finally pushed. I have made sure that recovery TAP tests, > upgrade tests and bin/ tests are working properly. Thanks for being interested an

Re: hot_standby_feedback vs excludeVacuum and snapshots

2018-09-13 Thread Kyotaro HORIGUCHI
At Sat, 4 Aug 2018 14:09:18 +1200, Thomas Munro wrote in > On Mon, Jul 9, 2018 at 6:51 AM, Noah Misch wrote: > > On Fri, Jul 06, 2018 at 04:32:56PM +0100, Simon Riggs wrote: > >> On 6 July 2018 at 03:30, Thomas Munro > >> wrote: > >> > On Thu, Jul 5, 2018 at 8:27 AM, Noah Misch wrote: > >> >

Re: Cache lookup errors with functions manipulation object addresses

2018-09-13 Thread Michael Paquier
On Fri, Sep 14, 2018 at 11:22:12AM +0900, Michael Paquier wrote: > Attached are rebased versions. This has been around for some time, so I > am planning to move on with this patch set pretty soon as that's mainly > cleanup for getObjectIdentity as it triggers elog("cache lookup") or > such for und

Re: Consistent segfault in complex query

2018-09-13 Thread Andrew Gierth
> "Tom" == Tom Lane writes: >> Your other idea of forcing initPlan parameters to be evaluated >> before we enter the EPQ execution environment is probably more >> workable. Tom> Concretely, the attached seems to be enough to fix it (though I Tom> only tried the simplest case you posted)

Re: Cache lookup errors with functions manipulation object addresses

2018-09-13 Thread Michael Paquier
On Mon, Jul 02, 2018 at 08:54:25PM +0900, Michael Paquier wrote: > I am fine with any conclusion. As the patch has rotten a bit, I > switched it from "Ready for committer" to "Needs Review" by the way. > That seems more appropriate as the thing has rotten a bit. Attached are rebased versions. Th

logical decoding bug when mapped relation with toast contents is rewritten repeatedly

2018-09-13 Thread Andres Freund
Hi, (Tomas, CCing you because you IIRC mentioned encountered an issue like this) I just spent quite a while debugging an issue where running logical decoding yielded a: ERROR: could not map filenode "base/$X/$Y" to relation OID error. After discarding like 30 different theories, I have found th

Re: stat() on Windows might cause error if target file is larger than 4GB

2018-09-13 Thread Michael Paquier
On Thu, Sep 13, 2018 at 02:23:47PM -0400, Robert Haas wrote: > This, to me, seems way too clever. Replacing 'struct stat' with > something else everywhere in the code is more churn, but far less > likely to have surprising consequences down the road. Or so I would > think, anyway. I don't have t

Re: Allowing printf("%m") only where it actually works

2018-09-13 Thread Michael Paquier
On Wed, Sep 12, 2018 at 01:40:01PM -0400, Tom Lane wrote: > Michael Paquier writes: > > I would have liked to look at this patch in details, but it failed to > > apply. Could you rebase? > > Ah, yeah, the dlopen patch touched a couple of the same places. > Rebase attached --- no substantive chan

Re: Getting ERROR: could not open file "base/13164/t3_16388" with partition table with ON COMMIT

2018-09-13 Thread Amit Langote
On 2018/09/13 23:13, Tom Lane wrote: > Amit Langote writes: >> On 2018/09/13 1:14, Tom Lane wrote: >>> That seems excessively restrictive. Anything that has storage (e.g. >>> matviews) ought to be truncatable, no? > >> Not by heap_truncate it seems. The header comment of heap_truncate says >> t

Re: Loaded footgun open_datasync on Windows

2018-09-13 Thread Michael Paquier
On Thu, Sep 13, 2018 at 03:55:20PM +0200, Laurenz Albe wrote: > Attached is the new, improved version of the patch. Thanks, finally pushed. I have made sure that recovery TAP tests, upgrade tests and bin/ tests are working properly. -- Michael signature.asc Description: PGP signature

Re: Indicate anti-wraparound autovacuum in log_autovacuum_min_duration

2018-09-13 Thread Michael Paquier
On Thu, Sep 13, 2018 at 12:00:49PM +0300, Sergei Kornilov wrote: > Looks better for me. Updated patch attached. Thanks Sergei for the new version, pushed. -- Michael signature.asc Description: PGP signature

Re: Consistent segfault in complex query

2018-09-13 Thread Tom Lane
Andrew Gierth writes: > "Tom" == Tom Lane writes: > Tom> Don't think that's going to work; the EPQ environment doesn't have > Tom> any way to know that an execPlan link is pointing to something in > Tom> a different estate. > But can't such a way be created? e.g. by pointing execPlan to a spe

Re: Consistent segfault in complex query

2018-09-13 Thread Tom Lane
I wrote: > Your other idea of forcing initPlan parameters to be evaluated before we > enter the EPQ execution environment is probably more workable. Concretely, the attached seems to be enough to fix it (though I only tried the simplest case you posted). I don't find anything to love about ExecEv

Re: Consistent segfault in complex query

2018-09-13 Thread Andrew Gierth
> "Tom" == Tom Lane writes: >> What I'm wondering is whether the param in the copied estate >> shouldn't rather be just a proxy for the one in the original estate >> - if we need to evaluate it, then do so in the original estate, >> store the value there, and copy the value back into the

Re: Consistent segfault in complex query

2018-09-13 Thread Tom Lane
Andrew Gierth writes: > "Tom" == Tom Lane writes: > Tom> The second of those; what we need is for any referenced InitPlans > Tom> to be executed afresh under EPQ rules. (I'm not entirely sure that > Tom> an InitPlan could need to see different input tuples under EPQ > Tom> than it'd see other

Re: Bug fix for glibc broke freebsd build in REL_11_STABLE

2018-09-13 Thread Andres Freund
On 2018-09-04 17:51:30 -0700, Andres Freund wrote: > My current proposal is thus to do add a check that does > #if defined(__clang__) && defined(__i386__) && !defined(__SSE2_MATH__) > something-that-fails > #endif > in an autoconf test, and have configure complain if that > fails. Something roughly

Re: Consistent segfault in complex query

2018-09-13 Thread Andrew Gierth
> "Tom" == Tom Lane writes: >> So I can see exactly where the problem is, but I'm not sure what the >> solution should be. >> EvalPlanQualStart copies the param_exec value list explicitly _not_ >> including the execPlan link, which obviously isn't going to work if >> the value has not b

Re: pg_dump test instability

2018-09-13 Thread Tom Lane
I wrote: > Peter Eisentraut writes: >> I see. Why not shift all items up to the i'th up by one place, instead >> of moving only the first one? That way the sortedness would be >> preserved. Otherwise we'd move the first one into the middle, then >> sorting would move it to the front again, etc.

Re: patch to allow disable of WAL recycling

2018-09-13 Thread Jerry Jelinek
Hi Peter, I'll take a look at that. I had been trying to keep the patch as minimal as possible, but I'm happy to work through this. Thanks, Jerry On Tue, Sep 11, 2018 at 9:39 AM, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 10/09/2018 16:10, Jerry Jelinek wrote: > > Thank y

Re: Index Skip Scan

2018-09-13 Thread Alexander Kuzmenkov
El 13/09/18 a las 18:39, Jesper Pedersen escribió: Yes, this doesn't look good. Using your test case I'm seeing that unique is being chosen when the group size is below 34, and skip above. This is with the standard initdb configuration; did you change something else ? Or did you force the def

Re: [Patch] Create a new session in postmaster by calling setsid()

2018-09-13 Thread Andres Freund
Hi, On 2018-09-13 15:27:58 +0800, Paul Guo wrote: > From the respective of users instead of developers, I think for such > important > service, tty should be dissociated, i.e. postmaster should be daemonized by > default (and even should have default log filename?) or be setsid()-ed at > least. I

Re: Consistent segfault in complex query

2018-09-13 Thread Tom Lane
Andrew Gierth writes: > So I can see exactly where the problem is, but I'm not sure what the > solution should be. > EvalPlanQualStart copies the param_exec value list explicitly _not_ > including the execPlan link, which obviously isn't going to work if the > value has not been computed yet. Sho

Re: [HACKERS] Bug in to_timestamp().

2018-09-13 Thread Tom Lane
Alexander Korotkov writes: > I've closed commitfest entry. I think we can add new commitfest entry if > required. Regarding FX part, it easy to extract it as separate patch, but > it's hard to find consensus. I think there are at least three possible > decisions. > 1) Change FX mode to require

Re: stat() on Windows might cause error if target file is larger than 4GB

2018-09-13 Thread Robert Haas
On Thu, Sep 13, 2018 at 10:29 AM, Tom Lane wrote: > What I was vaguely imagining is that win32_port.h could #include > whichever Windows header defines these functions and structs, and > then do > > #define stat __stat64 > > static inline ... __stat64(...) { return _stat64(...); } > > What would n

Re: [HACKERS] Bug in to_timestamp().

2018-09-13 Thread Alexander Korotkov
On Tue, Sep 11, 2018 at 6:18 PM Arthur Zakirov wrote: > On Sun, Sep 09, 2018 at 09:52:46PM +0300, Alexander Korotkov wrote: > > So, pushed! Thanks to every thread participant for review and feedback. > > Great! Should we close the commitfest entry? There is FX part of the > patch though. But it

Re: Avoid extra Sort nodes between WindowAggs when sorting can be reused

2018-09-13 Thread Andrew Gierth
Here's what I have queued up to push. My changes are: - added to the header comment of list_concat_unique that callers have ordering expectations. Didn't touch list_union, since I ended up sticking with list_concat_unique for this patch. - WindowClauseSortNode renamed WindowClauseSortDat

Re: [patch] Support LLVM 7

2018-09-13 Thread Andres Freund
On 2018-09-12 23:07:34 +0200, Christoph Berg wrote: > Re: Andres Freund 2018-09-12 > <20180912210338.h3vsss5lkuu26...@alap3.anarazel.de> > > Hi, > > > > On 2018-09-12 14:45:17 +0200, Christoph Berg wrote: > > > LLVM 7 landed in Debian unstable, this patch teaches ./configure to use > > > it. (Gen

Re: cache lookup failed for constraint when alter table referred by partition table

2018-09-13 Thread Alvaro Herrera
On 2018-Sep-13, Tom Lane wrote: > Alvaro Herrera writes: > > That's the problem all right. The solution is to drop all > > index/constraint objects together in one performMultipleDeletions() > > instead of performDeletion() one by one, as in the attached patch. > > Looks reasonable as far as it

PostgreSQL 11 {Beta 4, RC1} Release: 2018-09-20

2018-09-13 Thread Jonathan S. Katz
Hi, We are planning to have another release of PostgreSQL 11, either Beta 4 or RC1, next week on Thursday, 2018-09-20. The version will be determined based on the state of the open items list[1] around the time of stamping. Thanks, Jonathan [1] https://wiki.postgresql.org/wiki/PostgreSQL_11_Ope

Re: cache lookup failed for constraint when alter table referred by partition table

2018-09-13 Thread Tom Lane
Alvaro Herrera writes: > That's the problem all right. The solution is to drop all > index/constraint objects together in one performMultipleDeletions() > instead of performDeletion() one by one, as in the attached patch. Looks reasonable as far as it goes. Given that we no longer require any o

Re: cache lookup failed for constraint when alter table referred by partition table

2018-09-13 Thread Alvaro Herrera
On 2018-Sep-10, Alvaro Herrera wrote: > ATPostAlterTypeCleanup is trying to search the original constraint by > OID in order to drop it, but it's not there -- I suppose it has already > been dropped by recursion in a previous step. That's the problem all right. The solution is to drop all index/

Re: Index Skip Scan

2018-09-13 Thread Jesper Pedersen
Hi Alexander. On 9/13/18 9:01 AM, Alexander Kuzmenkov wrote: While testing this patch Thanks for the review ! I noticed that current implementation doesn't perform well when we have lots of small groups of equal values. Here is the execution time of index skip scan vs unique over index scan

Re: speeding up planning with partitions

2018-09-13 Thread Dilip Kumar
On Tue, Sep 4, 2018 at 12:44 PM, Amit Langote wrote: > Hi Dilip, > > Thanks for taking a look. > > On 2018/09/03 20:57, Dilip Kumar wrote: >> The idea looks interesting while going through the patch I observed >> this comment. >> >> /* >> * inheritance_planner >> * Generate Paths in the case w

Re: stat() on Windows might cause error if target file is larger than 4GB

2018-09-13 Thread Tom Lane
Michael Paquier writes: > On Wed, Sep 12, 2018 at 08:17:05PM -0400, Tom Lane wrote: >> And we can't just "#define stat __stat64" because >> that would break the calls to stat(). Or wait, maybe we could do that >> and also have a one-liner function named __stat64 that would catch the >> calls and

Re: Avoid extra Sort nodes between WindowAggs when sorting can be reused

2018-09-13 Thread Tom Lane
Andrew Gierth writes: > (aside: I itch to rewrite the comment that says "the spec requires that > there be only one sort" - number of sorts is an implementation detail > about which the spec is silent, what it _actually_ requires is that peer > rows must be presented in the same order in all order

Re: [HACKERS] Cutting initdb's runtime (Perl question embedded)

2018-09-13 Thread Tom Lane
Thomas Munro writes: > I noticed that the patch does a bunch of s/Olson/IANA/. That leaves > only one place in the tree that still refers to the "Olson" database: > dt_common.c. Might want to change that too? Ah, I didn't realize we were that close to wiping out the old terminology. Yeah, I'll

Re: Getting ERROR: could not open file "base/13164/t3_16388" with partition table with ON COMMIT

2018-09-13 Thread Tom Lane
Amit Langote writes: > On 2018/09/13 1:14, Tom Lane wrote: >> That seems excessively restrictive. Anything that has storage (e.g. >> matviews) ought to be truncatable, no? > Not by heap_truncate it seems. The header comment of heap_truncate says > that it concerns itself only with ON COMMIT tru

Re: Avoid extra Sort nodes between WindowAggs when sorting can be reused

2018-09-13 Thread Andrew Gierth
> "Tom" == Tom Lane writes: Tom> I'm also a bit suspicious as to whether the code is even correct; Tom> does it *really* match what will happen later when we create sort Tom> plan nodes? (Maybe it's fine; I haven't looked.) As things stand before the patch, the code that actually generate

Re: pg_dump test instability

2018-09-13 Thread Tom Lane
Peter Eisentraut writes: > On 12/09/2018 18:06, Tom Lane wrote: >> No. In both code paths, the array slot at index first_te is being >> physically dropped from the set of valid entries (by incrementing >> first_te). In the first path, that slot holds the item we want to >> remove logically from

Re: Loaded footgun open_datasync on Windows

2018-09-13 Thread Laurenz Albe
Michael Paquier wrote: > On Mon, Sep 10, 2018 at 04:46:40PM +0200, Laurenz Albe wrote: > > I didn't get pg_upgrade to work without the log file hacks; I suspect > > that there is more than just log file locking going on, but my Windows > > skills are limited. > > > > How shall I proceed? > > I do

Re: Index Skip Scan

2018-09-13 Thread Alexander Kuzmenkov
Hi Jesper, While testing this patch I noticed that current implementation doesn't perform well when we have lots of small groups of equal values. Here is the execution time of index skip scan vs unique over index scan, in ms, depending on the size of group. The benchmark script is attached.

Re: Protect syscache from bloating with negative cache entries

2018-09-13 Thread Kyotaro HORIGUCHI
Hello. Thank you for looking this. At Wed, 12 Sep 2018 05:16:52 +, "Ideriha, Takeshi" wrote in <4E72940DA2BF16479384A86D54D0988A6F197012@G01JPEXMBKW04> > Hi, > > >Subject: Re: Protect syscache from bloating with negative cache entries > > > >Hello. The previous v4 patchset was just broken

Re: Collation versioning

2018-09-13 Thread Stephen Frost
Greetings, * Christoph Berg (m...@debian.org) wrote: > Re: Peter Eisentraut 2018-09-13 > <4f60612c-a7b5-092d-1532-21ff7a106...@2ndquadrant.com> > > Moreover, the fix for a collation version mismatch is, in the simplest > > case, to go around and REINDEX everything. Making the collation or > > co

Re: executor relation handling

2018-09-13 Thread Jesper Pedersen
Hi Amit, On 9/13/18 12:58 AM, Amit Langote wrote: Attached updated patches. Beside the issue that caused eval-plan-qual isolation test to crash, I also spotted and fixed an oversight in the 0002 patch which would lead to EState.es_output_cid being set to wrong value and causing unexpected error

Re: Problem while setting the fpw with SIGHUP

2018-09-13 Thread Amit Kapila
On Mon, Sep 10, 2018 at 11:54 AM Amit Kapila wrote: > > Thanks, but what I wanted you to verify is the commit that broke it in > 9.5. On again looking at it, I think it is below code in commit > 2076db2aea that caused this problem. If possible, can you once test > it before and at this commit or

Re: [HACKERS] Restricting maximum keep segments by repslots

2018-09-13 Thread Kyotaro HORIGUCHI
Hello. Thank you for the comments, Sawada-san, Peter. At Mon, 10 Sep 2018 19:52:24 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI wrote in <20180910.195224.22629595.horiguchi.kyot...@lab.ntt.co.jp> > At Thu, 6 Sep 2018 22:32:21 +0200, Peter Eisentraut > wrote in > <29bbd79d-696b-509e-578a-0

Re: Indicate anti-wraparound autovacuum in log_autovacuum_min_duration

2018-09-13 Thread Sergei Kornilov
Hello > The brackets look rather useless to me, wouldn't it be better to remove > them? By doing so the longest message becomes: > "automatic aggressive vacuum to prevent wraparound of table blah.blah" Hmm, > 2018-09-13 11:48:09.303 MSK 6994 @ from [vxid:6/713 txid:0] [] LOG: > automatic aggre

Re: Collation versioning

2018-09-13 Thread Christoph Berg
Re: Peter Eisentraut 2018-09-13 <4f60612c-a7b5-092d-1532-21ff7a106...@2ndquadrant.com> > Moreover, the fix for a collation version mismatch is, in the simplest > case, to go around and REINDEX everything. Making the collation or > collation version global doesn't fix that. It would actually make

Re: Unused argument from execute_sql_string()

2018-09-13 Thread Michael Paquier
On Thu, Sep 13, 2018 at 03:47:26PM +0900, Tatsuo Ishii wrote: > Anyway, considering it's a static function, chance of breaking > backward compatibility is minimum, I think. > > So +1 to remove the unused argument. Same opinion and arguments here, so I have committed the patch. -- Michael signat

Re: pg_dump test instability

2018-09-13 Thread Peter Eisentraut
On 12/09/2018 18:06, Tom Lane wrote: > No. In both code paths, the array slot at index first_te is being > physically dropped from the set of valid entries (by incrementing > first_te). In the first path, that slot holds the item we want to > remove logically from the set, so that incrementing fi

Re: [Patch] Create a new session in postmaster by calling setsid()

2018-09-13 Thread Paul Guo
On Thu, Sep 13, 2018 at 8:20 AM, Michael Paquier wrote: > On Wed, Sep 12, 2018 at 03:55:00PM -0400, Tom Lane wrote: > > Although pg_ctl could sneak it in between forking and execing, it seems > > like it'd be cleaner to have the postmaster proper do it in response to > > a switch that pg_ctl pass

Re: Collation versioning

2018-09-13 Thread Peter Eisentraut
On 12/09/2018 13:25, Christoph Berg wrote: > Re: Peter Eisentraut 2018-09-12 > <0447ec7b-cdb6-7252-7943-88a4664e7...@2ndquadrant.com> >>> Naive idea: make that catalog shared? Collations are system-wide after >>> all. >> >> By the same argument, extensions should be shared, but they are not. > >