Re: Query running for very long time (server hanged) with parallel append

On 7 February 2018 at 07:30, Kyotaro HORIGUCHI wrote: > At Tue, 6 Feb 2018 13:50:28 -0500, Robert Haas wrote > in >> On Tue, Feb 6, 2018 at 11:32 AM, Amit Khandekar

Re: Why does load_external_function() return PGFunction?

On 2018-02-06 15:43:29 -0500, Tom Lane wrote: > Andres Freund writes: > > We've several callers to load_external_function() that do not use the > > returned value as a PGFunction. I'd vote for changing the return type to > > void * and have fmgr.c cast it to PGFunction after

Re: Add RANGE with values and exclusions clauses to the Window Functions

I wrote: > [ 0001-window-frame-v14.patch ] Pushed after further hacking on the documentation and test cases. I went ahead with the "value" to "offset" terminology change, too. You mentioned upthread that you were interested in adding more in_range support functions. I think it'd be a great idea

Cached/global query plans, autopreparation

Hi all. Various versions of having PostgreSQL caching and/or autopreparing statement plans have been discussed ( https://www.postgresql.org/message-id/op.t9ggb3wacigqcu%40apollo13.peufeu.com , https://www.postgresql.org/message-id/8e76d8fc-8b8c-14bd-d4d1-e9cf193a74f5%40postgrespro.ru), without

Re: Add more information_schema columns

On Tue, Feb 06, 2018 at 10:45:52PM -0500, Peter Eisentraut wrote: > I think what I had meant to write was something like > > (t.tgtype & (1 | 66)) > > but maybe it's clearer to write it all out as you did. If you prefer that, that's fine for me as well. I tend to prefer the formulation

Re: [HACKERS] datetime.h defines like PM conflict with external libraries

Andres Freund wrote: > Medium term, it > seems to me, we should start actually move a lot of the adt code into a > library that can be included (or possibly just compiled?) both by > frontend and backend code. Which kinda seems to imply we'd need > compatible elog support for frontend code,

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

Hello, At Tue, 6 Feb 2018 10:41:22 -0300, Claudio Freire wrote in

Re: Add more information_schema columns

On 2/6/18 17:15, Michael Paquier wrote: > On Tue, Feb 06, 2018 at 03:16:59PM -0500, Robert Haas wrote: >> What possible point can there be to such an expression? It's always 0. >> >> rhaas=# select distinct tgtype::smallint & 1 & 66 from >> generate_series(-32768,32767) tgtype; >> ?column? >>

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

On 02/06/2018 11:15 AM, Mark Rofail wrote: A new patch including all the fixes is ready. Can you give the docs another look. I re-wrapped, re-indented  and changed all `Foreign Key Arrays` to `Array Element Foreign Keys` for consistency. Looks good to me so set it to ready for committer. I

Re: Failed to request an autovacuum work-item in silence

On Thu, Jan 25, 2018 at 10:41 AM, Masahiko Sawada wrote: > On Thu, Jan 25, 2018 at 12:14 AM, Fabrízio de Royes Mello > wrote: >> >> >> On Wed, Jan 24, 2018 at 12:31 PM, Fabrízio de Royes Mello >> wrote: >>> >>> >>> >>> On

Re: Query running for very long time (server hanged) with parallel append

At Tue, 6 Feb 2018 13:50:28 -0500, Robert Haas wrote in > On Tue, Feb 6, 2018 at 11:32 AM, Amit Khandekar > wrote: > > Yeah, I think it looks equally good that way, and like you

Re: PostgreSQL crashes with SIGSEGV

On Tue, Feb 6, 2018 at 5:54 PM, Craig Ringer wrote: >> In a quick look at the patches, WIP-kludge-fix.patch seems clearly >> unacceptable for back-patching because it changes the signature and >> behavior of ExecResetTupleTable, which external code might well be using. >

Re: PostgreSQL crashes with SIGSEGV

On 18 January 2018 at 03:23, Tom Lane wrote: > Aleksandr Parfenov writes: > > The new status of this patch is: Ready for Committer > > I don't feel particularly comfortable committing a patch that > was clearly labeled as a rushed draft by its

Re: [HACKERS] More stats about skipped vacuums

At Tue, 06 Feb 2018 19:24:37 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI wrote in <20180206.192437.229464841.horiguchi.kyot...@lab.ntt.co.jp> > At Tue, 6 Feb 2018 14:50:01 +0900, Masahiko Sawada > wrote in

RE: Temporary tables prevent autovacuum, leading to XID wraparound

From: Michael Paquier [mailto:michael.paqu...@gmail.com] > > postmaster deletes temporary relation files at startup by calling > > RemovePgTempFiles() regardless of whether it's in recovery. It > > doesn't call that function during auto restart after a crash when > > restart_after_crash is on. >

Re: Temporary tables prevent autovacuum, leading to XID wraparound

On Wed, Feb 07, 2018 at 12:37:50AM +, Tsunakawa, Takayuki wrote: > From: Michael Paquier [mailto:michael.paqu...@gmail.com] >> I am not sure that we would like to give up that easily the property that >> we have now to clean up past temporary files only at postmaster startup >> and only when

RE: Temporary tables prevent autovacuum, leading to XID wraparound

From: Michael Paquier [mailto:michael.paqu...@gmail.com] > I am not sure that we would like to give up that easily the property that > we have now to clean up past temporary files only at postmaster startup > and only when not in recovery. If you implement that, there is a risk that > the backend

Re: Temporary tables prevent autovacuum, leading to XID wraparound

On Tue, Feb 06, 2018 at 11:33:59AM -0500, Robert Haas wrote: > This is really two separate changes: > > 1. Teach backends to remove any leftover pg_temp_%d schema on > startup. I am not sure that we would like to give up that easily the property that we have now to clean up past temporary files

Re: PostgreSQL crashes with SIGSEGV

On Wed, Jan 17, 2018 at 2:23 PM, Peter Geoghegan wrote: > A complicating factor for this fix of mine is that mode_final() seems > to have its own ideas about tuple memory lifetime, over and above what > tuplesort_getdatum() explicitly promises, as can be seen here: > > /* > * Note:

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On 02/06/2018 10:39 PM, Peter Geoghegan wrote: > On Tue, Feb 6, 2018 at 1:30 PM, Tomas Vondra > wrote: >> I have little idea what -Og exactly means. It seems to be focused on >> debugging experience, and so still does some of the optimizations. > > As I understand

Re: Add more information_schema columns

On Tue, Feb 06, 2018 at 03:16:59PM -0500, Robert Haas wrote: > What possible point can there be to such an expression? It's always 0. > > rhaas=# select distinct tgtype::smallint & 1 & 66 from > generate_series(-32768,32767) tgtype; > ?column? > -- > 0 > (1 row) Of course you

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Tue, Feb 6, 2018 at 1:30 PM, Tomas Vondra wrote: > I have little idea what -Og exactly means. It seems to be focused on > debugging experience, and so still does some of the optimizations. As I understand it, -Og allows any optimization that does not hamper

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On 02/06/2018 10:14 PM, Peter Geoghegan wrote: > On Tue, Feb 6, 2018 at 1:04 PM, Tomas Vondra > wrote: >> Did you do a test with "-O0"? In my experience that makes valgrind tests >> much more reliable and repeatable. Some time ago we've seen cases that >> were

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Tue, Feb 6, 2018 at 1:04 PM, Tomas Vondra wrote: > Did you do a test with "-O0"? In my experience that makes valgrind tests > much more reliable and repeatable. Some time ago we've seen cases that > were failing for me but not for others, and I suspect it was due

Re: constraint exclusion and nulls in IN (..) clause

On Sun, Feb 4, 2018 at 11:20 PM, Ashutosh Bapat wrote: > On Thu, Feb 1, 2018 at 2:23 PM, Amit Langote > wrote: >> Yeah, the patch in its current form is wrong, because it will give wrong >> answers if the operator being used in a

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On 02/06/2018 09:56 PM, Peter Geoghegan wrote: > On Tue, Feb 6, 2018 at 12:53 PM, Robert Haas wrote: >>> Do you want somebody who does have a working valgrind installation >>> (ie me) to take responsibility for pushing this patch? >> >> I committed it before seeing this.

Re: git instructions

On Tue, Feb 6, 2018 at 1:46 PM, Stefan Kaltenbrunner < ste...@kaltenbrunner.cc> wrote: > > > > > Yes, this used to be the case, and is the reason behind the original > > recommendation. It's what they call the "dumb HTTP protocol" in the > > docs. This is not the case when you use

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Tue, Feb 6, 2018 at 12:53 PM, Robert Haas wrote: >> Do you want somebody who does have a working valgrind installation >> (ie me) to take responsibility for pushing this patch? > > I committed it before seeing this. It probably would've been better > if you had done it,

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Tue, Feb 6, 2018 at 2:11 PM, Tom Lane wrote: > Robert Haas writes: >> Unfortunately valgrind does not work at all on my laptop -- the server >> appears to start, but as soon as you try to connect, the whole thing >> dies with an error claiming that

Re: Incorrect grammar

On Tue, Feb 6, 2018 at 5:22 AM, Etsuro Fujita wrote: > While reviewing the > lazy-initialization-of-partition-info-for-tuple-routing patch, I ran > into a grammar mistake in a comment in ExecSetupChildParentMapForLeaf. > Attached is a patch for fixing that.

Re: git instructions

On 02/01/2018 05:35 PM, Magnus Hagander wrote: > > > On Thu, Feb 1, 2018 at 5:20 PM, Chapman Flack > wrote: > > On 02/01/2018 10:54 AM, Magnus Hagander wrote: > > > in saying that git:// is faster than https://. In fact, we have

Re: Why does load_external_function() return PGFunction?

Andres Freund writes: > We've several callers to load_external_function() that do not use the > returned value as a PGFunction. I'd vote for changing the return type to > void * and have fmgr.c cast it to PGFunction after verifying the > function's magic. void* isn't

Re: Add PGDLLIMPORT to enable_hashagg

On Tue, Feb 6, 2018 at 12:39 PM, Robert Haas wrote: > Yeah, let's get them all into one list and I'll commit the whole thing > together. +1 -- Peter Geoghegan

Re: Add PGDLLIMPORT to enable_hashagg

On Tue, Feb 6, 2018 at 3:34 PM, legrand legrand wrote: > Hello, > I know an other extension that would need the same for: > > src/include/optimizer/paths.h > geqo_threshold > enable_geqo > min_parallel_index_scan_size > min_parallel_table_scan_size > >

Re: Add PGDLLIMPORT to enable_hashagg

Hello, I know an other extension that would need the same for: src/include/optimizer/paths.h geqo_threshold enable_geqo min_parallel_index_scan_size min_parallel_table_scan_size src/include/optimizer/cost.h max_parallel_workers_per_gather enable_hashjoin enable_mergejoin enable_nestloop

Re: Add PGDLLIMPORT to enable_hashagg

On Mon, Feb 5, 2018 at 6:17 AM, Metin Doslu wrote: > There was already a discussion and commit for adding PGDLLIMPORT to some > variables which enables extensions to use them on Windows builds. For > reference, the previous thread:"Add PGDLLIMPORT lines to some variables". >

Re: Add more information_schema columns

On Tue, Feb 6, 2018 at 2:15 AM, Michael Paquier wrote: > Better to use parenthesis for (t.tgtype & 1 & 66) perhaps? You may want > to comment that this is to filter per row-statement first, and then with > after/before/instead of, which are what the 1 and the 66 are

Re: RelOptInfo -> Relation

On Tue, Feb 6, 2018 at 2:23 PM, Tom Lane wrote: >> If we're going to have to change this at some point (and I bet we >> are), I'd rather do it before people jam even more stuff into the >> current system rather than wait until it gets totally out of hand. > > While I'm

Why does load_external_function() return PGFunction?

Hi, We've several callers to load_external_function() that do not use the returned value as a PGFunction. I'd vote for changing the return type to void * and have fmgr.c cast it to PGFunction after verifying the function's magic. Greetings, Andres Freund

Re: RelOptInfo -> Relation

Robert Haas writes: > On Fri, Feb 2, 2018 at 7:06 PM, Tom Lane wrote: >> I'm disinclined to monkey with the way this works without someone >> presenting hard evidence that it creates enough of a performance problem >> to be worth spending a significant

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

Robert Haas writes: > Unfortunately valgrind does not work at all on my laptop -- the server > appears to start, but as soon as you try to connect, the whole thing > dies with an error claiming that the startup process has failed. So I > can't easily test this at the

Re: Query running for very long time (server hanged) with parallel append

On Tue, Feb 6, 2018 at 11:32 AM, Amit Khandekar wrote: > Yeah, I think it looks equally good that way, and like you said, the > current code does it that way. So in the attached patch, I have > swapped the two conditions. I prefer to avoid introducing 2 new variables and

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Mon, Feb 5, 2018 at 1:45 PM, Peter Geoghegan wrote: >> So, I guess another option might be to call VALGRIND_MAKE_MEM_DEFINED >> on the buffer. "We know what we're doing, trust us!" >> >> In some ways, that seems better than inserting a suppression, because >> it only affects the

Re: Cancelling parallel query leads to segfault

On 2018-02-06 12:01:08 -0500, Peter Eisentraut wrote: > On 2/1/18 20:35, Andres Freund wrote: > > On February 1, 2018 11:13:06 PM GMT+01:00, Peter Eisentraut > > wrote: > >>Here is a patch to implement that idea. Do you have a way to test it > >>repeatedly, or do

Re: Cancelling parallel query leads to segfault

On 2/1/18 20:35, Andres Freund wrote: > On February 1, 2018 11:13:06 PM GMT+01:00, Peter Eisentraut > wrote: >>Here is a patch to implement that idea. Do you have a way to test it >>repeatedly, or do you just randomly cancel queries? > > For me cancelling the

Re: [HACKERS] path toward faster partition pruning

On Tue, Feb 6, 2018 at 4:55 AM, Amit Langote wrote: >> I understand why COLLATION_MATCH think that a collation OID match is >> OK, but why is InvalidOid also OK? Can you add a comment? Maybe some >> test cases, too? > > partcollid == InvalidOid means the partition

Re: Temporary tables prevent autovacuum, leading to XID wraparound

On Sun, Feb 4, 2018 at 10:10 PM, Tsunakawa, Takayuki wrote: > The attached patch does the former. The small change in autovacuum.c is > mainly for autovac launcher and background workers which don't connect to a > database. I'll add this to the next CF. I'd

Re: Query running for very long time (server hanged) with parallel append

On 6 February 2018 at 16:11, Kyotaro HORIGUCHI wrote: > Thanks for the explanation. I'm fine with it. Well may I make > some comments on the patch? Sure, always welcome. > > - It seems to me that the if (!should_warp_around) block and else > block are better

Re: [HACKERS] [PATCH] Lockable views

On Tue, Feb 6, 2018 at 1:28 AM, Tatsuo Ishii wrote: >> But what does that have to do with locking? > > Well, if the view is not updatable, I think there will be less point > to allow to lock the base tables in the view because locking is > typically used in a case when updates

Re: RelOptInfo -> Relation

On Fri, Feb 2, 2018 at 7:06 PM, Tom Lane wrote: > We are not keeping a lock strong enough to prevent the relcache entry > from changing altogether, e.g. an ANALYZE could commit midway through > and change, say, the relpages/reltuples info. The consequences of > that for

Re: Crash in partition-wise join involving dummy partitioned relation

On Mon, Feb 5, 2018 at 11:02 PM, Ashutosh Bapat wrote: > The comment says why it checks both bounds and part_scheme, but it > doesn't explain why we check nparts, part_rels etc. My patch had that > explanation. Hmm, well, I couldn't understand it from your

Re: [HACKERS] MERGE SQL Statement for PG11

On Tue, Feb 6, 2018 at 9:40 AM, Tomas Vondra wrote: > I think we can do that for MERGE too, assuming we actually understand > > (1) why each of the pieces is missing > > (2) what would it take to make it work Right, I completely agree with that. For example, if we

Re: [HACKERS] MERGE SQL Statement for PG11

On 02/06/2018 05:20 AM, Peter Geoghegan wrote: > On Mon, Feb 5, 2018 at 7:56 PM, Robert Haas wrote: >> I don't think you get to make a unilateral decision to exclude >> features that work everywhere else from the scope of this patch. >> If there is agreement that those

Obsolete fmgr() declaration in fmgr.h

Hi Hackers, Commit 5ded4bd21403e143dd3eb66b92d52732fdac1945 removed support for version-0 function calling convention, and with it the fmgr() function. However, the declaration was left behind in fmgr.h. The attached patch finishes the cleanup. Regards, - ilmari -- "I use RMS as a guide in

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

On Tue, Feb 6, 2018 at 10:18 AM, Claudio Freire wrote: > On Tue, Feb 6, 2018 at 4:35 AM, Kyotaro HORIGUCHI > wrote: >>> It's starting to look like a timing effect indeed. >> >> It seems to be truncation skip, maybe caused by concurrent >>

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

On Sun, Feb 4, 2018 at 10:47 AM, Amit Kapila wrote: > On Fri, Feb 2, 2018 at 2:11 PM, amul sul wrote: >> On Fri, Jan 26, 2018 at 11:58 AM, Amit Kapila >> wrote: >> [] >>> I think you can manually (via debugger) hit this

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

On Tue, Feb 6, 2018 at 4:35 AM, Kyotaro HORIGUCHI wrote: >> It's starting to look like a timing effect indeed. > > It seems to be truncation skip, maybe caused by concurrent > autovacuum. Good point, I'll also disable autovacuum on vactst. > See

Re: [HACKERS] MERGE SQL Statement for PG11

On Tue, Feb 6, 2018 at 9:19 AM, Robert Haas wrote: > On Sun, Feb 4, 2018 at 3:41 AM, Simon Riggs wrote: >>> It is not clear to me what is exactly your concern if we try to follow >>> #2? To me, #2 seems like a natural choice. >> >> At first, but it

Re: [HACKERS] [PATCH] Vacuum: Update FSM more frequently

On Tue, Feb 6, 2018 at 4:56 AM, Masahiko Sawada wrote: > On Tue, Feb 6, 2018 at 2:55 AM, Claudio Freire wrote: >> On Mon, Feb 5, 2018 at 1:53 AM, Masahiko Sawada >> wrote: >>> On Fri, Feb 2, 2018 at 11:13 PM, Claudio Freire

Re: Query running for very long time (server hanged) with parallel append

At Tue, 6 Feb 2018 11:56:27 +0530, Amit Khandekar wrote in > On 6 February 2018 at 10:11, Kyotaro HORIGUCHI > wrote: > >> At Mon, 5 Feb 2018 15:29:27 +0530, Amit

Re: [HACKERS] More stats about skipped vacuums

At Tue, 6 Feb 2018 14:50:01 +0900, Masahiko Sawada wrote in > On Mon, Dec 11, 2017 at 8:15 PM, Kyotaro HORIGUCHI > wrote: > > I considered dshash for pgstat.c and the

Incorrect grammar

Hi, While reviewing the lazy-initialization-of-partition-info-for-tuple-routing patch, I ran into a grammar mistake in a comment in ExecSetupChildParentMapForLeaf. Attached is a patch for fixing that. Best regards, Etsuro Fujita diff --git a/src/backend/executor/execPartition.c

Re: [HACKERS] MERGE SQL Statement for PG11

Greetings, * Peter Geoghegan (p...@bowt.ie) wrote: > On Mon, Feb 5, 2018 at 7:56 PM, Robert Haas wrote: > > I don't think you get to make a unilateral decision to exclude > > features that work everywhere else from the scope of this patch. If > > there is agreement that

Re: [HACKERS] path toward faster partition pruning

On 2018/02/03 6:05, Robert Haas wrote: > On Fri, Feb 2, 2018 at 9:33 AM, Robert Haas wrote: >>> Updated set of patches attached (patches 0002 onward mostly unchanged, >>> except I incorporated the delta patch posted by David upthread). >> >> Committed 0001. Thanks. > >

Sv: Better Upgrades

På tirsdag 06. februar 2018 kl. 01:09:18, skrev David Fetter >: Folks, While chatting with Bruce about how to make something better than pg_upgrade, we (and by "we," I mean mostly Bruce) came up with the following. What needs improvement: -

Re: [HACKERS] Bug in to_timestamp().

On Fri, Feb 02, 2018 at 09:54:45PM +0100, Dmitry Dolgov wrote: > For some reason I can't apply it clean to the latest master: > > (Stripping trailing CRs from patch; use --binary to disable.) > patching file doc/src/sgml/func.sgml > (Stripping trailing CRs from patch; use --binary to