[HACKERS] Re: [COMMITTERS] pgsql: Perform only one ReadControlFile() during startup.

2017-09-16 Thread Andres Freund
s like someone didn't think hard enough about when to reset > ControlFile to null. Yep. Just adding a LocalProcessControlFile() reset_shared() works after removing the assert from LocalProcessControlFile(). That's not what I'm proposing, just confirming that that's the issue.

Re: [HACKERS] SendRowDescriptionMessage() is slow for queries with a lot of columns

2017-09-15 Thread Andres Freund
just this patch reverted: tps = 17342.006825 (excluding connections establishing) Regards, Andres >From a2325a2649da403dc562b8e93df972839823d924 Mon Sep 17 00:00:00 2001 From: Andres Freund Date: Wed, 13 Sep 2017 19:25:34 -0700 Subject: [PATCH 1/8] Speedup pgstat_report_activity

Re: [HACKERS] More efficient truncation of pg_stat_activity query strings

2017-09-15 Thread Andres Freund
On 2017-09-15 16:45:47 -0400, Tom Lane wrote: > Andres Freund writes: > > Version correcting these is attached. Thanks! > > I'd vote for undoing the s/st_activity/st_activity_raw/g business. > That may have been useful while writing the patch, to ensure you > found al

Re: [HACKERS] More efficient truncation of pg_stat_activity query strings

2017-09-15 Thread Andres Freund
On 2017-09-15 08:25:09 -0400, Robert Haas wrote: > On Thu, Sep 14, 2017 at 2:03 AM, Andres Freund wrote: > > Here's a patch that implements that idea. > > From the department of cosmetic nitpicking: > > + * All supported server-encodings allow to determine th

Re: [HACKERS] pgjdbc logical replication client throwing exception

2017-09-15 Thread Andres Freund
On 2017-09-15 20:00:34 +, Vladimir Sitnikov wrote: > ++pgjdbc dev list. > > >I am facing unusual connection breakdown problem. Here is the simple code > that I am using to read WAL file: > > Does it always fails? > Can you create a test case? For instance, if you file a pull request with > th

Re: [HACKERS] POC: Sharing record typmods between backends

2017-09-15 Thread Andres Freund
On 2017-09-15 15:39:49 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2017-09-14 23:29:05 -0400, Tom Lane wrote: > >> FWIW, I'm not on board with that. I think the version of typedefs.list > >> in the tree should reflect the last official pgindent run.

Re: [HACKERS] COMMIT TRIGGERs, take n, implemented with CONSTRAINT TRIGGERS

2017-09-15 Thread Andres Freund
re we're fairly sure), aren't liked much. If you want to argue for a change, it should happen on-list. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] COMMIT TRIGGERs, take n, implemented with CONSTRAINT TRIGGERS

2017-09-15 Thread Andres Freund
commit triggers in the past. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] SendRowDescriptionMessage() is slow for queries with a lot of columns

2017-09-15 Thread Andres Freund
lable_columns * #selected_columns) in colNameToVar()). Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] POC: Sharing record typmods between backends

2017-09-15 Thread Andres Freund
On 2017-09-14 23:29:05 -0400, Tom Lane wrote: > Thomas Munro writes: > > On Fri, Sep 15, 2017 at 3:03 PM, Andres Freund wrote: > >> - added typedefs to typedefs.list > > > Should I do this manually with future patches? I think there's sort of a circuit sp

Re: [HACKERS] POC: Sharing record typmods between backends

2017-09-14 Thread Andres Freund
Hi, On 2017-09-04 18:14:39 +1200, Thomas Munro wrote: > Thanks for the review and commits so far. Here's a rebased, debugged > and pgindented version of the remaining patches. I've pushed this with minor modifications: - added typedefs to typedefs.list - re-pgindented, there were some missing re

Re: [HACKERS] Trouble with amcheck

2017-09-14 Thread Andres Freund
On 2017-09-14 22:36:38 -0400, Stephen Frost wrote: > Andres, > > * Andres Freund (and...@anarazel.de) wrote: > > On 2017-09-15 02:22:49 +, Douglas Doole wrote: > > > Thanks all. Making and installing the contribs got me rolling again. (I > > > tried "mak

Re: [HACKERS] Trouble with amcheck

2017-09-14 Thread Andres Freund
ot;make installcheck-world" should imply that all prereqs are > met - that's certainsly the normal behaviour for make. I'm very unconvinced by this, given that one use of installcheck is to run against an existing server. For which one might not even have access to the relevant direct

Re: [HACKERS] Process startup infrastructure is a mess

2017-09-14 Thread Andres Freund
On 2017-09-15 01:06:54 +0100, Simon Riggs wrote: > On 14 September 2017 at 22:44, Andres Freund wrote: > > > The way we currently start and initialize individual postgres (sub-) > > processes is pretty complicated and duplicative. I've a couple > > complaints

[HACKERS] Process startup infrastructure is a mess

2017-09-14 Thread Andres Freund
come some thoughts on whether others consider this a serious problem or not, and what they think we should do about this, first. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [PATCH] Call RelationDropStorage() for broader range of object drops.

2017-09-14 Thread Andres Freund
On 2017-09-14 12:05:37 +0200, Alvaro Herrera wrote: > This sounds terrible. Welcome to the life of writing extensions. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] uninterruptible state in 10beta4

2017-09-14 Thread Andres Freund
On 2017-09-13 17:53:44 -0400, Tom Lane wrote: > Andres Freund writes: > > Indeed that seems plausible. I guess something like the attached should > > fix the issue? > > Ah, I see you came to the same conclusion I did. But see comment > about adding a comment. I'v

Re: [HACKERS] uninterruptible state in 10beta4

2017-09-14 Thread Andres Freund
On 2017-09-13 14:46:08 -0700, Jeff Janes wrote: > On Wed, Sep 13, 2017 at 2:41 PM, Andres Freund wrote: > > Indeed that seems plausible. I guess something like the attached should > > fix the issue? > > > > Yep, that fixes it. Cool. Pushed. Thanks for the report! Tha

[HACKERS] Binary search in fmgr_isbuiltin() is a bottleneck.

2017-09-13 Thread Andres Freund
ther hashtable... I was wondering about also replacing the C function hash with a simplehash, but given that I've not seen this in profiles, I've not bothered so far. Greetings, Andres Freund >From 2b3e06380d5a339efc94e748aa57985d3bb80223 Mon Sep 17 00:00:00 2001 From: Andres Freu

[HACKERS] SendRowDescriptionMessage() is slow for queries with a lot of columns

2017-09-13 Thread Andres Freund
e loops into separate functions. Imo also a good chunk more readable. Comments? Greetings, Andres Freund [1] http://archives.postgresql.org/message-id/ca+tgmobj72e_tg6w98h0oubccumoc4urmjocypbnwc2rxya...@mail.gmail.com [2] http://archives.postgresql.org/message-id/20

[HACKERS] Improve catcache/syscache performance.

2017-09-13 Thread Andres Freund
ughly 8% speedup in a workload that consists out of a fast query that returns a lot of columns. If I apply a few other performance patches, this patch itself starts to make a bigger difference, of around 11%. Greetings, Andres Freund [1] https://www.postgresql.org/message-id/20130905191323.gc

Re: [HACKERS] More efficient truncation of pg_stat_activity query strings

2017-09-13 Thread Andres Freund
On 2017-09-12 00:19:48 -0700, Andres Freund wrote: > Hi, > > I've recently seen a benchmark in which pg_mbcliplen() showed up > prominently. Which it will basically in any benchmark with longer query > strings, but fast queries. That's not that uncommon. > > I wo

Re: [HACKERS] Is it time to kill support for very old servers?

2017-09-13 Thread Andres Freund
Hi, On 2017-09-13 23:39:21 -0400, Tom Lane wrote: > Andres Freund writes: > > Re-upping this topic. > > > On 2016-10-07 10:06:07 -0400, Tom Lane wrote: > >> In the same line, maybe we should kill libpq's support for V2 protocol > >> (which would make

Re: [HACKERS] domain type smashing is expensive

2017-09-13 Thread Andres Freund
tements, but not for simple query execution. > Sure, but that's kinda my point. We've got to send a RowDescription > message for every query, and if that requires smashing domain types to > base types, we have to do it. What we don't have to do is repeat that > work for every execution of a prepared query. We also have done a bunch of those lookups in the planner already, so if we'd move it there it might still be be an advantage performancewise even for the single execution case. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Is it time to kill support for very old servers?

2017-09-13 Thread Andres Freund
q unable to parse such a message from an older server. Possibly we > could handle that specific case with a little special-purpose code and > still be able to take out most of fe-protocol2.c. We should really fix that so it reports the error as a v3 message, independent of ripping out libpq

Re: [HACKERS] uninterruptible state in 10beta4

2017-09-13 Thread Andres Freund
ate trigger foobar after insert on foo for each row execute procedure > notice(); > insert into foo select * from generate_series(1,1); > > Git bisect lays the blame here which certainly seems plausible: > > commit d47cfef7116fb36349949f5c757aa2112c249804 > Autho

Re: [HACKERS] increasing the default WAL segment size

2017-09-13 Thread Andres Freund
ickly agree on that, I think we need to refactor this a bit, I've done so in the attached, but it's untested. Could you please verify it works and if not fix it up? What do you think? Regards, Andres [1] http://archives.postgresql.org/message-id/20170913064824.rqflkadxwpbo

[HACKERS] Removing pg_standby #17.

2017-09-12 Thread Andres Freund
ther do with my time than understand its code and manually test it (there's no tests). Greetings, Andres Freund [1] https://www.postgresql.org/message-id/545946E9.8060504%40gmx.net [2] https://www.postgresql.org/message-id/54866BFC.4090804%40gmx.net [3] https://www.postgresql.org/CAOG9A

Re: [HACKERS] Automatic testing of patches in commit fest

2017-09-12 Thread Andres Freund
ms worthy of a good bit of attention. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] More efficient truncation of pg_stat_activity query strings

2017-09-12 Thread Andres Freund
lse len = 1; return len; } As you can see, only the first character (*s) is accessed to determine the length/width of the multibyte-character. That's afaict the case for all server-side encodings. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (p

[HACKERS] More efficient truncation of pg_stat_activity query strings

2017-09-12 Thread Andres Freund
fairly harmless. Faults in my thinking? Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] mysql_fdw + PG10: unrecognized node type: 217

2017-09-11 Thread Andres Freund
TICE: Found number Four > NOTICE: Found number Five > ! ERROR: unrecognized node type: 217 > ! CONTEXT: SQL statement "select bfrom numbers where a=x" > ! PL/pgSQL function test_param_where() line 6 at SQL statement > ! /* Could you pprint() the expression that's being initialized? Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Statement timeout behavior in extended queries

2017-09-10 Thread Andres Freund
Hi, On 2017-09-11 09:10:49 +0900, Tatsuo Ishii wrote: > If you don't mind, can you please commit/push the patch? Ok, will do so. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgr

[HACKERS] Constifying numeric.c's local vars

2017-09-10 Thread Andres Freund
27;t quite see a better solution? Leaving JIT aside, I think stuff like this is worthwhile on its own... Greetings, Andres Freund >From 2b114f2c91c7bbf80e48ef5c9653748c957bf15f Mon Sep 17 00:00:00 2001 From: Andres Freund Date: Sun, 10 Sep 2017 16:20:41 -0700 Subject: [PATCH] Const

Re: [HACKERS] Fix performance of generic atomics

2017-09-06 Thread Andres Freund
On 2017-09-06 15:25:20 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2017-09-06 15:12:13 -0400, Tom Lane wrote: > >> It looks to me like two of the three implementations promise no such > >> thing. > > > They're volatile vars, so why not? >

Re: [HACKERS] [BUGS] Bug in Physical Replication Slots (at least 9.5)?

2017-09-06 Thread Andres Freund
On 2017-09-06 17:36:02 +0900, Kyotaro HORIGUCHI wrote: > Hi, > > At Mon, 4 Sep 2017 17:17:19 +0900, Michael Paquier > wrote in > > > On Mon, Sep 4, 2017 at 4:04 PM, Andres Freund wrote: > > > I've not read through the thread, but this seems like the wrong

Re: [HACKERS] Fix performance of generic atomics

2017-09-06 Thread Andres Freund
On 2017-09-06 15:12:13 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2017-09-06 14:31:26 -0400, Tom Lane wrote: > >> However, if that's the reasoning, why don't we make all of these > >> use simple reads? It seems unlikely that a locked read is fr

Re: [HACKERS] Fix performance of generic atomics

2017-09-06 Thread Andres Freund
o need to pay the extra cost of a > locked read instead of an unlocked one. > > However, if that's the reasoning, why don't we make all of these > use simple reads? It seems unlikely that a locked read is free. We don't really use locked reads? All the _atom

[HACKERS] Changing Jobs

2017-09-06 Thread Andres Freund
y mentioning it to avoid the appearance of a concealed bias when reviewing/committing patches by other contributors from EDB. Besides that, I'll continue to have an advisor position at Citus Data. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.

Re: [HACKERS] increasing the default WAL segment size

2017-09-05 Thread Andres Freund
tch, but XLogReadDetermineTimeline() could really use some simplifying of repetitive expresssions - XLOGShmemInit shouldn't memcpy to temp_cfile and such, why not just save previous pointer in a local variable? - could you fill in the Reviewed-By: line in the commit message? Running out of c

Re: [HACKERS] JIT compiling expressions/deform + inlining prototype v2.0

2017-09-05 Thread Andres Freund
lumn value, that's independent of NULLness e) one addi adding the length to the offset Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] JIT compiling expressions/deform + inlining prototype v2.0

2017-09-05 Thread Andres Freund
On 2017-09-05 13:58:56 +0300, Konstantin Knizhnik wrote: > > > On 04.09.2017 23:52, Andres Freund wrote: > > > > Yea, I've changed that already, although it's currently added earlier, > > because the alignment is needed before, to access the column co

Re: [HACKERS] Function to move the position of a replication slot

2017-09-04 Thread Andres Freund
port(ERROR, > +(errmsg("Only physical replication slots can be > advanced."))); > ERRCODE_FEATURE_NOT_SUPPORTED, no? Seither of these seem to follow the message guidelines. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@

Re: [HACKERS] JIT compiling expressions/deform + inlining prototype v2.0

2017-09-04 Thread Andres Freund
; expected to be less then zero wjhich means that previous attribute is varlen > field. Yea, I've changed that already, although it's currently added earlier, because the alignment is needed before, to access the column correctly. I've also made number of efficiency improvements, pri

Re: [HACKERS] [BUGS] Bug in Physical Replication Slots (at least 9.5)?

2017-09-04 Thread Andres Freund
Hi, On 2017-09-04 15:51:51 +0900, Kyotaro HORIGUCHI wrote: > SpinLockAcquire(&walsnd->mutex); > + oldFlushPtr = walsnd->flush; > walsnd->write = writePtr; > walsnd->flush = flushPtr; > walsnd->apply = applyPtr; > @@ -1821,7 +1836,

Re: [HACKERS] JIT & function naming

2017-09-03 Thread Andres Freund
On 2017-09-03 10:11:37 -0400, Tom Lane wrote: > Andres Freund writes: > > Currently there's essentially a per EState counter and the generated > > functions get named deform$n and evalexpr$n. That allows for profiling > > of a single query, because differen

Re: [HACKERS] JIT & function naming

2017-09-02 Thread Andres Freund
Hi, On 2017-08-31 23:41:31 -0700, Andres Freund wrote: > I previously had an early prototype of JITing [1] expression evaluation > and tuple deforming. I've since then worked a lot on this. > > Here's an initial, not really pretty but functional, submission. One of the

Re: [HACKERS] Function to move the position of a replication slot

2017-09-02 Thread Andres Freund
On 2017-09-02 18:31:10 -0400, Tom Lane wrote: > Andres Freund writes: > > I don't quite see how you'd get corruption from a physical slot being > > forwarded? I mean you surely can get into the situation that there's > > missing WAL from wherever a stand

Re: [HACKERS] Function to move the position of a replication slot

2017-09-02 Thread Andres Freund
On 2017-09-01 23:37:07 -0400, Tom Lane wrote: > Peter Eisentraut writes: > > On 8/31/17 08:19, Magnus Hagander wrote: > >> I think that, in the end, covered all the comments? > > > I didn't see any explanation of what this would actually be useful for. > > I suppose you could skip over some chang

Re: [HACKERS] Upcoming commit fest will begin soon

2017-09-01 Thread Andres Freund
On 2017-09-01 16:14:25 +0200, Daniel Gustafsson wrote: > > On 01 Sep 2017, at 05:33, Michael Paquier wrote: > > > > Another question is: who would like to become the CF manager for this > > time? This commit fest is very large, so it is going to be difficult > > for one person to do the whole thi

Re: [HACKERS] segment size depending *_wal_size defaults (was increasing the default WAL segment size)

2017-08-30 Thread Andres Freund
Hi, To make it clear: I don't have a strong opinion on these, I'm happy enough to commit the patch as is, minus one unrelated change. I just think it should be discussed. On 2017-08-30 07:01:54 -0400, Peter Eisentraut wrote: > On 8/30/17 00:45, Andres Freund wrote: > > 1) Cu

Re: [HACKERS] segment size depending *_wal_size defaults (was increasing the default WAL segment size)

2017-08-29 Thread Andres Freund
On 2017-08-30 12:52:26 +0900, Michael Paquier wrote: > On Wed, Aug 30, 2017 at 11:20 AM, Peter Eisentraut > wrote: > > On 8/29/17 20:36, Andres Freund wrote: > >> So the question is whether we want {max,min}_wal_size be sized in > >> multiples of segment sizes

Re: [HACKERS] segment size depending *_wal_size defaults (was increasing the default WAL segment size)

2017-08-29 Thread Andres Freund
On 2017-08-30 10:14:22 +0900, Michael Paquier wrote: > On Wed, Aug 30, 2017 at 10:06 AM, Andres Freund wrote: > > On 2017-08-30 09:49:14 +0900, Michael Paquier wrote: > >> Do you think that we should worry about wal segment sizes higher than > >> 2GB? Support for

Re: [HACKERS] segment size depending *_wal_size defaults (was increasing the default WAL segment size)

2017-08-29 Thread Andres Freund
On 2017-08-30 09:49:14 +0900, Michael Paquier wrote: > On Wed, Aug 30, 2017 at 9:36 AM, Andres Freund wrote: > > So the question is whether we want {max,min}_wal_size be sized in > > multiples of segment sizes or as a proper byte size. I'm leaning > > towards the lat

[HACKERS] segment size depending *_wal_size defaults (was increasing the default WAL segment size)

2017-08-29 Thread Andres Freund
Hi, intentionally breaking the thread here, I want this one point to get a bit wider audience. The excerpt of the relevant discussion is: On 2017-08-23 12:13:15 +0530, Beena Emerson wrote: > >> + /* set default max_wal_size and min_wal_size */ > >> + snprintf(repltok, sizeof(repltok), "m

Re: [HACKERS] error-handling in ReorderBufferCommit() seems somewhat broken

2017-08-29 Thread Andres Freund
On 2017-08-30 02:11:10 +0200, Tomas Vondra wrote: > > > On 08/30/2017 01:34 AM, Andres Freund wrote: > > Hi, > > > > On 2017-08-30 01:27:34 +0200, Tomas Vondra wrote: > >> I've been investigating some failures in test_decoding regression tests, >

Re: [HACKERS] error-handling in ReorderBufferCommit() seems somewhat broken

2017-08-29 Thread Andres Freund
SubTransaction() call in the try branch always > fails, triggering the issue. So, IIUC, there's no live problem in postgres core, besides some ugly & undocumented assumptions? Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] increasing the default WAL segment size

2017-08-29 Thread Andres Freund
Hi, On 2017-08-23 12:13:15 +0530, Beena Emerson wrote: > >> + /* > >> + * The calculation of XLOGbuffers requires the run-time > >> parameter > >> + * XLogSegSize which is set from the control file. This > >> value is > >> + * required to creat

Re: [HACKERS] [PATCH] Fix drop replication slot blocking instead of returning error

2017-08-29 Thread Andres Freund
re the previous behavior. > > > > Changing that behavior was the entire point of the cited commit. > > Sorry, I was thinking that the new behavior was needed for internal > future functions since the doc wasn't changed. FWIW, I also don't think it's ok to jus

Re: [HACKERS] pgbench: faster version of tpcb-like transaction

2017-08-27 Thread Andres Freund
at your point is? Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] POC: Sharing record typmods between backends

2017-08-24 Thread Andres Freund
*/ RecordCacheArray[typmod] = tupdesc; Uhm. Isn't that highly highly problematic? E.g. tdrefcount manipulations which are done by all lookups (cf. lookup_rowtype_tupdesc()) would in that case manipulate shared memory in a

Re: [HACKERS] MCXT_ALLOC_NO_OOM -> DSA_ALLOC_NO_OOM in dsa.c

2017-08-24 Thread Andres Freund
Hi, On 2017-08-24 23:08:52 +1200, Thomas Munro wrote: > I spotted a (harmless) thinko in dsa.c. Please see attached. Pushed to 10 and master. Thanks. - Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [HACKERS] Regression stoping PostgreSQL 9.4.13 if a walsender is running

2017-08-24 Thread Andres Freund
dler > function. > > The trivial patch is the following: Pushed, thanks! And sorry again. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Proposal: global index

2017-08-24 Thread Andres Freund
tions. Depending on the desired deadlock behaviour one might end up doing speculative insertions in addition. The foreign key constraint checking is fairly simple, essentially one "just" need to remove the ONLY from the generated check query. Greetings, Andres Freund --

Re: [HACKERS] POC: Sharing record typmods between backends

2017-08-23 Thread Andres Freund
On 2017-08-23 09:45:38 -0400, Robert Haas wrote: > On Wed, Aug 23, 2017 at 1:46 AM, Andres Freund wrote: > > For later commits in the series: > > - Afaict the whole shared tupledesc stuff, as tqueue.c before, is > > entirely untested. This baffles me. See also [1]. I can fo

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2017-08-23 Thread Andres Freund
Hi, On 2017-08-10 15:23:06 +1000, Vaishnavi Prabakaran wrote: > Andres Freund wrote : > > If you were to send a gigabyte of queries, it'd buffer them all up in > memory... So some > >more intelligence is going to be needed. > > Firstly, sorry for the delayed resp

Re: [HACKERS] Regression stoping PostgreSQL 9.4.13 if a walsender is running

2017-08-23 Thread Andres Freund
On 2017-08-23 09:52:45 +0900, Michael Paquier wrote: > On Wed, Aug 23, 2017 at 2:28 AM, Marco Nenciarini > wrote: > > I have noticed that after the 9.4.13 release PostgreSQL reliably fails > > to shutdown with smart and fast method if there is a running walsender. > > > > The postmaster continues

Re: [HACKERS] POC: Sharing record typmods between backends

2017-08-22 Thread Andres Freund
On 2017-08-22 16:41:23 -0700, Andres Freund wrote: > On 2017-08-21 11:02:52 +1200, Thomas Munro wrote: > > On Mon, Aug 21, 2017 at 6:17 AM, Andres Freund wrote: > > > Pushing 0001, 0002 now. > > > > > > - rebased after conflicts > > > - fixed a signifi

Re: [HACKERS] POC: Sharing record typmods between backends

2017-08-22 Thread Andres Freund
On 2017-08-21 11:02:52 +1200, Thomas Munro wrote: > On Mon, Aug 21, 2017 at 6:17 AM, Andres Freund wrote: > > Pushing 0001, 0002 now. > > > > - rebased after conflicts > > - fixed a significant number of too long lines > > - removed a number of now superflous lin

[HACKERS] Blessed TupleDescs and constraints

2017-08-22 Thread Andres Freund
constraints) and proposed future code (it'd be considerably more complicated to share constraints too). Greetings, Andres Freund [1] http://archives.postgresql.org/message-id/CAEepm%3D04LM87Ya_Avgw40934Wh3G4Oyy%2BmmthYHuMb9m5WZwaQ%40mail.gmail.com -- Sent via pgsql-hackers ma

Re: [HACKERS] Updating line length guidelines

2017-08-20 Thread Andres Freund
On 2017-08-21 11:36:43 +0900, Michael Paquier wrote: > On Mon, Aug 21, 2017 at 11:30 AM, Robert Haas wrote: > > On Sun, Aug 20, 2017 at 10:49 AM, Andres Freund wrote: > >> We currently still have the guideline that code should fit into an 80 > >> character window. But

Re: [HACKERS] POC: Sharing record typmods between backends

2017-08-20 Thread Andres Freund
Hi, Pushing 0001, 0002 now. - rebased after conflicts - fixed a significant number of too long lines - removed a number of now superflous linebreaks I think it'd be a good idea to backpatch the addition of TupleDescAttr(tupledesc, n) to make future backpatching easier. What do others think? Tho

Re: [HACKERS] Updating line length guidelines

2017-08-20 Thread Andres Freund
On 2017-08-20 09:29:39 -0700, Joshua D. Drake wrote: > On 08/20/2017 07:49 AM, Andres Freund wrote: > > Hi, > > > > We currently still have the guideline that code should fit into an 80 > > character window. But an increasing amount of the code, and code > > submi

[HACKERS] Updating line length guidelines

2017-08-20 Thread Andres Freund
messages. And there's less need for such a relatively tight limit these days. Perhaps we should up the guideline to 90 or 100 chars? Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresq

Re: [HACKERS] Function to move the position of a replication slot

2017-08-16 Thread Andres Freund
On 2017-08-16 21:25:48 -0400, Robert Haas wrote: > On Wed, Aug 16, 2017 at 5:55 PM, Andres Freund wrote: > > I think we should constrain the API to only allow later LSNs than > > currently in the slot, rather than arbitrary ones. That's why I was > > thinking of "fo

Re: [HACKERS] Function to move the position of a replication slot

2017-08-16 Thread Andres Freund
On 2017-08-16 18:14:45 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2017-08-16 17:06:42 -0400, Tom Lane wrote: > >> If I understand what this is meant to do, maybe better > >> pg_move_replication_slot_lsn() or pg_change_replication_slot_lsn() ? > >> T

Re: [HACKERS] Atomics for heap_parallelscan_nextpage()

2017-08-16 Thread Andres Freund
On August 16, 2017 3:09:27 PM PDT, Tom Lane wrote: >Heikki Linnakangas writes: >> On 08/17/2017 12:20 AM, Tom Lane wrote: >>> Indeed, gaur fails with >>> 2017-08-16 17:09:38.315 EDT [13043:11] PANIC: stuck spinlock >detected at pg_atomic_compare_exchange_u64_impl, atomics.c:196 > >> I was able

Re: [HACKERS] Function to move the position of a replication slot

2017-08-16 Thread Andres Freund
On 2017-08-16 17:06:42 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2017-08-16 12:24:11 -0400, Peter Eisentraut wrote: > >> On 5/4/17 08:05, Magnus Hagander wrote: > >>> PFA a patch that adds a new function, pg_move_replication_slot, that > >>> m

Re: [HACKERS] Function to move the position of a replication slot

2017-08-16 Thread Andres Freund
On 2017-08-16 12:24:11 -0400, Peter Eisentraut wrote: > On 5/4/17 08:05, Magnus Hagander wrote: > > PFA a patch that adds a new function, pg_move_replication_slot, that > > makes it possible to move the location of a replication slot without > > actually consuming all the WAL on it. > > The name k

Re: [HACKERS] Atomics for heap_parallelscan_nextpage()

2017-08-16 Thread Andres Freund
On 2017-08-16 16:20:28 +0300, Heikki Linnakangas wrote: > On 05/06/2017 04:57 PM, David Rowley wrote: > > Andres mentioned in [2] that it might be worth exploring using atomics > > to do the same job. So I went ahead and did that, and came up with the > > attached, which is a slight variation on wh

Re: [HACKERS] Atomics for heap_parallelscan_nextpage()

2017-08-16 Thread Andres Freund
r shm_toc_allocate, which both have bigger-than-MAXALIGN alignment > practices. But this needs to be documented. Well, one could argue the alignment checks in every function are that :). But yea, we probably should mention it more than that. Greetings, Andres Freund -- Sent

Re: [HACKERS] Atomics for heap_parallelscan_nextpage()

2017-08-16 Thread Andres Freund
On August 16, 2017 10:47:23 AM PDT, Robert Haas wrote: >On Wed, Aug 16, 2017 at 1:40 PM, Tom Lane wrote: >> I was wondering why the shm_toc code was using BUFFERALIGN and not >> MAXALIGN, and I now suspect that the answer is "it's an entirely >> undocumented kluge to make the atomics code not c

Re: [HACKERS] Atomics for heap_parallelscan_nextpage()

2017-08-16 Thread Andres Freund
On 2017-08-16 13:44:28 -0400, Tom Lane wrote: > Andres Freund writes: > > Don't think we require BUFFERALIGN - MAXALIGN ought to be > > sufficient. > > Uh, see my other message just now. Yup, you're right. > > The use of BUFFERALIGN presumably is t

Re: [HACKERS] Atomics for heap_parallelscan_nextpage()

2017-08-16 Thread Andres Freund
and I'm damn sure that it > shouldn't be undocumented. 8 byte alignment would be good enough, so BUFFERALIGN ought to be sufficient. But it'd be nicer to have a separate more descriptive knob. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Atomics for heap_parallelscan_nextpage()

2017-08-16 Thread Andres Freund
unks. The latter is already aligned properly, and the important bit is that we have enough space for e->space_for_chunks afterwards, and only padding is in th eway of that... Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Orphaned files in base/[oid]

2017-08-16 Thread Andres Freund
ldn't add weird check commands that require locks on pg_class, we should avoid leaving the orphaned files in the first place. I've upthread outlined approached how to do so. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Atomics for heap_parallelscan_nextpage()

2017-08-16 Thread Andres Freund
On 2017-08-16 09:57:35 -0700, Peter Geoghegan wrote: > On Wed, Aug 16, 2017 at 9:55 AM, Andres Freund wrote: > > On 2017-08-16 11:16:58 -0400, Tom Lane wrote: > >> Heikki Linnakangas writes: > >> > A couple of 32-bit x86 buildfarm members don't seem to be happy

Re: [HACKERS] Atomics for heap_parallelscan_nextpage()

2017-08-16 Thread Andres Freund
Robert, Tom, On 2017-08-16 09:55:15 -0700, Andres Freund wrote: > > Not sure if this is your bug or if it's exposing a pre-existing > > deficiency in the atomics code, viz, failure to ensure that > > pg_atomic_uint64 is actually a 64-bit-aligned type. Andres? >

Re: [HACKERS] Atomics for heap_parallelscan_nextpage()

2017-08-16 Thread Andres Freund
On 2017-08-16 11:16:58 -0400, Tom Lane wrote: > Heikki Linnakangas writes: > > A couple of 32-bit x86 buildfarm members don't seem to be happy with > > this. I'll investigate, but if anyone has a clue, I'm all ears... > > dromedary's issue seems to be alignment: > > TRAP: UnalignedPointer("(((u

Re: [HACKERS] POC: Sharing record typmods between backends

2017-08-15 Thread Andres Freund
On 2017-08-15 20:30:16 -0400, Robert Haas wrote: > On Tue, Aug 15, 2017 at 6:06 PM, Andres Freund wrote: > > Interesting. I was apparently thinking slightly differently. I'd have > > thought we'd have Session struct in statically allocated shared > > memor

Re: [HACKERS] POC: Sharing record typmods between backends

2017-08-15 Thread Andres Freund
void EnsureCurrentSession(void); duplicated. +/* + * We want to create a DSA area to store shared state that has the same extent + * as a session. So far, it's only used to hold the shared record type + * registry. We don't want it to have to create any DSM segments just yet in + * common cases, so we'll give it enough space to hold a very small + * SharedRecordTypmodRegistry. + */ +#define SESSION_DSA_SIZE 0x3 Same "extent"? Maybe lifetime? + +/* + * Make sure that there is a CurrentSession. + */ +void EnsureCurrentSession(void) +{ linebreak. +{ + if (CurrentSession == NULL) + { + MemoryContext old_context = MemoryContextSwitchTo(TopMemoryContext); + + CurrentSession = palloc0(sizeof(Session)); + MemoryContextSwitchTo(old_context); + } +} Isn't MemoryContextAllocZero easier? Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] increasing the default WAL segment size

2017-08-15 Thread Andres Freund
= XLogSegMinSize && size <= XLogSegMaxSize)) > + Please wrap references to size in parens. Should we consider making this an inline function instead? There's some multiple evaluation hazard here... > +#define XLogSegmentsPerXLogId(UINT64CONST(0x1) / XLogSegSize) > > #define XLogSegNoOffsetToRecPtr(segno, offset, dest) \ > - (dest) = (segno) * XLOG_SEG_SIZE + (offset) > + (dest) = (segno) * XLogSegSize + (offset) I don't think it's a good idea to implicitly reference a global variable in such a macro. IOW, I think this needs to grow another parameter, and callers should get adjusted. I know this'll affect a number of macros, but it still seems like the right thing to do. I'd welcome other opinions on this. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [BUGS] Replication to Postgres 10 on Windows is broken

2017-08-15 Thread Andres Freund
On August 15, 2017 8:07:43 AM PDT, Andres Freund wrote: > > >On August 15, 2017 7:04:59 AM PDT, Tom Lane wrote: >>Peter Eisentraut writes: >>> On 8/14/17 10:57, Tom Lane wrote: >>>> I think we could commit add-connected-event-2.patch and call this >>&

Re: [HACKERS] [BUGS] Replication to Postgres 10 on Windows is broken

2017-08-15 Thread Andres Freund
On August 15, 2017 7:04:59 AM PDT, Tom Lane wrote: >Peter Eisentraut writes: >> On 8/14/17 10:57, Tom Lane wrote: >>> I think we could commit add-connected-event-2.patch and call this >>> issue resolved. > >> Would you like to commit your patch then? > >It's really Andres' patch, but I can push

Re: [HACKERS] Pluggable storage

2017-08-14 Thread Andres Freund
Hi, On 2017-06-13 11:50:27 +1000, Haribabu Kommi wrote: > Here I attached WIP patches to support pluggable storage. The patch series > are may not work individually. Still so many things are under development. > These patches are just to share the approach of the current development. Making a pa

Re: [HACKERS] measuring the impact of increasing WAL segment size

2017-08-14 Thread Andres Freund
oughput drops a bit (by 10-20%), for > some segment sizes, and then recovers. The behavior seems to be smooth (not > just a sudden drop for one segment size) but the value varies depending on > the scale, test type (tpc-b /simple-update). That's interesting. I presume you

Re: [HACKERS] [COMMITTERS] pgsql: Avoid GatherMerge crash when there are no workers.

2017-08-14 Thread Andres Freund
Hi, On 2017-04-03 12:56:36 +0530, Rushabh Lathia wrote: > On my local environment I was getting coverage for the heap_compare_slots > with > existing test. But it seems like due to low number of record fetch only > leader get > evolved to the fetching tuples in the shared report. > > I modified t

Re: [HACKERS] Orphaned files in base/[oid]

2017-08-14 Thread Andres Freund
On 2017-08-14 14:40:46 -0400, Tom Lane wrote: > The core problem with zapping non-temp table files is that you can't > do that unless you're sure you have consistent, up-to-date pg_class > data that nobody else is busy adding to. It's hard to see an external > application being able to do that saf

Re: [HACKERS] Re: [COMMITTERS] pgsql: Don't force-assign transaction id when exporting a snapshot.

2017-08-14 Thread Andres Freund
On 2017-08-14 13:55:29 -0400, Peter Eisentraut wrote: > On 8/12/17 07:32, Petr Jelinek wrote: > > This commit has side effect that it makes it possible to export > > snapshots on the standbys. This makes it possible to do pg_dump -j on > > standby with consistent snapshot. Here is one line patch (+

Re: [HACKERS] shared memory based stat collector (was: Sharing record typmods between backends)

2017-08-14 Thread Andres Freund
On 2017-08-14 13:06:48 -0400, Robert Haas wrote: > On Mon, Aug 14, 2017 at 1:02 PM, Andres Freund wrote: > > I previously thought that an option to occasionally WAL log the stats > > file would be useful (e.g. just before a checkpoint). That'd make them > > persis

<    1   2   3   4   5   6   7   8   9   10   >