Re: [HACKERS] [COMMITTERS] pgsql: Replication lag tracking for walsenders

2017-04-21 Thread Andres Freund
in having it on the C89 animal). Very curious what it'll do to skink's runtime :/. I'll add a few more cores to the machine, otherwise it'll probably wreak havock. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscript

Re: [HACKERS] Unportable implementation of background worker start

2017-04-21 Thread Andres Freund
Tom, On 2017-04-21 13:49:27 -0400, Tom Lane wrote: > >> * If we are in PM_WAIT_DEAD_END state, then we don't want to accept > >> - * any new connections, so we don't call select(), and just > >> sleep. > >> + * any new connections, so we don't call WaitEventSetWait(), > >>

Re: [HACKERS] Unportable implementation of background worker start

2017-04-21 Thread Andres Freund
On 2017-04-21 14:08:21 -0400, Tom Lane wrote: > but I see that SUSv2 > mandates that fcntl.h provide both F_SETFD and FD_CLOEXEC, so by our own > coding rules it ought to be okay to assume they're there. I'm tempted to > rip out the quoted bit, as well as the #ifdef F_SETFD, from libpq and see >

Re: [HACKERS] Unportable implementation of background worker start

2017-04-21 Thread Andres Freund
92d2ff0 100644 > --- a/src/backend/storage/ipc/latch.c > +++ b/src/backend/storage/ipc/latch.c > @@ -62,6 +62,10 @@ > #include "storage/pmsignal.h" > #include "storage/shmem.h" > > +#ifndef FD_CLOEXEC > +#define FD_CLOEXEC 1 > +#endif Hm? Are we sure this is portable? Is there really cases that have F_SETFD, but not CLOEXEC? Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Unportable implementation of background worker start

2017-04-20 Thread Andres Freund
On 2017-04-20 20:10:41 -0400, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > > On 2017-04-20 20:05:02 -0400, Tom Lane wrote: > >> Also, if it's not there we'd fall back to using plain poll(), which is > >> not so awful that we need to work ha

[HACKERS] walsender & parallelism

2017-04-20 Thread Andres Freund
things. Greetings, Andres Freund [1] https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=7c4f52409a8c7d85ed169bbbc1f6092274d03920 [2] http://archives.postgresql.org/message-id/30242bc6-eca4-b7bb-670e-8d0458753a8c%402ndquadrant.com -- Sent via pgsql-hackers mailing list (pgsql

Re: [HACKERS] Unportable implementation of background worker start

2017-04-20 Thread Andres Freund
On 2017-04-20 20:05:02 -0400, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > > On 2017-04-20 19:53:02 -0400, Tom Lane wrote: > >> So ... what would you say to replacing epoll_create() with > >> epoll_create1(EPOLL_CLOEXEC) ? Then a WaitEventSet wou

Re: [HACKERS] Unportable implementation of background worker start

2017-04-20 Thread Andres Freund
On 2017-04-20 19:53:02 -0400, Tom Lane wrote: > I wrote: > > Andres Freund <and...@anarazel.de> writes: > >> On 2017-04-20 19:23:28 -0400, Tom Lane wrote: > >>> or are the HANDLEs in a Windows WaitEventSet not inheritable > >>> resources? > &

Re: [HACKERS] Unportable implementation of background worker start

2017-04-20 Thread Andres Freund
On 2017-04-20 00:50:13 -0400, Tom Lane wrote: > I wrote: > > Alvaro Herrera writes: > >> Tom Lane wrote: > >>> Hm. Do you have a more-portable alternative? > > >> I was thinking in a WaitEventSet from latch.c. > > My first reaction was that that sounded like a lot

Re: [HACKERS] Unportable implementation of background worker start

2017-04-20 Thread Andres Freund
On 2017-04-20 19:23:28 -0400, Tom Lane wrote: > or are the HANDLEs in a Windows WaitEventSet not inheritable > resources? I think we have control over that. According to https://msdn.microsoft.com/en-us/library/windows/desktop/ms724466(v=vs.85).aspx CreateProcess() has to be called with

Re: [HACKERS] Removing select(2) based latch (was Unportable implementation of background worker start)

2017-04-20 Thread Andres Freund
Hi, On 2017-04-20 17:27:42 -0400, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > > On 2017-04-19 20:06:05 -0400, Tom Lane wrote: > >> We should check the buildfarm to see if the select() implementation is > >> being tested at all. > > &g

Re: [HACKERS] OK, so culicidae is *still* broken

2017-04-20 Thread Andres Freund
On 2017-04-20 16:57:03 +0530, Amit Kapila wrote: > On Wed, Apr 19, 2017 at 9:01 PM, Andres Freund <and...@anarazel.de> wrote: > > On 2017-04-19 10:15:31 -0400, Tom Lane wrote: > >> Amit Kapila <amit.kapil...@gmail.com> writes: > >> > On Sun, Apr 16, 201

[HACKERS] Removing select(2) based latch (was Unportable implementation of background worker start)

2017-04-19 Thread Andres Freund
On 2017-04-19 20:06:05 -0400, Tom Lane wrote: > > BTW, we IIRC had discussed removing the select() backed latch > > implementation in this release. I'll try to dig up that discussion. > > Might be sensible. Even my pet dinosaurs have poll(2). I can't find the discussion anymore, but I'm fairly

Re: [HACKERS] snapbuild woes

2017-04-19 Thread Andres Freund
On 2017-04-17 21:16:57 -0700, Andres Freund wrote: > I think we might need some more tests for this to be committable, so > it might not become committable tomorrow. I'm working on some infrastructure around this. Not sure if it needs to be committed, but it's certainly useful for eval

Re: [HACKERS] Unportable implementation of background worker start

2017-04-19 Thread Andres Freund
On 2017-04-19 18:56:26 -0400, Tom Lane wrote: > Alvaro Herrera writes: > > Tom Lane wrote: > >> Hm. Do you have a more-portable alternative? > > > I was thinking in a WaitEventSet from latch.c. > > Yeah, some googling turns up the suggestion that a self-pipe is a

Re: [HACKERS] Highly Variable Planning Times

2017-04-19 Thread Andres Freund
On 2017-04-19 16:43:07 -0700, Michael Malis wrote: > > The profile seems to confirm that this is largely due to cache misses. > > Can you elaborate? Cache misses of what? Why is the planning time so > variable? Postgres uses caches for a lot of metadata of tables, indexes, ... Some actions, like

Re: [HACKERS] Highly Variable Planning Times

2017-04-19 Thread Andres Freund
gt; information you need let me know. This was on Ubuntu 16.04 The profile seems to confirm that this is largely due to cache misses. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Highly Variable Planning Times

2017-04-19 Thread Andres Freund
On 2017-04-19 14:14:54 -0700, Michael Malis wrote: > > Could you also get a profile using perf record -g? The strace isn't > > telling us all that much. > > I've attached the perf.data file from `perf record -g -p > `. I'm not that familiar with perf so if there is more > information you need

Re: [HACKERS] Highly Variable Planning Times

2017-04-19 Thread Andres Freund
was slow, there would be a single semaphore which would make > up most of the `semop` calls. Here is a snippet of the output when I > ran `strace -r` on a query with high planning time: Could you also get a profile using perf record -g? The strace isn't telling us all that much. Greetin

Re: [HACKERS] Relation cache invalidation on replica

2017-04-19 Thread Andres Freund
On 2017-04-19 17:07:49 +0300, Victor Yegorov wrote: > 2017-03-13 9:22 GMT+02:00 Andres Freund <and...@anarazel.de>: > > > >I think we're hitting this particular issue quite frequently when > > >rebuilding indexes on master after big-volume data manipulations.

Re: [HACKERS] OK, so culicidae is *still* broken

2017-04-19 Thread Andres Freund
On 2017-04-19 10:15:31 -0400, Tom Lane wrote: > Amit Kapila writes: > > On Sun, Apr 16, 2017 at 3:04 AM, Tom Lane wrote: > >> Obviously, any such fix would be a lot more likely to be reliable in > >> 64-bit machines. There's probably not enough

Re: [HACKERS] Continuous buildfarm failures on hamster with bin-check

2017-04-18 Thread Andres Freund
a tmpfs? If hamster fails this regularly I think we have to do something about it, rather than paper over it. What's the storage situation currently like? Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Re: Query fails when SRFs are part of FROM clause (Commit id: 69f4b9c85f)

2017-04-17 Thread Andres Freund
ur SRF behaviour in any sort of detail ([1], [2]), particularly not documenting where SRFs are legal, and how nested SRFs are supposed to behave. I'm not sure in how much detail we want to go? If we want do document that, where? The closest seems to be "4.2.14. Expression Evaluation Rules&qu

Re: [HACKERS] snapbuild woes

2017-04-17 Thread Andres Freund
On 2017-04-16 22:04:04 -0400, Noah Misch wrote: > On Thu, Apr 13, 2017 at 12:58:12AM -0400, Noah Misch wrote: > > On Wed, Apr 12, 2017 at 10:21:51AM -0700, Andres Freund wrote: > > > On 2017-04-12 11:03:57 -0400, Peter Eisentraut wrote: > > > > On 4/12/17 02:31, Noah

Re: [HACKERS] Cutting initdb's runtime (Perl question embedded)

2017-04-17 Thread Andres Freund
On 2017-04-17 17:49:54 +0100, Dagfinn Ilmari Mannsåker wrote: > Tom Lane writes: > > > There's certainly lots more that could be done in the genbki code, > > but I think all we can justify at this stage of the development > > cycle is to get the low-hanging fruit for testing

Re: [HACKERS] Cutting initdb's runtime (Perl question embedded)

2017-04-17 Thread Andres Freund
On 2017-04-17 17:49:54 +0100, Dagfinn Ilmari Mannsåker wrote: > Tom Lane writes: > > > There's certainly lots more that could be done in the genbki code, > > but I think all we can justify at this stage of the development > > cycle is to get the low-hanging fruit for testing

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-04-17 Thread Andres Freund
On 2017-04-17 18:28:16 +0200, Petr Jelinek wrote: > On 17/04/17 18:02, Andres Freund wrote: > > On 2017-04-15 02:33:59 +0900, Fujii Masao wrote: > >> On Fri, Apr 14, 2017 at 10:33 PM, Petr Jelinek > >> <petr.jeli...@2ndquadrant.com> wrote: > >>> On

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-04-17 Thread Andres Freund
On 2017-04-15 02:33:59 +0900, Fujii Masao wrote: > On Fri, Apr 14, 2017 at 10:33 PM, Petr Jelinek > wrote: > > On 12/04/17 15:55, Fujii Masao wrote: > >> Hi, > >> > >> When I shut down the publisher while I repeated creating and dropping > >> the subscription in the

Re: [HACKERS] OK, so culicidae is *still* broken

2017-04-17 Thread Andres Freund
On 2017-04-15 14:34:28 -0700, Andres Freund wrote: > On 2017-04-15 17:30:21 -0400, Tom Lane wrote: > > Andres Freund <and...@anarazel.de> writes: > > > On 2017-04-15 16:48:05 -0400, Tom Lane wrote: > > >> Concretely, I propose the attached patch. We'd have

Re: [HACKERS] Logical replication - TRAP: FailedAssertion in pgstat.c

2017-04-16 Thread Andres Freund
On 2017-04-16 10:46:21 +0200, Erik Rijkers wrote: > On 2017-04-15 04:47, Erik Rijkers wrote: > > > > 0001-Reserve-global-xmin-for-create-slot-snasphot-export.patch + > > 0002-Don-t-use-on-disk-snapshots-for-snapshot-export-in-l.patch+ > >

Re: [HACKERS] OpenSSL support in our back branches

2017-04-15 Thread Andres Freund
On 2017-04-15 18:04:00 -0400, Tom Lane wrote: > I see that buildfarm member anchovy has been failing in pre-9.5 > branches for the last several weeks, with symptoms suggesting that > it's been updated to openssl 1.1.0 or later. The failures are > unsurprising given that commit 593d4e47d wasn't

Re: [HACKERS] OK, so culicidae is *still* broken

2017-04-15 Thread Andres Freund
On 2017-04-15 17:30:21 -0400, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > > On 2017-04-15 16:48:05 -0400, Tom Lane wrote: > >> Concretely, I propose the attached patch. We'd have to put it into > >> all supported branches, since culicidae is

Re: [HACKERS] OK, so culicidae is *still* broken

2017-04-15 Thread Andres Freund
On 2017-04-15 17:24:54 -0400, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > > On 2017-04-15 17:09:38 -0400, Tom Lane wrote: > >> Why doesn't Windows' ability to map the segment into the new process > >> before it executes take care of that? &

Re: [HACKERS] OK, so culicidae is *still* broken

2017-04-15 Thread Andres Freund
On 2017-04-15 17:09:38 -0400, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > > That seems quite reasonable. I'm afraid we're going to have to figure > > out something similar, but more robust, for windows soon-ish :/ > > Why doesn't Windows'

Re: [HACKERS] OK, so culicidae is *still* broken

2017-04-15 Thread Andres Freund
t; /* OK, should be able to attach to the segment */ > - memAddress = shmat(shmid, NULL, PG_SHMAT_FLAGS); > + memAddress = shmat(shmid, requestedAddress, PG_SHMAT_FLAGS); > > if (memAddress == (void *) -1) > elog(FATAL, "shmat(id=%d) failed: %m&quo

Re: [HACKERS] OK, so culicidae is *still* broken

2017-04-14 Thread Andres Freund
On April 14, 2017 9:42:41 PM PDT, Tom Lane wrote: >Per >https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=culicidae=2017-04-15%2004%3A00%3A02 > >2017-04-15 04:31:21.657 GMT [16792] FATAL: could not reattach to >shared memory (key=6280001, addr=0x7f692fece000): Invalid

[HACKERS] Make defines in pg_config_manual.h conditional?

2017-04-14 Thread Andres Freund
Hi, ATM the defines in pg_config_manual.h are largely unconditional - which means the file has to be edited instead of being able to override things via CFLAGS/COPT. Is there any good reason for doing so? Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] Cutting initdb's runtime (Perl question embedded)

2017-04-13 Thread Andres Freund
On 2017-04-13 14:05:43 -0400, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > > On 2017-04-13 12:56:14 -0400, Tom Lane wrote: > >> Andres Freund <and...@anarazel.de> writes: > >>> Cool. I wonder if we also should remove AtEOXact_CatC

Re: [HACKERS] Quals not pushed down into lateral

2017-04-13 Thread Andres Freund
On 2017-04-13 16:34:12 -0400, Robert Haas wrote: > On Thu, Mar 16, 2017 at 4:45 AM, Andres Freund <and...@anarazel.de> wrote: > > During citus development we noticed that restrictions aren't pushed down > > into lateral subqueries, even if they semantically could. For exampl

Re: [HACKERS] Re: Query fails when SRFs are part of FROM clause (Commit id: 69f4b9c85f)

2017-04-13 Thread Andres Freund
On 2017-04-13 12:53:27 -0400, Robert Haas wrote: > On Wed, Apr 12, 2017 at 6:58 PM, Andres Freund <and...@anarazel.de> wrote: > > This yields plenty weird behaviour in < v10. E.g. the following is > > disallowed: > > SELECT * FROM int4mul(generate_series(1,3), 1); &

Re: [HACKERS] Cutting initdb's runtime (Perl question embedded)

2017-04-13 Thread Andres Freund
On 2017-04-13 12:56:14 -0400, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > > Cool. I wonder if we also should remove AtEOXact_CatCache()'s > > cross-checks - the resowner replacement has been in place for a while, > > and seems robust enough. T

Re: [HACKERS] Cutting initdb's runtime (Perl question embedded)

2017-04-13 Thread Andres Freund
On 2017-04-13 12:13:30 -0400, Tom Lane wrote: > Andreas Karlsson writes: > > Here is my proof of concept patch. It does basically the same thing as > > Andres's patch except that it handles quoted values a bit better and > > does not try to support anything other than the

Re: [HACKERS] snapbuild woes

2017-04-12 Thread Andres Freund
On April 12, 2017 9:58:12 PM PDT, Noah Misch <n...@leadboat.com> wrote: >On Wed, Apr 12, 2017 at 10:21:51AM -0700, Andres Freund wrote: >> On 2017-04-12 11:03:57 -0400, Peter Eisentraut wrote: >> > On 4/12/17 02:31, Noah Misch wrote: >> > >>> But I hope

Re: [HACKERS] Function to control physical replication slot

2017-04-12 Thread Andres Freund
On 2017-04-12 20:15:52 -0400, Peter Eisentraut wrote: > On 4/11/17 05:15, Magnus Hagander wrote: > > Is there a particular reason we don't have a function to *set* the > > restart_lsn of a replication slot, other than to drop it and recreate it? > > I suppose there could be lots of problems if

Re: [HACKERS] snapbuild woes

2017-04-12 Thread Andres Freund
x = xid; > > @@ -975,10 +976,19 @@ SnapBuildCommitTxn(SnapBuild *builder, XLogRecPtr lsn, > TransactionId xid, > /* >* We could avoid treating !SnapBuildTxnIsRunning transactions > as > * timetravel ones, but we want to be able to export a snapshot > when > -

Re: [HACKERS] Re: Query fails when SRFs are part of FROM clause (Commit id: 69f4b9c85f)

2017-04-12 Thread Andres Freund
On 2017-04-05 09:39:37 -0400, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > > On 2017-04-05 02:47:55 -0400, Noah Misch wrote: > >> [Action required within three days. This is a generic notification.] > > > I've a very preliminary patch. I'd l

Re: [HACKERS] Query fails when SRFs are part of FROM clause (Commit id: 69f4b9c85f)

2017-04-12 Thread Andres Freund
wouldn't without additional work. I plan to continue to error out in either... Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Cutting initdb's runtime (Perl question embedded)

2017-04-12 Thread Andres Freund
On 2017-04-12 10:12:47 -0400, Tom Lane wrote: > Andres mentioned, and I've confirmed locally, that a large chunk of > initdb's runtime goes into regprocin's brute-force lookups of function > OIDs from function names. The recent discussion about cutting TAP test > time prompted me to look into

Re: [HACKERS] snapbuild woes

2017-04-12 Thread Andres Freund
On 2017-04-12 11:03:57 -0400, Peter Eisentraut wrote: > On 4/12/17 02:31, Noah Misch wrote: > >>> But I hope you mean to commit these snapbuild patches before the postgres > >>> 10 > >>> release? As far as I know, logical replication is still very broken > >>> without > >>> them (or at least

Re: [HACKERS] TAP tests take a long time

2017-04-11 Thread Andres Freund
On 2017-04-12 10:39:22 +0800, Craig Ringer wrote: > On 12 April 2017 at 08:22, Michael Paquier wrote: > > On Wed, Apr 12, 2017 at 9:12 AM, Craig Ringer > > wrote: > >> > >> We should also have a debug --no-fsync option for initdb, or maybe

Re: [HACKERS] Query fails when SRFs are part of FROM clause (Commit id: 69f4b9c85f)

2017-04-11 Thread Andres Freund
On 2017-04-11 17:25:52 -0400, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > > Tom, do you have any opinion on the volatility stuff? > > What was the previous behavior for such cases? If it was reasonably > sane, we probably have to preserve it.

Re: [HACKERS] Query fails when SRFs are part of FROM clause (Commit id: 69f4b9c85f)

2017-04-11 Thread Andres Freund
On 2017-01-30 18:54:50 -0500, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > > Wonder if we there's an argument to be made for implementing this > > roughly similarly to split_pathtarget_at_srf - instead of injecting a > > ProjectSet node we'd add a Funct

Re: [HACKERS] TAP tests take a long time

2017-04-11 Thread Andres Freund
On 2017-04-11 15:52:34 -0400, Tom Lane wrote: > Andrew Dunstan writes: > >> The other thing that might be useful here is to push on parallelizing > >> buildfarm runs. > > > One reason I haven't supported "make -j nn" everywhere (it is supported > > for making

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-04-10 Thread Andres Freund
Hi, On 2017-04-08 23:36:13 +0530, Pavan Deolasee wrote: > On Wed, Apr 5, 2017 at 11:57 PM, Andres Freund <and...@anarazel.de> wrote: > > > On 2017-04-05 09:36:47 -0400, Robert Haas wrote: > > > By the way, the "Converting WARM chains back to HOT chains&quo

Re: [HACKERS] "left shift of negative value" warnings

2017-04-10 Thread Andres Freund
On 2017-04-10 15:25:57 -0400, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > > On 2017-04-09 19:20:27 -0400, Tom Lane wrote: > >> As I read that, it's only "undefined" if overflow would occur (ie > >> the sign bit would change). Your comp

Re: [HACKERS] "left shift of negative value" warnings

2017-04-10 Thread Andres Freund
On 2017-04-09 19:20:27 -0400, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > > For a while I've been getting warnings like > > /home/andres/src/postgresql/src/backend/utils/adt/inet_cidr_ntop.c: In > > function ‘inet_cidr_ntop_ipv6’: > > /home/

Re: [HACKERS] valgrind errors around dsa.c

2017-04-10 Thread Andres Freund
On 2017-04-08 14:46:04 +1200, Thomas Munro wrote: > Fix attached. Thanks. Pushed! Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Some thoughts about SCRAM implementation

2017-04-10 Thread Andres Freund
. I know we're very late in the CF cycle > but, again, this would be a real shame. That can equally be said about about a lot of features. If we don't stop at some point... Also, we're not late in the CF cycle, the CF cycle for v10 is over. It's not like the non-existance of channel binding removes p

Re: Adding lfirst_node (was Re: [HACKERS] [sqlsmith] Planner crash on foreign table join)

2017-04-10 Thread Andres Freund
On 2017-04-10 12:20:16 -0400, Tom Lane wrote: > Barring objections, I'll push this shortly. +1, to just about all of it -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] GCC 7 warnings

2017-04-10 Thread Andres Freund
On 2017-04-10 09:10:07 -0700, Andres Freund wrote: > Hi, > > On 2017-04-10 11:03:23 -0400, Peter Eisentraut wrote: > > The release of GCC 7 is approaching [0], and the number of warnings in > > PostgreSQL has gone up since we last looked [1]. Output attached. (My > >

Re: [HACKERS] GCC 7 warnings

2017-04-10 Thread Andres Freund
Hi, On 2017-04-10 11:03:23 -0400, Peter Eisentraut wrote: > The release of GCC 7 is approaching [0], and the number of warnings in > PostgreSQL has gone up since we last looked [1]. Output attached. (My > version is 7.0.1 20170408.) > > Most of the issues have to do with concatenating two or

[HACKERS] "left shift of negative value" warnings

2017-04-09 Thread Andres Freund
t); If I understand C99 correctly, the behaviour of a left-shift of a negative number is undefined (6.5.7 4.). In C89 the spec was very unclear about that. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: h

Re: [HACKERS] partitioned tables and contrib/sepgsql

2017-04-09 Thread Andres Freund
etings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] missing isinf declaration on solaris

2017-04-09 Thread Andres Freund
On 2014-09-24 16:26:33 -0400, Tom Lane wrote: > Peter Eisentraut writes: > > On 9/24/14 9:21 AM, Tom Lane wrote: > >> Agreed, but what about non-GCC compilers? > > > Stick AC_PROG_CC_C99 into configure.in. > > I think that's a bad idea, unless you mean to do it only on Solaris. >

Re: [HACKERS] [sqlsmith] Planner crash on foreign table join

2017-04-08 Thread Andres Freund
On 2017-04-08 17:20:28 -0400, Tom Lane wrote: > Robert Haas writes: > > On Sat, Apr 8, 2017 at 3:57 PM, Tom Lane wrote: > >> This makes me wonder whether we were being penny-wise and pound-foolish > >> by not making Bitmapsets be a kind of Node, so that

Re: [HACKERS] snapbuild woes

2017-04-08 Thread Andres Freund
On 2017-04-08 16:29:10 +0200, Erik Rijkers wrote: > On 2017-04-08 15:56, Andres Freund wrote: > > On 2017-04-08 09:51:39 -0400, David Steele wrote: > > > On 3/2/17 7:54 PM, Petr Jelinek wrote: > > > > > > > > Yes the copy patch needs rebase as well. But

Re: [HACKERS] 2017-03 CF Closed

2017-04-08 Thread Andres Freund
Hi David, All, Thanks for your work on managing the fest! On 2017-04-08 10:25:06 -0400, David Steele wrote: > Final stats for the commitfest: > > Committed: 116 > Moved to next CF: 50 > Rejected: 2 > Returned with Feedback: 41 > Total: 209. Not a bad haul... - Andres -- Sent via

Re: [HACKERS] Remaining 2017-03 CF entries

2017-04-08 Thread Andres Freund
seem wrong to me - it's too large a patchset to just have been merged yesterday afternoon. I also suspect that some more design discussion will be needed next cycle... Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your su

Re: [HACKERS] Running make check-world in buildfarm (was Re: [COMMITTERS] pgsql: Use SASLprep to normalize passwords for SCRAM authentication.)

2017-04-08 Thread Andres Freund
On 2017-04-08 23:01:06 +0900, Michael Paquier wrote: > On Sat, Apr 8, 2017 at 7:33 PM, Heikki Linnakangas wrote: > > Hmm. It looks like none of the buildfarm members are running the > > authentication tests. Nor recovery tests, nor subscription tests. We're > > missing a trick

Re: [HACKERS] snapbuild woes

2017-04-08 Thread Andres Freund
On 2017-04-08 09:51:39 -0400, David Steele wrote: > On 3/2/17 7:54 PM, Petr Jelinek wrote: > > > > Yes the copy patch needs rebase as well. But these ones are fine. > > This bug has been moved to CF 2017-07. FWIW, as these are bug-fixes that need to be backpatched, I do plan to work on them

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2017-04-07 Thread Andres Freund
On 2017-04-07 22:06:13 -0300, Claudio Freire wrote: > On Fri, Apr 7, 2017 at 9:56 PM, Andres Freund <and...@anarazel.de> wrote: > > Hi, > > > > > > On 2017-04-07 19:43:39 -0300, Claudio Freire wrote: > >> On Fri, Apr 7, 2017 at 5:05 PM, Andres Fr

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2017-04-07 Thread Andres Freund
Hi, On 2017-04-07 19:43:39 -0300, Claudio Freire wrote: > On Fri, Apr 7, 2017 at 5:05 PM, Andres Freund <and...@anarazel.de> wrote: > > Hi, > > > > I've *not* read the history of this thread. So I really might be > > missin

Re: [HACKERS] [COMMITTERS] pgsql: Improve 64bit atomics support.

2017-04-07 Thread Andres Freund
On 2017-04-07 16:36:09 -0700, Andres Freund wrote: > On 2017-04-07 19:55:21 -0300, Alvaro Herrera wrote: > > Andres Freund wrote: > > > Improve 64bit atomics support. > > > > > > When adding atomics back in b64d92f1a, I added 64bit support as > >

Re: [HACKERS] [COMMITTERS] pgsql: Improve 64bit atomics support.

2017-04-07 Thread Andres Freund
On 2017-04-07 19:55:21 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > Improve 64bit atomics support. > > > > When adding atomics back in b64d92f1a, I added 64bit support as > > optional; there wasn't yet a direct user in sight. That turned out to > > be

[HACKERS] mvstats triggers 32bit warnings

2017-04-07 Thread Andres Freund
Hi, compiling on linux 32 bit I get a lot of warnings due to printf format. I suspect most of them should just be cured by using %zd or %zu instead of %ld. /home/andres/src/postgresql/src/backend/statistics/mvdistinct.c: In function ‘statext_ndistinct_deserialize’:

Re: [HACKERS] monitoring.sgml missing tag

2017-04-07 Thread Andres Freund
Hi, On 2017-04-07 23:00:01 +0200, Erik Rijkers wrote: > On 2017-04-07 22:50, Andres Freund wrote: > > On 2017-04-07 22:47:55 +0200, Erik Rijkers wrote: > > > monitoring.sgml has one tag missing > > > > Is that actually an issue? SGML allows skipping certain close

Re: [HACKERS] monitoring.sgml missing tag

2017-04-07 Thread Andres Freund
one-by-one. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2017-04-07 Thread Andres Freund
lazy_tid_reaped() -- is a particular tid deletable? > * > * This has the right signature to be an IndexBulkDeleteCallback. > * > - * Assumes dead_tuples array is in sorted order. > + * Assumes the dead_tuples multiarray is in sorted order, both &

Re: [HACKERS] Remaining 2017-03 CF entries

2017-04-07 Thread Andres Freund
On 2017-04-07 16:28:03 -0300, Alvaro Herrera wrote: > Peter Geoghegan wrote: > > > - can't move to next fest because it's waiting-on-author, which doesn't > > > allow that. Doesn't strike me as a useful restriction. > > > > I agree that that CF app restriction makes little sense. > > What the

Re: [HACKERS] Remaining 2017-03 CF entries

2017-04-07 Thread Andres Freund
On 2017-04-07 15:45:33 -0300, Alvaro Herrera wrote: > > Write Amplification Reduction Method (WARM) > > - fair number of people don't think it's ready for v10. > > I'm going over this one now with Pavan, with the intent of getting it in > committable shape. > > I may be biased, but the claimed

[HACKERS] Remaining 2017-03 CF entries

2017-04-07 Thread Andres Freund
Hi, When I started writing this, there were the following reamining CF items, minus bugfix ones which aren't bound by the code freeze. I think it makes sense to go through those and see whether it's realistic to commit any of them. Ready for Committer: Add GUCs for predicate lock promotion

Re: [HACKERS] UPDATE of partition key

2017-04-07 Thread Andres Freund
nk it's not entirely clear that we have consensus on the > design, so let's revisit it for next release. I was kind of looking for the appropriate status of "not entirely clear that we have consensus on the design" - which isn't really ready-for-committer, but no waiting-on-author eit

Re: [HACKERS] recent deadlock regression test failures

2017-04-07 Thread Andres Freund
On 2017-04-07 12:49:22 -0500, Kevin Grittner wrote: > On Fri, Apr 7, 2017 at 12:28 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > > Andrew Dunstan <andrew.duns...@2ndquadrant.com> writes: > >> On 04/07/2017 12:57 PM, Andres Freund wrote: > >>> I don't think

Re: [HACKERS] parallel bitmapscan isn't exercised in regression tests

2017-04-07 Thread Andres Freund
On 2017-04-06 13:43:55 -0700, Andres Freund wrote: > On 2017-04-06 10:00:32 +0530, Dilip Kumar wrote: > > On Tue, Apr 4, 2017 at 5:51 AM, Dilip Kumar <dilipbal...@gmail.com> wrote: > > > Sure I can do that, In attached patch, I only fixed the problem of not > > > e

[HACKERS] recent deadlock regression test failures

2017-04-07 Thread Andres Freund
Hi, There's two machines that recently report changes in deadlock detector output: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=hyrax=2017-04-05%2018%3A58%3A04 https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=friarbird=2017-04-07%2004%3A20%3A01 both just failed twice in a row:

[HACKERS] valgrind errors around dsa.c

2017-04-07 Thread Andres Freund
is the source of the undefined memory makes me doubt that. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [PATCH] Warn users about duplicate configuration parameters

2017-04-07 Thread Andres Freund
pecial case for postgresql.auto.conf. More useless warnings make actual warnings less useful. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [PATCH] Warn users about duplicate configuration parameters

2017-04-07 Thread Andres Freund
Also, with postgresql.auto.conf it's even more common to override parameters. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] src/interfaces/libpq shipping nmake-related Makefiles

2017-04-07 Thread Andres Freund
ant to build it, it certainly save a lot more than "a few minutes". > For somebody who doesn't have ready scripts it takes a *long* time to set > up a build environment to do our regular msvc builds. For a useful libpq you actually need a good chunk of dependencies, including

Re: [HACKERS] Supporting huge pages on Windows

2017-04-07 Thread Andres Freund
On 2017-04-07 13:57:07 +0200, Magnus Hagander wrote: > On Wed, Apr 5, 2017 at 9:15 AM, Tsunakawa, Takayuki < > tsunakawa.ta...@jp.fujitsu.com> wrote: > > > From: pgsql-hackers-ow...@postgresql.org > > > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Andres

Re: [HACKERS] Parallel Append implementation

2017-04-07 Thread Andres Freund
Hi, On 2017-04-07 11:44:39 +0530, Amit Khandekar wrote: > On 6 April 2017 at 07:33, Andres Freund <and...@anarazel.de> wrote: > > On 2017-04-05 14:52:38 +0530, Amit Khandekar wrote: > >> This is what the earlier versions of my patch had done : just add up > >>

Re: [HACKERS] Faster methods for getting SPI results (460% improvement)

2017-04-06 Thread Andres Freund
On 2017-04-07 00:11:59 -0400, Tom Lane wrote: > Jim Nasby writes: > > On 4/6/17 8:13 PM, Tom Lane wrote: > >> Given Peter's objections, I don't think this is getting into v10 anyway, > >> so we might as well take a bit more time and do it right. > > > Well, Peter's

Re: [HACKERS] src/interfaces/libpq shipping nmake-related Makefiles

2017-04-06 Thread Andres Freund
On 2017-04-07 13:07:59 +0900, Michael Paquier wrote: > On Fri, Apr 7, 2017 at 1:01 PM, Tom Lane wrote: > > Still, it's not very clear why we need to cater for building just libpq > > rather than the whole distribution, and a user of win32.mak presumably > > has the option to

Re: [HACKERS] Faster methods for getting SPI results (460% improvement)

2017-04-06 Thread Andres Freund
On 2017-04-06 21:06:59 -0700, Jim Nasby wrote: > On 4/6/17 9:04 PM, Andres Freund wrote: > > On 2017-04-06 09:14:43 -0700, Jim Nasby wrote: > > > On 4/6/17 9:04 AM, Peter Eisentraut wrote: > > > > On 4/6/17 03:50, Craig Ringer wrote: > > > > > But

Re: [HACKERS] src/interfaces/libpq shipping nmake-related Makefiles

2017-04-06 Thread Andres Freund
On 2017-04-07 00:01:01 -0400, Tom Lane wrote: > Still, it's not very clear why we need to cater for building just libpq > rather than the whole distribution, and a user of win32.mak presumably > has the option to do the latter. I think the raison d'etre for that infrastructure primarily comes

Re: [HACKERS] Faster methods for getting SPI results (460% improvement)

2017-04-06 Thread Andres Freund
On 2017-04-06 09:14:43 -0700, Jim Nasby wrote: > On 4/6/17 9:04 AM, Peter Eisentraut wrote: > > On 4/6/17 03:50, Craig Ringer wrote: > > > But otherwise, pending docs changes, I think it's ready for committer. > > > > My opinion is still that this is ultimately the wrong approach. The > > right

[HACKERS] Time to change pg_regress diffs to unified by default?

2017-04-06 Thread Andres Freund
are really common in our regression suite. I personally have PG_REGRESS_DIFF_OPTS set to -dU10, but that doesn't help much analyzing buildfarm output. Therefore I propose changing the defaults in pg_regress.c. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@post

Re: [HACKERS] tuplesort_gettuple_common() and *should_free argument

2017-04-06 Thread Andres Freund
On 2017-03-13 18:14:07 -0700, Peter Geoghegan wrote: > On Wed, Jan 25, 2017 at 3:11 PM, Peter Geoghegan wrote: > > On Wed, Jan 25, 2017 at 3:11 PM, Tom Lane wrote: > >> Please. You might want to hit the existing ones with a separate patch, > >> but it

Re: [HACKERS] parallel explain analyze support not exercised

2017-04-06 Thread Andres Freund
On 2017-04-06 17:33:49 -0400, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > > I guess we'll have to see. My personal conclusion is that greater > > coverage of parallelism is worth some very minor config trouble for > > people doing installcheck agains

Re: [HACKERS] [COMMITTERS] pgsql: Increase parallel bitmap scan test coverage.

2017-04-06 Thread Andres Freund
On 2017-04-06 20:43:48 +, Andres Freund wrote: > Increase parallel bitmap scan test coverage. > > Author: Dilip Kumar > Discussion: > https://postgr.es/m/20170331184603.qcp7t4md5bzxb...@alap3.anarazel.de This turned the !linux buildfarm red, because it re

<    3   4   5   6   7   8   9   10   11   12   >