Re: [HACKERS] Optional message to user when terminating/cancelling backend

2018-08-11 Thread Pavel Stehule
2018-08-12 0:17 GMT+02:00 Daniel Gustafsson : > > On 6 Aug 2018, at 09:47, Heikki Linnakangas wrote: > > > > Has there been any consideration to encodings? > > Thats a good point, no =/ > > > What happens if the message contains non-ASCII characters, and the > sending backend is connected to

Tid scan improvements

2018-08-11 Thread Edmund Horner
Hello, To scratch an itch, I have been working on teaching TidScan how to do range queries, i.e. those using >=, <, BETWEEN, etc. This means we can write, for instance, SELECT * FROM t WHERE ctid >= '(1000,0)' AND ctid < '(2000,0)'; instead of resorting to the old trick: SELECT * FROM

Re: Allowing printf("%m") only where it actually works

2018-08-11 Thread Tom Lane
I wrote: > At this point I'm inclined to push both of those patches so we can > see what the buildfarm makes of them. So I did that, and while not all of the buildfarm has reported in, enough of it has that I think we can draw conclusions. The only member that's showing any new warnings, AFAICT,

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2018-08-11 Thread Alexander Korotkov
Hi! On Fri, Aug 10, 2018 at 5:07 PM Alexander Korotkov wrote: > On Thu, Aug 9, 2018 at 11:26 PM Ivan Kartyshov > wrote: > > Alexander Korotkov писал 2018-06-20 20:54: > > > Thinking about that more I found that adding vacuum mark as an extra > > > argument to LockAcquireExtended is also wrong.

Re: NetBSD vs libxml2

2018-08-11 Thread Andrew Dunstan
On 08/11/2018 01:18 PM, Tom Lane wrote: In a moment of idle curiosity, I tried to build PG --with-libxml on NetBSD-current (well, mostly current, from May or so). The configure script blew up, complaining that it couldn't execute a test program. Investigation showed that xml2-config reports

Re: [HACKERS] Optional message to user when terminating/cancelling backend

2018-08-11 Thread Daniel Gustafsson
> On 6 Aug 2018, at 09:47, Heikki Linnakangas wrote: > > Has there been any consideration to encodings? Thats a good point, no =/ > What happens if the message contains non-ASCII characters, and the sending > backend is connected to database that uses a different encoding than the > backend

Re: Facility for detecting insecure object naming

2018-08-11 Thread Noah Misch
On Sat, Aug 11, 2018 at 03:32:23PM -0500, Nico Williams wrote: > On Sat, Aug 11, 2018 at 12:47:05PM -0700, Noah Misch wrote: > > -- (3) "SET search_path" with today's code. > > -- > > -- Security and reliability considerations are the same as (2). Today, this > > -- reduces performance by

Re: NetBSD vs libxml2

2018-08-11 Thread Nico Williams
On Sat, Aug 11, 2018 at 01:12:09PM -0500, Nico Williams wrote: > I'm willing to write a patch after lunch. In ./configure.in this: > > for pgac_option in `$XML2_CONFIG --libs`; do > case $pgac_option in > -L*) LDFLAGS="$LDFLAGS $pgac_option";; > esac > done > >

Re: Facility for detecting insecure object naming

2018-08-11 Thread Nico Williams
On Wed, Aug 08, 2018 at 10:47:04AM -0400, Tom Lane wrote: > Isaac Morland writes: > > While I'm asking, does anybody know why this isn't the default, especially > > for SECURITY DEFINER functions? > > It might fix some subset of security issues, but I think that changing > the default behavior

Re: Facility for detecting insecure object naming

2018-08-11 Thread Nico Williams
On Sat, Aug 11, 2018 at 12:47:05PM -0700, Noah Misch wrote: > -- (3) "SET search_path" with today's code. > -- > -- Security and reliability considerations are the same as (2). Today, this > -- reduces performance by suppressing optimizations like inlining. Out of curiosity, why does this

Re: Facility for detecting insecure object naming

2018-08-11 Thread Noah Misch
On Wed, Aug 08, 2018 at 09:58:38AM -0400, Tom Lane wrote: > I'm not sold on #2 either. That path leads to, for example, > s/=/OPERATOR(pg_catalog.=)/g everywhere, which is utterly catastrophic > to both readability and portability of your SQL code. We *must* find > a way to do better, not tell

Re: [sqlsmith] ERROR: plan should not reference subplan's variable

2018-08-11 Thread Tom Lane
Andreas Seltenreich writes: > sqlsmith caused another internal error while testing REL_11_STABLE at > 1b9d1b08fe. The query below on the regression DB yields "plan should > not reference subplan's variable" for me. Thanks for the report. Seems to be wrong order of operations in

Re: NetBSD vs libxml2

2018-08-11 Thread Nico Williams
On Sat, Aug 11, 2018 at 01:18:26PM -0400, Tom Lane wrote: > In a moment of idle curiosity, I tried to build PG --with-libxml > on NetBSD-current (well, mostly current, from May or so). > The configure script blew up, complaining that it couldn't execute > a test program. Investigation showed that

NetBSD vs libxml2

2018-08-11 Thread Tom Lane
In a moment of idle curiosity, I tried to build PG --with-libxml on NetBSD-current (well, mostly current, from May or so). The configure script blew up, complaining that it couldn't execute a test program. Investigation showed that xml2-config reports this: $ xml2-config --libs

[sqlsmith] ERROR: plan should not reference subplan's variable

2018-08-11 Thread Andreas Seltenreich
Hi, sqlsmith caused another internal error while testing REL_11_STABLE at 1b9d1b08fe. The query below on the regression DB yields "plan should not reference subplan's variable" for me. regards, Andreas delete from public.prt1_l where EXISTS ( select from public.xmltest as ref_10 ,

Re: libpq connection timeout mismanagement

2018-08-11 Thread Fabien COELHO
Hello Tom, The patch that taught libpq about allowing multiple target hosts modified connectDBComplete() with the intent of making the connect_timeout (if specified) apply per-host, not to the complete connection attempt. It did not do a very good job though, because the timeout only gets

Re: logical decoding / rewrite map vs. maxAllocatedDescs

2018-08-11 Thread Tomas Vondra
On 08/11/2018 04:15 PM, Tom Lane wrote: > Tomas Vondra writes: On 08/09/2018 07:47 PM, Alvaro Herrera wrote: > Actually, it seems to me that ApplyLogicalMappingFile is just leaking > the file descriptor for no good reason. > >> I think the fix can be as simple as attached ... I'm

Re: buildfarm: could not read block 3 in file "base/16384/2662": read only 0 of 8192 bytes

2018-08-11 Thread Tomas Vondra
On 08/11/2018 04:08 PM, Andres Freund wrote: > Hi, > > On 2018-08-11 15:40:19 +0200, Tomas Vondra wrote: >> For the record, I can actually reproduce this on 9.6 (haven't tried >> older releases, but I suspect it's there too). Instead of using the >> failing subscription, I've used another pgbench

Re: logical decoding / rewrite map vs. maxAllocatedDescs

2018-08-11 Thread Tom Lane
Tomas Vondra writes: >>> On 08/09/2018 07:47 PM, Alvaro Herrera wrote: Actually, it seems to me that ApplyLogicalMappingFile is just leaking the file descriptor for no good reason. > I think the fix can be as simple as attached ... I'm mostly afk for the > weekend, so I'll commit &

Re: buildfarm: could not read block 3 in file "base/16384/2662": read only 0 of 8192 bytes

2018-08-11 Thread Andres Freund
Hi, On 2018-08-11 15:40:19 +0200, Tomas Vondra wrote: > For the record, I can actually reproduce this on 9.6 (haven't tried > older releases, but I suspect it's there too). Instead of using the > failing subscription, I've used another pgbench script doing this: > SET statement_timeout = 5; >

Re: logical decoding / rewrite map vs. maxAllocatedDescs

2018-08-11 Thread Tomas Vondra
On 08/10/2018 11:13 PM, Andres Freund wrote: > On 2018-08-10 22:57:57 +0200, Tomas Vondra wrote: >> >> >> On 08/09/2018 07:47 PM, Alvaro Herrera wrote: >>> On 2018-Aug-09, Tomas Vondra wrote: >>> I suppose there are reasons why it's done this way, and admittedly the test that happens to

Re: buildfarm: could not read block 3 in file "base/16384/2662": read only 0 of 8192 bytes

2018-08-11 Thread Tomas Vondra
On 08/11/2018 03:16 PM, Tomas Vondra wrote: > On 08/11/2018 05:02 AM, Tom Lane wrote: >> Peter Geoghegan writes: >>> I'm concerned that this item has the potential to delay the release, >>> since, as you said, we're back to the drawing board. >> >> Me too. I will absolutely not vote to release

Re: FailedAssertion on partprune

2018-08-11 Thread David Rowley
On 9 August 2018 at 15:33, Tom Lane wrote: > Nonetheless, it's a damfool query plan, because we'll be going through > parallel worker startup/shutdown overhead 4 times for no benefit. > We really should put in some kind of logic to force those sibling > Gathers to be aggregated into one, or else

Re: buildfarm: could not read block 3 in file "base/16384/2662": read only 0 of 8192 bytes

2018-08-11 Thread Tomas Vondra
On 08/11/2018 05:02 AM, Tom Lane wrote: > Peter Geoghegan writes: >> I'm concerned that this item has the potential to delay the release, >> since, as you said, we're back to the drawing board. > > Me too. I will absolutely not vote to release 11.0 before we've > solved this ... > Not sure. I

Re: logical decoding / rewrite map vs. maxAllocatedDescs

2018-08-11 Thread Tomas Vondra
On 08/11/2018 09:46 AM, Andres Freund wrote: > On 2018-08-11 00:34:15 -0700, Andres Freund wrote: >> I've run this numerous times now, and I haven't triggered it yet :/ > > Heh, I hit it literally seconds after hitting send: > > 2018-08-11 00:35:52.804 PDT [2196][9/14864] LOG: automatic

Re: logical decoding / rewrite map vs. maxAllocatedDescs

2018-08-11 Thread Tomas Vondra
On 08/11/2018 09:34 AM, Andres Freund wrote: > Hi, > > On 2018-08-11 01:55:43 +0200, Tomas Vondra wrote: >> On 08/10/2018 11:59 PM, Tomas Vondra wrote: >>> >>> ... >>> >>> I suspect there's some other ingredient, e.g. some manipulation with the >>> subscription. Or maybe it's not needed at all

Re: libpq should not look up all host addresses at once

2018-08-11 Thread Fabien COELHO
* when the password is required, there is no way to know for which host/ip it is: sh> psql "host=127.0.0.17,local2.coelho.net,local.coelho.net" Password for user fabien: In the same vein on a wrong password: sh> psql "host=no-such-host,local2.coelho.net,local3.coelho.net" Password for

Re: libpq should not look up all host addresses at once

2018-08-11 Thread Fabien COELHO
Hello Tom, [...] So I think what this code should do is (1) look up each hostname as it needs it, not all at once, and (2) proceed on to the next hostname if it gets a DNS lookup failure, not fail the whole connection attempt immediately. As attached. A quick test, and very quick glance

Re: BUG #15182: Canceling authentication due to timeout aka Denial of Service Attack

2018-08-11 Thread Michael Paquier
Hi Nathan, On Fri, Jul 27, 2018 at 02:40:42PM +, Bossart, Nathan wrote: > I think I'm essentially suggesting what you have in 0002 but without > the new RangeVarGetRelidExtended() callback. I've attached a modified > version of 0002 that seems to fix the originally reported issue. (I >

pgbench - doCustom cleanup

2018-08-11 Thread Fabien COELHO
Hello pgdev, This patch rework & clarifies pgbench internal state machine. It was indirectly triggered by Heikki review of pgbench tap tests (https://commitfest.postgresql.org/19/1306/), and by Marina's patch about tx retry on some errors (https://commitfest.postgresql.org/19/1645/) which I

Re: logical decoding / rewrite map vs. maxAllocatedDescs

2018-08-11 Thread Andres Freund
Hi, On 2018-08-11 00:46:25 -0700, Andres Freund wrote: > Below you can find the bt full showing a bunch of nested invalidations. > Looking. Hm. I wonder if the attached fixes the issue for you. If it's this I don't understand why this doesn't occur on older branches... I've not yet been able

Re: logical decoding / rewrite map vs. maxAllocatedDescs

2018-08-11 Thread Andres Freund
On 2018-08-11 00:34:15 -0700, Andres Freund wrote: > I've run this numerous times now, and I haven't triggered it yet :/ Heh, I hit it literally seconds after hitting send: 2018-08-11 00:35:52.804 PDT [2196][9/14864] LOG: automatic analyze of table "postgres.pg_catalog.pg_depend" system usage:

Re: logical decoding / rewrite map vs. maxAllocatedDescs

2018-08-11 Thread Andres Freund
Hi, On 2018-08-11 01:55:43 +0200, Tomas Vondra wrote: > On 08/10/2018 11:59 PM, Tomas Vondra wrote: > > > > ... > > > > I suspect there's some other ingredient, e.g. some manipulation with the > > subscription. Or maybe it's not needed at all and I'm just imagining things. > > > > Indeed, the

Re: logical decoding / rewrite map vs. maxAllocatedDescs

2018-08-11 Thread Peter Geoghegan
On Sat, Aug 11, 2018 at 12:06 AM, Andres Freund wrote: > To the point that I wonder if we shouldn't just change the ERROR into a > PANIC on master (but not REL_11_STABLE), so the buildfarm gives us > feedback. I don't think the problem can fundamentally be related to > subscriptions, given the

Re: logical decoding / rewrite map vs. maxAllocatedDescs

2018-08-11 Thread Andres Freund
Hi, On 2018-08-11 01:55:43 +0200, Tomas Vondra wrote: > On 08/10/2018 11:59 PM, Tomas Vondra wrote: > > > > ... > > > > I suspect there's some other ingredient, e.g. some manipulation with the > > subscription. Or maybe it's not needed at all and I'm just imagining things. > > > > Indeed, the

Re: csv format for psql

2018-08-11 Thread Pavel Stehule
2018-08-10 12:25 GMT+02:00 Daniel Verite : > Pavel Stehule wrote: > > > > On the whole I'm inclined to resubmit the patch with > > > fieldsep_csv and some minor changes based on the rest > > > of the discussion. > > > > > > > +1 > > PFA an updated version. > Usage from the command line: >