Re: [HACKERS] SCRAM authentication, take three

2017-04-12 Thread Noah Misch
On Tue, Apr 11, 2017 at 08:10:23AM +0300, Heikki Linnakangas wrote: > On 04/11/2017 04:52 AM, Peter Eisentraut wrote: > >On 4/10/17 04:27, Heikki Linnakangas wrote: > >>One thing to consider is that we just made the decision that "md5" > >>actually means "md5 or scram-sha-256". Extrapolating from

Re: [HACKERS] snapbuild woes

2017-04-12 Thread Noah Misch
On Sat, Apr 08, 2017 at 07:30:59AM -0700, Andres Freund wrote: > On 2017-04-08 16:29:10 +0200, Erik Rijkers wrote: > > On 2017-04-08 15:56, Andres Freund wrote: > > > On 2017-04-08 09:51:39 -0400, David Steele wrote: > > > > On 3/2/17 7:54 PM, Petr Jelinek wrote: > > > > > > > > > > Yes the copy

Re: [HACKERS] Foreign Join pushdowns not working properly for outer joins

2017-04-12 Thread David Rowley
On 10 March 2017 at 17:33, Ashutosh Bapat wrote: > Yes and I also forgot to update the function prologue to refer to the > fpinfo_o/i instead of inner and outer relations. Attached patch > corrects it. Hi Ashutosh, This seems to have conflicted with 28b04787. Do

Re: [HACKERS] Supporting huge pages on Windows

2017-04-12 Thread Tsunakawa, Takayuki
Hello, Magnus cc: Andres From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Magnus Hagander > I think what you'd need to do is enumerate what privileges the user has > *before* calling CreateRestrictedToken(), using GetTokenInformation(). > And

[HACKERS] Tab completion support for ALTER SUBSCRIPTION REFRESH PUBLICATION

2017-04-12 Thread Masahiko Sawada
Hi, I've attached a patch for $subject. Please check it. Regards, -- Masahiko Sawada NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center fix_tab_completion_for_alter_sub.patch Description: Binary data -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Some thoughts about SCRAM implementation

2017-04-12 Thread Heikki Linnakangas
On 04/12/2017 11:22 AM, Magnus Hagander wrote: On Wed, Apr 12, 2017 at 3:25 AM, Bruce Momjian wrote: And which enterprises are using SSL without certificates? And I thought channel binding required certificates anyway, e.g.:

Re: [HACKERS] Foreign Join pushdowns not working properly for outer joins

2017-04-12 Thread Ashutosh Bapat
On Wed, Apr 12, 2017 at 12:18 PM, David Rowley wrote: > On 10 March 2017 at 17:33, Ashutosh Bapat > wrote: >> Yes and I also forgot to update the function prologue to refer to the >> fpinfo_o/i instead of inner and outer relations.

Re: [HACKERS] pg_dump emits ALTER TABLE ONLY partitioned_table

2017-04-12 Thread Amit Langote
Hi Stephen, On 2017/04/11 22:12, Stephen Frost wrote: > Amit, > > * Amit Langote (langote_amit...@lab.ntt.co.jp) wrote: >> On 2017/04/11 0:26, Robert Haas wrote: >>> Children can have constraints (including NOT NULL constraints) which >>> parents lack, and can have a different column order, but

Re: [HACKERS] Some thoughts about SCRAM implementation

2017-04-12 Thread Magnus Hagander
On Wed, Apr 12, 2017 at 11:13 AM, Heikki Linnakangas wrote: > On 04/12/2017 11:22 AM, Magnus Hagander wrote: > >> On Wed, Apr 12, 2017 at 3:25 AM, Bruce Momjian wrote: >> >> And which enterprises are using SSL without certificates? And I thought >>> channel

Re: [HACKERS] index-only count(*) for indexes supporting bitmap scans

2017-04-12 Thread Andrew Gierth
> "Alexander" == Alexander Kuzmenkov writes: Alexander> Structurally, the patch consists of two major parts: a Alexander> specialized executor node Why? It strikes me that the significant fact here is not that we're doing count(*), but that we don't need any

Re: [HACKERS] index-only count(*) for indexes supporting bitmap scans

2017-04-12 Thread Alexander Korotkov
On Wed, Apr 12, 2017 at 12:40 AM, Alexander Korotkov < a.korot...@postgrespro.ru> wrote: > On Tue, Apr 11, 2017 at 8:24 PM, Alexander Kuzmenkov < > a.kuzmen...@postgrespro.ru> wrote: > >> I would like to propose a patch that speeds up the queries of the form >> 'select >> count(*) ... where ...',

Re: [HACKERS] Vacuum full stats reporting

2017-04-12 Thread Robert Haas
On Mon, Apr 10, 2017 at 4:36 AM, Magnus Hagander wrote: > Right now, VACUUM FULL are not reported in pgstat. That seems bad:ish. I can > see two reasonable ways to proceed: > > 1. Start reporting VACUUM FULL as regular vacuums, so they count up > vacuum_count and last_vacuum

Re: [HACKERS] TAP tests take a long time

2017-04-12 Thread Amit Kapila
On Tue, Apr 11, 2017 at 9:38 PM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Apr 11, 2017 at 11:32 AM, Andrew Dunstan >> wrote: >>> This buildfarm run as you can see takes 33m32s, and the Tap tests take a >>>

Re: [HACKERS] Foreign Join pushdowns not working properly for outer joins

2017-04-12 Thread David Rowley
On 12 April 2017 at 21:45, Ashutosh Bapat wrote: > On Wed, Apr 12, 2017 at 12:18 PM, David Rowley > wrote: >> On 10 March 2017 at 17:33, Ashutosh Bapat >> wrote: >>> Yes and I also forgot to update

Re: [HACKERS] Cutting initdb's runtime (Perl question embedded)

2017-04-12 Thread Andreas Karlsson
On 04/12/2017 04:12 PM, Tom Lane wrote: 1. The best thing would still be to make genbki.pl do the conversion, and write numeric OIDs into postgres.bki. The core stumbling block here seems to be that for most catalogs, Catalog.pm and genbki.pl never really break down a DATA line into fields ---

Re: [HACKERS] Logical replication and inheritance

2017-04-12 Thread Robert Haas
On Wed, Apr 5, 2017 at 8:25 AM, Peter Eisentraut wrote: > After thinking about it some more, I think the behavior we want would be > that changes to inheritance would reflect in the publication membership. > So if you have a partitioned table, adding more

Re: [HACKERS] TAP tests take a long time

2017-04-12 Thread Tom Lane
Mithun Cy writes: > I have tried to optimize the tests maintaining the same coverage we were > able to get with it. This patch looks good to me: on my workstation, it reduces the total runtime of the parallel regression tests from ~20.5 to ~16.5 seconds. I concur that

Re: [HACKERS] Tab completion support for ALTER SUBSCRIPTION REFRESH PUBLICATION

2017-04-12 Thread Fujii Masao
On Wed, Apr 12, 2017 at 5:12 PM, Masahiko Sawada wrote: > Hi, > > I've attached a patch for $subject. Please check it. + COMPLETE_WITH_LIST8("WITH", "CONNECTION", "SET PUBLICATION", "ENABLE", + "DISABLE", "OWNER TO", "RENAME TO", "REFRESH PUBLICATION WITH"); "WITH" should

Re: [HACKERS] Undefined psql variables

2017-04-12 Thread Robert Haas
On Sun, Apr 2, 2017 at 3:56 PM, Tom Lane wrote: > So my view of this is that "send the expression to the server" ought > to be just one option for \if, not the only way to do it. I heartily agree. There should be some kind of client-side expression language, and one thing it

Re: [HACKERS] GCC 7 warnings

2017-04-12 Thread Peter Eisentraut
On 4/12/17 00:12, Tom Lane wrote: > The change in setup_formatted_log_time seems a bit weird: > > - charmsbuf[8]; > + charmsbuf[10]; > > The associated call is > > sprintf(msbuf, ".%03d", (int) (saved_timeval.tv_usec / 1000)); > > Now a human can see that

Re: [HACKERS] Interval for launching the table sync worker

2017-04-12 Thread Peter Eisentraut
On 4/12/17 00:48, Masahiko Sawada wrote: > On Wed, Apr 12, 2017 at 1:28 PM, Peter Eisentraut >> Perhaps instead of a global last_start_time, we store a per relation >> last_start_time in SubscriptionRelState? > > I was thinking the same. But a problem is that the list of > SubscriptionRelState is

Re: [HACKERS] TAP tests take a long time

2017-04-12 Thread Mithun Cy
On Wed, Apr 12, 2017 at 6:24 PM, Amit Kapila wrote: > > I have looked into the tests and I think we can do some optimization > without losing much on code coverage. First is we are doing both > Vacuum Full and Vacuum on hash_split_heap in the same test after > executing

Re: [HACKERS] logical replication apply to run with sync commit off by default

2017-04-12 Thread Peter Eisentraut
On 4/9/17 22:17, Noah Misch wrote: > [Action required within three days. This is a generic notification.] I'm expecting Petr to post an updated patch by the end of this week. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training

Re: [HACKERS] Some thoughts about SCRAM implementation

2017-04-12 Thread Craig Ringer
On 12 Apr. 2017 17:27, "Magnus Hagander" wrote: On Wed, Apr 12, 2017 at 11:13 AM, Heikki Linnakangas wrote: > On 04/12/2017 11:22 AM, Magnus Hagander wrote: > >> On Wed, Apr 12, 2017 at 3:25 AM, Bruce Momjian wrote: >> >> And which

Re: [HACKERS] Logical replication and inheritance

2017-04-12 Thread Peter Eisentraut
On 4/9/17 22:16, Noah Misch wrote: > On Wed, Apr 05, 2017 at 08:25:56AM -0400, Peter Eisentraut wrote: >> After thinking about it some more, I think the behavior we want would be >> that changes to inheritance would reflect in the publication membership. >> So if you have a partitioned table,

Re: [HACKERS] snapbuild woes

2017-04-12 Thread Peter Eisentraut
On 4/12/17 02:31, Noah Misch wrote: >>> But I hope you mean to commit these snapbuild patches before the postgres 10 >>> release? As far as I know, logical replication is still very broken without >>> them (or at least some of that set of 5 patches - I don't know which ones >>> are essential and

[HACKERS] the need to finish

2017-04-12 Thread Erik Rijkers
Logical replication emits logmessages like these: DETAIL: 90 transactions need to finish. DETAIL: 87 transactions need to finish. DETAIL: 70 transactions need to finish. Could we get rid of that 'need'? It strikes me as a bit off; something that people would say but not a mechanical

Re: [HACKERS] the need to finish

2017-04-12 Thread Tom Lane
Erik Rijkers writes: > Logical replication emits logmessages like these: > DETAIL: 90 transactions need to finish. > DETAIL: 87 transactions need to finish. > DETAIL: 70 transactions need to finish. > Could we get rid of that 'need'? It strikes me as a bit off; something >

Re: [HACKERS] Some thoughts about SCRAM implementation

2017-04-12 Thread Bruce Momjian
On Wed, Apr 12, 2017 at 12:13:03PM +0300, Heikki Linnakangas wrote: > >That said, I stand by my comment that I don't think it's the enterprises > >that need or want the channel binding. If they care about it, they have > >already put certificate validation in place, and it won't buy them anything.

Re: [HACKERS] Why does logical replication launcher set application_name?

2017-04-12 Thread Craig Ringer
On 12 April 2017 at 13:34, Kuntal Ghosh wrote: > For backend_type=background worker, application_name shows the name of > the background worker (BackgroundWorker->bgw_name). I think we need > some way to distinguish among different background workers. But, >

Re: [HACKERS] TAP tests take a long time

2017-04-12 Thread Craig Ringer
On 12 April 2017 at 13:27, Craig Ringer wrote: > On 12 April 2017 at 08:22, Michael Paquier wrote: >> On Wed, Apr 12, 2017 at 9:12 AM, Craig Ringer >> wrote: >>> Well, you can get a lot of information on

Re: [HACKERS] Add pgstathashindex() to get hash index table statistics.

2017-04-12 Thread Noah Misch
On Thu, Apr 06, 2017 at 10:45:26AM +0530, Ashutosh Sharma wrote: > >> Based on the earlier discussions, I have prepared a patch that would > >> allow pgstathashindex() to show the number of unused pages in hash > >> index. Please find the attached patch for the same. Thanks. > > > > My idea is

Re: [HACKERS] Some thoughts about SCRAM implementation

2017-04-12 Thread Magnus Hagander
On Wed, Apr 12, 2017 at 3:25 AM, Bruce Momjian wrote: > On Tue, Apr 11, 2017 at 08:58:30PM -0400, Stephen Frost wrote: > > > > But just a bit more is needed to make it really a big > announcement and > > > > provide real value to (I guess, mostly but very interesting) >

Re: [HACKERS] Merge join for GiST

2017-04-12 Thread Sergey Mirvoda
Thank you, Alexander! This is definitely the example we are looking for! Hat tip to Dmitry especially for this commit https://github.com/akorotkov/pgsphere/commit/971d2c5d61f17774a6d8d137ca3ad87e2883048f Regards, Sergey Mirvoda On Tue, Apr 11, 2017 at 2:17 PM, Alexander Korotkov <

Re: [HACKERS] [sqlsmith] ERROR: badly formatted node string "RESTRICTINFO...

2017-04-12 Thread Amit Kapila
On Wed, Apr 12, 2017 at 12:40 AM, Tom Lane wrote: > Amit Kapila writes: > >> However, the worker will >> never execute such a plan as we don't generate a plan where unsafe >> sublan/initplan is referenced in the node passed to the worker. If we >>

Re: [HACKERS] logical rep worker for table sync can start and stop very frequently unexpectedly

2017-04-12 Thread Fujii Masao
On Wed, Apr 12, 2017 at 2:40 AM, Masahiko Sawada wrote: > On Wed, Apr 12, 2017 at 2:05 AM, Fujii Masao wrote: >> Hi, >> >> I observed $subject when I ran the following steps. >> >> 1. start the publisher server >> 2. start the subscriber server >> 3.

[HACKERS] pg_statistic_ext.staenabled might not be the best column name

2017-04-12 Thread David Rowley
I'd been thinking that staenabled is not the most suitable column name for storing the types of statistics that are defined for the extended statistics. For me, this indicates that something can be disabled, but there's no syntax for that, and even if there was, if we were to enable/disable the

Re: [HACKERS] my open items vs. my vacation

2017-04-12 Thread Jan de Visser
On Tuesday, April 11, 2017 1:36:44 PM EDT Robert Haas wrote: > I apologize for any disruption this may cause, but I'm hopeful that it > won't be too bad. Spoken like a true American - apologizing for taking vacation :-) Enjoy your time off. You probably deserved it, and more than the week you're

Re: [pgsql-www] [HACKERS] Small issue in online devel documentation build

2017-04-12 Thread Bruce Momjian
On Wed, Apr 12, 2017 at 01:31:51PM +0200, Magnus Hagander wrote: > I think that only leaves the change to the javascript code that Bruce sent. > Let's see if we can figure out a way to do that one without requiring > javascript, but after that we have covered all listed issues I think? Well, we

[HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-04-12 Thread Fujii Masao
Hi, When I shut down the publisher while I repeated creating and dropping the subscription in the subscriber, the publisher emitted the following PANIC error during shutdown checkpoint. PANIC: concurrent transaction log activity while database system is shutting down The cause of this problem

Re: [HACKERS] index-only count(*) for indexes supporting bitmap scans

2017-04-12 Thread Alexander Kuzmenkov
On 12.04.2017 15:04, Tom Lane wrote: Andrew Gierth writes: "Alexander" == Alexander Kuzmenkov writes: Alexander> Structurally, the patch consists of two major parts: a Alexander> specialized executor node Why? It strikes me that the

[HACKERS] Cutting initdb's runtime (Perl question embedded)

2017-04-12 Thread Tom Lane
Andres mentioned, and I've confirmed locally, that a large chunk of initdb's runtime goes into regprocin's brute-force lookups of function OIDs from function names. The recent discussion about cutting TAP test time prompted me to look into that question again. We had had some grand plans for

Re: [HACKERS] index-only count(*) for indexes supporting bitmap scans

2017-04-12 Thread Tom Lane
Alexander Kuzmenkov writes: > With planner, the changes are more complex. Two things must be done > there. First, when the tlist is empty, we must use a different cost > function for bitmap heap scan, because the heap access pattern is > different. Second,

Re: [HACKERS] [PATCH v1] Add and report the new "in_hot_standby" GUC pseudo-variable.

2017-04-12 Thread Magnus Hagander
On Wed, Apr 12, 2017 at 2:36 PM, Robert Haas wrote: > On Tue, Apr 11, 2017 at 4:05 AM, Magnus Hagander > wrote: > > On Tue, Apr 11, 2017 at 3:26 AM, Michael Paquier < > michael.paqu...@gmail.com> wrote: > >> On Mon, Apr 10, 2017 at 5:47 PM, Magnus

Re: [HACKERS] error handling in RegisterBackgroundWorker

2017-04-12 Thread Robert Haas
On Tue, Apr 11, 2017 at 10:13 PM, Noah Misch wrote: > On Tue, Apr 11, 2017 at 11:33:34AM -0400, Tom Lane wrote: >> Peter Eisentraut writes: >> > I think there is no clear agreement here, and no historically consistent >> > behavior. I'm

Re: [HACKERS] snapbuild woes

2017-04-12 Thread Simon Riggs
On 3 March 2017 at 00:30, Petr Jelinek wrote: >> 0004 - Changes handling of the xl_running_xacts in initial snapshot >> build to what I wrote above and removes the extra locking from >> LogStandbySnapshot introduced by logical decoding. This seems OK and unlikely

Re: [HACKERS] the need to finish

2017-04-12 Thread Simon Riggs
On 12 April 2017 at 16:26, Tom Lane wrote: > Erik Rijkers writes: >> Logical replication emits logmessages like these: >> DETAIL: 90 transactions need to finish. >> DETAIL: 87 transactions need to finish. >> DETAIL: 70 transactions need to finish. > >>

Re: [HACKERS] Add pgstathashindex() to get hash index table statistics.

2017-04-12 Thread Robert Haas
On Wed, Apr 12, 2017 at 2:28 AM, Noah Misch wrote: > [Action required within three days. This is a generic notification.] > > The above-described topic is currently a PostgreSQL 10 open item. Robert, > since you committed the patch believed to have created it, you own this

Re: [HACKERS] Some thoughts about SCRAM implementation

2017-04-12 Thread Stephen Frost
Tom, all, * Tom Lane (t...@sss.pgh.pa.us) wrote: > ... which the user can't tell apart from having fat-fingered the password, > I suppose? Doesn't sound terribly friendly. A report of a certificate > mismatch is far more likely to lead people to realize there's a MITM. We might be able to

Re: [HACKERS] Partitioned tables and relfilenode

2017-04-12 Thread Robert Haas
On Tue, Apr 11, 2017 at 10:15 PM, Amit Langote wrote: > Alright. So I made it into two patches instead: 0001 fixes the bug that > validateCheckConstraint() tries to scan partitioned tables and 0002 makes > trying to convert a partitioned table to a view a user

Re: [HACKERS] Some thoughts about SCRAM implementation

2017-04-12 Thread Heikki Linnakangas
On 04/12/2017 06:26 PM, Bruce Momjian wrote: On Wed, Apr 12, 2017 at 12:13:03PM +0300, Heikki Linnakangas wrote: That said, I stand by my comment that I don't think it's the enterprises that need or want the channel binding. If they care about it, they have already put certificate validation in

Re: [HACKERS] Possible problem in Custom Scan API

2017-04-12 Thread Tom Lane
Dmitry Ivanov writes: >> Uh, no, construction of a custom plan node is entirely driven by the >> PlanCustomPath method as far as I can see. You're free to ignore what >> create_scan_plan did and insert your own tlist. > Are you sure? Even if it's true, this targetlist

Re: [HACKERS] Possible problem in Custom Scan API

2017-04-12 Thread Dmitry Ivanov
Tom Lane wrote: I'm coming around to the idea that it'd be better to disable physical tlists for custom scans. I've been thinking about this all along, and it seems that this is a decent decision. However, I've made a tiny bugfix patch which allows CustomScans to notify the core code that

Re: [HACKERS] Some thoughts about SCRAM implementation

2017-04-12 Thread Robert Haas
On Tue, Apr 11, 2017 at 9:20 AM, Álvaro Hernández Tortosa wrote: > LOL. Do you really want a half-baked Java programmer writing a patch in > C for PostgreSQL? I once tried that and Magnus said my code was Java code > that happened to compile with a C compiler ^_^ > >

Re: [HACKERS] GCC 7 warnings

2017-04-12 Thread Tom Lane
Peter Eisentraut writes: > On 4/12/17 00:12, Tom Lane wrote: >> Now a human can see that saved_timeval.tv_usec must be 0..99, so >> that the %d format item must always emit exactly 3 characters, which >> means that really 5 bytes would be enough. I wouldn't

Re: [HACKERS] Some thoughts about SCRAM implementation

2017-04-12 Thread Tom Lane
Heikki Linnakangas writes: > On 04/12/2017 06:26 PM, Bruce Momjian wrote: >> How does it do that? > Good question, crypto magic? I don't know the details, but the basic > idea is that you extract a blob of data that uniquely identifies the TLS > connection. Using some OpenSSL

Re: [HACKERS] index-only count(*) for indexes supporting bitmap scans

2017-04-12 Thread Tom Lane
Andrew Gierth writes: > "Alexander" == Alexander Kuzmenkov writes: > Alexander> Structurally, the patch consists of two major parts: a > Alexander> specialized executor node > Why? > It strikes me that the significant fact here is not

Re: [HACKERS] [PATCH v1] Add and report the new "in_hot_standby" GUC pseudo-variable.

2017-04-12 Thread Robert Haas
On Tue, Apr 11, 2017 at 4:05 AM, Magnus Hagander wrote: > On Tue, Apr 11, 2017 at 3:26 AM, Michael Paquier > wrote: >> On Mon, Apr 10, 2017 at 5:47 PM, Magnus Hagander >> wrote: >> > Based on that we seem to agree here,

Re: [HACKERS] Foreign Join pushdowns not working properly for outer joins

2017-04-12 Thread Ashutosh Bapat
Sorry, here's the right one. On Wed, Apr 12, 2017 at 6:27 PM, David Rowley wrote: > On 12 April 2017 at 21:45, Ashutosh Bapat > wrote: >> On Wed, Apr 12, 2017 at 12:18 PM, David Rowley >> wrote: >>> On

Re: [HACKERS] Reversed sync check in pg_receivewal

2017-04-12 Thread Magnus Hagander
On Tue, Apr 11, 2017 at 6:51 PM, Tom Lane wrote: > Magnus Hagander writes: > > Something like the attached? > > Not sure about > > + * All methods that have a failure path will set errno on failure. > > Given that you've got a getlasterror method, I

[HACKERS] TAP: fix undefined var warnings in PostgresNode with timeout

2017-04-12 Thread Craig Ringer
If PostgresNode::psql (from the TAP framework) is called with a timeout set and a timed_out reference, it will attempt to do bitwise AND and bitshifts on the $ret value from IPC::Run, which is undef if the command timed out. This produces annoying errors in the logs. Fix attached. Should be

Re: [HACKERS] [sqlsmith] ERROR: badly formatted node string "RESTRICTINFO...

2017-04-12 Thread Tom Lane
Amit Kapila writes: > On Wed, Apr 12, 2017 at 12:40 AM, Tom Lane wrote: >> Anyone want to draft a patch for this? > Please find patch attached based on above discussion. Thanks, I'll look at this later today. regards, tom

Re: [pgsql-www] [HACKERS] Small issue in online devel documentation build

2017-04-12 Thread Magnus Hagander
On Tue, Apr 11, 2017 at 4:30 PM, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 4/11/17 08:49, Magnus Hagander wrote: > > At the risk of being proven wrong again, won't this affect tags in > > the old documentation as well? And if so, is that something we actually > > want? > >

Re: [HACKERS] TAP tests take a long time

2017-04-12 Thread Tom Lane
Craig Ringer writes: > With > make PROVE_FLAGS="--timer -j 9" check > I don't see much difference with/without caching initdb results - > saves about 4 seconds, from 74 to 70 seconds, but hard to tell with > the error margins. > So if we're going to do anything,

Re: [HACKERS] TAP tests take a long time

2017-04-12 Thread Andrew Dunstan
On 04/12/2017 01:27 AM, Craig Ringer wrote: > > BTW, I suggest adding --timer to our default PROVE_FLAGS, so we can > collect more data from the buildfarm on what takes a while. Sample > output: > I'll add that to the commandline the buildfarm uses in the upcoming release. cheers andrew --

Re: [HACKERS] [PATCH v1] Add and report the new "in_hot_standby" GUC pseudo-variable.

2017-04-12 Thread Magnus Hagander
On Tue, Apr 11, 2017 at 2:38 PM, Simon Riggs wrote: > On 11 April 2017 at 09:05, Magnus Hagander wrote: > > On Tue, Apr 11, 2017 at 3:26 AM, Michael Paquier < > michael.paqu...@gmail.com> > > wrote: > >> > >> On Mon, Apr 10, 2017 at 5:47 PM, Magnus

Re: [HACKERS] Adding support for Default partition in partitioning

2017-04-12 Thread Robert Haas
On Thu, Apr 6, 2017 at 1:17 AM, Rushabh Lathia wrote: > I like the idea about having DEFAULT partition for the range partition. With > the > way partition is designed it can have holes into range partition. I think > DEFAULT > for the range partition is a good idea,

[HACKERS] Inadequate parallel-safety check for SubPlans

2017-04-12 Thread Tom Lane
While poking at the question of parallel_safe marking for Plans, I noticed that max_parallel_hazard_walker() does this: /* We can push the subplans only if they are parallel-safe. */ else if (IsA(node, SubPlan)) return !((SubPlan *) node)->parallel_safe; This is 100% wrong. It's

Re: [HACKERS] Query fails when SRFs are part of FROM clause (Commit id: 69f4b9c85f)

2017-04-12 Thread Andres Freund
On 2017-04-11 17:42:42 -0400, Tom Lane wrote: > Now, that old behavior matches what you got in the RangeFunction case: > > regression96=# select * from int4_tbl, cast(case when f1>0 then > generate_series(1,2) else null end as int); > f1 | int4 > -+-- >0 |

Re: [HACKERS] Cutting initdb's runtime (Perl question embedded)

2017-04-12 Thread Andreas Karlsson
On 04/12/2017 05:00 PM, Andreas Karlsson wrote: Looked at this an option 1 seems simple enough if I am not missing something. I might hack something up later tonight. Either way I think this improvement can be done separately from the proposed replacement of the catalog header files. Trying to

Re: [HACKERS] Adding support for Default partition in partitioning

2017-04-12 Thread Robert Haas
On Thu, Apr 6, 2017 at 7:30 AM, Rahila Syed wrote: > Thanks a lot for testing and reporting this. Please find attached an updated > patch with the fix. The patch also contains a fix > regarding operator used at the time of creating expression as default > partition

Re: [HACKERS] [sqlsmith] ERROR: badly formatted node string "RESTRICTINFO...

2017-04-12 Thread Tom Lane
Amit Kapila writes: > On Wed, Apr 12, 2017 at 12:40 AM, Tom Lane wrote: >> Anyone want to draft a patch for this? > Please find patch attached based on above discussion. This patch seems fairly incomplete: you can't just whack around major data

Re: [HACKERS] Some thoughts about SCRAM implementation

2017-04-12 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> ... which the user can't tell apart from having fat-fingered the password, >> I suppose? Doesn't sound terribly friendly. A report of a certificate >> mismatch is far more likely to lead people to realize

Re: [HACKERS] Some thoughts about SCRAM implementation

2017-04-12 Thread Álvaro Hernández Tortosa
On 12/04/17 18:09, Tom Lane wrote: Heikki Linnakangas writes: On 04/12/2017 06:26 PM, Bruce Momjian wrote: How does it do that? Good question, crypto magic? I don't know the details, but the basic idea is that you extract a blob of data that uniquely identifies the TLS

Re: [HACKERS] GSOC'17 project introduction: Parallel COPY execution with errors handling

2017-04-12 Thread Nicolas Barbier
2017-04-11 Robert Haas : > There's a nasty trade-off here between XID consumption (and the > aggressive vacuums it eventually causes) and preserving performance in > the face of errors - e.g. if you make k = 100,000 you consume 100x > fewer XIDs than if you make k = 1000,

Re: [HACKERS] the need to finish

2017-04-12 Thread Robert Haas
On Wed, Apr 12, 2017 at 11:41 AM, Simon Riggs wrote: > On 12 April 2017 at 16:26, Tom Lane wrote: >> Erik Rijkers writes: >>> Logical replication emits logmessages like these: >>> DETAIL: 90 transactions need to finish. >>> I would

Re: [HACKERS] index-only count(*) for indexes supporting bitmap scans

2017-04-12 Thread Alexander Kuzmenkov
On 12.04.2017 17:24, Tom Lane wrote: TBH, I'm not sure you need to do any of that work. Have you got evidence that the planner will fail to choose the right plan regardless? I'm particularly unconvinced that choose_bitmap_and is a critical problem, because once you're into having to AND

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-04-12 Thread Robert Haas
On Tue, Apr 11, 2017 at 1:20 PM, Pavan Deolasee wrote: > I don't know why you say that regressions are not addressed. Here are a few > things I did to address the regressions/reviews/concerns, apart from fixing > all the bugs discovered, but please let me know if there

Re: [HACKERS] snapbuild woes

2017-04-12 Thread Andres Freund
On 2017-04-12 11:03:57 -0400, Peter Eisentraut wrote: > On 4/12/17 02:31, Noah Misch wrote: > >>> But I hope you mean to commit these snapbuild patches before the postgres > >>> 10 > >>> release? As far as I know, logical replication is still very broken > >>> without > >>> them (or at least

Re: [HACKERS] Some thoughts about SCRAM implementation

2017-04-12 Thread Álvaro Hernández Tortosa
On 12/04/17 18:38, Robert Haas wrote: On Tue, Apr 11, 2017 at 9:20 AM, Álvaro Hernández Tortosa wrote: LOL. Do you really want a half-baked Java programmer writing a patch in C for PostgreSQL? I once tried that and Magnus said my code was Java code that happened to

Re: [HACKERS] GSOC'17 project introduction: Parallel COPY execution with errors handling

2017-04-12 Thread Stas Kelvich
> On 12 Apr 2017, at 20:23, Robert Haas wrote: > > On Wed, Apr 12, 2017 at 1:18 PM, Nicolas Barbier > wrote: >> 2017-04-11 Robert Haas : >>> If the data quality is poor (say, 50% of lines have errors) it's >>> almost

Re: [HACKERS] GSOC'17 project introduction: Parallel COPY execution with errors handling

2017-04-12 Thread Robert Haas
On Wed, Apr 12, 2017 at 1:18 PM, Nicolas Barbier wrote: > 2017-04-11 Robert Haas : >> There's a nasty trade-off here between XID consumption (and the >> aggressive vacuums it eventually causes) and preserving performance in >> the face of errors -

Re: [HACKERS] index-only count(*) for indexes supporting bitmap scans

2017-04-12 Thread Alexander Kuzmenkov
On 12.04.2017 12:29, Alexander Korotkov wrote: That's a cool feature for FTS users! Please, register this patch to the next commitfest. I've added this to the 2017-07 commitfest: https://commitfest.postgresql.org/14/1117/ Also, what is planning overhead of this patch? That's worth

Re: [HACKERS] Letting the client choose the protocol to use during a SASL exchange

2017-04-12 Thread Heikki Linnakangas
On 04/11/2017 02:32 PM, Álvaro Hernández Tortosa wrote: So I still see your proposal more awkward and less clear, mixing things that are separate. But again, your choice :) So, here's my more full-fledged proposal. The first patch refactors libpq code, by moving the responsibility of

Re: [HACKERS] Cutting initdb's runtime (Perl question embedded)

2017-04-12 Thread Andres Freund
On 2017-04-12 10:12:47 -0400, Tom Lane wrote: > Andres mentioned, and I've confirmed locally, that a large chunk of > initdb's runtime goes into regprocin's brute-force lookups of function > OIDs from function names. The recent discussion about cutting TAP test > time prompted me to look into

Re: [HACKERS] Some thoughts about SCRAM implementation

2017-04-12 Thread Heikki Linnakangas
On 04/12/2017 08:38 PM, Álvaro Hernández Tortosa wrote: - Even though I don't really care about SCRAM, and without having any prior knowledge about SCRAM, I volunteered some time ago to study SCRAM, give a lightning talk about SCRAM and later write a client implementation for the jdbc driver.

Re: [HACKERS] pg_dump emits ALTER TABLE ONLY partitioned_table

2017-04-12 Thread Robert Haas
On Wed, Apr 12, 2017 at 6:29 AM, Amit Langote wrote: > Actually, p1 is a partitioned table, so the error. And I realize that > that's a wrong behavior. Currently the check is performed using only the > relkind, which is bogus. Specifying ONLY should cause an

Re: [HACKERS] Some thoughts about SCRAM implementation

2017-04-12 Thread Robert Haas
On Wed, Apr 12, 2017 at 2:09 PM, Heikki Linnakangas wrote: > That is very much appreciated! You writing a second implementation of the > client-side support (libpq being the first) is very, very helpful, to > validate that the protocol is sane, unambiguous, and adequately

Re: [HACKERS] Undefined psql variables

2017-04-12 Thread Pavel Stehule
2017-04-12 17:05 GMT+02:00 Robert Haas : > On Sun, Apr 2, 2017 at 3:56 PM, Tom Lane wrote: > > So my view of this is that "send the expression to the server" ought > > to be just one option for \if, not the only way to do it. > > I heartily agree.

Re: [HACKERS] Adding support for Default partition in partitioning

2017-04-12 Thread Robert Haas
On Tue, Apr 11, 2017 at 9:41 AM, Jeevan Ladhe wrote: > I have checked for NULLs too, and the default partition can be created even > when there are partitions for each TRUE, FALSE and NULL. > > Consider the example below: > > postgres=# CREATE TABLE list_partitioned

Re: [HACKERS] pg_dump emits ALTER TABLE ONLY partitioned_table

2017-04-12 Thread Robert Haas
On Wed, Apr 12, 2017 at 3:29 PM, Stephen Frost wrote: > I'm not following what you're getting at here. > > There's already a constraint on the table, and ALTER TABLE ONLY doesn't > say anything about what happens later on (certainly it doesn't make new > tables created with

Re: [HACKERS] Foreign Join pushdowns not working properly for outer joins

2017-04-12 Thread Peter Eisentraut
Is this patch considered ready for review as a backpatch candidate? -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] pg_upgrade vs extension upgrades

2017-04-12 Thread Robert Haas
On Wed, Apr 12, 2017 at 6:30 PM, Peter Eisentraut wrote: > On 4/10/17 11:30, Magnus Hagander wrote: >> After you've run pg_upgrade, you have to loop through all your databases >> and do an "ALTER EXTENSION abc UPDATE" once for each extension. >> >> Is there a

Re: [HACKERS] Re: Query fails when SRFs are part of FROM clause (Commit id: 69f4b9c85f)

2017-04-12 Thread Andres Freund
On 2017-04-05 09:39:37 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2017-04-05 02:47:55 -0400, Noah Misch wrote: > >> [Action required within three days. This is a generic notification.] > > > I've a very preliminary patch. I'd like to only start polishing it up > >

Re: [pgsql-www] [HACKERS] Small issue in online devel documentation build

2017-04-12 Thread Peter Eisentraut
On 4/12/17 09:50, Bruce Momjian wrote: > On Wed, Apr 12, 2017 at 01:31:51PM +0200, Magnus Hagander wrote: >> I think that only leaves the change to the javascript code that Bruce sent. >> Let's see if we can figure out a way to do that one without requiring >> javascript, but after that we have

Re: [HACKERS] pg_upgrade vs extension upgrades

2017-04-12 Thread Peter Eisentraut
On 4/12/17 18:59, Robert Haas wrote: > I do think there might be some value in a tool that looked for old > extensions and tried to update them, but I'm not sure it should be > pg_dump. This reminds me a bit of the problem of upgrading all collations after an upgrade. Perhaps we can find similar

Re: [HACKERS] pg_statistic_ext.staenabled might not be the best column name

2017-04-12 Thread Tomas Vondra
On 04/12/2017 03:36 PM, David Rowley wrote: "stakind" seems like a better name. I'd have personally gone with "statype" but pg_statistic already thinks stakind is better. +1 to stakind -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-04-12 Thread Peter Eisentraut
On 4/12/17 09:55, Fujii Masao wrote: > To fix this issue, we should terminate walsender for logical replication > before shutdown checkpoint starts. Of course walsender for physical > replication still needs to keep running until shutdown checkpoint ends, > though. Can we turn it into a kind of

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-04-12 Thread Peter Geoghegan
On Wed, Apr 12, 2017 at 10:12 AM, Robert Haas wrote: >> I may have missed something, but there is no intention to ignore known >> regressions/reviews. Of course, I don't think that every regression will be >> solvable, like if you run a CPU-bound workload, setting it up in

Re: [HACKERS] SUBSCRIPTIONS and pg_upgrade

2017-04-12 Thread Peter Eisentraut
On 4/11/17 23:41, Noah Misch wrote: > On Tue, Apr 11, 2017 at 11:21:24PM -0400, Peter Eisentraut wrote: >> On 4/9/17 22:16, Noah Misch wrote: >>> [Action required within three days. This is a generic notification.] >> >> Patches have been posted. Discussion is still going on a bit. > > By what

  1   2   >