Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Alvaro Hernandez
On 25/09/17 22:08, Jignesh Shah wrote: On Mon, Sep 25, 2017 at 11:37 AM, Joshua D. Drake > wrote: On 09/25/2017 11:31 AM, Alvaro Hernandez wrote: Whether or not they are included in a managed environment

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Alvaro Hernandez
On 25/09/17 22:13, Magnus Hagander wrote: On Mon, Sep 25, 2017 at 8:20 PM, Alvaro Hernandez > wrote: On 25/09/17 20:18, Andres Freund wrote: On 2017-09-24 13:36:56 +0300, Alvaro Hernandez wrote: However, if DMS uses it

Re: [HACKERS] moving some partitioning code to executor

2017-09-25 Thread Amit Langote
On 2017/09/14 16:13, Amit Langote wrote: > Hi. > > It seems to me that some of the code in partition.c is better placed > somewhere under the executor directory. There was even a suggestion > recently [1] to introduce a execPartition.c to house some code around > tuple-routing. > > IMO,

[HACKERS] Runtime Partition Pruning

2017-09-25 Thread Beena Emerson
I have been working on implementing the runtime partition pruning which would increase the performance of queries involving partitioned table to a great extent. PFA the POC which can be applied over Amit's patch for faster partition pruning [1] and Dilip's refactor patch [2] on commit

Re: [HACKERS] Setting pd_lower in GIN metapage

2017-09-25 Thread Michael Paquier
On Mon, Sep 25, 2017 at 3:48 PM, Amit Langote wrote: > Trying to catch up. Glad to see you back. > Just to remind, I didn't actually start this thread [1] to address the > issue that the FPWs of meta pages written to WAL are not getting > compressed. An external

Re: [HACKERS] Enhancements to passwordcheck

2017-09-25 Thread Bossart, Nathan
On 9/25/17, 8:31 PM, "Michael Paquier" wrote: > Yes, I have developped a couple of years back a fork of passwordcheck > which had much similar enhancements, so getting something more modular > in upstream would be really welcome. Awesome. > On top of that I think that

Re: [HACKERS] Replication status in logical replication

2017-09-25 Thread Masahiko Sawada
On Tue, Sep 26, 2017 at 10:36 AM, Vaishnavi Prabakaran wrote: > Hi, > > On Wed, Sep 13, 2017 at 9:59 AM, Daniel Gustafsson wrote: >> >> >> I’m not entirely sure why this was flagged as "Waiting for Author” by the >> automatic run, the patch applies

Re: [HACKERS] Simplify ACL handling for large objects and removal of superuser() checks

2017-09-25 Thread Michael Paquier
On Tue, Sep 26, 2017 at 11:42 AM, Vaishnavi Prabakaran wrote: > I moved the cf entry to "ready for committer", and though my vote is for > keeping the existing API behavior with write implying read, I let the > committer decide whether the following behavior change

Re: [HACKERS] Simplify ACL handling for large objects and removal of superuser() checks

2017-09-25 Thread Vaishnavi Prabakaran
On Tue, Sep 26, 2017 at 11:45 AM, Michael Paquier wrote: > On Tue, Sep 26, 2017 at 9:04 AM, Vaishnavi Prabakaran > wrote: > > Yes, I did realize on further reading the patch and what led to the > > confusion is that in the 3rd patch ,

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Craig Ringer
On 26 September 2017 at 01:48, Joshua D. Drake wrote: > On 09/25/2017 10:43 AM, Andres Freund wrote: > >> On 2017-09-25 10:38:52 -0700, Joshua D. Drake wrote: >> >>> On 09/25/2017 10:32 AM, Petr Jelinek wrote: >>> On 25/09/17 19:26, Tom Lane wrote: > Alvaro

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Craig Ringer
On 26 September 2017 at 01:53, Andres Freund wrote: > On 2017-09-25 13:50:29 -0400, Tom Lane wrote: > > Andres Freund writes: > > >> On 25/09/17 19:26, Tom Lane wrote: > > >>> The problem with this type of argument is that it leads directly to > the > >

Re: [HACKERS] Setting pd_lower in GIN metapage

2017-09-25 Thread Amit Kapila
On Mon, Sep 25, 2017 at 12:18 PM, Amit Langote wrote: > Hi. > > Trying to catch up. > > On 2017/09/25 13:43, Michael Paquier wrote: >> On Sun, Sep 24, 2017 at 2:25 PM, Amit Kapila wrote: >>> Added and updated the comments for both btree and

Re: [HACKERS] Shaky coding for vacuuming partitioned relations

2017-09-25 Thread Michael Paquier
On Tue, Sep 26, 2017 at 10:55 AM, Amit Langote wrote: > On 2017/09/26 9:51, Michael Paquier wrote: >> On Tue, Sep 26, 2017 at 8:48 AM, Michael Paquier >> wrote: >>> On Mon, Sep 25, 2017 at 11:32 PM, Tom Lane wrote:

Re: [HACKERS] Shaky coding for vacuuming partitioned relations

2017-09-25 Thread Michael Paquier
On Tue, Sep 26, 2017 at 10:54 AM, Amit Langote wrote: > I think that's right, although, I don't see any new RangeVar created under > vacuum() at the moment. Maybe, you're referring to the Nathan's patch > that perhaps does that. Yes, you can check what it does

Re: [JDBC] [HACKERS] Channel binding support for SCRAM-SHA-256

2017-09-25 Thread Michael Paquier
On Mon, Sep 25, 2017 at 11:22 PM, Peter Eisentraut wrote: > Here is a review of the meat of the code, leaving aside the discussion > of the libpq connection parameters. > > Overall, the structure of the code makes sense and it fits in well with > the existing

Re: [HACKERS] Shaky coding for vacuuming partitioned relations

2017-09-25 Thread Amit Langote
On 2017/09/26 9:51, Michael Paquier wrote: > On Tue, Sep 26, 2017 at 8:48 AM, Michael Paquier > wrote: >> On Mon, Sep 25, 2017 at 11:32 PM, Tom Lane wrote: >>> Yeah, I'd noticed that while reviewing the vacuum-multiple-tables patch. >>> My thought

Re: [HACKERS] Shaky coding for vacuuming partitioned relations

2017-09-25 Thread Amit Langote
On 2017/09/25 18:37, Michael Paquier wrote: > On Mon, Sep 25, 2017 at 4:42 PM, Amit Langote > wrote: >> On 2017/09/25 12:10, Michael Paquier wrote: >> Hmm, I'm not sure if we need to lock the partitions, too. Locks taken by >> find_all_inheritors() will be gone

Re: [HACKERS] Simplify ACL handling for large objects and removal of superuser() checks

2017-09-25 Thread Michael Paquier
On Tue, Sep 26, 2017 at 9:04 AM, Vaishnavi Prabakaran wrote: > Yes, I did realize on further reading the patch and what led to the > confusion is that in the 3rd patch , updated documentation(copied below) > still says that reading from a descriptor opened with

Re: [HACKERS] Replication status in logical replication

2017-09-25 Thread Vaishnavi Prabakaran
Hi, On Wed, Sep 13, 2017 at 9:59 AM, Daniel Gustafsson wrote: > > I’m not entirely sure why this was flagged as "Waiting for Author” by the > automatic run, the patch applies for me and builds so resetting back to > “Needs > review”. > > This patch applies and build cleanly and

Re: [HACKERS] Enhancements to passwordcheck

2017-09-25 Thread Euler Taveira
2017-09-25 15:04 GMT-03:00 Bossart, Nathan : > Currently, the passwordcheck module provides a few basic checks to strengthen > passwords. However, any configuration must be ready at compile time, and many > common password requirements cannot be enforced without creating a

Re: [HACKERS] Enhancements to passwordcheck

2017-09-25 Thread Michael Paquier
On Tue, Sep 26, 2017 at 3:04 AM, Bossart, Nathan wrote: > Currently, the passwordcheck module provides a few basic checks to strengthen > passwords. However, any configuration must be ready at compile time, and many > common password requirements cannot be enforced without

Re: [HACKERS] Shaky coding for vacuuming partitioned relations

2017-09-25 Thread Michael Paquier
On Tue, Sep 26, 2017 at 8:48 AM, Michael Paquier wrote: > On Mon, Sep 25, 2017 at 11:32 PM, Tom Lane wrote: >> Yeah, I'd noticed that while reviewing the vacuum-multiple-tables patch. >> My thought about fixing it was to pass a null RangeVar when

Re: [HACKERS] Simplify ACL handling for large objects and removal of superuser() checks

2017-09-25 Thread Vaishnavi Prabakaran
Hi, On Tue, Sep 19, 2017 at 5:12 PM, Michael Paquier wrote: > > >>@@ -163,22 +150,16 @@ lo_read(int fd, char *buf, int len) > >> > >> + if ((lobj->flags & IFS_RDLOCK) == 0) > >>+ ereport(ERROR, > >>+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), > >>+

Re: [HACKERS] Shaky coding for vacuuming partitioned relations

2017-09-25 Thread Bossart, Nathan
On 9/25/17, 6:51 PM, "Michael Paquier" wrote: >> +* Take a lock here for the relation lookup. If ANALYZE or >> VACUUM spawn >> +* multiple transactions, the lock taken here will be gone >> once the >> +* current

Re: [HACKERS] Shaky coding for vacuuming partitioned relations

2017-09-25 Thread Michael Paquier
On Tue, Sep 26, 2017 at 1:47 AM, Bossart, Nathan wrote: > On 9/24/17, 10:12 PM, "Michael Paquier" wrote: >> Attached is a proposal of patch. > > The patch seems reasonable to me, and I haven't encountered any issues in > my tests, even after

Re: [HACKERS] Shaky coding for vacuuming partitioned relations

2017-09-25 Thread Michael Paquier
On Mon, Sep 25, 2017 at 11:32 PM, Tom Lane wrote: > Yeah, I'd noticed that while reviewing the vacuum-multiple-tables patch. > My thought about fixing it was to pass a null RangeVar when handling a > table we'd identified through inheritance or pg_class-scanning, to > indicate

Re: [HACKERS] COMMIT TRIGGERs, take n, implemented with CONSTRAINT TRIGGERS

2017-09-25 Thread Nico Williams
I guess this got lost over the weekend and subsequent week (I was on vacation). On Fri, Sep 15, 2017 at 04:03:35PM -0500, Nico Williams wrote: > On Fri, Sep 15, 2017 at 04:07:33PM -0400, Tom Lane wrote: > > Nico Williams writes: > > > On Fri, Sep 15, 2017 at 02:19:29PM

Re: [HACKERS] Row Level Security Documentation

2017-09-25 Thread Stephen Frost
Dean, * Dean Rasheed (dean.a.rash...@gmail.com) wrote: > On 5 August 2017 at 10:03, Fabien COELHO wrote: > > Patch applies cleanly, make html ok, new table looks good to me. > > So I started looking at this patch, but before even considering the > new table proposed, I

Re: [HACKERS] CREATE COLLATION does not sanitize ICU's BCP 47 language tags. Should it?

2017-09-25 Thread Peter Geoghegan
On Mon, Sep 25, 2017 at 12:52 PM, Peter Geoghegan wrote: > That must have been the real reason why you canonicalized > pg_collation.collname (I doubt it had anything to do with how keyword > variants used to be created during initdb, as you suggested). As Tom > pointed out recently,

Re: [HACKERS] Server crash due to SIGBUS(Bus Error) when trying to access the memory created using dsm_create().

2017-09-25 Thread Thomas Munro
On Tue, Sep 26, 2017 at 10:12 AM, Tom Lane wrote: > Thomas Munro writes: >> I think the problem here is that posix_fallocate() doesn't set errno. > > Huh. So the fact that it worked for me is likely because glibc's > emulation *does* allow

Re: [HACKERS] Server crash due to SIGBUS(Bus Error) when trying to access the memory created using dsm_create().

2017-09-25 Thread Tom Lane
Thomas Munro writes: > I think the problem here is that posix_fallocate() doesn't set errno. Huh. So the fact that it worked for me is likely because glibc's emulation *does* allow errno to get set. > Will write a patch. Thanks, I'm out of time for today.

Re: [HACKERS] Server crash due to SIGBUS(Bus Error) when trying to access the memory created using dsm_create().

2017-09-25 Thread Thomas Munro
On Tue, Sep 26, 2017 at 9:57 AM, Thomas Munro wrote: >> On Tue, Sep 26, 2017 at 9:13 AM, Tom Lane wrote: >>> Pushed with that change; we'll soon see what the buildfarm thinks. > > Hmm. One failure in the test modules: > >

Re: [HACKERS] Server crash due to SIGBUS(Bus Error) when trying to access the memory created using dsm_create().

2017-09-25 Thread Thomas Munro
> On Tue, Sep 26, 2017 at 9:13 AM, Tom Lane wrote: >> Pushed with that change; we'll soon see what the buildfarm thinks. Hmm. One failure in the test modules: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=rhinoceros=2017-09-25%2020%3A45%3A02 2017-09-25

Re: [HACKERS] Server crash due to SIGBUS(Bus Error) when trying to access the memory created using dsm_create().

2017-09-25 Thread Thomas Munro
On Tue, Sep 26, 2017 at 9:13 AM, Tom Lane wrote: > I wrote: >> Rather than dig into the guts of glibc to find that out, though, I think >> we should just s/fallocate/posix_fallocate/g on this patch. The argument >> for using the former seemed pretty thin to begin with. > >

Re: [HACKERS] PATCH: pgbench - option to build using ppoll() for larger connection counts

2017-09-25 Thread Rady, Doug
On 9/25/17, 11:07, "Andres Freund" wrote: On 2017-09-25 18:01:40 +, Rady, Doug wrote: > This patch enables building pgbench to use ppoll() instead of select() > to allow for more than (FD_SETSIZE - 10) connections. As implemented, > when using ppoll(),

Re: [HACKERS] Server crash due to SIGBUS(Bus Error) when trying to access the memory created using dsm_create().

2017-09-25 Thread Thomas Munro
On Tue, Sep 26, 2017 at 7:56 AM, Tom Lane wrote: > I wrote: >> Hmm, so I tested this patch on my RHEL6 box (kernel 2.6.32) and it >> immediately fell over with >> 2017-09-25 14:23:48.410 EDT [325] FATAL: could not resize shared memory >> segment "/PostgreSQL.1682054886" to

Re: [HACKERS] Server crash due to SIGBUS(Bus Error) when trying to access the memory created using dsm_create().

2017-09-25 Thread Tom Lane
I wrote: > Rather than dig into the guts of glibc to find that out, though, I think > we should just s/fallocate/posix_fallocate/g on this patch. The argument > for using the former seemed pretty thin to begin with. Pushed with that change; we'll soon see what the buildfarm thinks. I suspect

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Petr Jelinek
On 25/09/17 19:48, Joshua D. Drake wrote: > On 09/25/2017 10:43 AM, Andres Freund wrote: >> On 2017-09-25 10:38:52 -0700, Joshua D. Drake wrote: >>> On 09/25/2017 10:32 AM, Petr Jelinek wrote: On 25/09/17 19:26, Tom Lane wrote: > Alvaro Hernandez writes: >>>

Re: [HACKERS] CREATE COLLATION does not sanitize ICU's BCP 47 language tags. Should it?

2017-09-25 Thread Peter Geoghegan
On Mon, Sep 25, 2017 at 12:14 PM, Peter Geoghegan wrote: > But then our users categorically have to know about both formats, > without any practical benefit to make up for it. You will also get > people that don't realize that only one format is supported on some > versions if go

Re: [HACKERS] Row Level Security Documentation

2017-09-25 Thread Dean Rasheed
On 5 August 2017 at 10:03, Fabien COELHO wrote: > Patch applies cleanly, make html ok, new table looks good to me. > So I started looking at this patch, but before even considering the new table proposed, I think there are multiple issues that need to be resolved with the

Re: [HACKERS] Patch to address concerns about ICU collcollate stability in v10 (Was: CREATE COLLATION does not sanitize ICU's BCP 47 language tags. Should it?)

2017-09-25 Thread Peter Geoghegan
On Mon, Sep 25, 2017 at 11:42 AM, Peter Eisentraut wrote: > On 9/25/17 00:24, Peter Geoghegan wrote: >> * Creates root collation as root-x-icu (collcollate "root"), not >> und-x-icu. "und" means undefined language. > > I'm curious about this point. "und" is

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Christopher Browne
On Sep 25, 2017 1:39 PM, "Joshua D. Drake" wrote: > On 09/25/2017 10:32 AM, Petr Jelinek wrote: > >> On 25/09/17 19:26, Tom Lane wrote: >> >>> Alvaro Hernandez writes: >>> >> > >> There is already about 3 million output plugins out there so I think we >>

Re: [HACKERS] Reading backup label file for checkpoint and redo location during crash recovery

2017-09-25 Thread Satyanarayana Narlapuram
Thank you! Got it. -Original Message- From: Stephen Frost [mailto:sfr...@snowman.net] Sent: Monday, September 25, 2017 10:57 AM To: Magnus Hagander Cc: Satyanarayana Narlapuram ; PostgreSQL-development

Re: [HACKERS] CREATE COLLATION does not sanitize ICU's BCP 47 language tags. Should it?

2017-09-25 Thread Peter Geoghegan
On Mon, Sep 25, 2017 at 11:40 AM, Peter Eisentraut wrote: > On 9/22/17 16:46, Peter Geoghegan wrote: >> But you are *already* canonicalizing ICU collation names as BCP 47. My >> point here is: Why not finish the job off, and *also* canonicalize >> colcollate in

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Magnus Hagander
On Mon, Sep 25, 2017 at 8:20 PM, Alvaro Hernandez wrote: > > > On 25/09/17 20:18, Andres Freund wrote: > >> On 2017-09-24 13:36:56 +0300, Alvaro Hernandez wrote: >> >>> However, if DMS uses it for what I'd call production use, I assume >>> it is >>> actually production

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Jignesh Shah
On Mon, Sep 25, 2017 at 11:37 AM, Joshua D. Drake wrote: > On 09/25/2017 11:31 AM, Alvaro Hernandez wrote: > >> >> > Whether or not they are included in a managed environment is generally >>> based on two things: >>> >>> 1. Safety (why RDS doesn't allow certain C

Re: [HACKERS] Server crash due to SIGBUS(Bus Error) when trying to access the memory created using dsm_create().

2017-09-25 Thread Tom Lane
I wrote: > Hmm, so I tested this patch on my RHEL6 box (kernel 2.6.32) and it > immediately fell over with > 2017-09-25 14:23:48.410 EDT [325] FATAL: could not resize shared memory > segment "/PostgreSQL.1682054886" to 6928 bytes: Operation not supported > during startup. I wonder whether we

Re: [HACKERS] Fix number skipping in to_number

2017-09-25 Thread Oliver Ford
On Monday, 25 September 2017, Nathan Wagner wrote: > On Thu, Aug 17, 2017 at 12:33:02PM +0100, Oliver Ford wrote: > > > Ok I've made that change in the attached v3. I'm not sure as I'm on > > en_US.UTF-8 locale too. Maybe something Windows specific? > > This patch applies

Re: [HACKERS] [PATCH] Generic type subscripting

2017-09-25 Thread Oleg Bartunov
On Fri, Sep 22, 2017 at 3:51 PM, Peter Eisentraut wrote: > On 9/21/17 11:24, Dmitry Dolgov wrote: >> One last thing that I need to clarify. Initially there was an idea to >> minimize changes in `pg_type` > > I see, but there is no value in that if it makes

Re: [HACKERS] [BUGS] BUG #14825: enum type: unsafe use?

2017-09-25 Thread Tom Lane
Andrew Dunstan writes: > OK, that seems to be the consensus. So let's apply the blacklist patch > and then separately remove the 'created in the same transaction' test. > We'll need to adjust the regression tests and docs accordingly. Agreed. I'll work on that in

Re: [HACKERS] Patch to address concerns about ICU collcollate stability in v10 (Was: CREATE COLLATION does not sanitize ICU's BCP 47 language tags. Should it?)

2017-09-25 Thread Peter Eisentraut
On 9/25/17 00:24, Peter Geoghegan wrote: > * Creates root collation as root-x-icu (collcollate "root"), not > und-x-icu. "und" means undefined language. I'm curious about this point. "und" is defined in BCP 47. I don't see "root" defined anywhere. ICU converts the root collation to "und",

Re: [HACKERS] CREATE COLLATION does not sanitize ICU's BCP 47 language tags. Should it?

2017-09-25 Thread Peter Eisentraut
On 9/22/17 16:46, Peter Geoghegan wrote: > But you are *already* canonicalizing ICU collation names as BCP 47. My > point here is: Why not finish the job off, and *also* canonicalize > colcollate in the same way? This won't break ucol_open() if we take > appropriate precautions when we go to use

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Andres Freund
On 2017-09-25 21:31:11 +0300, Alvaro Hernandez wrote: > > > > - Distribution and testing are non-trivial: many OS/archs combinations. > > > > > > > > Yes, it is. Why would we want to increase that burden to this community? > > >     That's a different story, and one I cannot argue against. If

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Joshua D. Drake
On 09/25/2017 11:31 AM, Alvaro Hernandez wrote: Whether or not they are included in a managed environment is generally based on two things: 1. Safety (why RDS doesn't allow certain C extensions) 2. Community/Popularity (Exactly why RDS has PostGIS)     A. Demand with a

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Alvaro Hernandez
On 25/09/17 20:31, Joshua D. Drake wrote: On 09/25/2017 10:19 AM, Petr Jelinek wrote: On 25/09/17 18:48, Alvaro Hernandez wrote: In my opinion, logical decoding plugins that don't come with core are close to worthless (don't get me wrong): I respectfully disagree. As do I.    

Re: [HACKERS] Server crash due to SIGBUS(Bus Error) when trying to access the memory created using dsm_create().

2017-09-25 Thread Tom Lane
Robert Haas writes: > On Mon, Sep 25, 2017 at 10:22 AM, Tom Lane wrote: >> I think we don't really have a lot of choice. I propose applying this >> as far back as 9.6 --- anyone think differently? > +1. If applies to 9.5 and 9.4 without a lot of

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Alvaro Hernandez
On 25/09/17 20:18, Andres Freund wrote: On 2017-09-24 13:36:56 +0300, Alvaro Hernandez wrote:     However, if DMS uses it for what I'd call production use, I assume it is actually production quality. I bet they do enough testing, and don't ship software to potentially millions of customers

Re: [HACKERS] [BUGS] BUG #14825: enum type: unsafe use?

2017-09-25 Thread Andrew Dunstan
On 09/25/2017 01:34 PM, David E. Wheeler wrote: > On Sep 25, 2017, at 10:55, Andrew Dunstan > wrote: > >> Let's ask a couple of users who I think are or have been actually >> hurting on this point. Christophe and David, any opinions? > If I understand the issue

Re: [HACKERS] PATCH: pgbench - option to build using ppoll() for larger connection counts

2017-09-25 Thread Fabien COELHO
Hello Again, Two patches attached. One based on REL9_6_STABLE. I'd be surprise that there would be a backport unless there is a bug, so this one might not be useful. One based on 'master' which can also apply to REL_10_STABLE. Could you add your patches to the next CF? -- Fabien. --

Re: [HACKERS] PATCH: pgbench - option to build using ppoll() for larger connection counts

2017-09-25 Thread Andres Freund
On 2017-09-25 18:01:40 +, Rady, Doug wrote: > This patch enables building pgbench to use ppoll() instead of select() > to allow for more than (FD_SETSIZE - 10) connections. As implemented, > when using ppoll(), the only connection limitation is system resources. Hm, is there any need of

Re: [HACKERS] PATCH: pgbench - break out timing data for initialization phases

2017-09-25 Thread Fabien COELHO
Hello Doug, total time: 316.03 s (insert 161.60 s, commit 0.64 s, vacuum 60.77 s, index 93.01 s) Definitely interesting. There is a "ready for committers" patch in the CF which extensively rework the initialization: it becomes customizable, and this approach may not work as is after

[HACKERS] Enhancements to passwordcheck

2017-09-25 Thread Bossart, Nathan
Hi hackers, Currently, the passwordcheck module provides a few basic checks to strengthen passwords. However, any configuration must be ready at compile time, and many common password requirements cannot be enforced without creating a custom version of this module. I think there are a number of

Re: [HACKERS] Reading backup label file for checkpoint and redo location during crash recovery

2017-09-25 Thread Stephen Frost
* Magnus Hagander (mag...@hagander.net) wrote: > On Mon, Sep 25, 2017 at 7:43 PM, Stephen Frost wrote: > > * Satyanarayana Narlapuram (satyanarayana.narlapu...@microsoft.com) wrote: > > > During crash recovery, last checkpoint record information is obtained > > from the backup

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Stephen Frost
Andres, all, * Andres Freund (and...@anarazel.de) wrote: > On 2017-09-25 19:32:29 +0200, Petr Jelinek wrote: > > On 25/09/17 19:26, Tom Lane wrote: > > > Alvaro Hernandez writes: > > >> In my opinion, logical decoding plugins that don't come with core > > >> are close to

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Andres Freund
On 2017-09-25 13:50:29 -0400, Tom Lane wrote: > Andres Freund writes: > >> On 25/09/17 19:26, Tom Lane wrote: > >>> The problem with this type of argument is that it leads directly to the > >>> conclusion that every feature users want must be in core. > > > ... I don't think

Re: [HACKERS] psql \d sequence display

2017-09-25 Thread Fabien COELHO
Hello, This should be fixed for PG10, so if you have any feedback on the design, please let me know soon. Works for me on head against a 9.6 server, which is good. My 0.02 €: \d+ does not show more. Maybe Type, Min, Max, Inc & Cycles are enough for \d? The next/future or last/previous

Re: [HACKERS] Reading backup label file for checkpoint and redo location during crash recovery

2017-09-25 Thread Magnus Hagander
On Mon, Sep 25, 2017 at 7:43 PM, Stephen Frost wrote: > Greetings Satya, > > * Satyanarayana Narlapuram (satyanarayana.narlapu...@microsoft.com) wrote: > > During crash recovery, last checkpoint record information is obtained > from the backup label if present, instead of

Re: [HACKERS] logical replication and statistics

2017-09-25 Thread Pavel Stehule
2017-09-25 19:23 GMT+02:00 Petr Jelinek : > On 25/09/17 19:19, Tom Lane wrote: > > Pavel Stehule writes: > >> I had two instances on one server with different port. I am sure, so > >> replication was functional. Only one issue is statistics

[HACKERS][BUG] Cache invalidation for queries that contains const of temporary composite type

2017-09-25 Thread Maksim Milyutin
Hello everyone! I have found out the problem when try to sequentially call the function that casts constant to composite type of temporary table that is deleted ateach transaction termination (i.e. at each function call completion). For example, we have the following function 'test':

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Tom Lane
Andres Freund writes: >> On 25/09/17 19:26, Tom Lane wrote: >>> The problem with this type of argument is that it leads directly to the >>> conclusion that every feature users want must be in core. > ... I don't think that should mean that there's no possible output > plugin

Re: [HACKERS] Reading backup label file for checkpoint and redo location during crash recovery

2017-09-25 Thread Andres Freund
On 2017-09-25 13:43:32 -0400, Stephen Frost wrote: > Greetings Satya, > > * Satyanarayana Narlapuram (satyanarayana.narlapu...@microsoft.com) wrote: > > During crash recovery, last checkpoint record information is obtained from > > the backup label if present, instead of getting it from the

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Joshua D. Drake
On 09/25/2017 10:43 AM, Andres Freund wrote: On 2017-09-25 10:38:52 -0700, Joshua D. Drake wrote: On 09/25/2017 10:32 AM, Petr Jelinek wrote: On 25/09/17 19:26, Tom Lane wrote: Alvaro Hernandez writes: There is already about 3 million output plugins out there so I think

Re: [HACKERS] Server crash due to SIGBUS(Bus Error) when trying to access the memory created using dsm_create().

2017-09-25 Thread Tom Lane
Robert Haas writes: > On Mon, Sep 25, 2017 at 10:22 AM, Tom Lane wrote: >> Thomas Munro writes: >>> So, do we want this patch? >> I think we don't really have a lot of choice. I propose applying this >> as far back as

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Andres Freund
On 2017-09-25 19:32:29 +0200, Petr Jelinek wrote: > On 25/09/17 19:26, Tom Lane wrote: > > Alvaro Hernandez writes: > >> In my opinion, logical decoding plugins that don't come with core > >> are close to worthless (don't get me wrong): > > > >> - They very unlikely will be

Re: [HACKERS] Reading backup label file for checkpoint and redo location during crash recovery

2017-09-25 Thread Stephen Frost
Greetings Satya, * Satyanarayana Narlapuram (satyanarayana.narlapu...@microsoft.com) wrote: > During crash recovery, last checkpoint record information is obtained from > the backup label if present, instead of getting it from the control file. > This behavior is causing PostgreSQL database

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Andres Freund
On 2017-09-25 10:38:52 -0700, Joshua D. Drake wrote: > On 09/25/2017 10:32 AM, Petr Jelinek wrote: > > On 25/09/17 19:26, Tom Lane wrote: > > > Alvaro Hernandez writes: > > > > > There is already about 3 million output plugins out there so I think we > > did reasonable job

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Joshua D. Drake
On 09/25/2017 10:32 AM, Petr Jelinek wrote: On 25/09/17 19:26, Tom Lane wrote: Alvaro Hernandez writes: There is already about 3 million output plugins out there so I think we did reasonable job there. The fact that vast majority of that are various json ones gives

[HACKERS] Reading backup label file for checkpoint and redo location during crash recovery

2017-09-25 Thread Satyanarayana Narlapuram
Hi there, During crash recovery, last checkpoint record information is obtained from the backup label if present, instead of getting it from the control file. This behavior is causing PostgreSQL database cluster not to come up until the backup label file is deleted (as the error message says).

Re: [HACKERS] [BUGS] BUG #14825: enum type: unsafe use?

2017-09-25 Thread David E. Wheeler
On Sep 25, 2017, at 10:55, Andrew Dunstan wrote: > Let's ask a couple of users who I think are or have been actually > hurting on this point. Christophe and David, any opinions? If I understand the issue correctly, I think I’d be fine with requiring ALTER TYPE

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Petr Jelinek
On 25/09/17 19:26, Tom Lane wrote: > Alvaro Hernandez writes: >> In my opinion, logical decoding plugins that don't come with core >> are close to worthless (don't get me wrong): > >> - They very unlikely will be installed in managed environments (an area >> growing

Re: [HACKERS] [BUGS] BUG #14825: enum type: unsafe use?

2017-09-25 Thread Christophe Pettus
> On Sep 25, 2017, at 07:55, Andrew Dunstan > wrote: > Let's ask a couple of users who I think are or have been actually > hurting on this point. Christophe and David, any opinions? Since about 90% of what I encounter in this area are automatically-generated

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Joshua D. Drake
On 09/25/2017 10:19 AM, Petr Jelinek wrote: On 25/09/17 18:48, Alvaro Hernandez wrote:     In my opinion, logical decoding plugins that don't come with core are close to worthless (don't get me wrong): I respectfully disagree. As do I. - They very unlikely will be installed in

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Tom Lane
Alvaro Hernandez writes: > In my opinion, logical decoding plugins that don't come with core > are close to worthless (don't get me wrong): > - They very unlikely will be installed in managed environments (an area > growing significantly). > - As anything that is not in core,

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Joshua D. Drake
On 09/25/2017 10:15 AM, Gregory Brail wrote: Yes. I'm advocating something "built-in" to Postgres. Any or all of those are likely a great starting point. I don't see a benefit to having this "in postgres". The whole reason we have built out a mature and extensible product is so that not

Re: [HACKERS] logical replication and statistics

2017-09-25 Thread Petr Jelinek
On 25/09/17 19:19, Tom Lane wrote: > Pavel Stehule writes: >> I had two instances on one server with different port. I am sure, so >> replication was functional. Only one issue is statistics > >> Master: > >> CREATE TABLE foo(id int primary key, a int); >> CREATE

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Petr Jelinek
On 25/09/17 18:48, Alvaro Hernandez wrote: > > > On 25/09/17 19:39, Petr Jelinek wrote: >> >> Well, test_decoding is not meant for production use anyway, no need for >> middleware to support it. The pgoutput is primarily used for internal >> replication purposes, which is why we need something

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Andres Freund
Hi, On 2017-09-25 12:56:00 -0400, Andrew Dunstan wrote: > A general purpose lower bandwidth plugin might one supporting Protocol > Buffers. The downside is that unlike json it's not self-contained, you > need the message definitions to interpret the stream, AIUI. I think that makes it a

Re: [HACKERS] logical replication and statistics

2017-09-25 Thread Tom Lane
Pavel Stehule writes: > I had two instances on one server with different port. I am sure, so > replication was functional. Only one issue is statistics > Master: > CREATE TABLE foo(id int primary key, a int); > CREATE PUBLICATION test_pub FOR TABLE foo; > INSERT INTO

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Andres Freund
On 2017-09-24 13:36:56 +0300, Alvaro Hernandez wrote: >     However, if DMS uses it for what I'd call production use, I assume it is > actually production quality. I bet they do enough testing, and don't ship > software to potentially millions of customers if it doesn't work well. So... > first,

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Gregory Brail
Yes. I'm advocating something "built-in" to Postgres. Any or all of those are likely a great starting point. As for protobuf, I'm a big advocate -- it is easy to use, fast, extensible, runs on lots of platforms, and produces very compact output. However it introduces a few dependencies to the

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Alvaro Hernandez
On 25/09/17 19:56, Andrew Dunstan wrote: On 09/25/2017 12:48 PM, Alvaro Hernandez wrote: On 25/09/17 19:39, Petr Jelinek wrote: Well, test_decoding is not meant for production use anyway, no need for middleware to support it. The pgoutput is primarily used for internal replication

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Joshua D. Drake
On 09/25/2017 09:59 AM, Gregory Brail wrote: However, I can't find any docs for the output format of pgoutput, which is going to make it less likely for people to be able to consume it. Is anyone working on docs? I know that it's a painful process. I also think that a JSON-format (or

Re: [HACKERS] Shaky coding for vacuuming partitioned relations

2017-09-25 Thread Bossart, Nathan
On 9/24/17, 10:12 PM, "Michael Paquier" wrote: > Attached is a proposal of patch. The patch seems reasonable to me, and I haven't encountered any issues in my tests, even after applying the vacuum-multiple-relations patch on top of it. +* Take a lock

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Gregory Brail
I'm encouraged that pgoutput exists and I'm sorry that I missed it before. I think it's fine as a binary-only format. If someone can write a client for the Postgres wire protocol as documented in Chapter 52 of the docs, then they should have no trouble consuming the output from pgoutput. However,

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Andrew Dunstan
On 09/25/2017 12:48 PM, Alvaro Hernandez wrote: > > > On 25/09/17 19:39, Petr Jelinek wrote: >> >> Well, test_decoding is not meant for production use anyway, no need for >> middleware to support it. The pgoutput is primarily used for internal >> replication purposes, which is why we need

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Alvaro Hernandez
On 25/09/17 19:39, Petr Jelinek wrote: Well, test_decoding is not meant for production use anyway, no need for middleware to support it. The pgoutput is primarily used for internal replication purposes, which is why we need something with more interoperability in mind in the first place. The

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Petr Jelinek
On 23/09/17 19:01, Alvaro Hernandez wro> On 23/09/17 18:42, Euler Taveira wrote: >> 2017-09-22 19:28 GMT-03:00 Gregory Brail : >>> We have been working on a project that makes extensive use of logical >>> replication for use inside Apigee (which is a very small part of >>>

Re: [HACKERS] CREATE COLLATION does not sanitize ICU's BCP 47 language tags. Should it?

2017-09-25 Thread Peter Geoghegan
On Mon, Sep 25, 2017 at 9:06 AM, Robert Haas wrote: >> The big concern I have here is that this feels a lot like something that >> we'll regret at leisure, if it's not right in the first release. I'd >> much rather be restrictive in v10 and then loosen the rules later,

Re: [HACKERS] logical replication and statistics

2017-09-25 Thread Pavel Stehule
2017-09-25 18:30 GMT+02:00 Petr Jelinek : > On 25/09/17 13:33, Pavel Stehule wrote: > > > > > > 2017-09-25 13:12 GMT+02:00 Masahiko Sawada > >: > > > > On Mon, Sep 25, 2017 at 12:58 AM, Pavel Stehule > >

Re: bgw_type (was Re: [HACKERS] Why does logical replication launcher set application_name?)

2017-09-25 Thread Petr Jelinek
On 25/09/17 16:45, Peter Eisentraut wrote: > On 8/31/17 23:22, Michael Paquier wrote: >>> One open question is how to treat a missing (empty) bgw_type. I >>> currently fill in bgw_name as a fallback. We could also treat it as an >>> error or a warning as a transition measure. >> >> Hm. Why not

  1   2   >