Re: [HACKERS] Server crash due to SIGBUS(Bus Error) when trying to access the memory created using dsm_create().

2017-09-25 Thread Thomas Munro
> On Tue, Sep 26, 2017 at 9:13 AM, Tom Lane wrote: >> Pushed with that change; we'll soon see what the buildfarm thinks. Hmm. One failure in the test modules: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=rhinoceros=2017-09-25%2020%3A45%3A02 2017-09-25

Re: [HACKERS] Server crash due to SIGBUS(Bus Error) when trying to access the memory created using dsm_create().

2017-09-25 Thread Thomas Munro
On Tue, Sep 26, 2017 at 9:57 AM, Thomas Munro wrote: >> On Tue, Sep 26, 2017 at 9:13 AM, Tom Lane wrote: >>> Pushed with that change; we'll soon see what the buildfarm thinks. > > Hmm. One failure in the test modules: > >

Re: [HACKERS] Server crash due to SIGBUS(Bus Error) when trying to access the memory created using dsm_create().

2017-09-25 Thread Thomas Munro
On Tue, Sep 26, 2017 at 10:12 AM, Tom Lane wrote: > Thomas Munro writes: >> I think the problem here is that posix_fallocate() doesn't set errno. > > Huh. So the fact that it worked for me is likely because glibc's > emulation *does* allow

Re: [HACKERS] Row Level Security Documentation

2017-09-25 Thread Stephen Frost
Dean, * Dean Rasheed (dean.a.rash...@gmail.com) wrote: > On 5 August 2017 at 10:03, Fabien COELHO wrote: > > Patch applies cleanly, make html ok, new table looks good to me. > > So I started looking at this patch, but before even considering the > new table proposed, I

Re: [HACKERS] Shaky coding for vacuuming partitioned relations

2017-09-25 Thread Bossart, Nathan
On 9/25/17, 6:51 PM, "Michael Paquier" wrote: >> +* Take a lock here for the relation lookup. If ANALYZE or >> VACUUM spawn >> +* multiple transactions, the lock taken here will be gone >> once the >> +* current

Re: [HACKERS] Simplify ACL handling for large objects and removal of superuser() checks

2017-09-25 Thread Vaishnavi Prabakaran
Hi, On Tue, Sep 19, 2017 at 5:12 PM, Michael Paquier wrote: > > >>@@ -163,22 +150,16 @@ lo_read(int fd, char *buf, int len) > >> > >> + if ((lobj->flags & IFS_RDLOCK) == 0) > >>+ ereport(ERROR, > >>+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), > >>+

Re: [HACKERS] Shaky coding for vacuuming partitioned relations

2017-09-25 Thread Michael Paquier
On Mon, Sep 25, 2017 at 11:32 PM, Tom Lane wrote: > Yeah, I'd noticed that while reviewing the vacuum-multiple-tables patch. > My thought about fixing it was to pass a null RangeVar when handling a > table we'd identified through inheritance or pg_class-scanning, to > indicate

Re: [HACKERS] PATCH: pgbench - option to build using ppoll() for larger connection counts

2017-09-25 Thread Rady, Doug
On 9/25/17, 11:07, "Andres Freund" wrote: On 2017-09-25 18:01:40 +, Rady, Doug wrote: > This patch enables building pgbench to use ppoll() instead of select() > to allow for more than (FD_SETSIZE - 10) connections. As implemented, > when using ppoll(),

Re: [HACKERS] COMMIT TRIGGERs, take n, implemented with CONSTRAINT TRIGGERS

2017-09-25 Thread Nico Williams
I guess this got lost over the weekend and subsequent week (I was on vacation). On Fri, Sep 15, 2017 at 04:03:35PM -0500, Nico Williams wrote: > On Fri, Sep 15, 2017 at 04:07:33PM -0400, Tom Lane wrote: > > Nico Williams writes: > > > On Fri, Sep 15, 2017 at 02:19:29PM

Re: [HACKERS] Server crash due to SIGBUS(Bus Error) when trying to access the memory created using dsm_create().

2017-09-25 Thread Thomas Munro
On Tue, Sep 26, 2017 at 7:56 AM, Tom Lane wrote: > I wrote: >> Hmm, so I tested this patch on my RHEL6 box (kernel 2.6.32) and it >> immediately fell over with >> 2017-09-25 14:23:48.410 EDT [325] FATAL: could not resize shared memory >> segment "/PostgreSQL.1682054886" to

Re: [HACKERS] Shaky coding for vacuuming partitioned relations

2017-09-25 Thread Michael Paquier
On Tue, Sep 26, 2017 at 1:47 AM, Bossart, Nathan wrote: > On 9/24/17, 10:12 PM, "Michael Paquier" wrote: >> Attached is a proposal of patch. > > The patch seems reasonable to me, and I haven't encountered any issues in > my tests, even after

Re: [HACKERS] Shaky coding for vacuuming partitioned relations

2017-09-25 Thread Michael Paquier
On Tue, Sep 26, 2017 at 10:54 AM, Amit Langote wrote: > I think that's right, although, I don't see any new RangeVar created under > vacuum() at the moment. Maybe, you're referring to the Nathan's patch > that perhaps does that. Yes, you can check what it does

Re: [HACKERS] Shaky coding for vacuuming partitioned relations

2017-09-25 Thread Michael Paquier
On Tue, Sep 26, 2017 at 8:48 AM, Michael Paquier wrote: > On Mon, Sep 25, 2017 at 11:32 PM, Tom Lane wrote: >> Yeah, I'd noticed that while reviewing the vacuum-multiple-tables patch. >> My thought about fixing it was to pass a null RangeVar when

Re: [HACKERS] Enhancements to passwordcheck

2017-09-25 Thread Euler Taveira
2017-09-25 15:04 GMT-03:00 Bossart, Nathan : > Currently, the passwordcheck module provides a few basic checks to strengthen > passwords. However, any configuration must be ready at compile time, and many > common password requirements cannot be enforced without creating a

Re: [HACKERS] Replication status in logical replication

2017-09-25 Thread Vaishnavi Prabakaran
Hi, On Wed, Sep 13, 2017 at 9:59 AM, Daniel Gustafsson wrote: > > I’m not entirely sure why this was flagged as "Waiting for Author” by the > automatic run, the patch applies for me and builds so resetting back to > “Needs > review”. > > This patch applies and build cleanly and

Re: [HACKERS] Shaky coding for vacuuming partitioned relations

2017-09-25 Thread Amit Langote
On 2017/09/26 9:51, Michael Paquier wrote: > On Tue, Sep 26, 2017 at 8:48 AM, Michael Paquier > wrote: >> On Mon, Sep 25, 2017 at 11:32 PM, Tom Lane wrote: >>> Yeah, I'd noticed that while reviewing the vacuum-multiple-tables patch. >>> My thought

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Craig Ringer
On 26 September 2017 at 01:53, Andres Freund wrote: > On 2017-09-25 13:50:29 -0400, Tom Lane wrote: > > Andres Freund writes: > > >> On 25/09/17 19:26, Tom Lane wrote: > > >>> The problem with this type of argument is that it leads directly to > the > >

Re: [HACKERS] Simplify ACL handling for large objects and removal of superuser() checks

2017-09-25 Thread Michael Paquier
On Tue, Sep 26, 2017 at 11:42 AM, Vaishnavi Prabakaran wrote: > I moved the cf entry to "ready for committer", and though my vote is for > keeping the existing API behavior with write implying read, I let the > committer decide whether the following behavior change

[HACKERS] Runtime Partition Pruning

2017-09-25 Thread Beena Emerson
I have been working on implementing the runtime partition pruning which would increase the performance of queries involving partitioned table to a great extent. PFA the POC which can be applied over Amit's patch for faster partition pruning [1] and Dilip's refactor patch [2] on commit

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Alvaro Hernandez
On 25/09/17 22:13, Magnus Hagander wrote: On Mon, Sep 25, 2017 at 8:20 PM, Alvaro Hernandez > wrote: On 25/09/17 20:18, Andres Freund wrote: On 2017-09-24 13:36:56 +0300, Alvaro Hernandez wrote: However, if DMS uses it

Re: [HACKERS] Simplify ACL handling for large objects and removal of superuser() checks

2017-09-25 Thread Michael Paquier
On Tue, Sep 26, 2017 at 9:04 AM, Vaishnavi Prabakaran wrote: > Yes, I did realize on further reading the patch and what led to the > confusion is that in the 3rd patch , updated documentation(copied below) > still says that reading from a descriptor opened with

Re: [JDBC] [HACKERS] Channel binding support for SCRAM-SHA-256

2017-09-25 Thread Michael Paquier
On Mon, Sep 25, 2017 at 11:22 PM, Peter Eisentraut wrote: > Here is a review of the meat of the code, leaving aside the discussion > of the libpq connection parameters. > > Overall, the structure of the code makes sense and it fits in well with > the existing

Re: [HACKERS] Setting pd_lower in GIN metapage

2017-09-25 Thread Amit Kapila
On Mon, Sep 25, 2017 at 12:18 PM, Amit Langote wrote: > Hi. > > Trying to catch up. > > On 2017/09/25 13:43, Michael Paquier wrote: >> On Sun, Sep 24, 2017 at 2:25 PM, Amit Kapila wrote: >>> Added and updated the comments for both btree and

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Craig Ringer
On 26 September 2017 at 01:48, Joshua D. Drake wrote: > On 09/25/2017 10:43 AM, Andres Freund wrote: > >> On 2017-09-25 10:38:52 -0700, Joshua D. Drake wrote: >> >>> On 09/25/2017 10:32 AM, Petr Jelinek wrote: >>> On 25/09/17 19:26, Tom Lane wrote: > Alvaro

Re: [HACKERS] Replication status in logical replication

2017-09-25 Thread Masahiko Sawada
On Tue, Sep 26, 2017 at 10:36 AM, Vaishnavi Prabakaran wrote: > Hi, > > On Wed, Sep 13, 2017 at 9:59 AM, Daniel Gustafsson wrote: >> >> >> I’m not entirely sure why this was flagged as "Waiting for Author” by the >> automatic run, the patch applies

Re: [HACKERS] Enhancements to passwordcheck

2017-09-25 Thread Bossart, Nathan
On 9/25/17, 8:31 PM, "Michael Paquier" wrote: > Yes, I have developped a couple of years back a fork of passwordcheck > which had much similar enhancements, so getting something more modular > in upstream would be really welcome. Awesome. > On top of that I think that

Re: [JDBC] [HACKERS] Channel binding support for SCRAM-SHA-256

2017-09-25 Thread Peter Eisentraut
On 9/10/17 22:37, Michael Paquier wrote: >>> With the tests directly in the patch, things are easy to run. WIth >>> PG10 stabilization work, of course I don't expect much feedback :) >>> But this set of patches looks like the direction we want to go so as >>> JDBC and libpq users can take

Re: [HACKERS] Server crash due to SIGBUS(Bus Error) when trying to access the memory created using dsm_create().

2017-09-25 Thread Tom Lane
Thomas Munro writes: > So, do we want this patch? I think we don't really have a lot of choice. I propose applying this as far back as 9.6 --- anyone think differently? regards, tom lane -- Sent via pgsql-hackers mailing list

Re: [HACKERS] GSoC 2017: weekly progress reports (week 4) and patch for hash index

2017-09-25 Thread Shubham Barai
Hi Thomas, I have attached the rebased version of patch here. Kind Regards, Shubham On 8 September 2017 at 06:37, Thomas Munro wrote: > Hi Shubham, > > On Tue, Jun 27, 2017 at 9:21 PM, Shubham Barai > wrote: > > If these two hash keys

Re: [HACKERS] Small improvement to compactify_tuples

2017-09-25 Thread Claudio Freire
On Sat, Sep 23, 2017 at 5:56 AM, Sokolov Yura wrote: > Hello, Claudio. > > Thank you for review and confirm of improvement. > > > On 2017-09-23 01:12, Claudio Freire wrote: >> >> >> Patch 1 applies cleanly, builds, and make check runs fine. >> >> The code looks

Re: [HACKERS] [BUGS] BUG #14825: enum type: unsafe use?

2017-09-25 Thread Tom Lane
Andrew Dunstan writes: > On 09/24/2017 07:06 PM, Tom Lane wrote: >> So I think we should just stop with the blacklist test for v10, >> and then see if we still get complaints (and exactly what they're >> about) so that we can judge how much more work the problem

Re: [HACKERS] [BUGS] BUG #14825: enum type: unsafe use?

2017-09-25 Thread Andrew Dunstan
On 09/25/2017 10:14 AM, Tom Lane wrote: > Andrew Dunstan writes: >> On 09/24/2017 07:06 PM, Tom Lane wrote: >>> So I think we should just stop with the blacklist test for v10, >>> and then see if we still get complaints (and exactly what they're >>> about) so

Re: [HACKERS] Shaky coding for vacuuming partitioned relations

2017-09-25 Thread Tom Lane
Amit Langote writes: > On 2017/09/25 12:10, Michael Paquier wrote: >> As long as I don't forget... Another thing currently on HEAD and >> REL_10_STABLE is that OIDs of partitioned tables are used, but the >> RangeVar of the parent is used for error reports. This

Re: [HACKERS] [BUGS] BUG #14825: enum type: unsafe use?

2017-09-25 Thread Tom Lane
Andrew Dunstan writes: > On 09/25/2017 10:14 AM, Tom Lane wrote: >> Oh ... I did not think we were on the same page, because your patch >> didn't include removal of the same-transaction heuristic. It'd be >> sensible to do that as a separate patch, though, to make

Re: bgw_type (was Re: [HACKERS] Why does logical replication launcher set application_name?)

2017-09-25 Thread Peter Eisentraut
On 8/31/17 23:22, Michael Paquier wrote: >> One open question is how to treat a missing (empty) bgw_type. I >> currently fill in bgw_name as a fallback. We could also treat it as an >> error or a warning as a transition measure. > > Hm. Why not reporting an empty type string as NULL at SQL

Re: [HACKERS] [BUGS] BUG #14825: enum type: unsafe use?

2017-09-25 Thread Andrew Dunstan
On 09/25/2017 10:42 AM, Tom Lane wrote: > Andrew Dunstan writes: >> On 09/25/2017 10:14 AM, Tom Lane wrote: >>> Oh ... I did not think we were on the same page, because your patch >>> didn't include removal of the same-transaction heuristic. It'd be >>> sensible

[HACKERS] Re: CREATE COLLATION does not sanitize ICU's BCP 47 language tags. Should it?

2017-09-25 Thread Noah Misch
On Tue, Sep 19, 2017 at 07:01:47PM -0700, Peter Geoghegan wrote: > On Tue, Sep 19, 2017 at 5:52 PM, Peter Eisentraut > wrote: > > On 9/18/17 18:46, Peter Geoghegan wrote: > >> As I pointed out a couple of times already [1], we don't currently > >> sanitize ICU's

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-09-25 Thread Masahiko Sawada
On Fri, Sep 22, 2017 at 5:31 PM, Kyotaro HORIGUCHI wrote: > At Fri, 22 Sep 2017 17:21:04 +0900, Masahiko Sawada > wrote in >> On Fri, Sep 22, 2017 at 4:16 PM, Kyotaro

Re: [HACKERS] Shaky coding for vacuuming partitioned relations

2017-09-25 Thread Masahiko Sawada
On Mon, Sep 25, 2017 at 12:10 PM, Michael Paquier wrote: > On Sat, Sep 23, 2017 at 4:13 AM, Tom Lane wrote: >> Somebody inserted this into vacuum.c's get_rel_oids(): >> >> tuple = SearchSysCache1(RELOID, ObjectIdGetDatum(relid)); >>

Re: [HACKERS] comments improvements

2017-09-25 Thread Masahiko Sawada
On Mon, Sep 25, 2017 at 5:36 AM, Erik Rijkers wrote: > comments improvements > For triggers.sql.20170924.diff file, you need to update expected/triggers.out file as well. -- Regards, -- Masahiko Sawada NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center

Re: [HACKERS] CREATE COLLATION does not sanitize ICU's BCP 47 language tags. Should it?

2017-09-25 Thread Robert Haas
On Fri, Sep 22, 2017 at 11:56 PM, Tom Lane wrote: > FWIW, the release is a week from Monday, not Monday. (Or if it is > Monday, somebody else is wrapping it.) Oops. > We have some other embarrassingly critical things to fix, like bug #14825, > so I can certainly sympathize

Re: [HACKERS] Server crash due to SIGBUS(Bus Error) when trying to access the memory created using dsm_create().

2017-09-25 Thread Robert Haas
On Mon, Sep 25, 2017 at 10:22 AM, Tom Lane wrote: > Thomas Munro writes: >> So, do we want this patch? > > I think we don't really have a lot of choice. I propose applying this > as far back as 9.6 --- anyone think differently? +1. If applies

Re: bgw_type (was Re: [HACKERS] Why does logical replication launcher set application_name?)

2017-09-25 Thread Petr Jelinek
On 25/09/17 16:45, Peter Eisentraut wrote: > On 8/31/17 23:22, Michael Paquier wrote: >>> One open question is how to treat a missing (empty) bgw_type. I >>> currently fill in bgw_name as a fallback. We could also treat it as an >>> error or a warning as a transition measure. >> >> Hm. Why not

Re: [HACKERS] logical replication and statistics

2017-09-25 Thread Pavel Stehule
2017-09-25 18:30 GMT+02:00 Petr Jelinek : > On 25/09/17 13:33, Pavel Stehule wrote: > > > > > > 2017-09-25 13:12 GMT+02:00 Masahiko Sawada > >: > > > > On Mon, Sep 25, 2017 at 12:58 AM, Pavel Stehule > >

[HACKERS] psql \d sequence display

2017-09-25 Thread Peter Eisentraut
In PostgreSQL 10, the sequence metadata moved from the sequence "relation" to a system catalog. The psql \d sequence command was not updated for that. (It just did SELECT * FROM seq and there were no tests, so this was missed.) Attached is a patch that fixes that up, taking the opportunity to

Re: [HACKERS] logical replication and statistics

2017-09-25 Thread Petr Jelinek
On 25/09/17 13:33, Pavel Stehule wrote: > > > 2017-09-25 13:12 GMT+02:00 Masahiko Sawada >: > > On Mon, Sep 25, 2017 at 12:58 AM, Pavel Stehule > > wrote: > > Hi > > >

Re: [HACKERS] [PATCH]make pg_rewind to not copy useless WAL files

2017-09-25 Thread chenhj
On 2017-09-23 01:59:0, "Alexander Korotkov" wrote: On Fri, Sep 22, 2017 at 7:16 PM, chenhj wrote: This is the new pacth with TAP test and use Macro XLOGDIR. Good. I took a quick look over the patch. Why do you need master_query(), standby_query()

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Petr Jelinek
On 23/09/17 19:01, Alvaro Hernandez wro> On 23/09/17 18:42, Euler Taveira wrote: >> 2017-09-22 19:28 GMT-03:00 Gregory Brail : >>> We have been working on a project that makes extensive use of logical >>> replication for use inside Apigee (which is a very small part of >>>

Re: [HACKERS] CREATE COLLATION does not sanitize ICU's BCP 47 language tags. Should it?

2017-09-25 Thread Peter Geoghegan
On Mon, Sep 25, 2017 at 9:06 AM, Robert Haas wrote: >> The big concern I have here is that this feels a lot like something that >> we'll regret at leisure, if it's not right in the first release. I'd >> much rather be restrictive in v10 and then loosen the rules later,

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Alvaro Hernandez
On 25/09/17 19:39, Petr Jelinek wrote: Well, test_decoding is not meant for production use anyway, no need for middleware to support it. The pgoutput is primarily used for internal replication purposes, which is why we need something with more interoperability in mind in the first place. The

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Alvaro Hernandez
On 25/09/17 22:08, Jignesh Shah wrote: On Mon, Sep 25, 2017 at 11:37 AM, Joshua D. Drake > wrote: On 09/25/2017 11:31 AM, Alvaro Hernandez wrote: Whether or not they are included in a managed environment

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Joshua D. Drake
On 09/25/2017 09:59 AM, Gregory Brail wrote: However, I can't find any docs for the output format of pgoutput, which is going to make it less likely for people to be able to consume it. Is anyone working on docs? I know that it's a painful process. I also think that a JSON-format (or

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Alvaro Hernandez
On 25/09/17 19:56, Andrew Dunstan wrote: On 09/25/2017 12:48 PM, Alvaro Hernandez wrote: On 25/09/17 19:39, Petr Jelinek wrote: Well, test_decoding is not meant for production use anyway, no need for middleware to support it. The pgoutput is primarily used for internal replication

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Tom Lane
Alvaro Hernandez writes: > In my opinion, logical decoding plugins that don't come with core > are close to worthless (don't get me wrong): > - They very unlikely will be installed in managed environments (an area > growing significantly). > - As anything that is not in core,

[HACKERS] Reading backup label file for checkpoint and redo location during crash recovery

2017-09-25 Thread Satyanarayana Narlapuram
Hi there, During crash recovery, last checkpoint record information is obtained from the backup label if present, instead of getting it from the control file. This behavior is causing PostgreSQL database cluster not to come up until the backup label file is deleted (as the error message says).

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Joshua D. Drake
On 09/25/2017 10:32 AM, Petr Jelinek wrote: On 25/09/17 19:26, Tom Lane wrote: Alvaro Hernandez writes: There is already about 3 million output plugins out there so I think we did reasonable job there. The fact that vast majority of that are various json ones gives

Re: [HACKERS] Server crash due to SIGBUS(Bus Error) when trying to access the memory created using dsm_create().

2017-09-25 Thread Tom Lane
Robert Haas writes: > On Mon, Sep 25, 2017 at 10:22 AM, Tom Lane wrote: >> Thomas Munro writes: >>> So, do we want this patch? >> I think we don't really have a lot of choice. I propose applying this >> as far back as

Re: [HACKERS] logical replication and statistics

2017-09-25 Thread Pavel Stehule
2017-09-25 19:23 GMT+02:00 Petr Jelinek : > On 25/09/17 19:19, Tom Lane wrote: > > Pavel Stehule writes: > >> I had two instances on one server with different port. I am sure, so > >> replication was functional. Only one issue is statistics

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Stephen Frost
Andres, all, * Andres Freund (and...@anarazel.de) wrote: > On 2017-09-25 19:32:29 +0200, Petr Jelinek wrote: > > On 25/09/17 19:26, Tom Lane wrote: > > > Alvaro Hernandez writes: > > >> In my opinion, logical decoding plugins that don't come with core > > >> are close to

Re: [HACKERS] [BUGS] BUG #14825: enum type: unsafe use?

2017-09-25 Thread Andrew Dunstan
On 09/25/2017 01:34 PM, David E. Wheeler wrote: > On Sep 25, 2017, at 10:55, Andrew Dunstan > wrote: > >> Let's ask a couple of users who I think are or have been actually >> hurting on this point. Christophe and David, any opinions? > If I understand the issue

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Andrew Dunstan
On 09/25/2017 12:48 PM, Alvaro Hernandez wrote: > > > On 25/09/17 19:39, Petr Jelinek wrote: >> >> Well, test_decoding is not meant for production use anyway, no need for >> middleware to support it. The pgoutput is primarily used for internal >> replication purposes, which is why we need

Re: [HACKERS] Shaky coding for vacuuming partitioned relations

2017-09-25 Thread Bossart, Nathan
On 9/24/17, 10:12 PM, "Michael Paquier" wrote: > Attached is a proposal of patch. The patch seems reasonable to me, and I haven't encountered any issues in my tests, even after applying the vacuum-multiple-relations patch on top of it. +* Take a lock

Re: [HACKERS] logical replication and statistics

2017-09-25 Thread Petr Jelinek
On 25/09/17 19:19, Tom Lane wrote: > Pavel Stehule writes: >> I had two instances on one server with different port. I am sure, so >> replication was functional. Only one issue is statistics > >> Master: > >> CREATE TABLE foo(id int primary key, a int); >> CREATE

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Joshua D. Drake
On 09/25/2017 10:19 AM, Petr Jelinek wrote: On 25/09/17 18:48, Alvaro Hernandez wrote:     In my opinion, logical decoding plugins that don't come with core are close to worthless (don't get me wrong): I respectfully disagree. As do I. - They very unlikely will be installed in

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Petr Jelinek
On 25/09/17 19:26, Tom Lane wrote: > Alvaro Hernandez writes: >> In my opinion, logical decoding plugins that don't come with core >> are close to worthless (don't get me wrong): > >> - They very unlikely will be installed in managed environments (an area >> growing

Re: [HACKERS] [BUGS] BUG #14825: enum type: unsafe use?

2017-09-25 Thread Christophe Pettus
> On Sep 25, 2017, at 07:55, Andrew Dunstan > wrote: > Let's ask a couple of users who I think are or have been actually > hurting on this point. Christophe and David, any opinions? Since about 90% of what I encounter in this area are automatically-generated

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Andres Freund
On 2017-09-25 10:38:52 -0700, Joshua D. Drake wrote: > On 09/25/2017 10:32 AM, Petr Jelinek wrote: > > On 25/09/17 19:26, Tom Lane wrote: > > > Alvaro Hernandez writes: > > > > > There is already about 3 million output plugins out there so I think we > > did reasonable job

Re: [HACKERS] Reading backup label file for checkpoint and redo location during crash recovery

2017-09-25 Thread Stephen Frost
Greetings Satya, * Satyanarayana Narlapuram (satyanarayana.narlapu...@microsoft.com) wrote: > During crash recovery, last checkpoint record information is obtained from > the backup label if present, instead of getting it from the control file. > This behavior is causing PostgreSQL database

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Joshua D. Drake
On 09/25/2017 10:43 AM, Andres Freund wrote: On 2017-09-25 10:38:52 -0700, Joshua D. Drake wrote: On 09/25/2017 10:32 AM, Petr Jelinek wrote: On 25/09/17 19:26, Tom Lane wrote: Alvaro Hernandez writes: There is already about 3 million output plugins out there so I think

Re: [HACKERS] Reading backup label file for checkpoint and redo location during crash recovery

2017-09-25 Thread Andres Freund
On 2017-09-25 13:43:32 -0400, Stephen Frost wrote: > Greetings Satya, > > * Satyanarayana Narlapuram (satyanarayana.narlapu...@microsoft.com) wrote: > > During crash recovery, last checkpoint record information is obtained from > > the backup label if present, instead of getting it from the

[HACKERS][BUG] Cache invalidation for queries that contains const of temporary composite type

2017-09-25 Thread Maksim Milyutin
Hello everyone! I have found out the problem when try to sequentially call the function that casts constant to composite type of temporary table that is deleted ateach transaction termination (i.e. at each function call completion). For example, we have the following function 'test':

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Tom Lane
Andres Freund writes: >> On 25/09/17 19:26, Tom Lane wrote: >>> The problem with this type of argument is that it leads directly to the >>> conclusion that every feature users want must be in core. > ... I don't think that should mean that there's no possible output > plugin

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Andres Freund
On 2017-09-25 13:50:29 -0400, Tom Lane wrote: > Andres Freund writes: > >> On 25/09/17 19:26, Tom Lane wrote: > >>> The problem with this type of argument is that it leads directly to the > >>> conclusion that every feature users want must be in core. > > > ... I don't think

Re: [HACKERS] PATCH: pgbench - option to build using ppoll() for larger connection counts

2017-09-25 Thread Andres Freund
On 2017-09-25 18:01:40 +, Rady, Doug wrote: > This patch enables building pgbench to use ppoll() instead of select() > to allow for more than (FD_SETSIZE - 10) connections. As implemented, > when using ppoll(), the only connection limitation is system resources. Hm, is there any need of

Re: [HACKERS] PATCH: pgbench - break out timing data for initialization phases

2017-09-25 Thread Fabien COELHO
Hello Doug, total time: 316.03 s (insert 161.60 s, commit 0.64 s, vacuum 60.77 s, index 93.01 s) Definitely interesting. There is a "ready for committers" patch in the CF which extensively rework the initialization: it becomes customizable, and this approach may not work as is after

Re: [HACKERS] PATCH: pgbench - option to build using ppoll() for larger connection counts

2017-09-25 Thread Fabien COELHO
Hello Again, Two patches attached. One based on REL9_6_STABLE. I'd be surprise that there would be a backport unless there is a bug, so this one might not be useful. One based on 'master' which can also apply to REL_10_STABLE. Could you add your patches to the next CF? -- Fabien. --

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Gregory Brail
I'm encouraged that pgoutput exists and I'm sorry that I missed it before. I think it's fine as a binary-only format. If someone can write a client for the Postgres wire protocol as documented in Chapter 52 of the docs, then they should have no trouble consuming the output from pgoutput. However,

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Gregory Brail
Yes. I'm advocating something "built-in" to Postgres. Any or all of those are likely a great starting point. As for protobuf, I'm a big advocate -- it is easy to use, fast, extensible, runs on lots of platforms, and produces very compact output. However it introduces a few dependencies to the

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Andres Freund
Hi, On 2017-09-25 12:56:00 -0400, Andrew Dunstan wrote: > A general purpose lower bandwidth plugin might one supporting Protocol > Buffers. The downside is that unlike json it's not self-contained, you > need the message definitions to interpret the stream, AIUI. I think that makes it a

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Petr Jelinek
On 25/09/17 18:48, Alvaro Hernandez wrote: > > > On 25/09/17 19:39, Petr Jelinek wrote: >> >> Well, test_decoding is not meant for production use anyway, no need for >> middleware to support it. The pgoutput is primarily used for internal >> replication purposes, which is why we need something

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Andres Freund
On 2017-09-24 13:36:56 +0300, Alvaro Hernandez wrote: >     However, if DMS uses it for what I'd call production use, I assume it is > actually production quality. I bet they do enough testing, and don't ship > software to potentially millions of customers if it doesn't work well. So... > first,

Re: [HACKERS] logical replication and statistics

2017-09-25 Thread Tom Lane
Pavel Stehule writes: > I had two instances on one server with different port. I am sure, so > replication was functional. Only one issue is statistics > Master: > CREATE TABLE foo(id int primary key, a int); > CREATE PUBLICATION test_pub FOR TABLE foo; > INSERT INTO

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Joshua D. Drake
On 09/25/2017 10:15 AM, Gregory Brail wrote: Yes. I'm advocating something "built-in" to Postgres. Any or all of those are likely a great starting point. I don't see a benefit to having this "in postgres". The whole reason we have built out a mature and extensible product is so that not

Re: [HACKERS] [BUGS] BUG #14825: enum type: unsafe use?

2017-09-25 Thread David E. Wheeler
On Sep 25, 2017, at 10:55, Andrew Dunstan wrote: > Let's ask a couple of users who I think are or have been actually > hurting on this point. Christophe and David, any opinions? If I understand the issue correctly, I think I’d be fine with requiring ALTER TYPE

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Andres Freund
On 2017-09-25 19:32:29 +0200, Petr Jelinek wrote: > On 25/09/17 19:26, Tom Lane wrote: > > Alvaro Hernandez writes: > >> In my opinion, logical decoding plugins that don't come with core > >> are close to worthless (don't get me wrong): > > > >> - They very unlikely will be

Re: [HACKERS] Reading backup label file for checkpoint and redo location during crash recovery

2017-09-25 Thread Magnus Hagander
On Mon, Sep 25, 2017 at 7:43 PM, Stephen Frost wrote: > Greetings Satya, > > * Satyanarayana Narlapuram (satyanarayana.narlapu...@microsoft.com) wrote: > > During crash recovery, last checkpoint record information is obtained > from the backup label if present, instead of

Re: [HACKERS] psql \d sequence display

2017-09-25 Thread Fabien COELHO
Hello, This should be fixed for PG10, so if you have any feedback on the design, please let me know soon. Works for me on head against a 9.6 server, which is good. My 0.02 €: \d+ does not show more. Maybe Type, Min, Max, Inc & Cycles are enough for \d? The next/future or last/previous

Re: [HACKERS] Reading backup label file for checkpoint and redo location during crash recovery

2017-09-25 Thread Stephen Frost
* Magnus Hagander (mag...@hagander.net) wrote: > On Mon, Sep 25, 2017 at 7:43 PM, Stephen Frost wrote: > > * Satyanarayana Narlapuram (satyanarayana.narlapu...@microsoft.com) wrote: > > > During crash recovery, last checkpoint record information is obtained > > from the backup

[HACKERS] Enhancements to passwordcheck

2017-09-25 Thread Bossart, Nathan
Hi hackers, Currently, the passwordcheck module provides a few basic checks to strengthen passwords. However, any configuration must be ready at compile time, and many common password requirements cannot be enforced without creating a custom version of this module. I think there are a number of

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Alvaro Hernandez
On 25/09/17 20:18, Andres Freund wrote: On 2017-09-24 13:36:56 +0300, Alvaro Hernandez wrote:     However, if DMS uses it for what I'd call production use, I assume it is actually production quality. I bet they do enough testing, and don't ship software to potentially millions of customers

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Alvaro Hernandez
On 25/09/17 20:31, Joshua D. Drake wrote: On 09/25/2017 10:19 AM, Petr Jelinek wrote: On 25/09/17 18:48, Alvaro Hernandez wrote: In my opinion, logical decoding plugins that don't come with core are close to worthless (don't get me wrong): I respectfully disagree. As do I.    

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Joshua D. Drake
On 09/25/2017 11:31 AM, Alvaro Hernandez wrote: Whether or not they are included in a managed environment is generally based on two things: 1. Safety (why RDS doesn't allow certain C extensions) 2. Community/Popularity (Exactly why RDS has PostGIS)     A. Demand with a

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-25 Thread Andres Freund
On 2017-09-25 21:31:11 +0300, Alvaro Hernandez wrote: > > > > - Distribution and testing are non-trivial: many OS/archs combinations. > > > > > > > > Yes, it is. Why would we want to increase that burden to this community? > > >     That's a different story, and one I cannot argue against. If

Re: [HACKERS] Patch to address concerns about ICU collcollate stability in v10 (Was: CREATE COLLATION does not sanitize ICU's BCP 47 language tags. Should it?)

2017-09-25 Thread Peter Eisentraut
On 9/25/17 00:24, Peter Geoghegan wrote: > * Creates root collation as root-x-icu (collcollate "root"), not > und-x-icu. "und" means undefined language. I'm curious about this point. "und" is defined in BCP 47. I don't see "root" defined anywhere. ICU converts the root collation to "und",

Re: [HACKERS] [PATCH] Generic type subscripting

2017-09-25 Thread Oleg Bartunov
On Fri, Sep 22, 2017 at 3:51 PM, Peter Eisentraut wrote: > On 9/21/17 11:24, Dmitry Dolgov wrote: >> One last thing that I need to clarify. Initially there was an idea to >> minimize changes in `pg_type` > > I see, but there is no value in that if it makes

Re: [HACKERS] Server crash due to SIGBUS(Bus Error) when trying to access the memory created using dsm_create().

2017-09-25 Thread Tom Lane
I wrote: > Hmm, so I tested this patch on my RHEL6 box (kernel 2.6.32) and it > immediately fell over with > 2017-09-25 14:23:48.410 EDT [325] FATAL: could not resize shared memory > segment "/PostgreSQL.1682054886" to 6928 bytes: Operation not supported > during startup. I wonder whether we

Re: [HACKERS] Reading backup label file for checkpoint and redo location during crash recovery

2017-09-25 Thread Satyanarayana Narlapuram
Thank you! Got it. -Original Message- From: Stephen Frost [mailto:sfr...@snowman.net] Sent: Monday, September 25, 2017 10:57 AM To: Magnus Hagander Cc: Satyanarayana Narlapuram ; PostgreSQL-development

Re: [HACKERS] Server crash due to SIGBUS(Bus Error) when trying to access the memory created using dsm_create().

2017-09-25 Thread Tom Lane
Robert Haas writes: > On Mon, Sep 25, 2017 at 10:22 AM, Tom Lane wrote: >> I think we don't really have a lot of choice. I propose applying this >> as far back as 9.6 --- anyone think differently? > +1. If applies to 9.5 and 9.4 without a lot of

Re: [HACKERS] CREATE COLLATION does not sanitize ICU's BCP 47 language tags. Should it?

2017-09-25 Thread Peter Eisentraut
On 9/22/17 16:46, Peter Geoghegan wrote: > But you are *already* canonicalizing ICU collation names as BCP 47. My > point here is: Why not finish the job off, and *also* canonicalize > colcollate in the same way? This won't break ucol_open() if we take > appropriate precautions when we go to use

Re: [HACKERS] [BUGS] BUG #14825: enum type: unsafe use?

2017-09-25 Thread Tom Lane
Andrew Dunstan writes: > OK, that seems to be the consensus. So let's apply the blacklist patch > and then separately remove the 'created in the same transaction' test. > We'll need to adjust the regression tests and docs accordingly. Agreed. I'll work on that in

  1   2   >