Re: [HACKERS] Contents of "backup_label" and "*.backup" in pg_wal location

2016-11-04 Thread Venkata B Nagothi
> On Fri, Nov 4, 2016 at 7:04 PM, Venkata B Nagothi > wrote: > > Sure. I will look at the possibility of using XLOG_BACKUP_END in my > patch. > > I am looking at the possibility of keeping the backup_label at source > until > > pg_stop_backup() > > is executed and then write the STOP information a

Re: [HACKERS] WAL consistency check facility

2016-11-04 Thread Michael Paquier
On Fri, Nov 4, 2016 at 6:02 PM, Michael Paquier wrote: > On Fri, Nov 4, 2016 at 5:02 PM, Michael Paquier > wrote: >> On Thu, Nov 3, 2016 at 11:17 PM, Kuntal Ghosh >> wrote: >>> I've updated the patch for review. >> >> Thank you for the new patch. This will be hopefully the last round of >> revie

Re: [HACKERS] macaddr 64 bit (EUI-64) datatype support

2016-11-04 Thread Peter Eisentraut
On 11/4/16 4:55 AM, Shay Rojansky wrote: > 1. Does everyone agrees that renaming the existing datatype without > changing the OID? > > > As I said before, Npgsql for one loads data types by name, not by OID. > So this would definitely cause breakage. Why would that cause breakage? -- P

Re: [HACKERS] add more NLS to bin

2016-11-04 Thread Peter Eisentraut
On 11/3/16 7:17 PM, Michael Paquier wrote: > This patch not being complicated, so I would vote for those being > addressed now so as they are not forgotten even if there is a FIXME > flag added. Perhaps you don't think so, and as that's a minor issue > I'll be fine with your judgement as well. OK,

Re: [HACKERS] Applying XLR_INFO_MASK correctly when looking at WAL record information

2016-11-04 Thread Michael Paquier
On Sat, Nov 5, 2016 at 2:29 AM, Tom Lane wrote: > Michael Paquier writes: >> I have just been trapped by the fact that in some code paths we look >> at the RMGR information of a WAL record (record->xl_info) but the >> filter dedicated to it, ~XLR_INFO_MASK, is not applied. This is >> harmful now,

Re: [HACKERS] btree_gin and btree_gist for enums

2016-11-04 Thread Andrew Dunstan
On 11/04/2016 04:14 PM, Emre Hasegeli wrote: Here is a patch to add enum support to btree_gin and btree_gist. I didn't include distance operations, as I didn't think it terribly important, and there isn't a simple way to compute it sanely and efficiently, so no KNN support. I don't think we ca

Re: [HACKERS] Add PGDLLEXPORT to PG_FUNCTION_INFO_V1

2016-11-04 Thread Tom Lane
Albe Laurenz writes: >> Anyway, I have prepared a patch along the lines you suggest. Pushed, we'll see if the buildfarm likes this iteration any better. > It occurred to me that the documentation still suggests that you should > add a declaration to a C function; I have fixed that too. I didn't

Re: [HACKERS] Improving executor performance

2016-11-04 Thread Doug Doole
On 07/13/2016 06:18 PM, Andres Freund wrote: Attached (in patch 0003) is a proof-of-concept implementing an expression evalution framework that doesn't use recursion. Instead ExecInitExpr2 computes a number of 'steps' necessary to compute an expression. These steps are stored in a linear array, a

Re: [HACKERS] Fix bug in handling of dropped columns in pltcl triggers

2016-11-04 Thread Tom Lane
Michael Paquier writes: > On Tue, Nov 1, 2016 at 4:17 AM, Jim Nasby wrote: >> While reviewing code coverage in pltcl, I uncovered a bug in trigger >> function return handling. If you returned the munged name of a dropped >> column, that would silently be ignored. It would be unusual to hit this,

[HACKERS] Tweak cost_merge_append to reflect 7a2fe9bd?

2016-11-04 Thread Thomas Munro
Hi As noted in a nearby review of some similar code[1], commit 7a2fe9bd made merge append very slightly more efficient, but nobody told cost_merge_append about the change. I doubt it makes much difference to the final cost in practice but I figured it might be worth correcting the comment. Does

Re: [HACKERS] Calculation of param_source_rels in add_paths_to_joinrel

2016-11-04 Thread Tom Lane
Ashutosh Bapat writes: > There's code in add_paths_to_joinrel() which computes the set of > target relations that should overlap parameterization of any proposed > join path. > ... > The calculations that follow are based on joinrel->relids (baserels > covered by the join) and SpecialJoinInfo list

Re: [HACKERS] Patch to implement pg_current_logfile() function

2016-11-04 Thread Karl O. Pinc
On Fri, 4 Nov 2016 16:58:45 +0100 Gilles Darold wrote: > I attached a v12 patch Attached is a comment patch which improves the comment describing CURRENT_LOG_FILENAME. It's been bugging me. I should have made this change long ago when I looked at all the other code comments but neglected to.

Re: [HACKERS] btree_gin and btree_gist for enums

2016-11-04 Thread Emre Hasegeli
>> Here is a patch to add enum support to btree_gin and btree_gist. I didn't >> include distance operations, as I didn't think it terribly important, and >> there isn't a simple way to compute it sanely and efficiently, so no KNN >> support. I don't think we can implement a meaningful distance ope

Re: [HACKERS] Gather Merge

2016-11-04 Thread Thomas Munro
On Sat, Nov 5, 2016 at 2:42 AM, Tom Lane wrote: > Michael Paquier writes: >> On Fri, Nov 4, 2016 at 12:00 PM, Thomas Munro >> wrote: >>> Shouldn't this say just "(c) 2016, PostgreSQL Global Development >>> Group"? Are we supposed to be blaming the University of California >>> for new files? > >

Re: [HACKERS] Hash Indexes

2016-11-04 Thread Amit Kapila
On Fri, Nov 4, 2016 at 9:37 PM, Robert Haas wrote: > On Tue, Nov 1, 2016 at 9:09 PM, Robert Haas wrote: >> On Mon, Oct 24, 2016 at 10:30 AM, Amit Kapila >> wrote: >>> [ new patches ] >> >> I looked over parts of this today, mostly the hashinsert.c changes. > > Some more review... > > @@ -656,6

Re: [HACKERS] Gather Merge

2016-11-04 Thread Thomas Munro
On Sat, Nov 5, 2016 at 1:55 AM, Robert Haas wrote: > On Thu, Nov 3, 2016 at 11:00 PM, Thomas Munro > wrote: >> + /* >> + * Avoid log(0)... >> + */ >> + N = (path->num_workers < 2) ? 2.0 : (double) path->num_workers; >> + logN = LOG2(N); >> ... >> + /* Per-tuple heap maintenance cost */ >> + run_c

Re: [HACKERS] Mention column name in error messages

2016-11-04 Thread David G. Johnston
On Fri, Nov 4, 2016 at 12:15 PM, Tom Lane wrote: > Michael Paquier writes: > > I am passing that down to a committer for review. The patch looks > > large, but at 95% it involves diffs in the regression tests, > > alternative outputs taking a large role in the bloat. > > This is kind of cute, bu

Re: [HACKERS] Mention column name in error messages

2016-11-04 Thread Tom Lane
Michael Paquier writes: > I am passing that down to a committer for review. The patch looks > large, but at 95% it involves diffs in the regression tests, > alternative outputs taking a large role in the bloat. This is kind of cute, but it doesn't seem to cover very much territory, because it onl

Re: [HACKERS] Bug in to_timestamp().

2016-11-04 Thread Tom Lane
Artur Zakirov writes: > I attached new version of the patch, which fix is_char_separator() > declaration too. I did some experimenting using http://rextester.com/l/oracle_online_compiler It appears that Oracle will consider a single space in the pattern to match zero or more spaces in the input,

Re: [HACKERS] Hash Indexes

2016-11-04 Thread Amit Kapila
On Fri, Nov 4, 2016 at 6:37 PM, Robert Haas wrote: > On Thu, Nov 3, 2016 at 6:25 AM, Amit Kapila wrote: >>> +nblkno = _hash_get_newblk(rel, pageopaque); >>> >>> I think this is not a great name for this function. It's not clear >>> what "new blocks" refers to, exactly. I suggest >>> FIN

Re: [HACKERS] Applying XLR_INFO_MASK correctly when looking at WAL record information

2016-11-04 Thread Tom Lane
Michael Paquier writes: > I have just been trapped by the fact that in some code paths we look > at the RMGR information of a WAL record (record->xl_info) but the > filter dedicated to it, ~XLR_INFO_MASK, is not applied. This is > harmful now, but this could lead to problems if in the future new >

Re: [HACKERS] delta relations in AFTER triggers

2016-11-04 Thread Kevin Grittner
On Sun, Oct 30, 2016 at 10:35 AM, Kevin Grittner wrote: > On Sun, Oct 2, 2016 at 11:20 PM, Michael Paquier > wrote: >> Not as big as I thought, only 2k when both patches are combined... The >> patch without noapi in its name needs to be applied first, and after >> the patch with noapi can be ap

Re: [HACKERS] Hash Indexes

2016-11-04 Thread Robert Haas
On Tue, Nov 1, 2016 at 9:09 PM, Robert Haas wrote: > On Mon, Oct 24, 2016 at 10:30 AM, Amit Kapila wrote: >> [ new patches ] > > I looked over parts of this today, mostly the hashinsert.c changes. Some more review... @@ -656,6 +678,10 @@ _hash_squeezebucket(Relation rel, IndexTuple

Re: [HACKERS] Password identifiers, protocol aging and SCRAM protocol

2016-11-04 Thread Peter Eisentraut
The organization of these patches makes sense to me. On 10/20/16 1:14 AM, Michael Paquier wrote: > - 0001, moving all the SHA2 functions to src/common/ and introducing a > PG-like interface. No actual changes here. That's probably alright, although the patch contains a lot more changes than I wou

Re: [HACKERS] Patch to implement pg_current_logfile() function

2016-11-04 Thread Gilles Darold
Le 04/11/2016 à 14:17, Karl O. Pinc a écrit : > On Mon, 31 Oct 2016 10:19:18 +0100 > Gilles Darold wrote: > >> Le 31/10/2016 à 04:35, Karl O. Pinc a écrit : >>> Attached is a patch to apply on top of the v10 patch. >>> >>> It updates current_logfiles only once per log rotation. >>> I see no reason

Re: [HACKERS] Proposal for changes to recovery.conf API

2016-11-04 Thread Abhijit Menon-Sen
At 2016-11-04 10:35:05 +, si...@2ndquadrant.com wrote: > > Since the "lots of parameters" approach is almost exactly what we have > now, I think we should do that first, get this patch committed and > then sit back and discuss an improved API and see what downsides it > introduces. Thanks, I a

Re: [HACKERS] Hash Indexes

2016-11-04 Thread Robert Haas
On Fri, Oct 28, 2016 at 12:33 AM, Amit Kapila wrote: > On Fri, Oct 28, 2016 at 2:52 AM, Robert Haas wrote: >> On Mon, Oct 24, 2016 at 10:30 AM, Amit Kapila >> wrote: Amit, can you please split the buffer manager changes in this patch into a separate patch? >>> >>> Sure, attached patch

Re: [HACKERS] Proposal for changes to recovery.conf API

2016-11-04 Thread Robert Haas
On Fri, Nov 4, 2016 at 6:04 AM, Michael Paquier wrote: > On Tue, Nov 1, 2016 at 11:31 PM, Robert Haas wrote: >> I liked Heikki's suggestion (at some point quite a while ago now) of >> recovery_target = 'xid 123' or recovery_target='lsn 0/723' or >> whatever. > > My vote goes for having two separa

Re: [HACKERS] Gather Merge

2016-11-04 Thread Tom Lane
Michael Paquier writes: > On Fri, Nov 4, 2016 at 12:00 PM, Thomas Munro > wrote: >> Shouldn't this say just "(c) 2016, PostgreSQL Global Development >> Group"? Are we supposed to be blaming the University of California >> for new files? > If the new file contains a portion of code from this age

Re: [HACKERS] Improve hash-agg performance

2016-11-04 Thread Andres Freund
On 2016-11-04 15:18:49 +0200, Heikki Linnakangas wrote: > On 11/03/2016 01:07 PM, Andres Freund wrote: > > Hi, > > > > There's two things I found while working on faster expression > > evaluation, slot deforming and batched execution. As those two issues > > often seemed quite dominant cost-wise i

Re: [HACKERS] Improve hash-agg performance

2016-11-04 Thread Heikki Linnakangas
On 11/03/2016 01:07 PM, Andres Freund wrote: Hi, There's two things I found while working on faster expression evaluation, slot deforming and batched execution. As those two issues often seemed quite dominant cost-wise it seemed worthwhile to evaluate them independently. 1) We atm do one ExecPr

Re: [HACKERS] Logical Replication WIP

2016-11-04 Thread Andres Freund
Hi, (btw, I vote against tarballing patches) + + + + Name + Type + References + Description + + + + + + oid + oid + + Row identifier (hidden attribute; must be explicitly selected) + + + + subpublications +

Re: [HACKERS] Patch to implement pg_current_logfile() function

2016-11-04 Thread Karl O. Pinc
On Mon, 31 Oct 2016 10:19:18 +0100 Gilles Darold wrote: > Le 31/10/2016 à 04:35, Karl O. Pinc a écrit : > > Attached is a patch to apply on top of the v10 patch. > > > > It updates current_logfiles only once per log rotation. > > I see no reason to open and write the file twice > > if both csvlo

Re: [HACKERS] Hash Indexes

2016-11-04 Thread Robert Haas
On Thu, Nov 3, 2016 at 6:25 AM, Amit Kapila wrote: >> +nblkno = _hash_get_newblk(rel, pageopaque); >> >> I think this is not a great name for this function. It's not clear >> what "new blocks" refers to, exactly. I suggest >> FIND_SPLIT_BUCKET(metap, bucket) or OLD_BUCKET_TO_NEW_BUCKET(m

Re: [HACKERS] Typo in comment in contrib/postgres_fdw/deparse.c

2016-11-04 Thread Robert Haas
On Fri, Nov 4, 2016 at 7:20 AM, Etsuro Fujita wrote: > On 2016/11/02 5:22, Robert Haas wrote: >> On Tue, Nov 1, 2016 at 8:20 AM, Etsuro Fujita >> wrote: >>> >>> I ran into a typo in a comment in contrib/postgres_fdw/deparse.c. Please >>> find attached a patch. > >> Committed. > > Thanks! > > I fo

Re: [HACKERS] Logical Replication WIP

2016-11-04 Thread Andres Freund
Hi, + + pg_publication_rel + + + pg_publication_rel + + + + The pg_publication_rel catalog contains + mapping between tables and publications in the database. This is many to + many mapping. + I wonder if we shouldn't abstract this a bit away from relations to allow other object

Re: [HACKERS] Gather Merge

2016-11-04 Thread Robert Haas
On Thu, Nov 3, 2016 at 11:00 PM, Thomas Munro wrote: > + /* > + * Avoid log(0)... > + */ > + N = (path->num_workers < 2) ? 2.0 : (double) path->num_workers; > + logN = LOG2(N); > ... > + /* Per-tuple heap maintenance cost */ > + run_cost += path->path.rows * comparison_cost * 2.0 * logN; > > Why m

Re: [HACKERS] Logical Replication WIP

2016-11-04 Thread Andres Freund
Hi, /* Prototypes for interface functions */ -static void libpqrcv_connect(char *conninfo); -static char *libpqrcv_get_conninfo(void); -static void libpqrcv_identify_system(TimeLineID *primary_tli); -static void libpqrcv_readtimelinehistoryfile(TimeLineID tli, char **filename, char **content, in

Re: [HACKERS] Patch to implement pg_current_logfile() function

2016-11-04 Thread Karl O. Pinc
On Thu, 3 Nov 2016 18:34:50 -0500 "Karl O. Pinc" wrote: > On Mon, 31 Oct 2016 09:26:27 +0100 > Gilles Darold wrote: > > > Le 30/10/2016 à 08:04, Karl O. Pinc a écrit : > > > > Have you given any thought to my proposal to change > > > CURRENT_LOG_FILENAME to LOG_METAINFO_FILE? > > Yes, I

Re: [HACKERS] Logical Replication WIP

2016-11-04 Thread Andres Freund
/* * Replication slot on-disk data structure. @@ -225,10 +226,25 @@ ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlot *slot = NULL; int i; - Assert(MyReplicationSlot == NULL); + /* Only aka ephemeral slots can survive across

Re: [HACKERS] Typo in comment in contrib/postgres_fdw/deparse.c

2016-11-04 Thread Etsuro Fujita
On 2016/11/02 5:22, Robert Haas wrote: On Tue, Nov 1, 2016 at 8:20 AM, Etsuro Fujita wrote: I ran into a typo in a comment in contrib/postgres_fdw/deparse.c. Please find attached a patch. Committed. Thanks! I found another typo in postgres_fdw.c. Attached is a patch for fixing that. B

Re: [HACKERS] Push down more full joins in postgres_fdw

2016-11-04 Thread Etsuro Fujita
On 2016/11/04 13:08, Ashutosh Bapat wrote: On Fri, Nov 4, 2016 at 9:19 AM, Etsuro Fujita wrote: On 2016/11/03 18:52, Ashutosh Bapat wrote: I wrote: Here is the updated version, Other than the above issue and the alias issue we discussed, I addressed all your comments except one on testing

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2016-11-04 Thread Ashutosh Bapat
On Mon, Oct 31, 2016 at 6:37 PM, Robert Haas wrote: > On Fri, Oct 28, 2016 at 3:09 AM, Ashutosh Bapat > wrote: >> I think there are going to be two kinds of partitioning use-cases. >> First, carefully hand-crafted by DBAs so that every partition is >> different from other and so is every join bet

Re: [HACKERS] Proposal for changes to recovery.conf API

2016-11-04 Thread Simon Riggs
On 4 November 2016 at 10:04, Michael Paquier wrote: > On Tue, Nov 1, 2016 at 11:31 PM, Robert Haas wrote: >> I liked Heikki's suggestion (at some point quite a while ago now) of >> recovery_target = 'xid 123' or recovery_target='lsn 0/723' or >> whatever. > > My vote goes for having two separate

Re: [HACKERS] Contents of "backup_label" and "*.backup" in pg_wal location

2016-11-04 Thread Michael Paquier
On Fri, Nov 4, 2016 at 7:04 PM, Venkata B Nagothi wrote: > Sure. I will look at the possibility of using XLOG_BACKUP_END in my patch. > I am looking at the possibility of keeping the backup_label at source until > pg_stop_backup() > is executed and then write the STOP information and then move it

Re: [HACKERS] Contents of "backup_label" and "*.backup" in pg_wal location

2016-11-04 Thread Venkata B Nagothi
On Fri, Nov 4, 2016 at 3:44 PM, Michael Paquier wrote: > On Fri, Nov 4, 2016 at 1:18 PM, Venkata B Nagothi > wrote: > > I see the following contents in the file > > "00010044.0060.backup" which was generated in the > pg_wal > > location during the online backup. When pg_stop_

Re: [HACKERS] Proposal for changes to recovery.conf API

2016-11-04 Thread Michael Paquier
On Tue, Nov 1, 2016 at 11:31 PM, Robert Haas wrote: > I liked Heikki's suggestion (at some point quite a while ago now) of > recovery_target = 'xid 123' or recovery_target='lsn 0/723' or > whatever. My vote goes for having two separate parameters, because as we know that there will be always two

Re: [HACKERS] [RFC] Should we fix postmaster to avoid slow shutdown?

2016-11-04 Thread Ashutosh Bapat
On Tue, Oct 4, 2016 at 2:35 PM, Tsunakawa, Takayuki wrote: > From: pgsql-hackers-ow...@postgresql.org >> [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Tom Lane > FWIW, I'm pretty much -1 on messing with the timing of the socket close >> actions. I broke that once within recent memory,

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-11-04 Thread Amit Kapila
On Thu, Nov 3, 2016 at 8:38 PM, Robert Haas wrote: > On Tue, Nov 1, 2016 at 11:31 PM, Tomas Vondra >> The difference is that both the fast-path locks and msgNumLock went into >> 9.2, so that end users probably never saw that regression. But we don't know >> if that happens for clog and WAL. >> >>

Re: [HACKERS] Danger of automatic connection reset in psql

2016-11-04 Thread Oleksandr Shulgin
On Thu, Nov 3, 2016 at 12:26 PM, Ashutosh Bapat < ashutosh.ba...@enterprisedb.com> wrote: > > A couple of doubts/suggestions: > 1. Should pset.conn_was_reset be set to false, when we make a > connection in do_connection()? Usually pset.conn_was_reset would be > initialised with 0 since it's a glob

Re: [HACKERS] WAL consistency check facility

2016-11-04 Thread Michael Paquier
On Fri, Nov 4, 2016 at 5:02 PM, Michael Paquier wrote: > On Thu, Nov 3, 2016 at 11:17 PM, Kuntal Ghosh > wrote: >> I've updated the patch for review. > > Thank you for the new patch. This will be hopefully the last round of > reviews, we are getting close to something that has an acceptable > sha

Re: [HACKERS] macaddr 64 bit (EUI-64) datatype support

2016-11-04 Thread Shay Rojansky
> > Yes. Before doing this change, it is better to confirm the approach and > then do all the changes. > > 1. Does everyone agrees that renaming the existing datatype without > changing the OID? > As I said before, Npgsql for one loads data types by name, not by OID. So this would definitely cause

Re: [HACKERS] Re: [sqlsmith] FailedAssertion("!(XLogCtl->Insert.exclusiveBackup)", File: "xlog.c", Line: 10200)

2016-11-04 Thread Kyotaro HORIGUCHI
Hello, (the header of this message is crafted so it might be isolate this message from the thread) The patch still applies on the current master with disaplacements. > On Tue, Aug 30, 2016 at 1:44 PM, Fujii Masao wrote: > > On Tue, Aug 30, 2016 at 1:32 PM, Michael Paquier > > wrote: > >> On Mo

Re: [HACKERS] Patch to implement pg_current_logfile() function

2016-11-04 Thread Gilles Darold
Le 04/11/2016 à 00:34, Karl O. Pinc a écrit : > On Mon, 31 Oct 2016 09:26:27 +0100 > Gilles Darold wrote: > >> Le 30/10/2016 à 08:04, Karl O. Pinc a écrit : >>> Have you given any thought to my proposal to change >>> CURRENT_LOG_FILENAME to LOG_METAINFO_FILE? >> Yes, I don't think the informatio

Re: [HACKERS] WAL consistency check facility

2016-11-04 Thread Michael Paquier
On Thu, Nov 3, 2016 at 11:17 PM, Kuntal Ghosh wrote: > I've updated the patch for review. Thank you for the new patch. This will be hopefully the last round of reviews, we are getting close to something that has an acceptable shape. + + + + + + Did you try to compile

Re: [HACKERS] Radix tree for character conversion

2016-11-04 Thread Kyotaro HORIGUCHI
Thank you for looling this. At Mon, 31 Oct 2016 17:11:17 +0100, Daniel Gustafsson wrote in <3fc648b5-2b7f-4585-9615-207a44b73...@yesql.se> > > On 27 Oct 2016, at 09:23, Kyotaro HORIGUCHI > > wrote: > > Perl scripts are to be messy, I believe. Anyway the duplicate > > check as been built into t