Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2016-08-31 Thread Bruce Momjian
On Wed, Aug 31, 2016 at 10:15:33PM +0530, Pavan Deolasee wrote: > Instead, what I would like to propose and the patch currently implements is to > restrict WARM update to once per chain. So the first non-HOT update to a tuple > or a HOT chain can be a WARM update. The chain can further be HOT

Re: [HACKERS] [Patch] Temporary tables that do not bloat pg_catalog (a.k.a fast temp tables)

2016-08-31 Thread Andres Freund
On August 31, 2016 3:00:15 PM PDT, Tomas Vondra wrote: > > >On 08/31/2016 11:43 PM, Andres Freund wrote: >> On 2016-08-31 23:40:46 +0200, Tomas Vondra wrote: >>> It's an improvement (and it's pretty much exactly what I proposed >>> upthread). But it does not solve

Re: [HACKERS] [Patch] Temporary tables that do not bloat pg_catalog (a.k.a fast temp tables)

2016-08-31 Thread Andres Freund
On 2016-08-31 23:40:46 +0200, Tomas Vondra wrote: > It's an improvement (and it's pretty much exactly what I proposed > upthread). But it does not solve the problems with pg_statistic for > example (each backend needs it's own statistics. So we'd either bloat > the pg_statistic (if we manage to

Re: [HACKERS] _mdfd_getseg can be expensive

2016-08-31 Thread Peter Geoghegan
On Wed, Aug 31, 2016 at 2:09 PM, Peter Geoghegan wrote: > The only thing that stuck out to any degree is that we don't grow the > "reln->md_seg_fds[forknum]" array within the new _fdvec_resize() > function geometrically. That new function looks like this: > +static void >

Re: [HACKERS] ICU integration

2016-08-31 Thread Doug Doole
Hi all. I’m new to the PostgreSQL code and the mailing list, but I’ve had a lot of experience with using ICU in a different database product. So while I’m not up to speed on the code yet, I can offer some insights on using ICU. > On Aug 30, 2016, at 9:12 PM, Peter Eisentraut >

Re: [HACKERS] _mdfd_getseg can be expensive

2016-08-31 Thread Andres Freund
On 2016-08-31 14:09:47 -0700, Peter Geoghegan wrote: > On Thu, Aug 18, 2016 at 5:26 PM, Andres Freund wrote: > > Rebased version attached. A review would be welcome. Plan to push this > > forward otherwise in the not too far away future. > > This looks good. Thanks for

Re: [HACKERS] _mdfd_getseg can be expensive

2016-08-31 Thread Peter Geoghegan
On Thu, Aug 18, 2016 at 5:26 PM, Andres Freund wrote: > Rebased version attached. A review would be welcome. Plan to push this > forward otherwise in the not too far away future. This looks good. The only thing that stuck out to any degree is that we don't grow the

Re: [HACKERS] [PATCH] Reload SSL certificates on SIGHUP

2016-08-31 Thread Peter Geoghegan
On Sun, Nov 22, 2015 at 7:29 PM, Andreas Karlsson wrote: > Sorry for dropping this patch, but now I have started looking at it again. Any chance of picking this up again soon, Andreas? I think it's an important project. I would like to review it. -- Peter Geoghegan --

[HACKERS] pg_basebackup stream xlog to tar

2016-08-31 Thread Magnus Hagander
Attached patch adds support for -X stream to work with .tar and .tar.gz file formats. If tar mode is specified, a separate pg_xlog.tar (or .tar.gz) file is created and the data is streamed into it. Regular mode is (should not) see any changes in how it works. The implementation creates a

Re: [HACKERS] WAL consistency check facility

2016-08-31 Thread Michael Paquier
On Wed, Aug 31, 2016 at 10:32 PM, Simon Riggs wrote: > On 27 August 2016 at 12:09, Kuntal Ghosh wrote: > * wal_consistency_mask = 511 /* Enable consistency check mask bit*/ >>> >>> What does this mean? (No docs) >> >> I was using this

Re: [HACKERS] autonomous transactions

2016-08-31 Thread Vik Fearing
On 08/31/2016 03:09 PM, Joel Jacobson wrote: > On Wed, Aug 31, 2016 at 6:41 AM, Jaime Casanova > wrote: >> >> On 30 August 2016 at 23:10, Joel Jacobson wrote: >>> >>> There should be a way to within the session and/or txn permanently >>> block

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2016-08-31 Thread Bruce Momjian
On Wed, Aug 31, 2016 at 04:03:29PM -0400, Bruce Momjian wrote: > Why not just remember the tid of chains converted from WARM to HOT, then > use "amrecheck" on an index entry matching that tid to see if the index > matches one of the entries in the chain. (It will match all of them or > none of

Re: [HACKERS] Replace use malloc() & friend by memory contexts for plperl and pltcl

2016-08-31 Thread Tom Lane
Michael Paquier writes: > Cleanup $subject has been raised a couple of times, like one year ago here: > https://www.postgresql.org/message-id/cab7npqrxvq+q66ufzd9wa5uaftyn4wauadbjxkfrync96kf...@mail.gmail.com > And more recently here while working on the NULL checks for

Re: [HACKERS] [Patch] Temporary tables that do not bloat pg_catalog (a.k.a fast temp tables)

2016-08-31 Thread Tomas Vondra
On 08/31/2016 09:20 PM, Vik Fearing wrote: > On 08/24/2016 06:16 PM, Robert Haas wrote: >> On Tue, Aug 23, 2016 at 6:11 PM, Tomas Vondra >> wrote: >>> Could someone please explain how the unlogged tables are supposed to fix the >>> catalog bloat problem, as stated

Re: [HACKERS] _mdfd_getseg can be expensive

2016-08-31 Thread Peter Geoghegan
On Wed, Aug 31, 2016 at 3:08 PM, Andres Freund wrote: > On August 31, 2016 3:06:23 PM PDT, Peter Geoghegan wrote: > >>In other painfully pedantic news, I should point out that >>sizeof(size_t) isn't necessarily word size (the most generic >>definition of word

Re: [HACKERS] delta relations in AFTER triggers

2016-08-31 Thread Kevin Grittner
I have merged in the changes since v4 (a year and a half ago) and cured all bit-rot I found, to get the attached v5 which runs `make check world` without problem -- including the tests added for this feature. I did remove the contrib code that David Fetter wrote to demonstrate the correctness and

Re: [HACKERS] [Patch] Temporary tables that do not bloat pg_catalog (a.k.a fast temp tables)

2016-08-31 Thread Tomas Vondra
On 08/31/2016 11:43 PM, Andres Freund wrote: > On 2016-08-31 23:40:46 +0200, Tomas Vondra wrote: >> It's an improvement (and it's pretty much exactly what I proposed >> upthread). But it does not solve the problems with pg_statistic for >> example (each backend needs it's own statistics. So we'd

Re: [HACKERS] _mdfd_getseg can be expensive

2016-08-31 Thread Peter Geoghegan
On Wed, Aug 31, 2016 at 2:37 PM, Andres Freund wrote: >> This looks good. > > Thanks for looking! No problem. In other painfully pedantic news, I should point out that sizeof(size_t) isn't necessarily word size (the most generic definition of word size for the architecture),

Re: [HACKERS] _mdfd_getseg can be expensive

2016-08-31 Thread Andres Freund
On August 31, 2016 3:06:23 PM PDT, Peter Geoghegan wrote: >In other painfully pedantic news, I should point out that >sizeof(size_t) isn't necessarily word size (the most generic >definition of word size for the architecture), contrary to my reading >of the 0002-* patch

Re: [HACKERS] [Patch] Temporary tables that do not bloat pg_catalog (a.k.a fast temp tables)

2016-08-31 Thread Corey Huinker
On Wed, Aug 31, 2016 at 6:07 PM, Andres Freund wrote: > > In my experience pg attribute is usually the worst affected. Many tech > takes won't even have stays entries... > > Mine too. One database currently has a 400GB pg_attribute table, because we chew through temp tables

[HACKERS] restoration after crash slowness, any way to improve?

2016-08-31 Thread Joshua D. Drake
-hackers, So this is more of a spit balling thread than anything. As I understand it, if we have a long running transaction or a large number of wal logs and we crash, we then have to restore those logs on restart to the last known good transaction. No problem. I recently ran a very long

Re: [HACKERS] Replace use malloc() & friend by memory contexts for plperl and pltcl

2016-08-31 Thread Michael Paquier
On Thu, Sep 1, 2016 at 8:57 AM, Tom Lane wrote: > I wrote: >> Michael Paquier writes: >>> Attached are a set of patches to replace those memory system calls by >>> proper memory contexts: >>> - 0001 does the cleanup work for pltcl >>> - 0002 does

Re: [HACKERS] Notice lock waits

2016-08-31 Thread Jeff Janes
On Tue, Aug 9, 2016 at 5:17 PM, Jim Nasby wrote: > On 8/5/16 12:00 PM, Jeff Janes wrote: > >> So I created a new guc, notice_lock_waits, which acts like >> log_lock_waits but sends the message as NOTICE so it will show up on >> interactive connections like psql. >> > >

Re: [HACKERS] WAL consistency check facility

2016-08-31 Thread Peter Geoghegan
On Wed, Aug 31, 2016 at 8:26 PM, Amit Kapila wrote: >> As far as I am understanding things, we are aiming at something that >> could be used on production systems. >> > > I don't think you can enable it by default in production systems. > Enabling it will lead to

[HACKERS] [PATCH] Logical decoding timeline following take II

2016-08-31 Thread Craig Ringer
Hi all Attached is a rebased and updated logical decoding timeline following patch for 10.0. This is a pre-requisite for the pending work on logical decoding on standby servers and simplified failover of logical decoding. Restating the commit message: __ Follow timeline switches in

Re: [HACKERS] WAL consistency check facility

2016-08-31 Thread Amit Kapila
On Thu, Sep 1, 2016 at 8:30 AM, Michael Paquier wrote: > On Thu, Sep 1, 2016 at 11:32 AM, Amit Kapila wrote: >> On Wed, Aug 31, 2016 at 7:02 PM, Simon Riggs wrote: >>> On 27 August 2016 at 12:09, Kuntal Ghosh

Re: [HACKERS] Replace use malloc() & friend by memory contexts for plperl and pltcl

2016-08-31 Thread Tom Lane
I wrote: > Michael Paquier writes: >> Attached are a set of patches to replace those memory system calls by >> proper memory contexts: >> - 0001 does the cleanup work for pltcl >> - 0002 does this cleanup for plperl > Off to look at 0002 next. Pushed 0002 as well.

Re: [HACKERS] WAL consistency check facility

2016-08-31 Thread Michael Paquier
On Thu, Sep 1, 2016 at 11:32 AM, Amit Kapila wrote: > On Wed, Aug 31, 2016 at 7:02 PM, Simon Riggs wrote: >> On 27 August 2016 at 12:09, Kuntal Ghosh wrote: >> > * wal_consistency_mask = 511 /* Enable consistency

Re: [HACKERS] WAL consistency check facility

2016-08-31 Thread Amit Kapila
On Thu, Sep 1, 2016 at 8:02 AM, Amit Kapila wrote: > On Wed, Aug 31, 2016 at 7:02 PM, Simon Riggs wrote: >> On 27 August 2016 at 12:09, Kuntal Ghosh wrote: >> > * wal_consistency_mask = 511 /* Enable consistency

Re: [HACKERS] Logical Replication WIP

2016-08-31 Thread Erik Rijkers
On 2016-08-31 22:51, Petr Jelinek wrote: Hi, and one more version with bug fixes, improved code docs and couple more tests, some general cleanup and also rebased on current master for the start of CF. Clear, well-written docs, thanks. Here are some small changes to logical-replication.sgml

Re: [HACKERS] WAL consistency check facility

2016-08-31 Thread Amit Kapila
On Wed, Aug 31, 2016 at 7:02 PM, Simon Riggs wrote: > On 27 August 2016 at 12:09, Kuntal Ghosh wrote: > * wal_consistency_mask = 511 /* Enable consistency check mask bit*/ >>> >>> What does this mean? (No docs) >> >> I was using this

Re: [HACKERS] Missing checks when malloc returns NULL...

2016-08-31 Thread Michael Paquier
On Thu, Sep 1, 2016 at 12:14 AM, Tom Lane wrote: > I still think it'd be better to fix that as attached, because it > represents a net reduction not net addition of code, and it provides > a defense against future repetitions of the same omission. If only > 4 out of 11

[HACKERS] pg_recvlogical --endpos

2016-08-31 Thread Craig Ringer
Hi all Here's a rebased version of my pg_recvlogical --endpos patch from the 9.5 series, updated to incoroprate Álvaro's changes. This will be mainly useful for recovery tests as we start adding more logical decoding testing. See original post for more detail:

[HACKERS] Re: [COMMITTERS] pgsql: Add putenv support for msvcrt from Visual Studio 2013

2016-08-31 Thread Michael Paquier
On Wed, Apr 27, 2016 at 2:39 AM, Christian Ullrich wrote: > * Christian Ullrich wrote: > >> wrong even without considering the debug/release split. If we load a >> compiled extension built with a CRT we have not seen yet, _after_ the >> first call to pgwin32_putenv(), that

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2016-08-31 Thread Craig Ringer
On 29 August 2016 at 15:53, Craig Ringer wrote: > Said better approach attached in revised series. Thanks. Here's another minor update to the txid_status() and txid_convert_if_recent() patches. The only change is moving get_xid_in_recent_past from

Re: [HACKERS] Proposal for changes to recovery.conf API

2016-08-31 Thread Michael Paquier
On Thu, Sep 1, 2016 at 1:15 AM, Simon Riggs wrote: > This is a summary of proposed changes to the recovery.conf API for > v10. These are based in part on earlier discussions, and represent a > minimal modification to current usage. > > Proposed changes (with reference to

Re: [HACKERS] Declarative partitioning - another take

2016-08-31 Thread Amit Langote
On 2016/08/25 16:15, Ashutosh Bapat wrote: > On Thu, Aug 25, 2016 at 12:22 PM, Amit Langote wrote: >> b) >> when accumulating append subpaths, do not flatten a subpath that is itself >> an append when ((AppendPath *) subpath)->path.parent is a RelOptInfo with >> non-NULL partitioning info.Is the

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2016-08-31 Thread Craig Ringer
On 1 September 2016 at 13:08, Craig Ringer wrote: > On 29 August 2016 at 15:53, Craig Ringer wrote: > >> Said better approach attached in revised series. Thanks. > > Here's another minor update to the txid_status() and > txid_convert_if_recent()

Re: [HACKERS] WAL consistency check facility

2016-08-31 Thread Amit Kapila
On Thu, Sep 1, 2016 at 9:43 AM, Peter Geoghegan wrote: > On Wed, Aug 31, 2016 at 8:26 PM, Amit Kapila wrote: >>> As far as I am understanding things, we are aiming at something that >>> could be used on production systems. >>> >> >> I don't think you can

Re: [HACKERS] ICU integration

2016-08-31 Thread Peter Geoghegan
On Tue, Aug 30, 2016 at 7:46 PM, Peter Eisentraut wrote: > Here is a patch I've been working on to allow the use of ICU for sorting > and other locale things. I'm really happy that you're working on this. This is more important than is widely appreciated, and

Re: [HACKERS] pg_sequence catalog

2016-08-31 Thread Tom Lane
and...@anarazel.de (Andres Freund) writes: > On 2016-08-31 14:25:43 -0300, Alvaro Herrera wrote: >> Yes, sure, we're still improving even if we stick to one-seq-per-bufpage, >> but while we're at it, we could as well find a way to make it as best as >> we can. And allowing multiple seqs per page

Re: [HACKERS] pg_sequence catalog

2016-08-31 Thread Andres Freund
On 2016-08-31 13:59:48 -0400, Tom Lane wrote: > and...@anarazel.de (Andres Freund) writes: > > On 2016-08-31 14:25:43 -0300, Alvaro Herrera wrote: > >> Yes, sure, we're still improving even if we stick to one-seq-per-bufpage, > >> but while we're at it, we could as well find a way to make it as

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2016-08-31 Thread Pavan Deolasee
On Wed, Aug 31, 2016 at 10:38 PM, Claudio Freire wrote: > > On Wed, Aug 31, 2016 at 1:45 PM, Pavan Deolasee > wrote: > >> We discussed a few ideas to address the "Duplicate Scan" problem. For >> example, we can teach Index AMs to discard any

[HACKERS] Re: [COMMITTERS] pgsql: Use static inline functions for float <-> Datum conversions.

2016-08-31 Thread Heikki Linnakangas
On 08/31/2016 08:27 PM, Tom Lane wrote: Heikki Linnakangas writes: Use static inline functions for float <-> Datum conversions. Hmm, it looks like narwhal for one is not happy with this:

Re: [HACKERS] pg_sequence catalog

2016-08-31 Thread Alvaro Herrera
Andres Freund wrote: > Hi, > > On 2016-08-31 12:53:30 -0400, Tom Lane wrote: > > Improving on the space wastage is exactly the point IMO. If it's still > > going to be 8k per sequence on disk (*and* in shared buffers, remember), > > I'm not sure it's worth all the work to change things at all. >

Re: [HACKERS] pg_sequence catalog

2016-08-31 Thread Andres Freund
On 2016-08-31 14:25:43 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > Hi, > > > > On 2016-08-31 12:53:30 -0400, Tom Lane wrote: > > > Improving on the space wastage is exactly the point IMO. If it's still > > > going to be 8k per sequence on disk (*and* in shared buffers, remember), > >

Re: [HACKERS] [COMMITTERS] pgsql: Use static inline functions for float <-> Datum conversions.

2016-08-31 Thread Tom Lane
Heikki Linnakangas writes: > Use static inline functions for float <-> Datum conversions. Hmm, it looks like narwhal for one is not happy with this: http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=narwhal=2016-08-31%2016%3A00%3A03 I suspect the failure traces

Re: [HACKERS] ICU integration

2016-08-31 Thread Peter Geoghegan
On Tue, Aug 30, 2016 at 7:46 PM, Peter Eisentraut wrote: > In initdb, I initialize the default collation set as before from the > `locale -a` output, but also add all available ICU locales with a "%icu" > appended (so "fr_FR%icu"). I suppose one could create a

Re: [HACKERS] [COMMITTERS] pgsql: Use static inline functions for float <-> Datum conversions.

2016-08-31 Thread Tom Lane
Heikki Linnakangas writes: > On 08/31/2016 08:27 PM, Tom Lane wrote: >> We had a bunch of similar problems in the recent work on exact degree trig >> functions, and eventually found that storing values into volatile float8 >> variables was the most reliable way to force rounding

Re: [HACKERS] Proposal for changes to recovery.conf API

2016-08-31 Thread Abhijit Menon-Sen
At 2016-08-31 17:15:59 +0100, si...@2ndquadrant.com wrote: > > * Recovery parameters would now be part of the main postgresql.conf > infrastructure > Any parameters set in $DATADIR/recovery.conf will be read after the > main parameter file, similar to the way that postgresql.conf.auto is > read. >

Re: [HACKERS] [Patch] Temporary tables that do not bloat pg_catalog (a.k.a fast temp tables)

2016-08-31 Thread Vik Fearing
On 08/24/2016 06:16 PM, Robert Haas wrote: > On Tue, Aug 23, 2016 at 6:11 PM, Tomas Vondra > wrote: >> Could someone please explain how the unlogged tables are supposed to fix the >> catalog bloat problem, as stated in the initial patch submission? We'd still >> need

Re: [HACKERS] [Patch] Temporary tables that do not bloat pg_catalog (a.k.a fast temp tables)

2016-08-31 Thread Corey Huinker
On Wed, Aug 24, 2016 at 12:39 PM, Andres Freund wrote: > > > On August 24, 2016 9:32:48 AM PDT, Tomas Vondra < > tomas.von...@2ndquadrant.com> wrote: > > > > > >On 08/24/2016 12:20 AM, Andres Freund wrote: > >> On 2016-08-23 19:18:04 -0300, Claudio Freire wrote: > >>> On Tue,

Re: [HACKERS] pg_sequence catalog

2016-08-31 Thread Tom Lane
Andres Freund writes: > On 2016-08-31 13:59:48 -0400, Tom Lane wrote: >> You are ignoring the performance costs associated with eating 100x more >> shared buffer space than necessary. > I doubt that's measurable in any real-world scenario. You seldomly have > hundreds of

Re: [HACKERS] pg_sequence catalog

2016-08-31 Thread Andres Freund
On 2016-08-31 14:23:41 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2016-08-31 13:59:48 -0400, Tom Lane wrote: > >> You are ignoring the performance costs associated with eating 100x more > >> shared buffer space than necessary. > > > I doubt that's measurable in any

Re: [HACKERS] autonomous transactions

2016-08-31 Thread Serge Rielau
> On Aug 31, 2016, at 6:46 AM, Greg Stark wrote: > > Using a background worker mean that the autonomous transaction can't > access any state from the process memory. Parameters in plpgsql are a > symptom of this but I suspect there will be others. What happens if a > statement

Re: [HACKERS] Declarative partitioning - another take

2016-08-31 Thread Amit Langote
On 2016/08/29 20:53, Robert Haas wrote: > On Fri, Aug 26, 2016 at 1:33 PM, Amit Langote > wrote: >> We do take a lock on the parent because we would be changing its partition >> descriptor (relcache). I changed MergeAttributes() such that an >> AccessExclusiveLock

Re: [HACKERS] Confusing docs about GetForeignUpperPaths in fdwhandler.sgml

2016-08-31 Thread Etsuro Fujita
On 2016/08/01 21:14, Etsuro Fujita wrote: I noticed that the following note about direct modification via GetForeignUpperPaths in fdwhandler.sgml is a bit confusing. We have another approach using PlanDirectModify, so that should be reflected in the note as well. Please find attached a patch.

Re: [HACKERS] Patch: Implement failover on libpq connect level.

2016-08-31 Thread Victor Wagner
On Fri, 26 Aug 2016 10:10:33 +0530 Mithun Cy wrote: > On Thu, Mar 17, 2016 at 4:47 AM, David Steele > wrote: > >Since there has been no response from the author I have marked this > >patch > "returned with feedback". Please feel free >to

Re: [HACKERS] Patch: Implement failover on libpq connect level.

2016-08-31 Thread Victor Wagner
On Tue, 30 Aug 2016 14:54:57 +0530 Mithun Cy wrote: > On Fri, Aug 26, 2016 at 10:10 AM, Mithun Cy > wrote: > > > > >rpath,'/home/mithun/edbsrc/patch6bin/lib',--enable-new-dtags > > >-lecpg > > -lpgtypes -lpq -lpgcommon -lpgport -lz

Re: [HACKERS] Aggregate Push Down - Performing aggregation on foreign server

2016-08-31 Thread Robert Haas
On Wed, Aug 31, 2016 at 11:56 AM, Pavel Stehule wrote: > It is pity - lot of performance issues are related to this missing feature. I don't think you are being very clear about what feature you are talking about. The feature that Jeevan has implemented is pushing

Re: [HACKERS] Declarative partitioning - another take

2016-08-31 Thread Amit Langote
On 2016/08/31 16:17, Robert Haas wrote: > On Wed, Aug 31, 2016 at 12:37 PM, Amit Langote wrote: >> What I was trying to understand is why this would not be possible >> with a design where partition bound is stored in the catalog as a property >> of individual partitions instead of a design where

Re: [HACKERS] Aggregate Push Down - Performing aggregation on foreign server

2016-08-31 Thread Pavel Stehule
2016-08-31 10:03 GMT+02:00 Amit Langote : > On 2016/08/31 16:42, Pavel Stehule wrote: > > 2016-08-31 9:00 GMT+02:00 Robert Haas : > > > >> On Wed, Aug 31, 2016 at 11:56 AM, Pavel Stehule < > pavel.steh...@gmail.com> > >> wrote: > >>> It is

Re: [HACKERS] Logical decoding restart problems

2016-08-31 Thread Stas Kelvich
> On 31 Aug 2016, at 03:28, Craig Ringer wrote: > > On 25 Aug. 2016 20:03, "Stas Kelvich" wrote: > > > > Thanks for clarification about how restart_lsn is working. > > > > Digging slightly deeper into this topic revealed that problem was in two

Re: [HACKERS] Missing checks when malloc returns NULL...

2016-08-31 Thread Michael Paquier
On Wed, Aug 31, 2016 at 2:15 AM, Tom Lane wrote: > Michael Paquier writes: >> And with an actual patch things are better. > > Working through this patch, it suddenly strikes me that we are going > about fixing the callers of simple_prompt the wrong

Re: [HACKERS] Declarative partitioning - another take

2016-08-31 Thread Robert Haas
On Wed, Aug 31, 2016 at 12:37 PM, Amit Langote wrote: >>> If we need an AccessExclusiveLock on parent to add/remove a partition >>> (IOW, changing that child table's partitioning information), then do we >>> need to lock the individual partitions when reading

Re: [HACKERS] Aggregate Push Down - Performing aggregation on foreign server

2016-08-31 Thread Pavel Stehule
2016-08-31 9:00 GMT+02:00 Robert Haas : > On Wed, Aug 31, 2016 at 11:56 AM, Pavel Stehule > wrote: > > It is pity - lot of performance issues are related to this missing > feature. > > I don't think you are being very clear about what feature you

Re: [HACKERS] Small patch for snapmgr.c

2016-08-31 Thread Aleksander Alekseev
> Hi Aleksander, > > This has already been fixed with commit > 4f9f495889d3d410195c9891b58228727b340189 > > Thanks Agree, it's fixed now. Thank you! -- Best regards, Aleksander Alekseev -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Aggregate Push Down - Performing aggregation on foreign server

2016-08-31 Thread Jeevan Chalke
On Tue, Aug 30, 2016 at 6:51 PM, Pavel Stehule wrote: > Hi > > 2016-08-30 15:02 GMT+02:00 Jeevan Chalke : > >> Hi all, >> >> Attached is the patch which adds support to push down aggregation and >> grouping >> to the foreign server for

Re: [HACKERS] comment fix for CUSTOMPATH_* flags

2016-08-31 Thread Robert Haas
On Mon, Aug 29, 2016 at 6:04 AM, Kouhei Kaigai wrote: > I noticed the source code comment around CustomPath structure says "see above" > for definition of CUSTOMPATH_* flags. It was originally right, but it was > moved > to nodes/extensible.h on the further development. So,

[HACKERS] Replace use malloc() & friend by memory contexts for plperl and pltcl

2016-08-31 Thread Michael Paquier
Hi all, Cleanup $subject has been raised a couple of times, like one year ago here: https://www.postgresql.org/message-id/cab7npqrxvq+q66ufzd9wa5uaftyn4wauadbjxkfrync96kf...@mail.gmail.com And more recently here while working on the NULL checks for malloc():

Re: [HACKERS] Small patch for snapmgr.c

2016-08-31 Thread Jeevan Chalke
Hi Aleksander, This has already been fixed with commit 4f9f495889d3d410195c9891b58228727b340189 Thanks On Fri, Apr 8, 2016 at 6:02 PM, Aleksander Alekseev < a.aleks...@postgrespro.ru> wrote: > Hello > > Currently there is a following piece of code in snapmgr.c: > > ``` > /* Copy all required

Re: [HACKERS] Aggregate Push Down - Performing aggregation on foreign server

2016-08-31 Thread Pavel Stehule
2016-08-31 8:17 GMT+02:00 Jeevan Chalke : > > > On Tue, Aug 30, 2016 at 6:51 PM, Pavel Stehule > wrote: > >> Hi >> >> 2016-08-30 15:02 GMT+02:00 Jeevan Chalke >> : >> >>> Hi all, >>> >>> Attached is the

Re: [HACKERS] Comment on GatherPath.single_copy

2016-08-31 Thread Robert Haas
On Tue, Aug 30, 2016 at 6:38 PM, Tom Lane wrote: > Kyotaro HORIGUCHI writes: >> - boolsingle_copy;/* path must not be executed >1x */ >> + boolsingle_copy;/* path must not span on multiple >> processes

Re: [HACKERS] Missing checks when malloc returns NULL...

2016-08-31 Thread Michael Paquier
On Wed, Aug 31, 2016 at 7:47 AM, Tom Lane wrote: > Michael Paquier writes: >> [ malloc-nulls-v5.patch ] > > I've committed some form of all of these changes Thanks! > except the one in > adt/pg_locale.c. I'm not entirely sure whether we need to

Re: [HACKERS] Aggregate Push Down - Performing aggregation on foreign server

2016-08-31 Thread Amit Langote
On 2016/08/31 16:42, Pavel Stehule wrote: > 2016-08-31 9:00 GMT+02:00 Robert Haas : > >> On Wed, Aug 31, 2016 at 11:56 AM, Pavel Stehule >> wrote: >>> It is pity - lot of performance issues are related to this missing >> feature. >> >> I don't

[HACKERS] Exclude schema during pg_restore

2016-08-31 Thread Michael Banck
Hi, attached is a small patch that adds an -N option to pg_restore, in order to exclude a schema, in addition to -n for the restriction to a schema. In principle, this could be extended to -t etc., but I think having this for schemas would be the most useful with the least effort. One use case

Re: [HACKERS] 9.5.4: Segfault (signal 11) while running ALTER TABLE

2016-08-31 Thread Devrim Gündüz
Hi Tom, On Tue, 2016-08-30 at 08:18 -0400, Tom Lane wrote: > The above isn't ever likely to work for any large value of "work", > because the function would be confused about what the table rowtype > is.  I thought we had adequate defenses in there to throw an error > if you try to access a

Re: [HACKERS] GIN logging GIN_SEGMENT_UNMODIFIED actions?

2016-08-31 Thread Fujii Masao
On Wed, Aug 31, 2016 at 12:10 AM, Tom Lane wrote: > Fujii Masao writes: >> I found that pg_xlogdump code for XLOG_GIN_INSERT record with >> GIN_INSERT_ISLEAF flag has the same issue, i.e., >> "unknown action 0" error is thrown for that record. >> The

Re: [HACKERS] Comment on GatherPath.single_copy

2016-08-31 Thread Tom Lane
Robert Haas writes: > On Tue, Aug 30, 2016 at 6:38 PM, Tom Lane wrote: >> Robert, could you fix the documentation for that field so it's >> intelligible? > Uh, maybe. The trick, as you've already noted, is finding something > better. Maybe this: > -

Re: [HACKERS] Use static inline functions for Float <-> Datum conversions

2016-08-31 Thread Heikki Linnakangas
On 08/31/2016 02:38 PM, Tom Lane wrote: Heikki Linnakangas writes: Now that we are OK with static inline functions, we can save some cycles from floating-point functions, by turning Float4GetDatum, Float8GetDatum, and DatumGetFloat8 into static inlines. Looks good to me.

Re: [HACKERS] Use static inline functions for Float <-> Datum conversions

2016-08-31 Thread Tom Lane
Heikki Linnakangas writes: > Now that we are OK with static inline functions, we can save some cycles > from floating-point functions, by turning Float4GetDatum, > Float8GetDatum, and DatumGetFloat8 into static inlines. Looks good to me. I wonder whether there is a

Re: [HACKERS] GIN logging GIN_SEGMENT_UNMODIFIED actions?

2016-08-31 Thread Tom Lane
Fujii Masao writes: > On Wed, Aug 31, 2016 at 12:10 AM, Tom Lane wrote: >> Hmm, comparing gin_desc() to ginRedoInsert() makes me think there are more >> problems there than that one. Aren't the references to "payload" wrong >> in all three branches of

[HACKERS] Use static inline functions for Float <-> Datum conversions

2016-08-31 Thread Heikki Linnakangas
Hi, Now that we are OK with static inline functions, we can save some cycles from floating-point functions, by turning Float4GetDatum, Float8GetDatum, and DatumGetFloat8 into static inlines. They are only a few instructions, but couldn't be implemented as macros before, because they need a

Re: [HACKERS] Exclude schema during pg_restore

2016-08-31 Thread Fabrízio de Royes Mello
Em quarta-feira, 31 de agosto de 2016, Michael Banck escreveu: > Hi, > > attached is a small patch that adds an -N option to pg_restore, in order > to exclude a schema, in addition to -n for the restriction to a schema. > > In principle, this could be extended to -t etc., but I

Re: [HACKERS] INSERT .. SET syntax

2016-08-31 Thread Marko Tiikkaja
Hello hello, Here's a rebased and updated patch for $SUBJECT for the September commit fest. .m *** a/doc/src/sgml/ref/insert.sgml --- b/doc/src/sgml/ref/insert.sgml *** *** 22,33 PostgreSQL documentation [ WITH [ RECURSIVE ] with_query [, ...] ] ! INSERT INTO

Re: [HACKERS] standalone backend PANICs during recovery

2016-08-31 Thread Tom Lane
Michael Paquier writes: > On Tue, Aug 30, 2016 at 11:00 PM, Tom Lane wrote: >> Thinking about it some more ... what we actually need to prevent, AFAICS, >> is standby_mode becoming true in a standalone backend. > I have spent some time playing with

Re: [HACKERS] [WIP] Patches to enable extraction state of query execution from external session

2016-08-31 Thread Maksim Milyutin
On 2016-08-30 11:22:43 +0300, Maksim Milyutin wrote: Hi, On 2016-08-29 18:22:56 +0300, maksim wrote: Now I complete extension that provides facility to see the current state of query execution working on external session in form of EXPLAIN ANALYZE output. This extension works on 9.5 version,

Re: [HACKERS] autonomous transactions

2016-08-31 Thread Pavel Stehule
2016-08-31 15:09 GMT+02:00 Joel Jacobson : > On Wed, Aug 31, 2016 at 6:41 AM, Jaime Casanova > wrote: > > > > On 30 August 2016 at 23:10, Joel Jacobson wrote: > > > > > > There should be a way to within the session and/or txn

Re: [HACKERS] some requests on auditing

2016-08-31 Thread David Steele
On 8/30/16 10:12 AM, Pavel Stehule wrote: I am working on pgaudit customization for one my customer. There are few requests: 1. flat format without complex types, without nesting - CSV is ideal. 2. all important attributes should be separated - is not possible to search in original queries:

Re: [HACKERS] Use static inline functions for Float <-> Datum conversions

2016-08-31 Thread Tom Lane
Heikki Linnakangas writes: > On 08/31/2016 02:38 PM, Tom Lane wrote: >> I wonder whether there is a compiler-dependent way of avoiding the union >> trick ... or maybe gcc is already smart enough that it doesn't matter? > It seems to compile into a single instruction, so it can't

Re: [HACKERS] pg_sequence catalog

2016-08-31 Thread Craig Ringer
On 31 August 2016 at 21:17, Peter Eisentraut wrote: > While I was hacking around sequence stuff, I felt the urge to look into > an old peeve: That sequence metadata is not stored in a proper catalog. > Right now in order to find out some metadata about sequences

[HACKERS] Leftover member in openssl part of Port struct

2016-08-31 Thread Daniel Gustafsson
When SSL renegotiation was removed in 426746b9 the only consumer of the openssl specific count member in the Port struct was removed, but the member was left together with a few updates to it which are unused. Attached patch removes the leftovers which now serves no purpose unless I’m missing

Re: [HACKERS] pg_sequence catalog

2016-08-31 Thread Tom Lane
Craig Ringer writes: > On 31 August 2016 at 21:17, Peter Eisentraut > wrote: >> I don't know if this is a net improvement. Maybe this introduces as >> many new issues as it removes. But I figured I'll post this, so that at >> least we

Re: [HACKERS] autonomous transactions

2016-08-31 Thread Craig Ringer
On 31 August 2016 at 21:46, Greg Stark wrote: > On Wed, Aug 31, 2016 at 2:50 AM, Peter Eisentraut > wrote: >> - A API interface to open a "connection" to a background worker, run >> queries, get results: AutonomousSessionStart(),

Re: [HACKERS] Leftover member in openssl part of Port struct

2016-08-31 Thread Tom Lane
Daniel Gustafsson writes: > When SSL renegotiation was removed in 426746b9 the only consumer of the > openssl > specific count member in the Port struct was removed, but the member was left > together with a few updates to it which are unused. Attached patch removes > the >

Re: [HACKERS] Exclude schema during pg_restore

2016-08-31 Thread Michael Banck
Hi, Am Mittwoch, den 31.08.2016, 07:59 -0300 schrieb Fabrízio de Royes Mello: > Please add it to the next open commitfest. I had done so already: https://commitfest.postgresql.org/10/762/ Regards, Michael -- Michael Banck Projektleiter / Senior Berater Tel.: +49 2166 9901-171 Fax: +49

Re: [HACKERS] WAL consistency check facility

2016-08-31 Thread Simon Riggs
On 27 August 2016 at 12:09, Kuntal Ghosh wrote: >>> * wal_consistency_mask = 511 /* Enable consistency check mask bit*/ >> >> What does this mean? (No docs) > > I was using this parameter as a masking integer to indicate the > operations(rmgr list) for which we need

Re: [HACKERS] some requests on auditing

2016-08-31 Thread David Steele
On 8/31/16 9:39 AM, David Steele wrote: On 8/30/16 10:12 AM, Pavel Stehule wrote: #3 is not likely without changes to logging in Postgres. However, there are plenty of tools for log analysis (e.g. ELK) that might help and a Postgres extension that allows log messages to be directed elsewhere

[HACKERS] make async slave to wait for lsn to be replayed

2016-08-31 Thread Ivan Kartyshov
Hi hackers, Few days earlier I've finished my work on WAITLSN statement utility, so I’d like to share it. Introduction Our clients who deal with 9.5 and use asynchronous master-slave replication, asked to make the wait-mechanism on the slave side to prevent the situation when

Re: [HACKERS] Leftover member in openssl part of Port struct

2016-08-31 Thread Daniel Gustafsson
> On 31 Aug 2016, at 15:12, Tom Lane wrote: > > Daniel Gustafsson writes: >> When SSL renegotiation was removed in 426746b9 the only consumer of the >> openssl >> specific count member in the Port struct was removed, but the member was left >> together

  1   2   >