Re: [HACKERS] UPDATE of partition key

2017-11-02 Thread Amit Langote
Hi Amit. Thanks a lot for updated patches and sorry that I couldn't get to looking at your emails sooner. Note that I'm replying here to both of your emails, but looking at only the latest v22 patch. On 2017/10/24 0:15, Amit Khandekar wrote: > On 16 October 2017 at 08:28, Amit Langote > wrote:

Re: [HACKERS] SQL/JSON in PostgreSQL

2017-11-02 Thread Pavel Stehule
Hi 2017-11-02 3:39 GMT+01:00 Peter Eisentraut : > Could someone clarify the status of this patch set? It has been in > "Waiting" mode since the previous CF and no new patch, just a few > questions from the author. > There was a state "needs review". I looked to the patch, and found some issues,

Re: [HACKERS] Try to fix endless loop in ecpg with informix mode

2017-11-02 Thread Michael Meskes
> I am afraid the changes may separate "7.a" to "7" and "a", then error > out > with "invalid input syntax for type int" for "a". Which is correct, is it not? > How about changes as below? (use following the if to decide true or > false) > ... >return false; > +} while (is

Re: [HACKERS] pg_basebackup fails on Windows when using tablespace mapping

2017-11-02 Thread Michael Paquier
On Thu, Nov 2, 2017 at 2:05 AM, Peter Eisentraut wrote: > On 11/1/17 10:29, Michael Paquier wrote: >> On Wed, Nov 1, 2017 at 2:24 PM, Peter Eisentraut >> wrote: >>> Committed to master. I suppose this should be backpatched? >> >> Thanks! Yes this should be back-patched. > > OK, done for 10, 9.6,

Re: [HACKERS] [PATCH] A hook for session start

2017-11-02 Thread Aleksandr Parfenov
The following review has been posted through the commitfest application: make installcheck-world: not tested Implements feature: not tested Spec compliant: not tested Documentation:not tested Hi, Unfortunately, patches 0001 and 0002 don't apply to current master. The

Re: [HACKERS] [POC] hash partitioning

2017-11-02 Thread Robert Haas
On Wed, Nov 1, 2017 at 3:46 PM, amul sul wrote: > Although partition constraints become more simple, there isn't any performance > gain with 0005 patch. Also I am little skeptic about logic in 0005 where we > copied extended hash function info from the partition key, what if parent is > changed wh

Re: [HACKERS] Mapping MERGE onto CTEs (Re: MERGE SQL Statement for PG11)

2017-11-02 Thread Craig Ringer
On 2 November 2017 at 01:14, Alvaro Herrera wrote: > Nico Williams wrote: > >> As an aside, I'd like to be able to control which CTEs are view-like and >> which are table-like. In SQLite3, for example, they are all view-like, >> and the optimizer will act accordingly, whereas in PG they are all >

Re: [HACKERS] [POC] hash partitioning

2017-11-02 Thread amul sul
On Thu, Nov 2, 2017 at 1:35 PM, Robert Haas wrote: > On Wed, Nov 1, 2017 at 3:46 PM, amul sul wrote: >> Although partition constraints become more simple, there isn't any >> performance >> gain with 0005 patch. Also I am little skeptic about logic in 0005 where we >> copied extended hash functio

Re: [HACKERS] Statement-level rollback

2017-11-02 Thread Vladimir Sitnikov
Tsunakawa>So the statement-level rollback is newer to users, isn't it? Technically speaking, the feature was listed in the changelog. Tsunakawa>Doesn't PgJDBC execute RELEASE after each SQL statement? It does not. Tsunakawa>That said, even with RELEASE, the server memory bloat is not solved. T

Re: [HACKERS] [POC] hash partitioning

2017-11-02 Thread Ashutosh Bapat
On Thu, Nov 2, 2017 at 1:35 PM, Robert Haas wrote: > On Wed, Nov 1, 2017 at 3:46 PM, amul sul wrote: >> Although partition constraints become more simple, there isn't any >> performance >> gain with 0005 patch. Also I am little skeptic about logic in 0005 where we >> copied extended hash functio

Re: [HACKERS] Statement-level rollback

2017-11-02 Thread Simon Riggs
On 2 November 2017 at 01:33, Peter Eisentraut wrote: > The proposed statement-level rollback feature works in a slightly > different context. It does not change when or how a transaction or > transaction block begins and ends. It only changes what happens inside > explicit transaction blocks.

Re: [HACKERS] Walsender timeouts and large transactions

2017-11-02 Thread Robert Haas
On Wed, Nov 1, 2017 at 8:19 PM, Petr Jelinek wrote: > sorry for the delay but I didn't have much time in past weeks to follow > this thread. +TimestampTz now = GetCurrentTimestamp(); + /* output previously gathered data in a CopyData packet */ pq_putmessage_noblock('d', ctx->out->da

Re: [HACKERS] [POC] hash partitioning

2017-11-02 Thread Robert Haas
On Thu, Nov 2, 2017 at 1:45 PM, amul sul wrote: > Yes, you are correct, column involved in the partitioning are immutable. > > I was just worried about any change in the partition key column that > might change selected hash function. Any such change, even if it were allowed, would have to take A

Re: [HACKERS] [POC] hash partitioning

2017-11-02 Thread Robert Haas
On Thu, Nov 2, 2017 at 1:52 PM, Ashutosh Bapat wrote: > Right now partition keys are immutable but we don't have much code > written with that assumption. All the code usually keeps a lock on the > parent till the time they use the information in the partition key. In > a distant future, which may

Re: [HACKERS] Custom compression methods

2017-11-02 Thread Ildus Kurbangaliev
On Wed, 1 Nov 2017 17:05:58 -0400 Peter Eisentraut wrote: > On 9/12/17 10:55, Ildus Kurbangaliev wrote: > >> The patch also includes custom compression method for tsvector > >> which is used in tests. > >> > >> [1] > >> https://www.postgresql.org/message-id/CAPpHfdsdTA5uZeq6MNXL5ZRuNx%2BSig4ykWzW

Re: [HACKERS] Add some const decorations to prototypes

2017-11-02 Thread Robert Haas
On Tue, Oct 31, 2017 at 8:26 PM, Tom Lane wrote: > ... but I'm not sure that it's an improvement in cases where you have to > cast away the const somewhere else. I agree. I guess I may be in the minority here but I don't really like decorating things with const too much because I have tended to

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-11-02 Thread Andres Freund
Hi, On 2017-09-28 14:47:53 +, Alvaro Herrera wrote: > Fix freezing of a dead HOT-updated tuple > > Vacuum calls page-level HOT prune to remove dead HOT tuples before doing > liveness checks (HeapTupleSatisfiesVacuum) on the remaining tuples. But > concurrent transaction commit/abort may turn

[HACKERS] Client Connection redirection support for PostgreSQL

2017-11-02 Thread Satyanarayana Narlapuram
Proposal: Add the ability to the PostgreSQL server instance to route the traffic to a different server instance based on the rules defined in server's pg_bha.conf configuration file. At a high level this enables offloading the user requests to a different server instance based on the rules defin

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-11-02 Thread Robert Haas
On Thu, Nov 2, 2017 at 4:50 PM, Andres Freund wrote: > I think a5736bf754c82d8b86674e199e232096c679201d might be dangerous in > the face of previously corrupted tuple chains and pg_upgraded clusters - > it can lead to tuples being considered related, even though they they're > from entirely indepe

Re: [HACKERS] proposal: schema variables

2017-11-02 Thread Robert Haas
On Thu, Oct 26, 2017 at 12:51 PM, Pavel Stehule wrote: > The variables can be modified by SQL command SET (this is taken from > standard, and it natural) > > SET varname = expression; Overloading SET to handle both variables and GUCs seems likely to create problems, possibly including security pr

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-11-02 Thread Alvaro Herrera
Andres Freund wrote: > I spent some time discussing this with Robert today (with both of us > alternating between feeling the other and ourselves as stupid), and the > conclusion I think is that the problem is on the pruning, rather than > the freezing side. Thanks both for spending some more tim

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-11-02 Thread Andres Freund
Hi, On 2017-11-02 13:49:47 +0100, Alvaro Herrera wrote: > Andres Freund wrote: > > I think the problem is on the pruning, rather than the freezing side. We > > can't freeze a tuple if it has an alive predecessor - rather than > > weakining this, we should be fixing the pruning to not have the aliv

Re: [HACKERS] list of credits for release notes

2017-11-02 Thread Peter Eisentraut
On 10/3/17 12:31, Dang Minh Huong wrote: > Sorry but please change "Huong Dangminh"(my name) to "Dang Minh Huong". done -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pg

Re: [HACKERS] [PATCH] Add two-arg for of current_setting(NAME, FALLBACK)

2017-11-02 Thread Peter Eisentraut
On 11/1/17 14:02, Nico Williams wrote: > There already _is_ a two-argument form of current_setting() that yours > somewhat conflicts with: > >current_setting(setting_name [, missing_ok ]) > > https://www.postgresql.org/docs/current/static/functions-admin.html > > I often use > > coalesce(

Re: [HACKERS] Client Connection redirection support for PostgreSQL

2017-11-02 Thread David Fetter
On Thu, Nov 02, 2017 at 06:02:43AM +, Satyanarayana Narlapuram wrote: > Proposal: > Add the ability to the PostgreSQL server instance to route the > traffic to a different server instance based on the rules defined in > server's pg_bha.conf configuration file. At a high level this > enables off

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-11-02 Thread Tom Lane
Andres Freund writes: > Do we care about people upgrading to unreleased versions? We could do > nothing, document it in the release notes, or ??? Do nothing. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

[HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread David Rowley
Hackers, Our List implementation internally uses linked lists which are certainly good for some things, but pretty bad at other things. Linked lists are pretty bad when you want to fetch the Nth element, as it means looping ever each previous element until you get to the Nth. They're also pretty b

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-11-02 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Andres Freund writes: > > Do we care about people upgrading to unreleased versions? We could do > > nothing, document it in the release notes, or ??? > > Do nothing. Agreed. Not much we can do there. Thanks! Stephen signature.asc Description: Digital

Re: [HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread Tom Lane
David Rowley writes: > Comments on the design are welcome, but I was too late to the > commitfest, so there are other priorities. However, if you have a > strong opinion, feel free to voice it. I do not like replacing Lists piecemeal; that's a recipe for ongoing API breakage and back-patching pai

Re: [HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread David Rowley
On 3 November 2017 at 03:17, Tom Lane wrote: > We've jacked up the List API and driven a new implementation underneath > once before. Maybe it's time to do that again. Maybe, but the new implementation is not going to do well with places where we perform lcons(). Probably many of those places co

Re: [HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread Craig Ringer
On 2 November 2017 at 22:17, Tom Lane wrote: > David Rowley writes: >> Comments on the design are welcome, but I was too late to the >> commitfest, so there are other priorities. However, if you have a >> strong opinion, feel free to voice it. > > I do not like replacing Lists piecemeal; that's a

Re: [HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread Craig Ringer
On 2 November 2017 at 22:22, David Rowley wrote: > On 3 November 2017 at 03:17, Tom Lane wrote: >> We've jacked up the List API and driven a new implementation underneath >> once before. Maybe it's time to do that again. > > Maybe, but the new implementation is not going to do well with places >

Re: [HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread Stephen Frost
David, * David Rowley (david.row...@2ndquadrant.com) wrote: > Our List implementation internally uses linked lists which are > certainly good for some things, but pretty bad at other things. Linked > lists are pretty bad when you want to fetch the Nth element, as it > means looping ever each previ

Re: [HACKERS] pgbench - use enum for meta commands

2017-11-02 Thread Aleksandr Parfenov
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed Hi, Looks good to me. The only thing I'm not quite sure abo

Re: [HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread Tom Lane
David Rowley writes: > On 3 November 2017 at 03:17, Tom Lane wrote: >> We've jacked up the List API and driven a new implementation underneath >> once before. Maybe it's time to do that again. > Maybe, but the new implementation is not going to do well with places > where we perform lcons(). Pr

Re: [HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread David Rowley
On 3 November 2017 at 03:38, Tom Lane wrote: > David Rowley writes: >> On 3 November 2017 at 03:17, Tom Lane wrote: >>> We've jacked up the List API and driven a new implementation underneath >>> once before. Maybe it's time to do that again. > >> Maybe, but the new implementation is not going

Re: [HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread Tom Lane
David Rowley writes: >> It seems to me that a whole lot of the complaints about this could be >> resolved simply by improving the List infrastructure to allocate ListCells >> in batches. That would address the question of "too much palloc traffic" >> and greatly improve the it-accesses-all-over-m

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-11-02 Thread Alvaro Herrera
Stephen Frost wrote: > * Tom Lane (t...@sss.pgh.pa.us) wrote: > > Andres Freund writes: > > > Do we care about people upgrading to unreleased versions? We could do > > > nothing, document it in the release notes, or ??? > > > > Do nothing. > > Agreed. Not much we can do there. Pushed the rever

Re: [HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread David Rowley
On 3 November 2017 at 03:27, Stephen Frost wrote: > * David Rowley (david.row...@2ndquadrant.com) wrote: >> We could get around a few of these problems if Lists were implemented >> internally as arrays, however, arrays are pretty bad if we want to >> delete an item out the middle as we'd have to s

Re: [HACKERS] Custom compression methods

2017-11-02 Thread Craig Ringer
On 2 November 2017 at 17:41, Ildus Kurbangaliev wrote: > In this patch compression methods is suitable for MAIN and EXTENDED > storages like in current implementation in postgres. Just instead only > of LZ4 you can specify any other compression method. We've had this discussion before. Please r

Re: [HACKERS] proposal: schema variables

2017-11-02 Thread Craig Ringer
On 26 October 2017 at 15:21, Pavel Stehule wrote: > Hi, > > I propose a new database object - a variable. Didn't we have a pretty long discussion about this already in Yeah. https://www.postgresql.org/message-id/flat/CAMsr%2BYF0G8_FehQyFS8gSfnEer9OPsMOvpfniDJOVGQzJzHzsw%40mail.gmail.com#camsr+

Re: [HACKERS] Client Connection redirection support for PostgreSQL

2017-11-02 Thread Craig Ringer
On 2 November 2017 at 14:02, Satyanarayana Narlapuram wrote: > Proposal: > > Add the ability to the PostgreSQL server instance to route the traffic to a > different server instance based on the rules defined in server’s pg_bha.conf > configuration file. At a high level this enables offloading the

Re: [HACKERS] proposal: schema variables

2017-11-02 Thread Nico Williams
On Thu, Nov 02, 2017 at 06:05:54PM +0530, Robert Haas wrote: > On Thu, Oct 26, 2017 at 12:51 PM, Pavel Stehule > wrote: > > The variables can be modified by SQL command SET (this is taken from > > standard, and it natural) > > > > SET varname = expression; > > Overloading SET to handle both vari

Re: [HACKERS] proposal: schema variables

2017-11-02 Thread Pavel Stehule
2017-11-02 16:35 GMT+01:00 Nico Williams : > On Thu, Nov 02, 2017 at 06:05:54PM +0530, Robert Haas wrote: > > On Thu, Oct 26, 2017 at 12:51 PM, Pavel Stehule > wrote: > > > The variables can be modified by SQL command SET (this is taken from > > > standard, and it natural) > > > > > > SET varname

Re: [HACKERS] proposal: schema variables

2017-11-02 Thread Pavel Stehule
2017-11-02 16:07 GMT+01:00 Craig Ringer : > On 26 October 2017 at 15:21, Pavel Stehule > wrote: > > Hi, > > > > I propose a new database object - a variable. > > Didn't we have a pretty long discussion about this already in > > Yeah. > > https://www.postgresql.org/message-id/flat/CAMsr%2BYF0G8_

Re: [HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread Andres Freund
On 2017-11-02 10:38:34 -0400, Tom Lane wrote: > David Rowley writes: > > On 3 November 2017 at 03:17, Tom Lane wrote: > >> We've jacked up the List API and driven a new implementation underneath > >> once before. Maybe it's time to do that again. > > > Maybe, but the new implementation is not g

Re: [HACKERS] proposal: schema variables

2017-11-02 Thread Tom Lane
Nico Williams writes: > With access controls, GUCs could become schema variables, and settings > from postgresql.conf could move into the database itself (which I think > would be nice). People re-propose some variant of that every so often, but it never works, because it ignores the fact that so

Re: [HACKERS] proposal: schema variables

2017-11-02 Thread Pavel Stehule
2017-11-02 13:35 GMT+01:00 Robert Haas : > On Thu, Oct 26, 2017 at 12:51 PM, Pavel Stehule > wrote: > > The variables can be modified by SQL command SET (this is taken from > > standard, and it natural) > > > > SET varname = expression; > > Overloading SET to handle both variables and GUCs seems

Re: [HACKERS] Document pgstattuple privileges without ambiguity

2017-11-02 Thread Peter Eisentraut
On 8/21/17 03:47, Feike Steenbergen wrote: > When installing pgstattuple on 10, the documentation about its > privileges was unclear to me. (Does the pg_stat_scan_tables role get > EXECUTE privileges by default or not?). I agree that this has gotten a bit confusing after apparently being patched a

[HACKERS] Minor comment issue in receivelog.c

2017-11-02 Thread Bernd Helmle
Please find a minor comment fix for receivelog.c, HandleCopyStream(). The comments talks about a START_STREAMING command, but i think START_REPLICATION is what's meant here. Bernd diff --git a/src/bin/pg_basebackup/receivelog.c b/src/bin/pg_basebackup/receivelog.c index 888458f4a9..befbb

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-11-02 Thread Tom Lane
Alvaro Herrera writes: >> Hmm, I'm pretty sure we stress-tested brin in pretty much the same way. >> But I see this misbehavior too. Looking ... > Turns out that this is related to concurrent growth of the table while > the summarization process is scanning -- so new pages have appeared at > the

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-11-02 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera writes: > >> Hmm, I'm pretty sure we stress-tested brin in pretty much the same way. > >> But I see this misbehavior too. Looking ... > > > Turns out that this is related to concurrent growth of the table while > > the summarization process is scanning -- so new

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-11-02 Thread Tom Lane
Alvaro Herrera writes: > Tom Lane wrote: >> Where are we on this --- do you want me to push the brin_doupdate >> fix I proposed, or were you intending to merge that into a >> larger patch? > Please push your fixes, I'll post my proposed patch for the other bug > afterwards; they are unrelated pro

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-11-02 Thread Robert Haas
On Thu, Nov 2, 2017 at 8:25 PM, Alvaro Herrera wrote: > Pushed the reverts. > > I noticed while doing so that REL_10_STABLE contains the bogus commits. > Does that change our opinion regarding what to do for people upgrading > to a version containing the broken commits? I don't think so, because

Re: [HACKERS] Removing wal_keep_segments as default configuration in PostgresNode.pm

2017-11-02 Thread Peter Eisentraut
On 9/11/17 21:55, Michael Paquier wrote: > I tend to think that while all the other parameters make sense to > deploy instances that need few resources, wal_keep_segments may cause > up to 350MB of WAL segments to be kept in each pg_wal's instance, > while max_wal_size is set at 128MB. The only tes

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-11-02 Thread Peter Geoghegan
On Thu, Nov 2, 2017 at 4:20 AM, Andres Freund wrote: > I think the problem is on the pruning, rather than the freezing side. We > can't freeze a tuple if it has an alive predecessor - rather than > weakining this, we should be fixing the pruning to not have the alive > predecessor. Excellent catc

Re: [HACKERS] Removing wal_keep_segments as default configuration in PostgresNode.pm

2017-11-02 Thread Michael Paquier
On Thu, Nov 2, 2017 at 4:47 PM, Peter Eisentraut wrote: > On 9/11/17 21:55, Michael Paquier wrote: >> I tend to think that while all the other parameters make sense to >> deploy instances that need few resources, wal_keep_segments may cause >> up to 350MB of WAL segments to be kept in each pg_wal'

Re: [HACKERS] pgbench - use enum for meta commands

2017-11-02 Thread Fabien COELHO
The only thing I'm not quite sure about is a comment "which meta command ...". Maybe it's better to write it without question word, something like "meta command identifier..."? Ok. I agree. Updated version attached. I also added a const on a function parameter. Just a note about the motivat

Re: [HACKERS] pgbench - use enum for meta commands

2017-11-02 Thread Fabien COELHO
Updated version attached. . Here is the patch. Sorry for the noise. -- Fabien.diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c index 5d8a01c..6bd3e52 100644 --- a/src/bin/pgbench/pgbench.c +++ b/src/bin/pgbench/pgbench.c @@ -373,11 +373,21 @@ typedef enum QueryMode static Q

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Robert Haas
On Thu, Nov 2, 2017 at 8:28 AM, Craig Ringer wrote: > On 1 November 2017 at 01:55, Peter Geoghegan wrote: >> The problem here is: Iff the first statement uses ON CONFLICT >> infrastructure, doesn't the absence of WHEN NOT MATCHED imply >> different semantics for the remaining updates and deletes

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-11-02 Thread Peter Geoghegan
On Thu, Nov 2, 2017 at 9:44 AM, Robert Haas wrote: > The second commit (22576734b805fb0952f9be841ca8f643694ee868) is where > I think things get a lot more dangerous. The problem (as Andres > pointed out to me this afternoon) is that it seems possible to end up > with a situation where there shoul

Re: [HACKERS] [COMMITTERS] pgsql: passwordcheck: Add test suite

2017-11-02 Thread Peter Eisentraut
On 9/15/17 00:20, Michael Paquier wrote: > On Fri, Sep 15, 2017 at 12:02 PM, Michael Paquier > wrote: >> On Fri, Sep 15, 2017 at 11:46 AM, Peter Eisentraut wrote: >>> passwordcheck: Add test suite >>> >>> Also improve one error message. >>> >>> Reviewed-by: David Steele >> >> Sorry for showing u

Re: [HACKERS] [PATCH] Minor patch to correct symbol name in logs

2017-11-02 Thread Peter Eisentraut
On 9/20/17 01:56, Vaishnavi Prabakaran wrote: > Backend's lo_functions were renamed to avoid conflicting with libpq > prototypes in commit - 6fc547960dbe0b8bd6cefae5ab7ec3605a5c46fc.  > > Logs inside those functions still refer to old symbol names, Here is the > small patch to update the same. I

Re: [HACKERS] pgbench - use enum for meta commands

2017-11-02 Thread a . parfenov
On Thu, 2 Nov 2017 17:50:52 +0100 (CET) Fabien COELHO wrote: > Updated version attached. . Here is the patch. Sorry for the noise. Everything alright. Patch is ready for commiter. -- Aleksandr Parfenov Postgres Professional: http://www.postgrespro.com Russian Postgres Company -- Sent via

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Robert Haas
On Tue, Oct 31, 2017 at 5:14 PM, Simon Riggs wrote: > I've proposed a SQL Standard compliant implementation that would do > much more than be new syntax over what we already have. > > So these two claims aren't accurate: "radical difference" and "syntax > sugar over a capability we have". I think

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-11-02 Thread Robert Haas
On Thu, Nov 2, 2017 at 10:26 PM, Peter Geoghegan wrote: > On Thu, Nov 2, 2017 at 9:44 AM, Robert Haas wrote: >> The second commit (22576734b805fb0952f9be841ca8f643694ee868) is where >> I think things get a lot more dangerous. The problem (as Andres >> pointed out to me this afternoon) is that it

[HACKERS] Parallel Plans and Cost of non-filter functions

2017-11-02 Thread Paul Ramsey
I'm working on a custom aggregate, that generates a serialized data format. The preparation of the geometry before being formatted is pretty intense, so it is probably a good thing for that work to be done in parallel, in partial aggregates. Here's an example SQL call: EXPLAIN analyze SELECT lengt

Re: [HACKERS] proposal: schema variables

2017-11-02 Thread Robert Haas
On Thu, Nov 2, 2017 at 9:05 PM, Nico Williams wrote: >> Overloading SET to handle both variables and GUCs seems likely to >> create problems, possibly including security problems. For example, >> maybe a security-definer function could leave behind variables to >> trick the calling code into fail

Re: [HACKERS] initdb w/ restart

2017-11-02 Thread Peter Eisentraut
On 9/28/17 15:40, Jesper Pedersen wrote: > In a case of > > initdb /tmp/pgsql > > followed by > > pg_ctl -D /tmp/pgsql/ -l /tmp/logfile restart > > you'll get > > pg_ctl: PID file "/tmp/pgsql/postmaster.pid" does not exist > Is server running? > starting server anyway > pg_ctl: could not read

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-11-02 Thread Alvaro Herrera
Tomas Vondra wrote: > Unfortunately, I think we still have a problem ... I've been wondering > if we end up producing correct indexes, so I've done a simple test. Here's a proposed patch that should fix this problem (and does, in my testing). Would you please give it a try? This patch changes t

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-11-02 Thread Tom Lane
Alvaro Herrera writes: > 1. in VACUUM or brin_summarize_new_values, we only process fully loaded > ranges, and ignore the partial range at end of table. OK. > 2. when summarization is requested on the partial range at the end of a > table, we acquire extension lock on the rel, then compute relat

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Peter Geoghegan
Robert Haas wrote: And if, in the meantime, MERGE can only handle the cases where there is a unique index, then it can only handle the cases INSERT .. ON CONFLICT UPDATE can cover, which makes it, as far as I can see, syntactic sugar over what we already have. Maybe it's not entirely - you migh

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Simon Riggs
On 1 November 2017 at 18:20, Peter Geoghegan wrote: > In Postgres, you can avoid duplicate violations with MERGE by using a > higher isolation level (these days, those are turned into a > serialization error at higher isolation levels when no duplicate is > visible to the xact's snapshot). So if

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Nico Williams
If nothing else, anyone needing MERGE can port their MERGE statements to a DML with DML-containing CTEs... The generic mapping would be something like this, I think: WITH rows AS (SELECT FROM WHERE ) , updated AS ( UPDATE SET ... WHERE IN (SELECT FROM rows) /* matched */

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-11-02 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera writes: > > 2. when summarization is requested on the partial range at the end of a > > table, we acquire extension lock on the rel, then compute relation size > > and run summarization with the lock held. This guarantees that we don't > > miss any pages. This i

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-11-02 Thread Tom Lane
Alvaro Herrera writes: > Tom Lane wrote: >> If VACUUM and brin_summarize_new_values both ignore the partial >> range, then what else would request this? Can't we just decree >> that we don't summarize the partial range, period? > brin_summarize_range() can do it. So what would happen if we just

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Nico Williams
On Thu, Nov 02, 2017 at 06:49:18PM +, Simon Riggs wrote: > On 1 November 2017 at 18:20, Peter Geoghegan wrote: > > In Postgres, you can avoid duplicate violations with MERGE by using a > > higher isolation level (these days, those are turned into a > > serialization error at higher isolation l

Re: [HACKERS] VACUUM and ANALYZE disagreeing on what reltuples means

2017-11-02 Thread Tom Lane
Haribabu Kommi writes: > The changes are fine and now it reports proper live tuples in both > pg_class and stats. The other issue of continuous vacuum operation > leading to decrease of number of live tuples is not related to this > patch and that can be handled separately. I did not like this pa

Re: [HACKERS] proposal: schema variables

2017-11-02 Thread Nico Williams
On Thu, Nov 02, 2017 at 11:48:44AM -0400, Tom Lane wrote: > Nico Williams writes: > > With access controls, GUCs could become schema variables, and settings > > from postgresql.conf could move into the database itself (which I think > > would be nice). > > People re-propose some variant of that e

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Peter Geoghegan
Nico Williams wrote: If you want to ignore conflicts arising from concurrency you could always add an ON CONFLICT DO NOTHING to the INSERT DML in the mapping I proposed earlier. Thus a MERGE CONCURRENTLY could just do that. Is there any reason not to map MERGE as I proposed? Performance, for

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Peter Geoghegan
Simon Riggs wrote: So if I understand you correctly, in your view MERGE should just fail with an ERROR if it runs concurrently with other DML? That's certainly my opinion on the matter. It seems like that might be the consensus, too. Obviously there are things that you as a user can do about

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-11-02 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera writes: > > Tom Lane wrote: > >> If VACUUM and brin_summarize_new_values both ignore the partial > >> range, then what else would request this? Can't we just decree > >> that we don't summarize the partial range, period? > > > brin_summarize_range() can do it. >

Re: [HACKERS] [PATCH] Add ALWAYS DEFERRED option for constraints

2017-11-02 Thread Peter Eisentraut
I haven't really thought about this feature too hard; I just want to give you a couple of code comments. I think the catalog structure, and relatedly also the parser structures, could be made more compact. We currently have condeferrable and condeferred to represent three valid states (NOT DEFERR

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Nico Williams
On Thu, Nov 02, 2017 at 12:51:45PM -0700, Peter Geoghegan wrote: > Nico Williams wrote: > >If you want to ignore conflicts arising from concurrency you could > >always add an ON CONFLICT DO NOTHING to the INSERT DML in the mapping I > >proposed earlier. Thus a MERGE CONCURRENTLY could just do tha

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Simon Riggs
On 2 November 2017 at 19:16, Peter Geoghegan wrote: > Simon Riggs wrote: >> >> So if I understand you correctly, in your view MERGE should just fail >> with an ERROR if it runs concurrently with other DML? > > > That's certainly my opinion on the matter. It seems like that might be > the consensu

Re: [HACKERS] Dynamic result sets from procedures

2017-11-02 Thread Daniel Verite
Peter Eisentraut wrote: > CREATE PROCEDURE pdrstest1() > LANGUAGE SQL > AS $$ > DECLARE c1 CURSOR WITH RETURN FOR SELECT * FROM cp_test2; > DECLARE c2 CURSOR WITH RETURN FOR SELECT * FROM cp_test3; > $$; > > CALL pdrstest1(); > > and that returns those two result sets to the client. If

Re: [HACKERS] Linking libpq statically to libssl

2017-11-02 Thread Peter Eisentraut
On 10/27/17 08:24, Daniele Varrazzo wrote: > I have a problem building binary packages for psycopg2. Binary > packages ship with their own copies of libpq and libssl; Aside from the advice of "don't do that" ... > however if > another python package links to libssl the library will be imported >

Re: [HACKERS] [PATCH] Add ALWAYS DEFERRED option for constraints

2017-11-02 Thread Nico Williams
On Thu, Nov 02, 2017 at 04:20:19PM -0400, Peter Eisentraut wrote: > I haven't really thought about this feature too hard; I just want to > give you a couple of code comments. Thanks! > I think the catalog structure, and relatedly also the parser structures, > could be made more compact. We curre

Re: [HACKERS] Deadlock in ALTER SUBSCRIPTION REFRESH PUBLICATION

2017-11-02 Thread Peter Eisentraut
On 10/24/17 13:13, Konstantin Knizhnik wrote: > The reason of this deadlock seems to be clear: ALTER SUBSCRIPTION starts > transaction at one node and tries to create slot at other node, which waiting > for completion of all active transaction while building scnapshpot. > Is there any way to avoi

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Peter Geoghegan
Simon Riggs wrote: I think people imagined you had worked out how to make MERGE run concurrently, I certainly did, but in fact you're just saying you don't believe it ever should. I'm certain that they didn't think that at all. But I'll let them speak for themselves. That is strange since th

Re: [HACKERS] Setting pd_lower in GIN metapage

2017-11-02 Thread Tom Lane
Amit Langote writes: > On 2017/09/26 16:30, Michael Paquier wrote: >> Cool, let's switch it back to a ready for committer status then. > Sure, thanks. Pushed with some cosmetic adjustments --- mostly, making the comments more explicit about why we need the apparently-redundant assignments to pd_

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-11-02 Thread Tom Lane
Alvaro Herrera writes: > Tom Lane wrote: >> So what would happen if we just don't summarize partial ranges? > Index scan would always have to read all the heap pages for that partial > range. Maybe not a big issue, but when you finish loading a table, it'd > be good to have a mechanism to summar

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Nico Williams
On Thu, Nov 02, 2017 at 02:05:19PM -0700, Peter Geoghegan wrote: > Simon Riggs wrote: > >So in your view we should make no attempt to avoid concurrent errors, > >even when we have the capability to do so (in some cases) and doing so > >would be perfectly compliant with the SQLStandard. > > Yes. T

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Peter Geoghegan
Nico Williams wrote: A MERGE mapped to a DML like this: WITH updated AS ( UPDATE SET ... WHERE RETURNING ) , inserted AS ( INSERT INTO SELECT ... WHERE NOT IN (SELECT FROM updated) AND .. ON CONFLICT DO NOTHING -- see bel

Re: [HACKERS] GnuTLS support

2017-11-02 Thread Andreas Karlsson
On 09/18/2017 07:04 PM, Jeff Janes wrote:> You fixed the first issue, but I still get the second one: be-secure-gnutls.c: In function 'get_peer_certificate': be-secure-gnutls.c:667: error: 'GNUTLS_X509_CRT_LIST_SORT' undeclared (first use in this function) be-secure-gnutls.c:667: error: (Each u

Re: [HACKERS] pgbench - use enum for meta commands

2017-11-02 Thread Tom Lane
Fabien COELHO writes: > [ pgbench-enum-meta-2.patch ] Pushed with some trivial cosmetic adjustments (pgindent changed it more than I did). regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Nico Williams
On Thu, Nov 02, 2017 at 03:25:48PM -0700, Peter Geoghegan wrote: > Nico Williams wrote: > >A MERGE mapped to a DML like this: > > This is a bad idea. An implementation like this is not at all > maintainable. Assuming the DELETE issue can be addressed, why would this not be maintainable? > >can

Re: [HACKERS] SQL/JSON in PostgreSQL

2017-11-02 Thread Piotr Stefaniak
On 2017-02-28 20:08, Oleg Bartunov wrote: > Attached patch is an implementation of SQL/JSON data model from SQL-2016 > standard (ISO/IEC 9075-2:2016(E)) I've faintly started looking into this. > We created repository for reviewing (ask for write access) - > https://github.com/postgrespro/sqljson/

Re: [HACKERS] [PATCH] A hook for session start

2017-11-02 Thread Fabrízio de Royes Mello
On Thu, Nov 2, 2017 at 5:42 AM, Aleksandr Parfenov < a.parfe...@postgrespro.ru> wrote: > > The following review has been posted through the commitfest application: > make installcheck-world: not tested > Implements feature: not tested > Spec compliant: not tested > Documentation:

Re: [HACKERS] Linking libpq statically to libssl

2017-11-02 Thread Stephen Frost
Peter, * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote: > On 10/27/17 08:24, Daniele Varrazzo wrote: > > I have a problem building binary packages for psycopg2. Binary > > packages ship with their own copies of libpq and libssl; > > Aside from the advice of "don't do that" ... > > >

  1   2   >