Re: [HACKERS] Setting pd_lower in GIN metapage

2017-11-02 Thread Michael Paquier
On Fri, Nov 3, 2017 at 1:10 AM, Amit Kapila wrote: > On Fri, Nov 3, 2017 at 2:54 AM, Tom Lane wrote: >> I've marked the CF entry closed. However, I'm not sure if we're quite >> done with this thread. Weren't we going to adjust nbtree and hash to >>

Re: [HACKERS] pgbench - use enum for meta commands

2017-11-02 Thread Fabien COELHO
[ pgbench-enum-meta-2.patch ] Pushed with some trivial cosmetic adjustments (pgindent changed it more than I did). Ok. Thanks. -- Fabien. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] PATCH: pgbench - option to build using ppoll() for larger connection counts

2017-11-02 Thread Fabien COELHO
Hello, Could you rebase the v11 patch? -- Fabien. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread David Rowley
On 3 November 2017 at 03:26, Craig Ringer wrote: > On 2 November 2017 at 22:22, David Rowley > wrote: >> Maybe, but the new implementation is not going to do well with places >> where we perform lcons(). Probably many of those places could be

Re: [HACKERS] SSL and Encryption

2017-11-02 Thread Michael Paquier
On Fri, Nov 3, 2017 at 3:19 AM, Craig Ringer wrote: > This is probably off topic for pgsql-hackers. > > For password crypto please go read the SCRAM thread and the PostgreSQL > 10 release notes. The SCRAM discussion is spread across two threads mainly with hundreds of

Re: [HACKERS] path toward faster partition pruning

2017-11-02 Thread David Rowley
On 31 October 2017 at 21:43, Amit Langote wrote: > Attached updated version of the patches addressing some of your comments I've spent a bit of time looking at these but I'm out of time for now. So far I have noted down the following; 1. This comment seem wrong.

Re: [HACKERS] SSL and Encryption

2017-11-02 Thread Craig Ringer
On 3 November 2017 at 11:16, chiru r wrote: > Hi , > > Please suggest the best chiper suite to configure openSSL for PostgreSQL > Server and client?. > > How to use other than md5 encryption algorithm to encrypt the passwords in > PostgreSQL? This is probably off topic for

[HACKERS] SSL and Encryption

2017-11-02 Thread chiru r
Hi , Please suggest the best chiper suite to configure openSSL for PostgreSQL Server and client?. How to use other than md5 encryption algorithm to encrypt the passwords in PostgreSQL? Thanks, Chiru

Re: [HACKERS] path toward faster partition pruning

2017-11-02 Thread David Rowley
On 31 October 2017 at 21:43, Amit Langote wrote: > Attached updated version of the patches addressing some of your comments > above and fixing a bug that Rajkumar reported [1]. As mentioned there, > I'm including here a patch (the 0005 of the attached) to tweak the

Re: [HACKERS] Small improvement to compactify_tuples

2017-11-02 Thread Tom Lane
Sokolov Yura writes: > [ 0001-Improve-compactify_tuples.patch, v5 or thereabouts ] I started to review this patch. I spent a fair amount of time on beautifying the code, because I found it rather ugly and drastically undercommented. Once I had it to the point where

[HACKERS] Skip unneeded temp file in 'make html'

2017-11-02 Thread David Fetter
Folks, Please find attached a patch for $Subject. Best, David. -- David Fetter http://fetter.org/ Phone: +1 415 235 3778 Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate Don't make an unneeded temp file In passing, make a slight

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2017-11-02 Thread Thomas Munro
On Fri, Nov 3, 2017 at 2:24 PM, Peter Geoghegan wrote: > Thomas Munro wrote: >> That way you don't have to opt in to BufFile's >> double buffering and segmentation schemes just to get shared file >> clean-up, if for some reason you want direct file

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2017-11-02 Thread Peter Geoghegan
Thomas Munro wrote: I'm going to make an item on my personal TODO list for that. No useful insights on that right now, though. I decided to try that, but it didn't really work: fd.h gets included by front-end code, so I can't very well define a struct and

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2017-11-02 Thread Thomas Munro
On Wed, Nov 1, 2017 at 2:11 PM, Peter Geoghegan wrote: > On Tue, Oct 31, 2017 at 5:07 PM, Thomas Munro > wrote: >> Another complaint is that perhaps fd.c >> knows too much about buffile.c's business. For example, >> RemovePgTempFilesInDir() knows

Re: [HACKERS] Setting pd_lower in GIN metapage

2017-11-02 Thread Amit Kapila
On Fri, Nov 3, 2017 at 2:54 AM, Tom Lane wrote: > Amit Langote writes: >> On 2017/09/26 16:30, Michael Paquier wrote: >>> Cool, let's switch it back to a ready for committer status then. > >> Sure, thanks. > > Pushed with some cosmetic

Re: [HACKERS] Parallel Hash take II

2017-11-02 Thread Thomas Munro
On Mon, Oct 30, 2017 at 1:49 PM, Thomas Munro wrote: > A couple of stupid things outstanding: > > 1. EXPLAIN ANALYZE for Parallel Hash "actual" shows the complete row > count, which is interesting to know (or not? maybe I should show it > somewhere else?), but the

Re: [HACKERS] Linking libpq statically to libssl

2017-11-02 Thread Stephen Frost
Peter, * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote: > On 10/27/17 08:24, Daniele Varrazzo wrote: > > I have a problem building binary packages for psycopg2. Binary > > packages ship with their own copies of libpq and libssl; > > Aside from the advice of "don't do that" ... > > >

Re: [HACKERS] [PATCH] A hook for session start

2017-11-02 Thread Fabrízio de Royes Mello
On Thu, Nov 2, 2017 at 5:42 AM, Aleksandr Parfenov < a.parfe...@postgrespro.ru> wrote: > > The following review has been posted through the commitfest application: > make installcheck-world: not tested > Implements feature: not tested > Spec compliant: not tested > Documentation:

Re: [HACKERS] SQL/JSON in PostgreSQL

2017-11-02 Thread Piotr Stefaniak
On 2017-02-28 20:08, Oleg Bartunov wrote: > Attached patch is an implementation of SQL/JSON data model from SQL-2016 > standard (ISO/IEC 9075-2:2016(E)) I've faintly started looking into this. > We created repository for reviewing (ask for write access) - >

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Nico Williams
On Thu, Nov 02, 2017 at 03:25:48PM -0700, Peter Geoghegan wrote: > Nico Williams wrote: > >A MERGE mapped to a DML like this: > > This is a bad idea. An implementation like this is not at all > maintainable. Assuming the DELETE issue can be addressed, why would this not

Re: [HACKERS] pgbench - use enum for meta commands

2017-11-02 Thread Tom Lane
Fabien COELHO writes: > [ pgbench-enum-meta-2.patch ] Pushed with some trivial cosmetic adjustments (pgindent changed it more than I did). regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to

Re: [HACKERS] GnuTLS support

2017-11-02 Thread Andreas Karlsson
On 09/18/2017 07:04 PM, Jeff Janes wrote:> You fixed the first issue, but I still get the second one: be-secure-gnutls.c: In function 'get_peer_certificate': be-secure-gnutls.c:667: error: 'GNUTLS_X509_CRT_LIST_SORT' undeclared (first use in this function) be-secure-gnutls.c:667: error: (Each

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Peter Geoghegan
Nico Williams wrote: A MERGE mapped to a DML like this: WITH updated AS ( UPDATE SET ... WHERE RETURNING ) , inserted AS ( INSERT INTO SELECT ... WHERE NOT IN (SELECT FROM updated) AND .. ON

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Nico Williams
On Thu, Nov 02, 2017 at 02:05:19PM -0700, Peter Geoghegan wrote: > Simon Riggs wrote: > >So in your view we should make no attempt to avoid concurrent errors, > >even when we have the capability to do so (in some cases) and doing so > >would be perfectly compliant with the

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-11-02 Thread Tom Lane
Alvaro Herrera writes: > Tom Lane wrote: >> So what would happen if we just don't summarize partial ranges? > Index scan would always have to read all the heap pages for that partial > range. Maybe not a big issue, but when you finish loading a table, it'd > be good to

Re: [HACKERS] Setting pd_lower in GIN metapage

2017-11-02 Thread Tom Lane
Amit Langote writes: > On 2017/09/26 16:30, Michael Paquier wrote: >> Cool, let's switch it back to a ready for committer status then. > Sure, thanks. Pushed with some cosmetic adjustments --- mostly, making the comments more explicit about why we need the

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Peter Geoghegan
Simon Riggs wrote: I think people imagined you had worked out how to make MERGE run concurrently, I certainly did, but in fact you're just saying you don't believe it ever should. I'm certain that they didn't think that at all. But I'll let them speak for themselves.

Re: [HACKERS] Deadlock in ALTER SUBSCRIPTION REFRESH PUBLICATION

2017-11-02 Thread Peter Eisentraut
On 10/24/17 13:13, Konstantin Knizhnik wrote: > The reason of this deadlock seems to be clear: ALTER SUBSCRIPTION starts > transaction at one node and tries to create slot at other node, which waiting > for completion of all active transaction while building scnapshpot. > Is there any way to

Re: [HACKERS] [PATCH] Add ALWAYS DEFERRED option for constraints

2017-11-02 Thread Nico Williams
On Thu, Nov 02, 2017 at 04:20:19PM -0400, Peter Eisentraut wrote: > I haven't really thought about this feature too hard; I just want to > give you a couple of code comments. Thanks! > I think the catalog structure, and relatedly also the parser structures, > could be made more compact. We

Re: [HACKERS] Linking libpq statically to libssl

2017-11-02 Thread Peter Eisentraut
On 10/27/17 08:24, Daniele Varrazzo wrote: > I have a problem building binary packages for psycopg2. Binary > packages ship with their own copies of libpq and libssl; Aside from the advice of "don't do that" ... > however if > another python package links to libssl the library will be imported >

Re: [HACKERS] Dynamic result sets from procedures

2017-11-02 Thread Daniel Verite
Peter Eisentraut wrote: > CREATE PROCEDURE pdrstest1() > LANGUAGE SQL > AS $$ > DECLARE c1 CURSOR WITH RETURN FOR SELECT * FROM cp_test2; > DECLARE c2 CURSOR WITH RETURN FOR SELECT * FROM cp_test3; > $$; > > CALL pdrstest1(); > > and that returns those two result sets to the client. If

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Simon Riggs
On 2 November 2017 at 19:16, Peter Geoghegan wrote: > Simon Riggs wrote: >> >> So if I understand you correctly, in your view MERGE should just fail >> with an ERROR if it runs concurrently with other DML? > > > That's certainly my opinion on the matter. It

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Nico Williams
On Thu, Nov 02, 2017 at 12:51:45PM -0700, Peter Geoghegan wrote: > Nico Williams wrote: > >If you want to ignore conflicts arising from concurrency you could > >always add an ON CONFLICT DO NOTHING to the INSERT DML in the mapping I > >proposed earlier. Thus a MERGE

Re: [HACKERS] [PATCH] Add ALWAYS DEFERRED option for constraints

2017-11-02 Thread Peter Eisentraut
I haven't really thought about this feature too hard; I just want to give you a couple of code comments. I think the catalog structure, and relatedly also the parser structures, could be made more compact. We currently have condeferrable and condeferred to represent three valid states (NOT

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-11-02 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera writes: > > Tom Lane wrote: > >> If VACUUM and brin_summarize_new_values both ignore the partial > >> range, then what else would request this? Can't we just decree > >> that we don't summarize the partial range, period? > > >

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Peter Geoghegan
Simon Riggs wrote: So if I understand you correctly, in your view MERGE should just fail with an ERROR if it runs concurrently with other DML? That's certainly my opinion on the matter. It seems like that might be the consensus, too. Obviously there are things that you

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Peter Geoghegan
Nico Williams wrote: If you want to ignore conflicts arising from concurrency you could always add an ON CONFLICT DO NOTHING to the INSERT DML in the mapping I proposed earlier. Thus a MERGE CONCURRENTLY could just do that. Is there any reason not to map MERGE as I

Re: [HACKERS] proposal: schema variables

2017-11-02 Thread Nico Williams
On Thu, Nov 02, 2017 at 11:48:44AM -0400, Tom Lane wrote: > Nico Williams writes: > > With access controls, GUCs could become schema variables, and settings > > from postgresql.conf could move into the database itself (which I think > > would be nice). > > People

Re: [HACKERS] VACUUM and ANALYZE disagreeing on what reltuples means

2017-11-02 Thread Tom Lane
Haribabu Kommi writes: > The changes are fine and now it reports proper live tuples in both > pg_class and stats. The other issue of continuous vacuum operation > leading to decrease of number of live tuples is not related to this > patch and that can be handled

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Nico Williams
On Thu, Nov 02, 2017 at 06:49:18PM +, Simon Riggs wrote: > On 1 November 2017 at 18:20, Peter Geoghegan wrote: > > In Postgres, you can avoid duplicate violations with MERGE by using a > > higher isolation level (these days, those are turned into a > > serialization error at

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-11-02 Thread Tom Lane
Alvaro Herrera writes: > Tom Lane wrote: >> If VACUUM and brin_summarize_new_values both ignore the partial >> range, then what else would request this? Can't we just decree >> that we don't summarize the partial range, period? > brin_summarize_range() can do it. So

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-11-02 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera writes: > > 2. when summarization is requested on the partial range at the end of a > > table, we acquire extension lock on the rel, then compute relation size > > and run summarization with the lock held. This guarantees that we don't >

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Nico Williams
If nothing else, anyone needing MERGE can port their MERGE statements to a DML with DML-containing CTEs... The generic mapping would be something like this, I think: WITH rows AS (SELECT FROM WHERE ) , updated AS ( UPDATE SET ... WHERE IN (SELECT FROM rows) /* matched */

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Simon Riggs
On 1 November 2017 at 18:20, Peter Geoghegan wrote: > In Postgres, you can avoid duplicate violations with MERGE by using a > higher isolation level (these days, those are turned into a > serialization error at higher isolation levels when no duplicate is > visible to the xact's

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Peter Geoghegan
Robert Haas wrote: And if, in the meantime, MERGE can only handle the cases where there is a unique index, then it can only handle the cases INSERT .. ON CONFLICT UPDATE can cover, which makes it, as far as I can see, syntactic sugar over what we already have. Maybe it's

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-11-02 Thread Tom Lane
Alvaro Herrera writes: > 1. in VACUUM or brin_summarize_new_values, we only process fully loaded > ranges, and ignore the partial range at end of table. OK. > 2. when summarization is requested on the partial range at the end of a > table, we acquire extension lock on

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-11-02 Thread Alvaro Herrera
Tomas Vondra wrote: > Unfortunately, I think we still have a problem ... I've been wondering > if we end up producing correct indexes, so I've done a simple test. Here's a proposed patch that should fix this problem (and does, in my testing). Would you please give it a try? This patch changes

Re: [HACKERS] initdb w/ restart

2017-11-02 Thread Peter Eisentraut
On 9/28/17 15:40, Jesper Pedersen wrote: > In a case of > > initdb /tmp/pgsql > > followed by > > pg_ctl -D /tmp/pgsql/ -l /tmp/logfile restart > > you'll get > > pg_ctl: PID file "/tmp/pgsql/postmaster.pid" does not exist > Is server running? > starting server anyway > pg_ctl: could not read

Re: [HACKERS] proposal: schema variables

2017-11-02 Thread Robert Haas
On Thu, Nov 2, 2017 at 9:05 PM, Nico Williams wrote: >> Overloading SET to handle both variables and GUCs seems likely to >> create problems, possibly including security problems. For example, >> maybe a security-definer function could leave behind variables to >> trick

[HACKERS] Parallel Plans and Cost of non-filter functions

2017-11-02 Thread Paul Ramsey
I'm working on a custom aggregate, that generates a serialized data format. The preparation of the geometry before being formatted is pretty intense, so it is probably a good thing for that work to be done in parallel, in partial aggregates. Here's an example SQL call: EXPLAIN analyze SELECT

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-11-02 Thread Robert Haas
On Thu, Nov 2, 2017 at 10:26 PM, Peter Geoghegan wrote: > On Thu, Nov 2, 2017 at 9:44 AM, Robert Haas wrote: >> The second commit (22576734b805fb0952f9be841ca8f643694ee868) is where >> I think things get a lot more dangerous. The problem (as Andres >>

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Robert Haas
On Tue, Oct 31, 2017 at 5:14 PM, Simon Riggs wrote: > I've proposed a SQL Standard compliant implementation that would do > much more than be new syntax over what we already have. > > So these two claims aren't accurate: "radical difference" and "syntax > sugar over a

Re: [HACKERS] pgbench - use enum for meta commands

2017-11-02 Thread a . parfenov
On Thu, 2 Nov 2017 17:50:52 +0100 (CET) Fabien COELHO wrote: > Updated version attached. . Here is the patch. Sorry for the noise. Everything alright. Patch is ready for commiter. -- Aleksandr Parfenov Postgres Professional: http://www.postgrespro.com Russian Postgres

Re: [HACKERS] [PATCH] Minor patch to correct symbol name in logs

2017-11-02 Thread Peter Eisentraut
On 9/20/17 01:56, Vaishnavi Prabakaran wrote: > Backend's lo_functions were renamed to avoid conflicting with libpq > prototypes in commit - 6fc547960dbe0b8bd6cefae5ab7ec3605a5c46fc.  > > Logs inside those functions still refer to old symbol names, Here is the > small patch to update the same. I

Re: [HACKERS] [COMMITTERS] pgsql: passwordcheck: Add test suite

2017-11-02 Thread Peter Eisentraut
On 9/15/17 00:20, Michael Paquier wrote: > On Fri, Sep 15, 2017 at 12:02 PM, Michael Paquier > wrote: >> On Fri, Sep 15, 2017 at 11:46 AM, Peter Eisentraut wrote: >>> passwordcheck: Add test suite >>> >>> Also improve one error message. >>> >>>

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-11-02 Thread Peter Geoghegan
On Thu, Nov 2, 2017 at 9:44 AM, Robert Haas wrote: > The second commit (22576734b805fb0952f9be841ca8f643694ee868) is where > I think things get a lot more dangerous. The problem (as Andres > pointed out to me this afternoon) is that it seems possible to end up > with a

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Robert Haas
On Thu, Nov 2, 2017 at 8:28 AM, Craig Ringer wrote: > On 1 November 2017 at 01:55, Peter Geoghegan wrote: >> The problem here is: Iff the first statement uses ON CONFLICT >> infrastructure, doesn't the absence of WHEN NOT MATCHED imply >> different semantics

Re: [HACKERS] pgbench - use enum for meta commands

2017-11-02 Thread Fabien COELHO
Updated version attached. . Here is the patch. Sorry for the noise. -- Fabien.diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c index 5d8a01c..6bd3e52 100644 --- a/src/bin/pgbench/pgbench.c +++ b/src/bin/pgbench/pgbench.c @@ -373,11 +373,21 @@ typedef enum QueryMode static

Re: [HACKERS] pgbench - use enum for meta commands

2017-11-02 Thread Fabien COELHO
The only thing I'm not quite sure about is a comment "which meta command ...". Maybe it's better to write it without question word, something like "meta command identifier..."? Ok. I agree. Updated version attached. I also added a const on a function parameter. Just a note about the

Re: [HACKERS] Removing wal_keep_segments as default configuration in PostgresNode.pm

2017-11-02 Thread Michael Paquier
On Thu, Nov 2, 2017 at 4:47 PM, Peter Eisentraut wrote: > On 9/11/17 21:55, Michael Paquier wrote: >> I tend to think that while all the other parameters make sense to >> deploy instances that need few resources, wal_keep_segments may cause >> up to 350MB of WAL

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-11-02 Thread Peter Geoghegan
On Thu, Nov 2, 2017 at 4:20 AM, Andres Freund wrote: > I think the problem is on the pruning, rather than the freezing side. We > can't freeze a tuple if it has an alive predecessor - rather than > weakining this, we should be fixing the pruning to not have the alive >

Re: [HACKERS] Removing wal_keep_segments as default configuration in PostgresNode.pm

2017-11-02 Thread Peter Eisentraut
On 9/11/17 21:55, Michael Paquier wrote: > I tend to think that while all the other parameters make sense to > deploy instances that need few resources, wal_keep_segments may cause > up to 350MB of WAL segments to be kept in each pg_wal's instance, > while max_wal_size is set at 128MB. The only

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-11-02 Thread Robert Haas
On Thu, Nov 2, 2017 at 8:25 PM, Alvaro Herrera wrote: > Pushed the reverts. > > I noticed while doing so that REL_10_STABLE contains the bogus commits. > Does that change our opinion regarding what to do for people upgrading > to a version containing the broken commits?

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-11-02 Thread Tom Lane
Alvaro Herrera writes: > Tom Lane wrote: >> Where are we on this --- do you want me to push the brin_doupdate >> fix I proposed, or were you intending to merge that into a >> larger patch? > Please push your fixes, I'll post my proposed patch for the other bug >

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-11-02 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera writes: > >> Hmm, I'm pretty sure we stress-tested brin in pretty much the same way. > >> But I see this misbehavior too. Looking ... > > > Turns out that this is related to concurrent growth of the table while > > the summarization

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-11-02 Thread Tom Lane
Alvaro Herrera writes: >> Hmm, I'm pretty sure we stress-tested brin in pretty much the same way. >> But I see this misbehavior too. Looking ... > Turns out that this is related to concurrent growth of the table while > the summarization process is scanning -- so new

[HACKERS] Minor comment issue in receivelog.c

2017-11-02 Thread Bernd Helmle
Please find a minor comment fix for receivelog.c, HandleCopyStream(). The comments talks about a START_STREAMING command, but i think START_REPLICATION is what's meant here. Bernd diff --git a/src/bin/pg_basebackup/receivelog.c b/src/bin/pg_basebackup/receivelog.c index

Re: [HACKERS] Document pgstattuple privileges without ambiguity

2017-11-02 Thread Peter Eisentraut
On 8/21/17 03:47, Feike Steenbergen wrote: > When installing pgstattuple on 10, the documentation about its > privileges was unclear to me. (Does the pg_stat_scan_tables role get > EXECUTE privileges by default or not?). I agree that this has gotten a bit confusing after apparently being patched

Re: [HACKERS] proposal: schema variables

2017-11-02 Thread Pavel Stehule
2017-11-02 13:35 GMT+01:00 Robert Haas : > On Thu, Oct 26, 2017 at 12:51 PM, Pavel Stehule > wrote: > > The variables can be modified by SQL command SET (this is taken from > > standard, and it natural) > > > > SET varname = expression; > >

Re: [HACKERS] proposal: schema variables

2017-11-02 Thread Tom Lane
Nico Williams writes: > With access controls, GUCs could become schema variables, and settings > from postgresql.conf could move into the database itself (which I think > would be nice). People re-propose some variant of that every so often, but it never works, because it

Re: [HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread Andres Freund
On 2017-11-02 10:38:34 -0400, Tom Lane wrote: > David Rowley writes: > > On 3 November 2017 at 03:17, Tom Lane wrote: > >> We've jacked up the List API and driven a new implementation underneath > >> once before. Maybe it's time to do that

Re: [HACKERS] proposal: schema variables

2017-11-02 Thread Pavel Stehule
2017-11-02 16:07 GMT+01:00 Craig Ringer : > On 26 October 2017 at 15:21, Pavel Stehule > wrote: > > Hi, > > > > I propose a new database object - a variable. > > Didn't we have a pretty long discussion about this already in > > Yeah. > >

Re: [HACKERS] proposal: schema variables

2017-11-02 Thread Pavel Stehule
2017-11-02 16:35 GMT+01:00 Nico Williams : > On Thu, Nov 02, 2017 at 06:05:54PM +0530, Robert Haas wrote: > > On Thu, Oct 26, 2017 at 12:51 PM, Pavel Stehule > wrote: > > > The variables can be modified by SQL command SET (this is taken from > > >

Re: [HACKERS] proposal: schema variables

2017-11-02 Thread Nico Williams
On Thu, Nov 02, 2017 at 06:05:54PM +0530, Robert Haas wrote: > On Thu, Oct 26, 2017 at 12:51 PM, Pavel Stehule > wrote: > > The variables can be modified by SQL command SET (this is taken from > > standard, and it natural) > > > > SET varname = expression; > >

Re: [HACKERS] Client Connection redirection support for PostgreSQL

2017-11-02 Thread Craig Ringer
On 2 November 2017 at 14:02, Satyanarayana Narlapuram wrote: > Proposal: > > Add the ability to the PostgreSQL server instance to route the traffic to a > different server instance based on the rules defined in server’s pg_bha.conf > configuration file. At

Re: [HACKERS] proposal: schema variables

2017-11-02 Thread Craig Ringer
On 26 October 2017 at 15:21, Pavel Stehule wrote: > Hi, > > I propose a new database object - a variable. Didn't we have a pretty long discussion about this already in Yeah.

Re: [HACKERS] Custom compression methods

2017-11-02 Thread Craig Ringer
On 2 November 2017 at 17:41, Ildus Kurbangaliev wrote: > In this patch compression methods is suitable for MAIN and EXTENDED > storages like in current implementation in postgres. Just instead only > of LZ4 you can specify any other compression method. We've had

Re: [HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread David Rowley
On 3 November 2017 at 03:27, Stephen Frost wrote: > * David Rowley (david.row...@2ndquadrant.com) wrote: >> We could get around a few of these problems if Lists were implemented >> internally as arrays, however, arrays are pretty bad if we want to >> delete an item out the

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-11-02 Thread Alvaro Herrera
Stephen Frost wrote: > * Tom Lane (t...@sss.pgh.pa.us) wrote: > > Andres Freund writes: > > > Do we care about people upgrading to unreleased versions? We could do > > > nothing, document it in the release notes, or ??? > > > > Do nothing. > > Agreed. Not much we can do

Re: [HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread Tom Lane
David Rowley writes: >> It seems to me that a whole lot of the complaints about this could be >> resolved simply by improving the List infrastructure to allocate ListCells >> in batches. That would address the question of "too much palloc traffic" >> and greatly

Re: [HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread David Rowley
On 3 November 2017 at 03:38, Tom Lane wrote: > David Rowley writes: >> On 3 November 2017 at 03:17, Tom Lane wrote: >>> We've jacked up the List API and driven a new implementation underneath >>> once before. Maybe it's time

Re: [HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread Tom Lane
David Rowley writes: > On 3 November 2017 at 03:17, Tom Lane wrote: >> We've jacked up the List API and driven a new implementation underneath >> once before. Maybe it's time to do that again. > Maybe, but the new implementation is not going to

Re: [HACKERS] pgbench - use enum for meta commands

2017-11-02 Thread Aleksandr Parfenov
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed Hi, Looks good to me. The only thing I'm not quite sure

Re: [HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread Stephen Frost
David, * David Rowley (david.row...@2ndquadrant.com) wrote: > Our List implementation internally uses linked lists which are > certainly good for some things, but pretty bad at other things. Linked > lists are pretty bad when you want to fetch the Nth element, as it > means looping ever each

Re: [HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread Craig Ringer
On 2 November 2017 at 22:22, David Rowley wrote: > On 3 November 2017 at 03:17, Tom Lane wrote: >> We've jacked up the List API and driven a new implementation underneath >> once before. Maybe it's time to do that again. > > Maybe, but the new

Re: [HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread Craig Ringer
On 2 November 2017 at 22:17, Tom Lane wrote: > David Rowley writes: >> Comments on the design are welcome, but I was too late to the >> commitfest, so there are other priorities. However, if you have a >> strong opinion, feel free to voice it. >

Re: [HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread David Rowley
On 3 November 2017 at 03:17, Tom Lane wrote: > We've jacked up the List API and driven a new implementation underneath > once before. Maybe it's time to do that again. Maybe, but the new implementation is not going to do well with places where we perform lcons(). Probably

Re: [HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread Tom Lane
David Rowley writes: > Comments on the design are welcome, but I was too late to the > commitfest, so there are other priorities. However, if you have a > strong opinion, feel free to voice it. I do not like replacing Lists piecemeal; that's a recipe for ongoing API

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-11-02 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Andres Freund writes: > > Do we care about people upgrading to unreleased versions? We could do > > nothing, document it in the release notes, or ??? > > Do nothing. Agreed. Not much we can do there. Thanks! Stephen signature.asc

[HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread David Rowley
Hackers, Our List implementation internally uses linked lists which are certainly good for some things, but pretty bad at other things. Linked lists are pretty bad when you want to fetch the Nth element, as it means looping ever each previous element until you get to the Nth. They're also pretty

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-11-02 Thread Tom Lane
Andres Freund writes: > Do we care about people upgrading to unreleased versions? We could do > nothing, document it in the release notes, or ??? Do nothing. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] Client Connection redirection support for PostgreSQL

2017-11-02 Thread David Fetter
On Thu, Nov 02, 2017 at 06:02:43AM +, Satyanarayana Narlapuram wrote: > Proposal: > Add the ability to the PostgreSQL server instance to route the > traffic to a different server instance based on the rules defined in > server's pg_bha.conf configuration file. At a high level this > enables

Re: [HACKERS] [PATCH] Add two-arg for of current_setting(NAME, FALLBACK)

2017-11-02 Thread Peter Eisentraut
On 11/1/17 14:02, Nico Williams wrote: > There already _is_ a two-argument form of current_setting() that yours > somewhat conflicts with: > >current_setting(setting_name [, missing_ok ]) > > https://www.postgresql.org/docs/current/static/functions-admin.html > > I often use > >

Re: [HACKERS] list of credits for release notes

2017-11-02 Thread Peter Eisentraut
On 10/3/17 12:31, Dang Minh Huong wrote: > Sorry but please change "Huong Dangminh"(my name) to "Dang Minh Huong". done -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-11-02 Thread Andres Freund
Hi, On 2017-11-02 13:49:47 +0100, Alvaro Herrera wrote: > Andres Freund wrote: > > I think the problem is on the pruning, rather than the freezing side. We > > can't freeze a tuple if it has an alive predecessor - rather than > > weakining this, we should be fixing the pruning to not have the

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-11-02 Thread Alvaro Herrera
Andres Freund wrote: > I spent some time discussing this with Robert today (with both of us > alternating between feeling the other and ourselves as stupid), and the > conclusion I think is that the problem is on the pruning, rather than > the freezing side. Thanks both for spending some more

Re: [HACKERS] proposal: schema variables

2017-11-02 Thread Robert Haas
On Thu, Oct 26, 2017 at 12:51 PM, Pavel Stehule wrote: > The variables can be modified by SQL command SET (this is taken from > standard, and it natural) > > SET varname = expression; Overloading SET to handle both variables and GUCs seems likely to create problems,

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-11-02 Thread Robert Haas
On Thu, Nov 2, 2017 at 4:50 PM, Andres Freund wrote: > I think a5736bf754c82d8b86674e199e232096c679201d might be dangerous in > the face of previously corrupted tuple chains and pg_upgraded clusters - > it can lead to tuples being considered related, even though they they're >

[HACKERS] Client Connection redirection support for PostgreSQL

2017-11-02 Thread Satyanarayana Narlapuram
Proposal: Add the ability to the PostgreSQL server instance to route the traffic to a different server instance based on the rules defined in server's pg_bha.conf configuration file. At a high level this enables offloading the user requests to a different server instance based on the rules

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-11-02 Thread Andres Freund
Hi, On 2017-09-28 14:47:53 +, Alvaro Herrera wrote: > Fix freezing of a dead HOT-updated tuple > > Vacuum calls page-level HOT prune to remove dead HOT tuples before doing > liveness checks (HeapTupleSatisfiesVacuum) on the remaining tuples. But > concurrent transaction commit/abort may turn

  1   2   >