Re: [HACKERS] Try to fix endless loop in ecpg with informix mode

2017-11-02 Thread 高增琦
Hi, I found the last commit changed as: ``` /* skip invalid characters */ do { (*scan_length)++; - } while (**scan_length != ' ' && **scan_length != '\0' && isdigit(**scan_length)); + } while (isdigit(**scan_length)); return

Re: [HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread Craig Ringer
On 3 November 2017 at 12:41, David Rowley wrote: > On 3 November 2017 at 03:26, Craig Ringer wrote: >> On 2 November 2017 at 22:22, David Rowley >> wrote: >>> Maybe, but the new implementation is not going to do well with places >>> where we perform lcons(). Probably many of those places could

Re: [HACKERS] Setting pd_lower in GIN metapage

2017-11-02 Thread Michael Paquier
On Fri, Nov 3, 2017 at 1:10 AM, Amit Kapila wrote: > On Fri, Nov 3, 2017 at 2:54 AM, Tom Lane wrote: >> I've marked the CF entry closed. However, I'm not sure if we're quite >> done with this thread. Weren't we going to adjust nbtree and hash to >> be more aggressive about labeling their metapa

Re: [HACKERS] pgbench - use enum for meta commands

2017-11-02 Thread Fabien COELHO
[ pgbench-enum-meta-2.patch ] Pushed with some trivial cosmetic adjustments (pgindent changed it more than I did). Ok. Thanks. -- Fabien. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-ha

Re: [HACKERS] PATCH: pgbench - option to build using ppoll() for larger connection counts

2017-11-02 Thread Fabien COELHO
Hello, Could you rebase the v11 patch? -- Fabien. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread David Rowley
On 3 November 2017 at 03:26, Craig Ringer wrote: > On 2 November 2017 at 22:22, David Rowley > wrote: >> Maybe, but the new implementation is not going to do well with places >> where we perform lcons(). Probably many of those places could be >> changed to lappend(), but I bet there's plenty tha

Re: [HACKERS] SSL and Encryption

2017-11-02 Thread Michael Paquier
On Fri, Nov 3, 2017 at 3:19 AM, Craig Ringer wrote: > This is probably off topic for pgsql-hackers. > > For password crypto please go read the SCRAM thread and the PostgreSQL > 10 release notes. The SCRAM discussion is spread across two threads mainly with hundreds of emails, which may discourage

Re: [HACKERS] path toward faster partition pruning

2017-11-02 Thread David Rowley
On 31 October 2017 at 21:43, Amit Langote wrote: > Attached updated version of the patches addressing some of your comments I've spent a bit of time looking at these but I'm out of time for now. So far I have noted down the following; 1. This comment seem wrong. /* * Since the clauses in rel->

Re: [HACKERS] SSL and Encryption

2017-11-02 Thread Craig Ringer
On 3 November 2017 at 11:16, chiru r wrote: > Hi , > > Please suggest the best chiper suite to configure openSSL for PostgreSQL > Server and client?. > > How to use other than md5 encryption algorithm to encrypt the passwords in > PostgreSQL? This is probably off topic for pgsql-hackers. For pas

[HACKERS] SSL and Encryption

2017-11-02 Thread chiru r
Hi , Please suggest the best chiper suite to configure openSSL for PostgreSQL Server and client?. How to use other than md5 encryption algorithm to encrypt the passwords in PostgreSQL? Thanks, Chiru

Re: [HACKERS] path toward faster partition pruning

2017-11-02 Thread David Rowley
On 31 October 2017 at 21:43, Amit Langote wrote: > Attached updated version of the patches addressing some of your comments > above and fixing a bug that Rajkumar reported [1]. As mentioned there, > I'm including here a patch (the 0005 of the attached) to tweak the default > range partition const

Re: [HACKERS] Small improvement to compactify_tuples

2017-11-02 Thread Tom Lane
Sokolov Yura writes: > [ 0001-Improve-compactify_tuples.patch, v5 or thereabouts ] I started to review this patch. I spent a fair amount of time on beautifying the code, because I found it rather ugly and drastically undercommented. Once I had it to the point where it seemed readable, I went to

[HACKERS] Skip unneeded temp file in 'make html'

2017-11-02 Thread David Fetter
Folks, Please find attached a patch for $Subject. Best, David. -- David Fetter http://fetter.org/ Phone: +1 415 235 3778 Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate Don't make an unneeded temp file In passing, make a slight correction to the regex.

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2017-11-02 Thread Thomas Munro
On Fri, Nov 3, 2017 at 2:24 PM, Peter Geoghegan wrote: > Thomas Munro wrote: >> That way you don't have to opt in to BufFile's >> double buffering and segmentation schemes just to get shared file >> clean-up, if for some reason you want direct file handles. > > Is that something that you really t

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2017-11-02 Thread Peter Geoghegan
Thomas Munro wrote: I'm going to make an item on my personal TODO list for that. No useful insights on that right now, though. I decided to try that, but it didn't really work: fd.h gets included by front-end code, so I can't very well define a struct and declare functions that deal in dsm_seg

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2017-11-02 Thread Thomas Munro
On Wed, Nov 1, 2017 at 2:11 PM, Peter Geoghegan wrote: > On Tue, Oct 31, 2017 at 5:07 PM, Thomas Munro > wrote: >> Another complaint is that perhaps fd.c >> knows too much about buffile.c's business. For example, >> RemovePgTempFilesInDir() knows about the ".set" directories created by >> buffil

Re: [HACKERS] Setting pd_lower in GIN metapage

2017-11-02 Thread Amit Kapila
On Fri, Nov 3, 2017 at 2:54 AM, Tom Lane wrote: > Amit Langote writes: >> On 2017/09/26 16:30, Michael Paquier wrote: >>> Cool, let's switch it back to a ready for committer status then. > >> Sure, thanks. > > Pushed with some cosmetic adjustments --- mostly, making the comments more > explicit a

Re: [HACKERS] Parallel Hash take II

2017-11-02 Thread Thomas Munro
On Mon, Oct 30, 2017 at 1:49 PM, Thomas Munro wrote: > A couple of stupid things outstanding: > > 1. EXPLAIN ANALYZE for Parallel Hash "actual" shows the complete row > count, which is interesting to know (or not? maybe I should show it > somewhere else?), but the costing shows the partial row e

Re: [HACKERS] Linking libpq statically to libssl

2017-11-02 Thread Stephen Frost
Peter, * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote: > On 10/27/17 08:24, Daniele Varrazzo wrote: > > I have a problem building binary packages for psycopg2. Binary > > packages ship with their own copies of libpq and libssl; > > Aside from the advice of "don't do that" ... > > >

Re: [HACKERS] [PATCH] A hook for session start

2017-11-02 Thread Fabrízio de Royes Mello
On Thu, Nov 2, 2017 at 5:42 AM, Aleksandr Parfenov < a.parfe...@postgrespro.ru> wrote: > > The following review has been posted through the commitfest application: > make installcheck-world: not tested > Implements feature: not tested > Spec compliant: not tested > Documentation:

Re: [HACKERS] SQL/JSON in PostgreSQL

2017-11-02 Thread Piotr Stefaniak
On 2017-02-28 20:08, Oleg Bartunov wrote: > Attached patch is an implementation of SQL/JSON data model from SQL-2016 > standard (ISO/IEC 9075-2:2016(E)) I've faintly started looking into this. > We created repository for reviewing (ask for write access) - > https://github.com/postgrespro/sqljson/

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Nico Williams
On Thu, Nov 02, 2017 at 03:25:48PM -0700, Peter Geoghegan wrote: > Nico Williams wrote: > >A MERGE mapped to a DML like this: > > This is a bad idea. An implementation like this is not at all > maintainable. Assuming the DELETE issue can be addressed, why would this not be maintainable? > >can

Re: [HACKERS] pgbench - use enum for meta commands

2017-11-02 Thread Tom Lane
Fabien COELHO writes: > [ pgbench-enum-meta-2.patch ] Pushed with some trivial cosmetic adjustments (pgindent changed it more than I did). regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] GnuTLS support

2017-11-02 Thread Andreas Karlsson
On 09/18/2017 07:04 PM, Jeff Janes wrote:> You fixed the first issue, but I still get the second one: be-secure-gnutls.c: In function 'get_peer_certificate': be-secure-gnutls.c:667: error: 'GNUTLS_X509_CRT_LIST_SORT' undeclared (first use in this function) be-secure-gnutls.c:667: error: (Each u

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Peter Geoghegan
Nico Williams wrote: A MERGE mapped to a DML like this: WITH updated AS ( UPDATE SET ... WHERE RETURNING ) , inserted AS ( INSERT INTO SELECT ... WHERE NOT IN (SELECT FROM updated) AND .. ON CONFLICT DO NOTHING -- see bel

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Nico Williams
On Thu, Nov 02, 2017 at 02:05:19PM -0700, Peter Geoghegan wrote: > Simon Riggs wrote: > >So in your view we should make no attempt to avoid concurrent errors, > >even when we have the capability to do so (in some cases) and doing so > >would be perfectly compliant with the SQLStandard. > > Yes. T

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-11-02 Thread Tom Lane
Alvaro Herrera writes: > Tom Lane wrote: >> So what would happen if we just don't summarize partial ranges? > Index scan would always have to read all the heap pages for that partial > range. Maybe not a big issue, but when you finish loading a table, it'd > be good to have a mechanism to summar

Re: [HACKERS] Setting pd_lower in GIN metapage

2017-11-02 Thread Tom Lane
Amit Langote writes: > On 2017/09/26 16:30, Michael Paquier wrote: >> Cool, let's switch it back to a ready for committer status then. > Sure, thanks. Pushed with some cosmetic adjustments --- mostly, making the comments more explicit about why we need the apparently-redundant assignments to pd_

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Peter Geoghegan
Simon Riggs wrote: I think people imagined you had worked out how to make MERGE run concurrently, I certainly did, but in fact you're just saying you don't believe it ever should. I'm certain that they didn't think that at all. But I'll let them speak for themselves. That is strange since th

Re: [HACKERS] Deadlock in ALTER SUBSCRIPTION REFRESH PUBLICATION

2017-11-02 Thread Peter Eisentraut
On 10/24/17 13:13, Konstantin Knizhnik wrote: > The reason of this deadlock seems to be clear: ALTER SUBSCRIPTION starts > transaction at one node and tries to create slot at other node, which waiting > for completion of all active transaction while building scnapshpot. > Is there any way to avoi

Re: [HACKERS] [PATCH] Add ALWAYS DEFERRED option for constraints

2017-11-02 Thread Nico Williams
On Thu, Nov 02, 2017 at 04:20:19PM -0400, Peter Eisentraut wrote: > I haven't really thought about this feature too hard; I just want to > give you a couple of code comments. Thanks! > I think the catalog structure, and relatedly also the parser structures, > could be made more compact. We curre

Re: [HACKERS] Linking libpq statically to libssl

2017-11-02 Thread Peter Eisentraut
On 10/27/17 08:24, Daniele Varrazzo wrote: > I have a problem building binary packages for psycopg2. Binary > packages ship with their own copies of libpq and libssl; Aside from the advice of "don't do that" ... > however if > another python package links to libssl the library will be imported >

Re: [HACKERS] Dynamic result sets from procedures

2017-11-02 Thread Daniel Verite
Peter Eisentraut wrote: > CREATE PROCEDURE pdrstest1() > LANGUAGE SQL > AS $$ > DECLARE c1 CURSOR WITH RETURN FOR SELECT * FROM cp_test2; > DECLARE c2 CURSOR WITH RETURN FOR SELECT * FROM cp_test3; > $$; > > CALL pdrstest1(); > > and that returns those two result sets to the client. If

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Simon Riggs
On 2 November 2017 at 19:16, Peter Geoghegan wrote: > Simon Riggs wrote: >> >> So if I understand you correctly, in your view MERGE should just fail >> with an ERROR if it runs concurrently with other DML? > > > That's certainly my opinion on the matter. It seems like that might be > the consensu

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Nico Williams
On Thu, Nov 02, 2017 at 12:51:45PM -0700, Peter Geoghegan wrote: > Nico Williams wrote: > >If you want to ignore conflicts arising from concurrency you could > >always add an ON CONFLICT DO NOTHING to the INSERT DML in the mapping I > >proposed earlier. Thus a MERGE CONCURRENTLY could just do tha

Re: [HACKERS] [PATCH] Add ALWAYS DEFERRED option for constraints

2017-11-02 Thread Peter Eisentraut
I haven't really thought about this feature too hard; I just want to give you a couple of code comments. I think the catalog structure, and relatedly also the parser structures, could be made more compact. We currently have condeferrable and condeferred to represent three valid states (NOT DEFERR

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-11-02 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera writes: > > Tom Lane wrote: > >> If VACUUM and brin_summarize_new_values both ignore the partial > >> range, then what else would request this? Can't we just decree > >> that we don't summarize the partial range, period? > > > brin_summarize_range() can do it. >

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Peter Geoghegan
Simon Riggs wrote: So if I understand you correctly, in your view MERGE should just fail with an ERROR if it runs concurrently with other DML? That's certainly my opinion on the matter. It seems like that might be the consensus, too. Obviously there are things that you as a user can do about

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Peter Geoghegan
Nico Williams wrote: If you want to ignore conflicts arising from concurrency you could always add an ON CONFLICT DO NOTHING to the INSERT DML in the mapping I proposed earlier. Thus a MERGE CONCURRENTLY could just do that. Is there any reason not to map MERGE as I proposed? Performance, for

Re: [HACKERS] proposal: schema variables

2017-11-02 Thread Nico Williams
On Thu, Nov 02, 2017 at 11:48:44AM -0400, Tom Lane wrote: > Nico Williams writes: > > With access controls, GUCs could become schema variables, and settings > > from postgresql.conf could move into the database itself (which I think > > would be nice). > > People re-propose some variant of that e

Re: [HACKERS] VACUUM and ANALYZE disagreeing on what reltuples means

2017-11-02 Thread Tom Lane
Haribabu Kommi writes: > The changes are fine and now it reports proper live tuples in both > pg_class and stats. The other issue of continuous vacuum operation > leading to decrease of number of live tuples is not related to this > patch and that can be handled separately. I did not like this pa

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Nico Williams
On Thu, Nov 02, 2017 at 06:49:18PM +, Simon Riggs wrote: > On 1 November 2017 at 18:20, Peter Geoghegan wrote: > > In Postgres, you can avoid duplicate violations with MERGE by using a > > higher isolation level (these days, those are turned into a > > serialization error at higher isolation l

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-11-02 Thread Tom Lane
Alvaro Herrera writes: > Tom Lane wrote: >> If VACUUM and brin_summarize_new_values both ignore the partial >> range, then what else would request this? Can't we just decree >> that we don't summarize the partial range, period? > brin_summarize_range() can do it. So what would happen if we just

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-11-02 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera writes: > > 2. when summarization is requested on the partial range at the end of a > > table, we acquire extension lock on the rel, then compute relation size > > and run summarization with the lock held. This guarantees that we don't > > miss any pages. This i

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Nico Williams
If nothing else, anyone needing MERGE can port their MERGE statements to a DML with DML-containing CTEs... The generic mapping would be something like this, I think: WITH rows AS (SELECT FROM WHERE ) , updated AS ( UPDATE SET ... WHERE IN (SELECT FROM rows) /* matched */

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Simon Riggs
On 1 November 2017 at 18:20, Peter Geoghegan wrote: > In Postgres, you can avoid duplicate violations with MERGE by using a > higher isolation level (these days, those are turned into a > serialization error at higher isolation levels when no duplicate is > visible to the xact's snapshot). So if

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Peter Geoghegan
Robert Haas wrote: And if, in the meantime, MERGE can only handle the cases where there is a unique index, then it can only handle the cases INSERT .. ON CONFLICT UPDATE can cover, which makes it, as far as I can see, syntactic sugar over what we already have. Maybe it's not entirely - you migh

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-11-02 Thread Tom Lane
Alvaro Herrera writes: > 1. in VACUUM or brin_summarize_new_values, we only process fully loaded > ranges, and ignore the partial range at end of table. OK. > 2. when summarization is requested on the partial range at the end of a > table, we acquire extension lock on the rel, then compute relat

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-11-02 Thread Alvaro Herrera
Tomas Vondra wrote: > Unfortunately, I think we still have a problem ... I've been wondering > if we end up producing correct indexes, so I've done a simple test. Here's a proposed patch that should fix this problem (and does, in my testing). Would you please give it a try? This patch changes t

Re: [HACKERS] initdb w/ restart

2017-11-02 Thread Peter Eisentraut
On 9/28/17 15:40, Jesper Pedersen wrote: > In a case of > > initdb /tmp/pgsql > > followed by > > pg_ctl -D /tmp/pgsql/ -l /tmp/logfile restart > > you'll get > > pg_ctl: PID file "/tmp/pgsql/postmaster.pid" does not exist > Is server running? > starting server anyway > pg_ctl: could not read

Re: [HACKERS] proposal: schema variables

2017-11-02 Thread Robert Haas
On Thu, Nov 2, 2017 at 9:05 PM, Nico Williams wrote: >> Overloading SET to handle both variables and GUCs seems likely to >> create problems, possibly including security problems. For example, >> maybe a security-definer function could leave behind variables to >> trick the calling code into fail

[HACKERS] Parallel Plans and Cost of non-filter functions

2017-11-02 Thread Paul Ramsey
I'm working on a custom aggregate, that generates a serialized data format. The preparation of the geometry before being formatted is pretty intense, so it is probably a good thing for that work to be done in parallel, in partial aggregates. Here's an example SQL call: EXPLAIN analyze SELECT lengt

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-11-02 Thread Robert Haas
On Thu, Nov 2, 2017 at 10:26 PM, Peter Geoghegan wrote: > On Thu, Nov 2, 2017 at 9:44 AM, Robert Haas wrote: >> The second commit (22576734b805fb0952f9be841ca8f643694ee868) is where >> I think things get a lot more dangerous. The problem (as Andres >> pointed out to me this afternoon) is that it

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Robert Haas
On Tue, Oct 31, 2017 at 5:14 PM, Simon Riggs wrote: > I've proposed a SQL Standard compliant implementation that would do > much more than be new syntax over what we already have. > > So these two claims aren't accurate: "radical difference" and "syntax > sugar over a capability we have". I think

Re: [HACKERS] pgbench - use enum for meta commands

2017-11-02 Thread a . parfenov
On Thu, 2 Nov 2017 17:50:52 +0100 (CET) Fabien COELHO wrote: > Updated version attached. . Here is the patch. Sorry for the noise. Everything alright. Patch is ready for commiter. -- Aleksandr Parfenov Postgres Professional: http://www.postgrespro.com Russian Postgres Company -- Sent via

Re: [HACKERS] [PATCH] Minor patch to correct symbol name in logs

2017-11-02 Thread Peter Eisentraut
On 9/20/17 01:56, Vaishnavi Prabakaran wrote: > Backend's lo_functions were renamed to avoid conflicting with libpq > prototypes in commit - 6fc547960dbe0b8bd6cefae5ab7ec3605a5c46fc.  > > Logs inside those functions still refer to old symbol names, Here is the > small patch to update the same. I

Re: [HACKERS] [COMMITTERS] pgsql: passwordcheck: Add test suite

2017-11-02 Thread Peter Eisentraut
On 9/15/17 00:20, Michael Paquier wrote: > On Fri, Sep 15, 2017 at 12:02 PM, Michael Paquier > wrote: >> On Fri, Sep 15, 2017 at 11:46 AM, Peter Eisentraut wrote: >>> passwordcheck: Add test suite >>> >>> Also improve one error message. >>> >>> Reviewed-by: David Steele >> >> Sorry for showing u

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-11-02 Thread Peter Geoghegan
On Thu, Nov 2, 2017 at 9:44 AM, Robert Haas wrote: > The second commit (22576734b805fb0952f9be841ca8f643694ee868) is where > I think things get a lot more dangerous. The problem (as Andres > pointed out to me this afternoon) is that it seems possible to end up > with a situation where there shoul

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Robert Haas
On Thu, Nov 2, 2017 at 8:28 AM, Craig Ringer wrote: > On 1 November 2017 at 01:55, Peter Geoghegan wrote: >> The problem here is: Iff the first statement uses ON CONFLICT >> infrastructure, doesn't the absence of WHEN NOT MATCHED imply >> different semantics for the remaining updates and deletes

Re: [HACKERS] pgbench - use enum for meta commands

2017-11-02 Thread Fabien COELHO
Updated version attached. . Here is the patch. Sorry for the noise. -- Fabien.diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c index 5d8a01c..6bd3e52 100644 --- a/src/bin/pgbench/pgbench.c +++ b/src/bin/pgbench/pgbench.c @@ -373,11 +373,21 @@ typedef enum QueryMode static Q

Re: [HACKERS] pgbench - use enum for meta commands

2017-11-02 Thread Fabien COELHO
The only thing I'm not quite sure about is a comment "which meta command ...". Maybe it's better to write it without question word, something like "meta command identifier..."? Ok. I agree. Updated version attached. I also added a const on a function parameter. Just a note about the motivat

Re: [HACKERS] Removing wal_keep_segments as default configuration in PostgresNode.pm

2017-11-02 Thread Michael Paquier
On Thu, Nov 2, 2017 at 4:47 PM, Peter Eisentraut wrote: > On 9/11/17 21:55, Michael Paquier wrote: >> I tend to think that while all the other parameters make sense to >> deploy instances that need few resources, wal_keep_segments may cause >> up to 350MB of WAL segments to be kept in each pg_wal'

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-11-02 Thread Peter Geoghegan
On Thu, Nov 2, 2017 at 4:20 AM, Andres Freund wrote: > I think the problem is on the pruning, rather than the freezing side. We > can't freeze a tuple if it has an alive predecessor - rather than > weakining this, we should be fixing the pruning to not have the alive > predecessor. Excellent catc

Re: [HACKERS] Removing wal_keep_segments as default configuration in PostgresNode.pm

2017-11-02 Thread Peter Eisentraut
On 9/11/17 21:55, Michael Paquier wrote: > I tend to think that while all the other parameters make sense to > deploy instances that need few resources, wal_keep_segments may cause > up to 350MB of WAL segments to be kept in each pg_wal's instance, > while max_wal_size is set at 128MB. The only tes

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-11-02 Thread Robert Haas
On Thu, Nov 2, 2017 at 8:25 PM, Alvaro Herrera wrote: > Pushed the reverts. > > I noticed while doing so that REL_10_STABLE contains the bogus commits. > Does that change our opinion regarding what to do for people upgrading > to a version containing the broken commits? I don't think so, because

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-11-02 Thread Tom Lane
Alvaro Herrera writes: > Tom Lane wrote: >> Where are we on this --- do you want me to push the brin_doupdate >> fix I proposed, or were you intending to merge that into a >> larger patch? > Please push your fixes, I'll post my proposed patch for the other bug > afterwards; they are unrelated pro

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-11-02 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera writes: > >> Hmm, I'm pretty sure we stress-tested brin in pretty much the same way. > >> But I see this misbehavior too. Looking ... > > > Turns out that this is related to concurrent growth of the table while > > the summarization process is scanning -- so new

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-11-02 Thread Tom Lane
Alvaro Herrera writes: >> Hmm, I'm pretty sure we stress-tested brin in pretty much the same way. >> But I see this misbehavior too. Looking ... > Turns out that this is related to concurrent growth of the table while > the summarization process is scanning -- so new pages have appeared at > the

[HACKERS] Minor comment issue in receivelog.c

2017-11-02 Thread Bernd Helmle
Please find a minor comment fix for receivelog.c, HandleCopyStream(). The comments talks about a START_STREAMING command, but i think START_REPLICATION is what's meant here. Bernd diff --git a/src/bin/pg_basebackup/receivelog.c b/src/bin/pg_basebackup/receivelog.c index 888458f4a9..befbb

Re: [HACKERS] Document pgstattuple privileges without ambiguity

2017-11-02 Thread Peter Eisentraut
On 8/21/17 03:47, Feike Steenbergen wrote: > When installing pgstattuple on 10, the documentation about its > privileges was unclear to me. (Does the pg_stat_scan_tables role get > EXECUTE privileges by default or not?). I agree that this has gotten a bit confusing after apparently being patched a

Re: [HACKERS] proposal: schema variables

2017-11-02 Thread Pavel Stehule
2017-11-02 13:35 GMT+01:00 Robert Haas : > On Thu, Oct 26, 2017 at 12:51 PM, Pavel Stehule > wrote: > > The variables can be modified by SQL command SET (this is taken from > > standard, and it natural) > > > > SET varname = expression; > > Overloading SET to handle both variables and GUCs seems

Re: [HACKERS] proposal: schema variables

2017-11-02 Thread Tom Lane
Nico Williams writes: > With access controls, GUCs could become schema variables, and settings > from postgresql.conf could move into the database itself (which I think > would be nice). People re-propose some variant of that every so often, but it never works, because it ignores the fact that so

Re: [HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread Andres Freund
On 2017-11-02 10:38:34 -0400, Tom Lane wrote: > David Rowley writes: > > On 3 November 2017 at 03:17, Tom Lane wrote: > >> We've jacked up the List API and driven a new implementation underneath > >> once before. Maybe it's time to do that again. > > > Maybe, but the new implementation is not g

Re: [HACKERS] proposal: schema variables

2017-11-02 Thread Pavel Stehule
2017-11-02 16:07 GMT+01:00 Craig Ringer : > On 26 October 2017 at 15:21, Pavel Stehule > wrote: > > Hi, > > > > I propose a new database object - a variable. > > Didn't we have a pretty long discussion about this already in > > Yeah. > > https://www.postgresql.org/message-id/flat/CAMsr%2BYF0G8_

Re: [HACKERS] proposal: schema variables

2017-11-02 Thread Pavel Stehule
2017-11-02 16:35 GMT+01:00 Nico Williams : > On Thu, Nov 02, 2017 at 06:05:54PM +0530, Robert Haas wrote: > > On Thu, Oct 26, 2017 at 12:51 PM, Pavel Stehule > wrote: > > > The variables can be modified by SQL command SET (this is taken from > > > standard, and it natural) > > > > > > SET varname

Re: [HACKERS] proposal: schema variables

2017-11-02 Thread Nico Williams
On Thu, Nov 02, 2017 at 06:05:54PM +0530, Robert Haas wrote: > On Thu, Oct 26, 2017 at 12:51 PM, Pavel Stehule > wrote: > > The variables can be modified by SQL command SET (this is taken from > > standard, and it natural) > > > > SET varname = expression; > > Overloading SET to handle both vari

Re: [HACKERS] Client Connection redirection support for PostgreSQL

2017-11-02 Thread Craig Ringer
On 2 November 2017 at 14:02, Satyanarayana Narlapuram wrote: > Proposal: > > Add the ability to the PostgreSQL server instance to route the traffic to a > different server instance based on the rules defined in server’s pg_bha.conf > configuration file. At a high level this enables offloading the

Re: [HACKERS] proposal: schema variables

2017-11-02 Thread Craig Ringer
On 26 October 2017 at 15:21, Pavel Stehule wrote: > Hi, > > I propose a new database object - a variable. Didn't we have a pretty long discussion about this already in Yeah. https://www.postgresql.org/message-id/flat/CAMsr%2BYF0G8_FehQyFS8gSfnEer9OPsMOvpfniDJOVGQzJzHzsw%40mail.gmail.com#camsr+

Re: [HACKERS] Custom compression methods

2017-11-02 Thread Craig Ringer
On 2 November 2017 at 17:41, Ildus Kurbangaliev wrote: > In this patch compression methods is suitable for MAIN and EXTENDED > storages like in current implementation in postgres. Just instead only > of LZ4 you can specify any other compression method. We've had this discussion before. Please r

Re: [HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread David Rowley
On 3 November 2017 at 03:27, Stephen Frost wrote: > * David Rowley (david.row...@2ndquadrant.com) wrote: >> We could get around a few of these problems if Lists were implemented >> internally as arrays, however, arrays are pretty bad if we want to >> delete an item out the middle as we'd have to s

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-11-02 Thread Alvaro Herrera
Stephen Frost wrote: > * Tom Lane (t...@sss.pgh.pa.us) wrote: > > Andres Freund writes: > > > Do we care about people upgrading to unreleased versions? We could do > > > nothing, document it in the release notes, or ??? > > > > Do nothing. > > Agreed. Not much we can do there. Pushed the rever

Re: [HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread Tom Lane
David Rowley writes: >> It seems to me that a whole lot of the complaints about this could be >> resolved simply by improving the List infrastructure to allocate ListCells >> in batches. That would address the question of "too much palloc traffic" >> and greatly improve the it-accesses-all-over-m

Re: [HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread David Rowley
On 3 November 2017 at 03:38, Tom Lane wrote: > David Rowley writes: >> On 3 November 2017 at 03:17, Tom Lane wrote: >>> We've jacked up the List API and driven a new implementation underneath >>> once before. Maybe it's time to do that again. > >> Maybe, but the new implementation is not going

Re: [HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread Tom Lane
David Rowley writes: > On 3 November 2017 at 03:17, Tom Lane wrote: >> We've jacked up the List API and driven a new implementation underneath >> once before. Maybe it's time to do that again. > Maybe, but the new implementation is not going to do well with places > where we perform lcons(). Pr

Re: [HACKERS] pgbench - use enum for meta commands

2017-11-02 Thread Aleksandr Parfenov
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed Hi, Looks good to me. The only thing I'm not quite sure abo

Re: [HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread Stephen Frost
David, * David Rowley (david.row...@2ndquadrant.com) wrote: > Our List implementation internally uses linked lists which are > certainly good for some things, but pretty bad at other things. Linked > lists are pretty bad when you want to fetch the Nth element, as it > means looping ever each previ

Re: [HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread Craig Ringer
On 2 November 2017 at 22:22, David Rowley wrote: > On 3 November 2017 at 03:17, Tom Lane wrote: >> We've jacked up the List API and driven a new implementation underneath >> once before. Maybe it's time to do that again. > > Maybe, but the new implementation is not going to do well with places >

Re: [HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread Craig Ringer
On 2 November 2017 at 22:17, Tom Lane wrote: > David Rowley writes: >> Comments on the design are welcome, but I was too late to the >> commitfest, so there are other priorities. However, if you have a >> strong opinion, feel free to voice it. > > I do not like replacing Lists piecemeal; that's a

Re: [HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread David Rowley
On 3 November 2017 at 03:17, Tom Lane wrote: > We've jacked up the List API and driven a new implementation underneath > once before. Maybe it's time to do that again. Maybe, but the new implementation is not going to do well with places where we perform lcons(). Probably many of those places co

Re: [HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread Tom Lane
David Rowley writes: > Comments on the design are welcome, but I was too late to the > commitfest, so there are other priorities. However, if you have a > strong opinion, feel free to voice it. I do not like replacing Lists piecemeal; that's a recipe for ongoing API breakage and back-patching pai

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-11-02 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Andres Freund writes: > > Do we care about people upgrading to unreleased versions? We could do > > nothing, document it in the release notes, or ??? > > Do nothing. Agreed. Not much we can do there. Thanks! Stephen signature.asc Description: Digital

[HACKERS] ArrayLists instead of List (for some things)

2017-11-02 Thread David Rowley
Hackers, Our List implementation internally uses linked lists which are certainly good for some things, but pretty bad at other things. Linked lists are pretty bad when you want to fetch the Nth element, as it means looping ever each previous element until you get to the Nth. They're also pretty b

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-11-02 Thread Tom Lane
Andres Freund writes: > Do we care about people upgrading to unreleased versions? We could do > nothing, document it in the release notes, or ??? Do nothing. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Client Connection redirection support for PostgreSQL

2017-11-02 Thread David Fetter
On Thu, Nov 02, 2017 at 06:02:43AM +, Satyanarayana Narlapuram wrote: > Proposal: > Add the ability to the PostgreSQL server instance to route the > traffic to a different server instance based on the rules defined in > server's pg_bha.conf configuration file. At a high level this > enables off

Re: [HACKERS] [PATCH] Add two-arg for of current_setting(NAME, FALLBACK)

2017-11-02 Thread Peter Eisentraut
On 11/1/17 14:02, Nico Williams wrote: > There already _is_ a two-argument form of current_setting() that yours > somewhat conflicts with: > >current_setting(setting_name [, missing_ok ]) > > https://www.postgresql.org/docs/current/static/functions-admin.html > > I often use > > coalesce(

Re: [HACKERS] list of credits for release notes

2017-11-02 Thread Peter Eisentraut
On 10/3/17 12:31, Dang Minh Huong wrote: > Sorry but please change "Huong Dangminh"(my name) to "Dang Minh Huong". done -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pg

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-11-02 Thread Andres Freund
Hi, On 2017-11-02 13:49:47 +0100, Alvaro Herrera wrote: > Andres Freund wrote: > > I think the problem is on the pruning, rather than the freezing side. We > > can't freeze a tuple if it has an alive predecessor - rather than > > weakining this, we should be fixing the pruning to not have the aliv

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-11-02 Thread Alvaro Herrera
Andres Freund wrote: > I spent some time discussing this with Robert today (with both of us > alternating between feeling the other and ourselves as stupid), and the > conclusion I think is that the problem is on the pruning, rather than > the freezing side. Thanks both for spending some more tim

Re: [HACKERS] proposal: schema variables

2017-11-02 Thread Robert Haas
On Thu, Oct 26, 2017 at 12:51 PM, Pavel Stehule wrote: > The variables can be modified by SQL command SET (this is taken from > standard, and it natural) > > SET varname = expression; Overloading SET to handle both variables and GUCs seems likely to create problems, possibly including security pr

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-11-02 Thread Robert Haas
On Thu, Nov 2, 2017 at 4:50 PM, Andres Freund wrote: > I think a5736bf754c82d8b86674e199e232096c679201d might be dangerous in > the face of previously corrupted tuple chains and pg_upgraded clusters - > it can lead to tuples being considered related, even though they they're > from entirely indepe

  1   2   >