Re: [HACKERS] PATCH: decreasing memory needlessly consumed by array_agg

2014-12-21 Thread Tomas Vondra
On 21.12.2014 02:54, Alvaro Herrera wrote: Tomas Vondra wrote: Attached is v5 of the patch, fixing an error with releasing a shared memory context (invalid flag values in a few calls). The functions that gain a new argument should get their comment updated, to explain what the new argument

Re: [HACKERS] pgbench -f and vacuum

2014-12-21 Thread Tatsuo Ishii
On Sun, Dec 14, 2014 at 11:43 AM, Tatsuo Ishii is...@postgresql.org wrote: If we care enough about that case to attempt the vacuum anyway then we need to do something about the error message; either squelch it or check for the existence of the tables before attempting to vacuum. Since there's

Re: [HACKERS] Add min and max execute statement time in pg_stat_statement

2014-12-21 Thread Andrew Dunstan
On 04/07/2014 04:19 PM, Tom Lane wrote: Alvaro Herrera alvhe...@2ndquadrant.com writes: I just noticed that this patch not only adds min,max,stddev, but it also adds the ability to reset an entry's counters. This hasn't been mentioned in this thread at all; there has been no discussion on

Re: [HACKERS] pgbench -f and vacuum

2014-12-21 Thread Fabrízio de Royes Mello
On Sun, Dec 21, 2014 at 12:58 PM, Tatsuo Ishii is...@postgresql.org wrote: On Sun, Dec 14, 2014 at 11:43 AM, Tatsuo Ishii is...@postgresql.org wrote: If we care enough about that case to attempt the vacuum anyway then we need to do something about the error message; either squelch it or

[HACKERS] Proposal VACUUM SCHEMA

2014-12-21 Thread Fabrízio de Royes Mello
Hi all, I work with some customer that have databases with a lot of schemas and sometimes we need to run manual VACUUM in one schema, and would be nice to have a new option to run vacuum in relations from a specific schema. The new syntax could be: VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] { [

Re: [HACKERS] TABLESAMPLE patch

2014-12-21 Thread Tomas Vondra
Hi, On 18.12.2014 13:14, Petr Jelinek wrote: Hi, v2 version of this patch is attached. I did a review of this v2 patch today. I plan to do a bit more testing, but these are my comments/questions so far: (0) There's a TABLESAMPLE page at the wiki, not updated since 2012:

Re: [HACKERS] PATCH: decreasing memory needlessly consumed by array_agg

2014-12-21 Thread Tom Lane
Tomas Vondra t...@fuzzy.cz writes: i.e. either destroy the whole context if possible, and just free the memory when using a shared memory context. But I'm afraid this would penalize the shared memory context, because that's intended for cases where all the build states coexist in parallel and

Re: [HACKERS] PrivateRefCount patch has got issues

2014-12-21 Thread Andres Freund
Hi, On 2014-12-16 18:25:13 -0500, Tom Lane wrote: I just happened to look into bufmgr.c for the first time in awhile, and noticed the privaterefcount-is-no-longer-a-simple-array stuff. It doesn't look too well thought out to me. In particular, PinBuffer_Locked calls GetPrivateRefCountEntry

Re: [HACKERS] PrivateRefCount patch has got issues

2014-12-21 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-12-16 18:25:13 -0500, Tom Lane wrote: I just happened to look into bufmgr.c for the first time in awhile, and noticed the privaterefcount-is-no-longer-a-simple-array stuff. It doesn't look too well thought out to me. In particular,

Re: [HACKERS] Add min and max execute statement time in pg_stat_statement

2014-12-21 Thread Alvaro Herrera
Andrew Dunstan wrote: On my blog Peter Geoghegan mentioned something about atomic fetch-and-add being useful here, but I'm not quite sure what that's referring to. Perhaps someone can give me a pointer. The point, I think, is that without atomic instructions you have to hold a lock while

Re: [HACKERS] Add min and max execute statement time in pg_stat_statement

2014-12-21 Thread Andres Freund
On December 21, 2014 7:23:27 PM CET, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Andrew Dunstan wrote: On my blog Peter Geoghegan mentioned something about atomic fetch-and-add being useful here, but I'm not quite sure what that's referring to. Perhaps someone can give me a pointer. The

Re: [HACKERS] Add min and max execute statement time in pg_stat_statement

2014-12-21 Thread Andrew Dunstan
On 12/21/2014 01:23 PM, Alvaro Herrera wrote: Andrew Dunstan wrote: On my blog Peter Geoghegan mentioned something about atomic fetch-and-add being useful here, but I'm not quite sure what that's referring to. Perhaps someone can give me a pointer. The point, I think, is that without atomic

Re: [HACKERS] Add min and max execute statement time in pg_stat_statement

2014-12-21 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: On 12/21/2014 01:23 PM, Alvaro Herrera wrote: The point, I think, is that without atomic instructions you have to hold a lock while incrementing the counters. Hmm, do we do that now? We already have a spinlock mutex around the counter adjustment

Re: [HACKERS] Proposal VACUUM SCHEMA

2014-12-21 Thread Tom Lane
=?UTF-8?Q?Fabr=C3=ADzio_de_Royes_Mello?= fabriziome...@gmail.com writes: I work with some customer that have databases with a lot of schemas and sometimes we need to run manual VACUUM in one schema, and would be nice to have a new option to run vacuum in relations from a specific schema. I'm

Re: [HACKERS] 9.5: Better memory accounting, towards memory-bounded HashAgg

2014-12-21 Thread Tomas Vondra
On 2.12.2014 06:14, Jeff Davis wrote: On Sun, 2014-11-30 at 17:49 -0800, Peter Geoghegan wrote: On Mon, Nov 17, 2014 at 11:39 PM, Jeff Davis pg...@j-davis.com wrote: I can also just move isReset there, and keep mem_allocated as a uint64. That way, if I find later that I want to track the

Re: [HACKERS] controlling psql's use of the pager a bit more

2014-12-21 Thread Andrew Dunstan
On 11/15/2014 05:56 PM, Andrew Dunstan wrote: On 11/13/2014 11:41 AM, Andrew Dunstan wrote: On 11/13/2014 11:09 AM, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: I often get annoyed because psql is a bit too aggressive when it decides whether to put output through the pager,

Re: [HACKERS] 9.5: Better memory accounting, towards memory-bounded HashAgg

2014-12-21 Thread Tomas Vondra
On 21.12.2014 20:19, Tomas Vondra wrote: However, I got a segfault on the very first query I tried :-( create table test_hash_agg as select i AS a, i AS b, i AS c, i AS d from generate_series(1,1000) s(i); analyze test_hash_agg; select a, count(*) from test_hash_agg where

Re: [HACKERS] Add min and max execute statement time in pg_stat_statement

2014-12-21 Thread Andrew Dunstan
On 12/21/2014 02:12 PM, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: On 12/21/2014 01:23 PM, Alvaro Herrera wrote: The point, I think, is that without atomic instructions you have to hold a lock while incrementing the counters. Hmm, do we do that now? We already have a

Re: [HACKERS] Final Patch for GROUPING SETS

2014-12-21 Thread Noah Misch
On Sat, Dec 13, 2014 at 04:37:48AM +, Andrew Gierth wrote: Tom == Tom Lane t...@sss.pgh.pa.us writes: I'd already explained in more detail way back when we posted the patch. But to reiterate: the ChainAggregate nodes pass through their input data unchanged, but on group boundaries

Re: [HACKERS] Proposal VACUUM SCHEMA

2014-12-21 Thread Fabrízio de Royes Mello
On Sun, Dec 21, 2014 at 5:18 PM, Tom Lane t...@sss.pgh.pa.us wrote: =?UTF-8?Q?Fabr=C3=ADzio_de_Royes_Mello?= fabriziome...@gmail.com writes: I work with some customer that have databases with a lot of schemas and sometimes we need to run manual VACUUM in one schema, and would be nice to

Re: [HACKERS] pgbench -f and vacuum

2014-12-21 Thread Tatsuo Ishii
- Error to apply to the current master: Works for me. $ git apply ~/pgbench-f-noexit-v2.patch $ Maybe git version difference or the patch file was malformed by mail client? +static void executeStatement2(PGconn *con, const char *sql, const char *table); I think we can use a better name

Re: [HACKERS] pgbench -f and vacuum

2014-12-21 Thread Tomas Vondra
Hi, On 21.12.2014 15:58, Tatsuo Ishii wrote: On Sun, Dec 14, 2014 at 11:43 AM, Tatsuo Ishii is...@postgresql.org wrote: If we care enough about that case to attempt the vacuum anyway then we need to do something about the error message; either squelch it or check for the existence of the

Re: [HACKERS] TABLESAMPLE patch

2014-12-21 Thread Michael Paquier
On Mon, Dec 22, 2014 at 2:38 AM, Tomas Vondra t...@fuzzy.cz wrote: (1) The patch adds a new catalog, but does not bump CATVERSION. FWIW, this part is managed by the committer when this patch is picked up. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] parallel mode and parallel contexts

2014-12-21 Thread Michael Paquier
On Thu, Dec 18, 2014 at 4:53 AM, Robert Haas robertmh...@gmail.com wrote: On Wed, Dec 17, 2014 at 9:16 AM, Simon Riggs si...@2ndquadrant.com wrote: On 12 December 2014 at 22:52, Robert Haas robertmh...@gmail.com wrote: I would be remiss if I failed to mention that this patch includes work by

Re: [HACKERS] PATCH: decreasing memory needlessly consumed by array_agg

2014-12-21 Thread Ali Akbar
Another positive benefit is that this won't break the code unless it uses the new API. This is a problem especially with external code (e.g. extensions), but the new API (initArray*) is not part of 9.4 so there's no such code. So that's nice. The one annoying thing is that this makes the

Re: [HACKERS] btree_gin and ranges

2014-12-21 Thread Michael Paquier
On Thu, Dec 18, 2014 at 4:13 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 10/22/2014 01:55 PM, Teodor Sigaev wrote: Suggested patch adds GIN support contains operator for ranges over scalar column. It allows more effective GIN scan. Currently, queries like SELECT * FROM

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

2014-12-21 Thread Michael Paquier
On Tue, Dec 16, 2014 at 7:04 PM, David Rowley dgrowle...@gmail.com wrote: It also looks like your OIDs have been nabbed by some jsonb stuff. DETAIL: Key (oid)=(3267) is duplicated. Use src/include/catalog/unused_oids to track the OIDs not yet used in the catalogs when adding new objects for a

Re: [HACKERS] Parallel Seq Scan

2014-12-21 Thread Jim Nasby
On 12/21/14, 12:42 AM, Amit Kapila wrote: On Fri, Dec 19, 2014 at 6:21 PM, Stephen Frost sfr...@snowman.net mailto:sfr...@snowman.net wrote: a. Instead of passing value array, just pass tuple id, but retain the buffer pin till master backend reads the tuple based on tupleid. This has side

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-12-21 Thread Michael Paquier
On Sun, Dec 21, 2014 at 6:56 AM, Peter Geoghegan p...@heroku.com wrote: On Thu, Dec 18, 2014 at 9:31 AM, Peter Geoghegan p...@heroku.com wrote: So I think there needs to be some kind of logic to de-recognize the table alias foo. Once I rewrote the query to use TARGET and EXCLUDED correctly,

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-12-21 Thread Peter Geoghegan
On Sun, Dec 21, 2014 at 6:10 PM, Michael Paquier michael.paqu...@gmail.com wrote: Looking at this thread, the last version of this patch is available here: http://www.postgresql.org/message-id/CAM3SWZRvkCKc=1Y6_Wn8mk97_Vi8+j-aX-RY-=msrjvu-ec...@mail.gmail.com And they do not apply correctly, so

Re: [HACKERS] Proposal VACUUM SCHEMA

2014-12-21 Thread Jim Nasby
On 12/21/14, 3:30 PM, Fabrízio de Royes Mello wrote: On Sun, Dec 21, 2014 at 5:18 PM, Tom Lane t...@sss.pgh.pa.us mailto:t...@sss.pgh.pa.us wrote: =?UTF-8?Q?Fabr=C3=ADzio_de_Royes_Mello?= fabriziome...@gmail.com mailto:fabriziome...@gmail.com writes: I work with some customer that have

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-12-21 Thread Michael Paquier
On Mon, Dec 22, 2014 at 11:20 AM, Peter Geoghegan p...@heroku.com wrote: On Sun, Dec 21, 2014 at 6:10 PM, Michael Paquier michael.paqu...@gmail.com wrote: Looking at this thread, the last version of this patch is available here:

Re: [HACKERS] PATCH: decreasing memory needlessly consumed by array_agg

2014-12-21 Thread Jim Nasby
On 12/21/14, 7:08 PM, Ali Akbar wrote: Another positive benefit is that this won't break the code unless it uses the new API. This is a problem especially with external code (e.g. extensions), but the new API (initArray*) is not part of 9.4 so there's no such code. So that's

Re: [HACKERS] Proposal VACUUM SCHEMA

2014-12-21 Thread Fabrízio de Royes Mello
Em segunda-feira, 22 de dezembro de 2014, Jim Nasby jim.na...@bluetreble.com escreveu: On 12/21/14, 3:30 PM, Fabrízio de Royes Mello wrote: On Sun, Dec 21, 2014 at 5:18 PM, Tom Lane t...@sss.pgh.pa.us mailto: t...@sss.pgh.pa.us wrote: =?UTF-8?Q?Fabr=C3=ADzio_de_Royes_Mello?=

Re: [HACKERS] Re[3]: [HACKERS] Patch: add recovery_timeout option to control timeout of restore_command nonzero status code

2014-12-21 Thread Michael Paquier
On Tue, Nov 4, 2014 at 6:25 AM, Alexey Vasiliev leopard...@inbox.ru wrote: Added new patch. Seems useful to me to be able to tune this interval of time. I would simply reword the documentation as follows: If varnamerestore_command/ returns nonzero exit status code, retry command after the

Re: [HACKERS] [PATCH] Add transforms feature

2014-12-21 Thread Michael Paquier
On Mon, Dec 15, 2014 at 3:10 PM, Peter Eisentraut pete...@gmx.net wrote: fixed This patch needs a rebase, it does not apply correctly in a couple of places on latest HEAD (699300a): ./src/include/catalog/catversion.h.rej ./src/include/catalog/pg_proc.h.rej ./src/pl/plpython/plpy_procedure.c.rej

Re: [HACKERS] Fractions in GUC variables

2014-12-21 Thread Michael Paquier
On Tue, Dec 16, 2014 at 12:19 AM, Peter Eisentraut pete...@gmx.net wrote: On 12/15/14 8:56 AM, Heikki Linnakangas wrote: Overall, I feel that this isn't really worth the trouble. We use fractions consistently now, so there isn't much room for confusion over what the current values mean. Using

Re: [HACKERS] no test programs in contrib

2014-12-21 Thread Michael Paquier
On Sat, Nov 29, 2014 at 5:54 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Peter Eisentraut wrote: On 11/27/14 3:10 PM, Tom Lane wrote: I'm not too happy with that approach, because packagers are going to tend to think they should package any files installed by install-world. The

Re: [HACKERS] Parallel Seq Scan

2014-12-21 Thread Amit Kapila
On Mon, Dec 22, 2014 at 7:34 AM, Jim Nasby jim.na...@bluetreble.com wrote: On 12/21/14, 12:42 AM, Amit Kapila wrote: On Fri, Dec 19, 2014 at 6:21 PM, Stephen Frost sfr...@snowman.net mailto:sfr...@snowman.net wrote: a. Instead of passing value array, just pass tuple id, but retain the buffer

Re: Suppressing elog.c context messages (was Re: [HACKERS] Wait free LW_SHARED acquisition)

2014-12-21 Thread Amit Kapila
On Fri, Dec 19, 2014 at 9:36 PM, Andres Freund and...@2ndquadrant.com wrote: Hi, When debugging lwlock issues I found PRINT_LWDEBUG/LOG_LWDEBUG rather painful to use because of the amount of elog contexts/statements emitted. Given the number of lwlock acquirations that's just not doable.

Re: [HACKERS] install libpq.dll in bin directory on Windows / Cygwin

2014-12-21 Thread Michael Paquier
On Wed, Feb 26, 2014 at 6:11 AM, Peter Eisentraut pete...@gmx.net wrote: On 2/1/14, 3:22 PM, Andrew Dunstan wrote: In the end I went with the way I had suggested, because that's what the MSVC system does - it doesn't copy any other DLLs to the bin directory. So doing that seemed sane for

Re: [HACKERS] documentation update for doc/src/sgml/func.sgml

2014-12-21 Thread Michael Paquier
On Sat, Oct 18, 2014 at 4:12 AM, Andreas 'ads' Scherbaum adsm...@wars-nicht.de wrote: On 09/14/2014 06:32 PM, Peter Eisentraut wrote: On 9/12/14 3:13 PM, Andreas 'ads' Scherbaum wrote: Of course a general rule how to link to WP would be nice ... I think Wikipedia links should be avoided

Re: [HACKERS] pgbench -f and vacuum

2014-12-21 Thread Tatsuo Ishii
Hi, On 21.12.2014 15:58, Tatsuo Ishii wrote: On Sun, Dec 14, 2014 at 11:43 AM, Tatsuo Ishii is...@postgresql.org wrote: If we care enough about that case to attempt the vacuum anyway then we need to do something about the error message; either squelch it or check for the existence of the

[HACKERS] ExplainModifyTarget doesn't work as expected

2014-12-21 Thread Etsuro Fujita
Hi, I think ExplainModifyTarget should show the parent of the inheritance tree in multi-target-table cases, as described there, but noticed that it doesn't always work like that. Here is an example. postgres=# create table parent (a int check (a 0) no inherit); CREATE TABLE postgres=# create

[HACKERS] Commit Fest 2014-12, Status after week 1

2014-12-21 Thread Michael Paquier
Hi all, I have been though the patches of the current CF, looking at their related threads and updating each patch status if needed. After one week in this CF, we have done progress on many patches, more than 2/3 of them getting some comments, reviews and/or refreshed versions. Looking at the CF