Re: [HACKERS][PROPOSAL] Covering + unique indexes.

2015-09-14 Thread Teodor Sigaev
Why not normal syntax with optional INCLUDE ? CREATE UNIQUE INDEX ON table (f1,f2,f3) INCLUDE (f4) That's not the same thing. Then you're including f3 in the unique constraint, which you may not want for uniqueness purposes, but may want in the index for sorting. But then saying that

Re: [HACKERS][PROPOSAL] Covering + unique indexes.

2015-09-14 Thread Teodor Sigaev
CREATE INDEX ... ON table (f1, f2, f3) UNIQUE(f1, f2) INCLUDE(f4); I don't see an advantage this form. What is f3 column? just order? and f4 will not be used for compare? At least now it requires additional checks that UNIQUE() fields are the same as first columns in definition. Non ordering

Re: [HACKERS] Review: GiST support for UUIDs

2015-09-14 Thread Teodor Sigaev
Paul Jungwirth wrote: Or something like this in pseudocode: numeric = int8_numeric(*(uint64 *)(&i->data[0])) * int8_numeric(MAX_INT64) + int8_numeric(*(uint64 *)(&i->data[8])) This is more like what I was hoping for, rather than converting to a string and back. Would you mind confirming for

Re: [HACKERS] Attach comments to functions' parameters and return value

2015-09-14 Thread Charles Clavadetscher
Hello Àlvaro On 14/09/2015 20:02, Alvaro Herrera wrote: Jim Nasby wrote: On 9/14/15 8:59 AM, Charles Clavadetscher wrote: To long time PostgreSQL developers this may look straightforward. For the moment I am not even sure if that is correct and if there are other places that would need addit

Re: [HACKERS] Attach comments to functions' parameters and return value

2015-09-14 Thread Charles Clavadetscher
Hello Jim On 14/09/2015 19:23, Jim Nasby wrote: On 9/14/15 8:59 AM, Charles Clavadetscher wrote: Hello In PostgreSQL it is possible to attach comments to almost everything. This made it possible for us to integrate the wiki that we use for our technical documentation directly with the database

Re: [HACKERS] row_security GUC, BYPASSRLS

2015-09-14 Thread Tom Lane
Noah Misch writes: > On Mon, Sep 14, 2015 at 07:30:33AM -0400, Robert Haas wrote: >> ...but I'm not sure I like this, either. Without row_security=force, >> it's hard for a table owner to test their policy, unless they have the >> ability to assume some other user ID, which some won't. If someon

Re: [HACKERS] Mention column name in error messages

2015-09-14 Thread Alvaro Herrera
Franck Verrot wrote: > On Wed, Aug 19, 2015 at 11:31 PM, Jeff Janes wrote: > > > > I took this for a test drive, and had some comments.on the user visible > > parts. > > [...] > > But I think these belong as CONTEXT or as DETAIL, not as HINT. The > > messages are giving me details about where (wh

Re: [HACKERS] Mention column name in error messages

2015-09-14 Thread Franck Verrot
On Wed, Aug 19, 2015 at 11:31 PM, Jeff Janes wrote: > > I took this for a test drive, and had some comments.on the user visible > parts. > [...] > But I think these belong as CONTEXT or as DETAIL, not as HINT. The > messages are giving me details about where (which column) the error > occurred,

Re: [HACKERS] row_security GUC, BYPASSRLS

2015-09-14 Thread Noah Misch
On Mon, Sep 14, 2015 at 07:30:33AM -0400, Robert Haas wrote: > On Mon, Sep 14, 2015 at 3:29 AM, Noah Misch wrote: > > Pondering it afresh this week, I see now that row_security=force itself is > > the > > problem. It's a new instance of the maligned "behavior-changing GUC". > > Function authors

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-09-14 Thread Thomas Munro
On Fri, Sep 11, 2015 at 6:41 AM, Beena Emerson wrote: > Hello, > > Please find attached the WIP patch for the proposed feature. It is built > based on the already discussed design. > > Changes made: > - add new parameter "sync_file" to provide the location of the pg_syncinfo > file. The default i

Re: [HACKERS] WIP: Make timestamptz_out less slow.

2015-09-14 Thread Alvaro Herrera
David Rowley wrote: > It's not like nothing is improved in float timestamps with this patch, it's > only appending the non-zero fractional seconds that I've left alone. Every > other portion of the timestamp has been improved. OK, sounds good enough to me. > I did, however spend some time a few

Re: [HACKERS] Parallel Seq Scan

2015-09-14 Thread Haribabu Kommi
On Thu, Sep 10, 2015 at 2:12 PM, Amit Kapila wrote: > On Thu, Sep 10, 2015 at 4:16 AM, Robert Haas wrote: >> >> On Wed, Sep 9, 2015 at 11:07 AM, Amit Kapila >> wrote: >> > On Wed, Sep 9, 2015 at 11:47 AM, Haribabu Kommi >> > >> > wrote: >> >> With subquery, parallel scan is having some problem,

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2015-09-14 Thread Andres Freund
On 2015-09-14 17:41:42 +0200, Andres Freund wrote: > I pointed out how you can actually make this safely lock-free giving you > the interesting code. And here's an actual implementation of that approach. It's definitely work-in-progress and could easily be optimized further. Don't have any big mac

Re: [HACKERS][PROPOSAL] Covering + unique indexes.

2015-09-14 Thread Gavin Flower
On 15/09/15 09:44, Jim Nasby wrote: On 9/14/15 1:50 PM, Thomas Munro wrote: CREATE [UNIQUE {ON FIRST {COLUMN | n_unique_column COLUMNS}} INDEX ON table_name (column_name1, column_name2 ...); I would use the first (simple) syntax and just throw an err

Re: [HACKERS] Multi-tenancy with RLS

2015-09-14 Thread Haribabu Kommi
On Fri, Sep 11, 2015 at 7:50 AM, Joe Conway wrote: > On 09/01/2015 11:25 PM, Haribabu Kommi wrote: >> If any user is granted any permissions on that object then that user >> can view it's meta data of that object from the catalog tables. >> To check the permissions of the user on the object, inste

Re: [HACKERS] Review: GiST support for UUIDs

2015-09-14 Thread Paul Jungwirth
Or something like this in pseudocode: numeric = int8_numeric(*(uint64 *)(&i->data[0])) * int8_numeric(MAX_INT64) + int8_numeric(*(uint64 *)(&i->data[8])) This is more like what I was hoping for, rather than converting to a string and back. Would you mind confirming for me: int8_numeric turns a

Re: [HACKERS] WIP: Make timestamptz_out less slow.

2015-09-14 Thread David Rowley
On 15 September 2015 at 05:52, Alvaro Herrera wrote: > Jim Nasby wrote: > > On 9/13/15 2:43 AM, David Rowley wrote: > > >Are you worried about this because I've not focused on optimising float > > >timestamps as much as int64 timestamps? Are there many people compiling > > >with float timestamps

Re: [HACKERS] Can extension build own SGML document?

2015-09-14 Thread Michael Paquier
On Tue, Sep 15, 2015 at 6:01 AM, Alvaro Herrera wrote: > I think the only way upstream Postgres could offer this is as a way to > build a separate "book", i.e. not a chapter/section within the main > book. I think it would require huge complications in doc/src/sgml's > Makefile. Not sure it's wor

Re: [HACKERS] [PATCH] add --log-output to pg_ctl on Windows

2015-09-14 Thread Michael Paquier
On Mon, Sep 14, 2015 at 7:42 PM, Egon Kocjan wrote: > Hello > > The patch implements a command line option for pg_ctl on Windows to redirect > logging of errors (write_stderr). The possible outputs are: default > (eventlog if running as a service, otherwise stderr), stderr or eventlog. > > The pre

Re: [HACKERS][PROPOSAL] Covering + unique indexes.

2015-09-14 Thread Thom Brown
On 14 September 2015 at 23:12, Oleg Bartunov wrote: > > > > On Tue, Sep 15, 2015 at 12:44 AM, Jim Nasby > wrote: > >> On 9/14/15 1:50 PM, Thomas Munro wrote: >> >>> CREATE [UNIQUE {ON FIRST {COLUMN | n_unique_column COLUMNS}} >>> INDEX ON >>> table_name (colum

Re: [HACKERS][PROPOSAL] Covering + unique indexes.

2015-09-14 Thread Thom Brown
On 14 September 2015 at 22:44, Jim Nasby wrote: > On 9/14/15 1:50 PM, Thomas Munro wrote: > >> CREATE [UNIQUE {ON FIRST {COLUMN | n_unique_column COLUMNS}} >> INDEX ON >> table_name (column_name1, column_name2 ...); >> >> >> I would use the first (simpl

Re: [HACKERS][PROPOSAL] Covering + unique indexes.

2015-09-14 Thread Oleg Bartunov
On Tue, Sep 15, 2015 at 12:44 AM, Jim Nasby wrote: > On 9/14/15 1:50 PM, Thomas Munro wrote: > >> CREATE [UNIQUE {ON FIRST {COLUMN | n_unique_column COLUMNS}} >> INDEX ON >> table_name (column_name1, column_name2 ...); >> >> >> I would use the first (si

Re: [HACKERS][PROPOSAL] Covering + unique indexes.

2015-09-14 Thread Jim Nasby
On 9/14/15 1:50 PM, Thomas Munro wrote: CREATE [UNIQUE {ON FIRST {COLUMN | n_unique_column COLUMNS}} INDEX ON table_name (column_name1, column_name2 ...); I would use the first (simple) syntax and just throw an error if the user tries to skip

Re: [HACKERS] Can extension build own SGML document?

2015-09-14 Thread Alvaro Herrera
Peter Eisentraut wrote: > On 9/14/15 5:35 AM, Kouhei Kaigai wrote: > > Hello, > > > > The pgxs makefile (pgxs.ml) says: > > > > # DOCS -- random files to install under $PREFIX/doc/$MODULEDIR > > > > It is a bunch of files to be copied to document directory, however, > > not looks like a variab

Re: [HACKERS] Can extension build own SGML document?

2015-09-14 Thread Peter Eisentraut
On 9/14/15 5:35 AM, Kouhei Kaigai wrote: > Hello, > > The pgxs makefile (pgxs.ml) says: > > # DOCS -- random files to install under $PREFIX/doc/$MODULEDIR > > It is a bunch of files to be copied to document directory, however, > not looks like a variable to specify SGML source as PostgreSQL co

Re: [HACKERS] exposing pg_controldata and pg_config as functions

2015-09-14 Thread Peter Eisentraut
On 9/8/15 4:56 PM, Andrew Dunstan wrote: > The problem is that at least this user's system had something odd about > it. so that I wouldn't entirely trust the output of > >select is_supported >from information_schema.sql_features >where feature_name = 'XML type'; > > to reflect the co

Re: [HACKERS] Review: GiST support for UUIDs

2015-09-14 Thread Teodor Sigaev
Paul Jungwirth wrote: 2) static double uuid2num(const pg_uuid_t *i) { return *((uint64 *)i); } It isn't looked as correct transformation for me. May be, it's better to transform to numeric type (UUID looks like a 16-digit hexademical number) and follow

Re: [HACKERS] Re: [COMMITTERS] pgsql: Check existency of table/schema for -t/-n option (pg_dump/pg_res

2015-09-14 Thread Robert Haas
On Mon, Sep 14, 2015 at 10:54 AM, Teodor Sigaev wrote: >> /* >> -* We use UNION ALL rather than UNION; this might sometimes result in >> -* duplicate entries in the OID list, but we don't care. >> +* this might sometimes result in duplicate entries in the OID list, >> +* but w

Re: [HACKERS] cache type info in json_agg and friends

2015-09-14 Thread Andrew Dunstan
On 09/14/2015 03:42 PM, Teodor Sigaev wrote: Currently, json_agg, jsonb_agg, json_object_agg and jsonb_object_agg do type classification on their arguments on each call to the transition function. This is quite unnecessary, as the argument types won't change. This patch remedies the defect by c

Re: [HACKERS] Review: GiST support for UUIDs

2015-09-14 Thread Paul Jungwirth
2) static double uuid2num(const pg_uuid_t *i) { return *((uint64 *)i); } It isn't looked as correct transformation for me. May be, it's better to transform to numeric type (UUID looks like a 16-digit hexademical number) and follow gbt_numeric_penalty() log

Re: [HACKERS] cache type info in json_agg and friends

2015-09-14 Thread Teodor Sigaev
Currently, json_agg, jsonb_agg, json_object_agg and jsonb_object_agg do type classification on their arguments on each call to the transition function. This is quite unnecessary, as the argument types won't change. This patch remedies the defect by caching the necessary values in the aggregate sta

Re: [HACKERS] cache type info in json_agg and friends

2015-09-14 Thread Alvaro Herrera
Andrew Dunstan wrote: > Currently, json_agg, jsonb_agg, json_object_agg and jsonb_object_agg do type > classification on their arguments on each call to the transition function. > This is quite unnecessary, as the argument types won't change. This patch > remedies the defect by caching the necessa

[HACKERS] cache type info in json_agg and friends

2015-09-14 Thread Andrew Dunstan
Currently, json_agg, jsonb_agg, json_object_agg and jsonb_object_agg do type classification on their arguments on each call to the transition function. This is quite unnecessary, as the argument types won't change. This patch remedies the defect by caching the necessary values in the aggregat

Re: [HACKERS] jsonb_set array append hack?

2015-09-14 Thread Andrew Dunstan
On 09/14/2015 01:29 PM, Thom Brown wrote: Hi, I've noticed that if you use a string for an element key in jsonb_set with create_missing set to true, you can use it to append to an array: postgres=# SELECT jsonb_set( '{"name": "Joe", "vehicle_types": ["car","van"]}'::jsonb, '{vehi

Re: [HACKERS][PROPOSAL] Covering + unique indexes.

2015-09-14 Thread Teodor Sigaev
It surprised me that you can INCLUDE extra columns on non-UNIQUE indexes, since you could just add them as regular indexed columns for the same effect. It looks like when you do that in SQL Server, the extra columns are only stored on btree leaf pages and so can't be used for searching or orderin

Re: [HACKERS][PROPOSAL] Covering + unique indexes.

2015-09-14 Thread Thomas Munro
On Tue, Sep 15, 2015 at 6:08 AM, Teodor Sigaev wrote: > CREATE INDEX index ON table (c1, c2, c3) UNIQUE ON (c1, c3); >>> >>> CREATE [UNIQUE {ON FIRST {COLUMN | n_unique_column COLUMNS}} INDEX ON >>> table_name (column_name1, column_name2 ...); >>> >> >> I would use the first (simple) syntax and j

Re: [HACKERS][PROPOSAL] Covering + unique indexes.

2015-09-14 Thread Teodor Sigaev
CREATE INDEX index ON table (c1, c2, c3) UNIQUE ON (c1, c3); CREATE [UNIQUE {ON FIRST {COLUMN | n_unique_column COLUMNS}} INDEX ON table_name (column_name1, column_name2 ...); I would use the first (simple) syntax and just throw an error if the user tries to skip a column on the UNIQUE clause.

Re: [HACKERS] Attach comments to functions' parameters and return value

2015-09-14 Thread Alvaro Herrera
Jim Nasby wrote: > On 9/14/15 8:59 AM, Charles Clavadetscher wrote: > >To long time PostgreSQL developers this may look straightforward. For the > >moment I am not even sure if that is correct and if there are other places > >that would need additions, apart from the obvious display in psql. > >

Re: [HACKERS] WIP: Make timestamptz_out less slow.

2015-09-14 Thread Alvaro Herrera
Jim Nasby wrote: > On 9/13/15 2:43 AM, David Rowley wrote: > >Are you worried about this because I've not focused on optimising float > >timestamps as much as int64 timestamps? Are there many people compiling > >with float timestamps in the real world? > > My $0.02: the default was changed some 5

[HACKERS] jsonb_set array append hack?

2015-09-14 Thread Thom Brown
Hi, I've noticed that if you use a string for an element key in jsonb_set with create_missing set to true, you can use it to append to an array: postgres=# SELECT jsonb_set( '{"name": "Joe", "vehicle_types": ["car","van"]}'::jsonb, '{vehicle_types,nonsense}', '"motorcycle"',

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-14 Thread Pavel Stehule
2015-09-14 18:46 GMT+02:00 Shulgin, Oleksandr : > On Mon, Sep 14, 2015 at 3:09 PM, Shulgin, Oleksandr < > oleksandr.shul...@zalando.de> wrote: > >> On Mon, Sep 14, 2015 at 2:11 PM, Tomas Vondra < >> tomas.von...@2ndquadrant.com> wrote: >> >>> Now the backend that has been signaled on the sec

Re: [HACKERS] Attach comments to functions' parameters and return value

2015-09-14 Thread Jim Nasby
On 9/14/15 8:59 AM, Charles Clavadetscher wrote: Hello In PostgreSQL it is possible to attach comments to almost everything. This made it possible for us to integrate the wiki that we use for our technical documentation directly with the database using the MediaWiki [1] extensions ExternalData [

Re: [HACKERS] WIP: Make timestamptz_out less slow.

2015-09-14 Thread Andres Freund
On 2015-09-14 12:03:31 -0500, Jim Nasby wrote: > On 9/13/15 2:43 AM, David Rowley wrote: > >Are you worried about this because I've not focused on optimising float > >timestamps as much as int64 timestamps? Are there many people compiling > >with float timestamps in the real world? > > My $0.02:

Re: [HACKERS] WIP: Make timestamptz_out less slow.

2015-09-14 Thread Jim Nasby
On 9/13/15 2:43 AM, David Rowley wrote: Are you worried about this because I've not focused on optimising float timestamps as much as int64 timestamps? Are there many people compiling with float timestamps in the real world? My $0.02: the default was changed some 5 years ago so FP time is pro

Re: [HACKERS] Spurious standby query cancellations

2015-09-14 Thread Jeff Janes
On Fri, Sep 4, 2015 at 3:25 PM, Simon Riggs wrote: > On 27 August 2015 at 22:55, Jeff Janes wrote: > >> In ResolveRecoveryConflictWithLock, there is the comment: >> >> /* >> * If blowing away everybody with conflicting locks doesn't work, >> after >> * the first two attempts then w

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-14 Thread Shulgin, Oleksandr
On Mon, Sep 14, 2015 at 3:09 PM, Shulgin, Oleksandr < oleksandr.shul...@zalando.de> wrote: > On Mon, Sep 14, 2015 at 2:11 PM, Tomas Vondra < > tomas.von...@2ndquadrant.com> wrote: > >> >>> Now the backend that has been signaled on the second call to >>> pg_cmdstatus (it can be either some other ba

Re: [HACKERS] Do Layered Views/Relations Preserve Sort Order ?

2015-09-14 Thread Jim Nasby
On 9/9/15 7:55 PM, Charles Sheridan wrote: The better question is how expensive is it to sort already sorted data. If its cheap, and it likely is, then placing explicit sorting where you care is the best solution regardless of your level of confidence that lower level sorting is being maintained

[HACKERS] [PATCH] add --log-output to pg_ctl on Windows

2015-09-14 Thread Egon Kocjan
Hello The patch implements a command line option for pg_ctl on Windows to redirect logging of errors (write_stderr). The possible outputs are: default (eventlog if running as a service, otherwise stderr), stderr or eventlog. The previous discussion in BUG #13594: http://www.postgresql.org/me

Re: [HACKERS][PROPOSAL] Covering + unique indexes.

2015-09-14 Thread Jim Nasby
On 9/11/15 7:45 AM, Anastasia Lubennikova wrote: This idea has obvious restriction. We can set unique only for first index columns. There is no clear way to maintain following index. CREATE INDEX index ON table (c1, c2, c3) UNIQUE ON (c1, c3); So I suggest following syntax: CREATE [UNIQUE {ON FI

Re: [HACKERS] RLS open items are vague and unactionable

2015-09-14 Thread Stephen Frost
Dean, * Dean Rasheed (dean.a.rash...@gmail.com) wrote: > On 14 September 2015 at 14:47, Stephen Frost wrote: > > Attached is a git format-patch built series which includes both commits, > > now broken out, for review. > > That looks OK to me. Excellent. > A minor point -- this comment isn't qu

Re: [HACKERS] RLS open items are vague and unactionable

2015-09-14 Thread Dean Rasheed
On 14 September 2015 at 14:47, Stephen Frost wrote: > Attached is a git format-patch built series which includes both commits, > now broken out, for review. > That looks OK to me. A minor point -- this comment isn't quite right: /* * For the target relation, when there is a returning l

Re: [HACKERS] Review: GiST support for UUIDs

2015-09-14 Thread Teodor Sigaev
Andres Freund wrote: Please post reviews to the original thread unless that's already humongously large. I've lost original mail. Sorry. Otherwise somebody needs to manually link up the threads in the CF application. Of course, I did it It also makes it much harder to follow the develo

Re: [HACKERS] Review: GiST support for UUIDs

2015-09-14 Thread Andres Freund
Please post reviews to the original thread unless that's already humongously large. Otherwise somebody needs to manually link up the threads in the CF application. It also makes it much harder to follow the development because there'll likely be a new version of the patch after a review. Which

[HACKERS] Review: GiST support for UUIDs

2015-09-14 Thread Teodor Sigaev
http://www.postgresql.org/message-id/flat/ca+renyvephxto1c7dfbvjp1gymuc0-3qdnwpn30-noo5mpy...@mail.gmail.com#ca+renyvephxto1c7dfbvjp1gymuc0-3qdnwpn30-noo5mpy...@mail.gmail.com Patch looks perfect but it's still needed some work. 0) rebase to current HEAD (done, in attach) 1) UUIDSIZE -> UUID_LE

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2015-09-14 Thread Andres Freund
On 2015-09-14 13:16:46 +0300, YUriy Zhuravlev wrote: > On Friday 11 September 2015 18:50:35 you wrote: > > a) As I said upthread there's a patch to remove these locks entirely > It is very interesting. Could you provide a link? http://archives.postgresql.org/message-id/CA%2BTgmoYE4t-Pt%2Bv08kMO5

Re: [HACKERS] Re: [COMMITTERS] pgsql: Check existency of table/schema for -t/-n option (pg_dump/pg_res

2015-09-14 Thread Teodor Sigaev
/* -* We use UNION ALL rather than UNION; this might sometimes result in -* duplicate entries in the OID list, but we don't care. +* this might sometimes result in duplicate entries in the OID list, +* but we don't care. */ This looks totally incoherent. You've removed

[HACKERS] Re: [COMMITTERS] pgsql: Check existency of table/schema for -t/-n option (pg_dump/pg_res

2015-09-14 Thread Robert Haas
On Mon, Sep 14, 2015 at 9:20 AM, Teodor Sigaev wrote: > Check existency of table/schema for -t/-n option (pg_dump/pg_restore) > > Patch provides command line option --strict-names which requires that at > least one table/schema should present for each -t/-n option. > > Pavel Stehule /* -

[HACKERS] Attach comments to functions' parameters and return value

2015-09-14 Thread Charles Clavadetscher
Hello In PostgreSQL it is possible to attach comments to almost everything. This made it possible for us to integrate the wiki that we use for our technical documentation directly with the database using the MediaWiki [1] extensions ExternalData [2] and MagicNoCache [3]. The result is a documentat

Re: [HACKERS] [COMMITTERS] pgsql: Fix an O(N^2) problem in foreign key references.

2015-09-14 Thread Tom Lane
Kevin Grittner writes: > Fix an O(N^2) problem in foreign key references. Judging from the buildfarm, this patch is broken under CLOBBER_CACHE_ALWAYS. See friarbird's results in particular. I might be too quick to finger this patch, but nothing else lately has touched foreign-key behavior, and t

Re: [HACKERS] RLS open items are vague and unactionable

2015-09-14 Thread Stephen Frost
Dean, * Dean Rasheed (dean.a.rash...@gmail.com) wrote: > It shouldn't be necessary to change get_policies_for_relation() at all > to support the RETURNING check. You just need to call it with > CMD_SELECT. Yup, that works well. > BTW, your change to change get_policies_for_relation() has > a bug

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-14 Thread Shulgin, Oleksandr
> > Well, I didn't attach the updated patch (doing that now). > This time for real. Sorry, it's Monday :-p diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c index 32ac58f..2e3beaf 100644 --- a/src/backend/storage/ipc/ipci.c +++ b/src/backend/storage/ipc/ipci.c @@ -43,6

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-09-14 Thread Thom Brown
On 11 September 2015 at 15:43, Syed, Rahila wrote: > > Hello, > > Please find attached updated VACUUM progress checker patch. > Following have been accomplished in the patch > > 1. Accounts for index pages count while calculating total progress of > VACUUM. > 2. Common location for storing progr

Re: [HACKERS] WIP: Rework access method interface

2015-09-14 Thread Petr Jelinek
On 2015-09-14 14:34, Oleg Bartunov wrote: Whhat I don't understand from this thread if we should wait 2ndQuadrant for their sequence and column AMs or just start to work on committing it ? Alvaro, where are you ? I don't see problems with this patch from the sequence am perspective. The nex

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-14 Thread Shulgin, Oleksandr
On Mon, Sep 14, 2015 at 2:11 PM, Tomas Vondra wrote: > >> Now the backend that has been signaled on the second call to >> pg_cmdstatus (it can be either some other backend, or the backend B >> again) will not find an unprocessed slot, thus it will not try to >> attach/detach the queue and the bac

Re: [HACKERS] Scaling PostgreSQL at multicore Power8

2015-09-14 Thread YUriy Zhuravlev
On Monday 31 August 2015 17:43:08 Tomas Vondra wrote: > Well, I could test the patch on a x86 machine with 4 sockets (64 cores), > but I wonder whether it makes sense at this point, as the patch really > is not correct (judging by what Andres says). Can you test patch from this thread: http://www

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-14 Thread Pavel Stehule
2015-09-14 14:11 GMT+02:00 Tomas Vondra : > > > On 09/14/2015 01:15 PM, Shulgin, Oleksandr wrote: > >> On Mon, Sep 14, 2015 at 11:53 AM, Tomas Vondra >> mailto:tomas.von...@2ndquadrant.com>> >> wrote: >> >> >> On 09/14/2015 10:23 AM, Shulgin, Oleksandr wrote: >> >> On Sat, Sep 12, 2015

Re: [HACKERS] WIP: Rework access method interface

2015-09-14 Thread Oleg Bartunov
On Fri, Sep 11, 2015 at 4:22 PM, Alexander Korotkov < a.korot...@postgrespro.ru> wrote: > On Mon, Sep 7, 2015 at 9:17 PM, Petr Jelinek wrote: > >> On 2015-09-04 16:26, Alexander Korotkov wrote: >> >>> >>> Attached patch is implementing this. It doesn't pretend to be fully >>> correct implementati

Re: [HACKERS] Double linking MemoryContext children

2015-09-14 Thread Jan Wieck
On 09/12/2015 11:35 AM, Kevin Grittner wrote: On the other hand, a grep indicates that there are two places that MemoryContextData.nextchild is set (and we therefore probably need to also set the new field), and Jan's proposed patch only changes one of them. If we do this, I think we need to ch

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-14 Thread Tomas Vondra
On 09/14/2015 01:15 PM, Shulgin, Oleksandr wrote: On Mon, Sep 14, 2015 at 11:53 AM, Tomas Vondra mailto:tomas.von...@2ndquadrant.com>> wrote: On 09/14/2015 10:23 AM, Shulgin, Oleksandr wrote: On Sat, Sep 12, 2015 at 11:50 AM, Tomas Vondra mailto:tomas.von...@2ndquadrant.c

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

2015-09-14 Thread Robert Haas
On Mon, Sep 14, 2015 at 5:32 AM, Alexander Korotkov wrote: > In order to build the consensus we need the roadmap for waits monitoring. > Would single byte in PgBackendStatus be the only way for tracking wait > events? Could we have pluggable infrastructure in waits monitoring: for > instance, hook

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

2015-09-14 Thread Alexander Korotkov
On Mon, Sep 14, 2015 at 2:12 PM, Amit Kapila wrote: > On Mon, Sep 14, 2015 at 2:25 PM, Alexander Korotkov > wrote: > >> On Sat, Sep 12, 2015 at 2:05 PM, Amit Kapila >> wrote: >> >>> On Thu, Aug 6, 2015 at 3:31 PM, Ildus Kurbangaliev < >>> i.kurbangal...@postgrespro.ru> wrote: >>> > >>> > On 08/

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

2015-09-14 Thread Alexander Korotkov
On Mon, Sep 14, 2015 at 2:25 PM, Amit Kapila wrote: > On Mon, Sep 14, 2015 at 3:02 PM, Alexander Korotkov > wrote: > >> On Sat, Sep 12, 2015 at 3:24 PM, Amit Kapila >> wrote: >> >>> On Fri, Aug 14, 2015 at 7:23 PM, Alexander Korotkov < >>> aekorot...@gmail.com> wrote: >>> > >>> > On Thu, Aug 6,

Re: [HACKERS] row_security GUC, BYPASSRLS

2015-09-14 Thread Robert Haas
On Mon, Sep 14, 2015 at 3:29 AM, Noah Misch wrote: > SECURITY DEFINER execution blocks SET ROLE, SET SESSION AUTHORIZATION, and > sometimes "GRANT role1 TO role2". Otherwise, it works like regular execution. > Adding exceptions, particularly silent behavior changes as opposed to hard > errors, is

Re: [HACKERS] PATCH: index-only scans with partial indexes

2015-09-14 Thread Tomas Vondra
On 09/14/2015 12:51 PM, Kyotaro HORIGUCHI wrote: I rethinked on this from the first. Sorry. Hi, this looks to be a bug of cost_index(). The attached patch would fix that. No, that's wrong. please forget the patch. The qual in qpquals should be indexquals which is excluded because it is no

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

2015-09-14 Thread Amit Kapila
On Mon, Sep 14, 2015 at 3:02 PM, Alexander Korotkov wrote: > On Sat, Sep 12, 2015 at 3:24 PM, Amit Kapila > wrote: > >> On Fri, Aug 14, 2015 at 7:23 PM, Alexander Korotkov >> wrote: >> > >> > On Thu, Aug 6, 2015 at 1:01 PM, Ildus Kurbangaliev < >> i.kurbangal...@postgrespro.ru> wrote: >> >> >>

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2015-09-14 Thread Robert Haas
On Fri, Sep 11, 2015 at 11:01 PM, Amit Kapila wrote: > If I am not wrong we need 1048576 number of transactions difference > for each record to make each CLOG access a disk access, so if we > increment XID counter by 100, then probably every 1th (or multiplier > of 1) transaction would go

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-14 Thread Shulgin, Oleksandr
On Mon, Sep 14, 2015 at 11:53 AM, Tomas Vondra wrote: > > On 09/14/2015 10:23 AM, Shulgin, Oleksandr wrote: > >> On Sat, Sep 12, 2015 at 11:50 AM, Tomas Vondra >> mailto:tomas.von...@2ndquadrant.com>> >> wrote: >> > ... > >> - Attempts to get plan for simple insert queries like this >> >>

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

2015-09-14 Thread Amit Kapila
On Mon, Sep 14, 2015 at 2:25 PM, Alexander Korotkov wrote: > On Sat, Sep 12, 2015 at 2:05 PM, Amit Kapila > wrote: > >> On Thu, Aug 6, 2015 at 3:31 PM, Ildus Kurbangaliev < >> i.kurbangal...@postgrespro.ru> wrote: >> > >> > On 08/05/2015 09:33 PM, Robert Haas wrote: >> >> >> >> >> >> You're miss

Re: [HACKERS] PATCH: index-only scans with partial indexes

2015-09-14 Thread Kyotaro HORIGUCHI
I rethinked on this from the first. > Sorry. > > > Hi, this looks to be a bug of cost_index(). The attached patch > > would fix that. > > No, that's wrong. please forget the patch. The qual in qpquals > should be indexquals which is excluded because it is not > necessary to be applied. The right

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

2015-09-14 Thread Amit Kapila
On Sun, Sep 13, 2015 at 8:06 AM, Robert Haas wrote: > > On Sat, Sep 12, 2015 at 8:24 AM, Amit Kapila wrote: > > 1. Modify the tranche mechanism so that information about LWLocks > > can be tracked easily. For this already there is some discussion, ideas > > and initial patch is floated in this t

Re: [HACKERS] [PATCH] Refactoring of LWLock tranches

2015-09-14 Thread Robert Haas
On Sun, Sep 13, 2015 at 5:05 PM, Ildus Kurbangaliev wrote: > Yes, that is because I tried to go with current convention working with > shmem in Postgres (there are one function that returns the size and > others that initialize that memory). But I like your suggestion about > API functions, in tha

Re: [HACKERS] Testing WAL replay by comparing before and after images again

2015-09-14 Thread Heikki Linnakangas
On 09/04/2015 09:30 PM, Simon Riggs wrote: On 4 September 2015 at 13:45, Heikki Linnakangas wrote: Another issue was with the new speculative insertions. Replaying a speculative insertion record sets the tuple's CTID to point to itself, like in a regular insertion. But in the original system,

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2015-09-14 Thread YUriy Zhuravlev
On Friday 11 September 2015 18:50:35 you wrote: > a) As I said upthread there's a patch to remove these locks entirely It is very interesting. Could you provide a link? And it's not very good, since there is a bottleneck PinBuffer / UnpinBuffer instead of LWLocks. > b) It doesn't matter anyway. No

Re: [HACKERS] PATCH: index-only scans with partial indexes

2015-09-14 Thread Kyotaro HORIGUCHI
Sorry. > Hi, this looks to be a bug of cost_index(). The attached patch > would fix that. No, that's wrong. please forget the patch. The qual in qpquals should be indexquals which is excluded because it is not necessary to be applied. The right way would be remove the cost for qpqual in cost_inde

Re: [HACKERS] PATCH: index-only scans with partial indexes

2015-09-14 Thread Kyotaro HORIGUCHI
Hi, this looks to be a bug of cost_index(). The attached patch would fix that. = The following part in cost_index, > cpu_per_tuple = cpu_tuple_cost + qpqual_cost.per_tuple; > > run_cost += cpu_per_tuple * tuples_fetched; Adds, *cpu_tuple_cost* (which is 0.01) + qpqual_cost.per_tuple (0.0025

Re: [HACKERS] Review: check existency of table for -t option (pg_dump) when pattern...

2015-09-14 Thread Pavel Stehule
2015-09-14 12:05 GMT+02:00 Teodor Sigaev : > (new version in attach), but patch shows some inconsistent output: >> % pg_dump -t 'aaa*' postgres >> pg_dump: No matching tables were found >> % pg_dump -t 'aaa*' --strict-names postgres >> pg_dump: Table "aaa*" not found. >> > > T

[HACKERS] Re: Review: check existency of table for -t option (pg_dump) when pattern...

2015-09-14 Thread Teodor Sigaev
(new version in attach), but patch shows some inconsistent output: % pg_dump -t 'aaa*' postgres pg_dump: No matching tables were found % pg_dump -t 'aaa*' --strict-names postgres pg_dump: Table "aaa*" not found. There are two different situation - first message says "there

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-14 Thread Tomas Vondra
On 09/14/2015 10:23 AM, Shulgin, Oleksandr wrote: On Sat, Sep 12, 2015 at 11:50 AM, Tomas Vondra mailto:tomas.von...@2ndquadrant.com>> wrote: ... - Attempts to get plan for simple insert queries like this INSERT INTO x SELECT * FROM x; end with a segfault, because Acti

Re: [HACKERS] Partitioned checkpointing

2015-09-14 Thread Takashi Horikawa
Hi, I wrote: > The original purpose is to mitigate full-page-write rush that occurs at > immediately after the beginning of each checkpoint. > The amount of FPW at each checkpoint is reduced to 1/16 by the > 'Partitioned checkpointing.' Let me show another set of measurement results that clearly i

[HACKERS] Can extension build own SGML document?

2015-09-14 Thread Kouhei Kaigai
Hello, The pgxs makefile (pgxs.ml) says: # DOCS -- random files to install under $PREFIX/doc/$MODULEDIR It is a bunch of files to be copied to document directory, however, not looks like a variable to specify SGML source as PostgreSQL core doing. Do we have way to build SGML source of extensi

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

2015-09-14 Thread Alexander Korotkov
On Sat, Sep 12, 2015 at 3:24 PM, Amit Kapila wrote: > On Fri, Aug 14, 2015 at 7:23 PM, Alexander Korotkov > wrote: > > > > On Thu, Aug 6, 2015 at 1:01 PM, Ildus Kurbangaliev < > i.kurbangal...@postgrespro.ru> wrote: > >> > >> > >> I've looked deeper and I found PgBackendStatus to be not a suitab

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

2015-09-14 Thread Alexander Korotkov
On Sat, Sep 12, 2015 at 2:05 PM, Amit Kapila wrote: > On Thu, Aug 6, 2015 at 3:31 PM, Ildus Kurbangaliev < > i.kurbangal...@postgrespro.ru> wrote: > > > > On 08/05/2015 09:33 PM, Robert Haas wrote: > >> > >> > >> You're missing the point. Those multi-byte fields have additional > >> synchronizat

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-14 Thread Pavel Stehule
2015-09-14 10:23 GMT+02:00 Shulgin, Oleksandr : > > > > I can think of something like: > > EXPLAIN [ ( option [, ...] ) ] PROCESS ; > > where option is extended with: > > QUERY > PROGRESS > BACKTRACE > > in addition to the usual ANALYZE, VERBOSE, FORMAT, etc. > > It can work Regards Pavel

Re: [HACKERS] extend pgbench expressions with functions

2015-09-14 Thread Kyotaro HORIGUCHI
Hi, I had a look on this, this gives amazing performance. (I myself haven't tried that yet:-p) But anyway the new syntax looks far smarter than preparing arbitraly metacommands. michael> I have moved this patch to the next CF. > > Hello Heikki, > > > >>> As soon as we add more functions, the way

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-14 Thread Shulgin, Oleksandr
On Sat, Sep 12, 2015 at 11:50 AM, Tomas Vondra wrote: > Hi, > > I did a quick initial review of this patch today, so here are my comments > so far: > Hi Tomas, First of all, thanks for the review! - ipcs.c should include utils/cmdstatus.h (the compiler complains > about implicit declaration

Re: [HACKERS] RLS open items are vague and unactionable

2015-09-14 Thread Dean Rasheed
On 13 September 2015 at 22:54, Stephen Frost wrote: > Not in front of my laptop and will review it when I get back in more detail, > but the original approach that I tried was changing > get_policies_for_relation to try and build everything necessary, which > didn't work as we need to OR the vario

Re: [HACKERS] PATCH: index-only scans with partial indexes

2015-09-14 Thread Tomas Vondra
On 09/14/2015 09:35 AM, Kyotaro HORIGUCHI wrote: Hi, ,,, Which is exactly the difference between costs from amcostestimate idx1: 4769.115000 + 0.015 * 297823 = 9236.46 idx2: 6258.23 + 0.010 * 297823 = 9236.46 These calculations are exactly right, but you overlooked the breakedown

Re: [HACKERS] Parser emits mysterious error message for very long tokens

2015-09-14 Thread Kyotaro HORIGUCHI
Hello, thank you for the opinion. At Fri, 11 Sep 2015 09:31:30 -0400, Tom Lane wrote in <884.1441978...@sss.pgh.pa.us> > Kyotaro HORIGUCHI writes: > > Hello, this is a problem on an extreme situation. > > When parser encounters very long tokens, it returns an error > > message which should be m

Re: [HACKERS] PATCH: index-only scans with partial indexes

2015-09-14 Thread Kyotaro HORIGUCHI
Hi, At Sun, 13 Sep 2015 23:21:30 +0200, Tomas Vondra wrote in <55f5e8da.8080...@2ndquadrant.com> > That's indeed strange, but after poking into that for a while, it > seems rather like a costing issue. Let me demonstrate: ... > Now, both plans are index only scans, but the first one has Index Co

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

2015-09-14 Thread Vladimir Borodin
> 12 сент. 2015 г., в 14:05, Amit Kapila написал(а): > > On Thu, Aug 6, 2015 at 3:31 PM, Ildus Kurbangaliev > mailto:i.kurbangal...@postgrespro.ru>> wrote: > > > > On 08/05/2015 09:33 PM, Robert Haas wrote: > >> > >> > >> You're missing the point. Those multi-byte fields have additional > >> s

  1   2   >