reorder pg_rewind control file sync

2019-03-22 Thread Fabien COELHO
Bonjour Michaël, On Sat, 23 Mar 2019, Michael Paquier wrote: On Fri, Mar 22, 2019 at 03:18:26PM +0100, Fabien COELHO wrote: Attached is a quick patch about "pg_rewind", so that the control file is updated after everything else is committed to disk. Could you start a new thread about that

Re: Transaction commits VS Transaction commits (with parallel) VS query mean time

2019-03-22 Thread Alvaro Herrera
On 2019-Mar-23, Amit Kapila wrote: > On Fri, Mar 22, 2019 at 10:04 AM Alvaro Herrera > wrote: > > Not count them if they're implementation details; otherwise count them. > > For example, IMO autovacuum transactions should definitely be counted > > (as one transaction, even if they run parallel

Re: Transaction commits VS Transaction commits (with parallel) VS query mean time

2019-03-22 Thread Amit Kapila
On Fri, Mar 22, 2019 at 10:04 AM Alvaro Herrera wrote: > On 2019-Mar-21, Robert Haas wrote: > > > I agree that it's a little funny to count the parallel worker commit > > as a separate transaction, since in a certain sense it is part of the > > same transaction. > > Right. So you don't count it.

Re: Ordered Partitioned Table Scans

2019-03-22 Thread David Rowley
On Sat, 23 Mar 2019 at 05:40, Tom Lane wrote: > BTW, another thing we could possibly do to answer this objection is to > give the ordered-Append node an artificially pessimistic startup cost, > such as the sum or the max of its children's startup costs. That's > pretty ugly and unprincipled, but

Re: Ordered Partitioned Table Scans

2019-03-22 Thread David Rowley
On Sat, 23 Mar 2019 at 04:56, Tom Lane wrote: > > David Rowley writes: > > Append has the additional > > saving of not having to determine to perform a sort on the top row > > from each subpath. > > Uh, what? sorted-Append and MergeAppend would need pre-sorts on > exactly the same set of

Re: psql display of foreign keys

2019-03-22 Thread Alvaro Herrera
On 2019-Mar-22, Alvaro Herrera wrote: > On 2019-Mar-05, Amit Langote wrote: > > > On 2019/03/05 4:41, Alvaro Herrera wrote: > > > Here's the patch I'm really interested about :-) > > > > Thanks for the updated patch. I applied it and rebased the > > foreign-keys-referencing-partitioned-tables

Re: Progress reporting for pg_verify_checksums

2019-03-22 Thread Michael Paquier
On Fri, Mar 22, 2019 at 02:23:17PM +0100, Michael Banck wrote: > The current version prints a newline when it progress reporting is > toggled off. Do you mean there is a hazard that this happens right when > we are printing the progress, so end up with a partly garbage line? I > don't think that'd

Re: Contribution to Perldoc for TestLib module in Postgres

2019-03-22 Thread Michael Paquier
On Fri, Mar 22, 2019 at 04:59:58PM +0530, Ramanarayana wrote: > Please find the first version of the patch for review. I was not sure what > some of the functions are used for and marked them with TODO. This is only adding some documentation to an internal perl module we ship, so it is far from

Re: Special role for subscriptions

2019-03-22 Thread Michael Paquier
On Fri, Mar 22, 2019 at 08:41:06PM +0800, Andrey Borodin wrote: > 22 марта 2019 г., в 19:17, Petr Jelinek > написал(а): >> I still don't like that we are running the subscription workers as >> superuser even for subscriptions created by regular user. That has >> plenty of privilege escalation

Re: propagating replica identity to partitions

2019-03-22 Thread Michael Paquier
On Fri, Mar 22, 2019 at 07:55:11PM +0100, Peter Eisentraut wrote: > If you are operating on a partitioned table and set the replica identity > to the primary key or a partitioned index of that partitioned table, > then I think, by definition of what it means to be a partitioned index, > that

Re: Introduce MIN/MAX aggregate functions to pg_lsn

2019-03-22 Thread Michael Paquier
On Fri, Mar 22, 2019 at 04:49:57PM -0300, Fabrízio de Royes Mello wrote: > So attached patch aims to introduce MIN/MAX aggregate functions to pg_lsn Fine by me. This looks helpful for monitoring. Please make sure to register it to the next commit fest: https://commitfest.postgresql.org/23/ It

Re: compiler warning in pgcrypto imath.c

2019-03-22 Thread Michael Paquier
On Fri, Mar 22, 2019 at 08:20:53PM -0400, Jeff Janes wrote: > PostgreSQL 12devel on aarch64-unknown-linux-gnu, compiled by gcc > (Ubuntu/Linaro 7.3.0-27ubuntu1~18.04) 7.3.0, 64-bit Adding Noah in CC as he has done the update of imath lately. > The attached patch adds PG_USED_FOR_ASSERTS_ONLY to

compiler warning in pgcrypto imath.c

2019-03-22 Thread Jeff Janes
When compiling on an AWS 64 bit Arm machine, I get this compiler warning: imath.c: In function 's_ksqr': imath.c:2590:6: warning: variable 'carry' set but not used [-Wunused-but-set-variable] carry; ^ With this version(): PostgreSQL 12devel on aarch64-unknown-linux-gnu, compiled

Re: Offline enabling/disabling of data checksums

2019-03-22 Thread Michael Paquier
On Fri, Mar 22, 2019 at 03:18:26PM +0100, Fabien COELHO wrote: > Attached is a quick patch about "pg_rewind", so that the control file is > updated after everything else is committed to disk. Could you start a new thread about that please? This one has already been used for too many things. --

Re: Offline enabling/disabling of data checksums

2019-03-22 Thread Michael Paquier
On Sat, Mar 23, 2019 at 08:16:07AM +0900, Michael Paquier wrote: > And committed the main part. I'll look after the --no-sync part in a > bit. --no-sync is committed as well now. -- Michael signature.asc Description: PGP signature

Re: Offline enabling/disabling of data checksums

2019-03-22 Thread Michael Paquier
On Fri, Mar 22, 2019 at 07:02:36PM +0100, Fabien COELHO wrote: > Indeed it does, and it is done in update_controlfile if the last argument is > true. Basically update_controlfile latest version always fsync the control > file, unless explicitely told not to do so. The options to do that are >

Re: Offline enabling/disabling of data checksums

2019-03-22 Thread Michael Paquier
On Fri, Mar 22, 2019 at 02:59:31PM +0100, Fabien COELHO wrote: > On write(), the error message is not translatable whereas it is for all > others. Fixed. > I agree that a BIG STRONG warning is needed about not to start the cluster > under pain of possible data corruption. I still think that

Re: Error message inconsistency

2019-03-22 Thread Fabrízio de Royes Mello
On Fri, Mar 22, 2019 at 2:25 PM Simon Riggs wrote: > > As noted by a PostgreSQL user to me, error messages for NOT NULL constraints are inconsistent - they do not mention the relation name in the message, as all other variants of this message do. e.g. > > postgres=# create table nn (id integer

Fix foreign key constraint check for partitioned tables

2019-03-22 Thread Hadi Moshayedi
Yesterday while doing some tests, I noticed that the following doesn't work properly: create role test_role with login; create table ref(a int primary key); grant references on ref to test_role; set role test_role; create table t1(a int, b int) partition by list (a); alter table t1 add constraint

RE: Planning counters in pg_stat_statements (using pgss_store)

2019-03-22 Thread legrand legrand
Hi, Here is a rebased and corrected version . Columns naming has not been modified, I would propose to change it to: - plans: ok - planning_time --> plan_time - calls: ok - total_time --> exec_time - {min,max,mean,stddev}_time: ok - new total_time (being the sum of plan_time and exec_time)

Re: psql display of foreign keys

2019-03-22 Thread Alvaro Herrera
On 2019-Mar-05, Amit Langote wrote: > On 2019/03/05 4:41, Alvaro Herrera wrote: > > Here's the patch I'm really interested about :-) > > Thanks for the updated patch. I applied it and rebased the > foreign-keys-referencing-partitioned-tables patch on top. Here's > something I think you may

Re: speeding up planning with partitions

2019-03-22 Thread Tom Lane
I wrote: > ... I also > don't like the undocumented way that you've got build_base_rel_tlists > working on something that's not the final tlist, and then going back > and doing more work of the same sort later. I wonder whether we can > postpone build_base_rel_tlists until after the other stuff

Re: Ordered Partitioned Table Scans

2019-03-22 Thread Julien Rouhaud
On Fri, Mar 22, 2019 at 7:19 PM Robert Haas wrote: > > On Fri, Mar 22, 2019 at 12:40 PM Tom Lane wrote: > > Robert Haas writes: > > > On Fri, Mar 22, 2019 at 11:56 AM Tom Lane wrote: > > >> In cases where, say, the first child requires no sort but also doesn't > > >> emit very many rows, while

Re: rename labels in heapam.c?

2019-03-22 Thread Tom Lane
Andres Freund writes: > On 2019-03-22 17:09:23 -0400, Tom Lane wrote: >> Is it practical to get rid of the goto's altogether? If not, >> renaming would be an improvement. > I don't think it'd be easy. Fair enough. I just wanted to be sure we considered getting rid of the pig before we put

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2019-03-22 Thread Peter Geoghegan
On Thu, Mar 21, 2019 at 10:28 AM Peter Geoghegan wrote: > I've committed the first 4 patches. Many thanks to Heikki for his very > valuable help! Thanks also to the other reviewers. > > I'll likely push the remaining two patches on Sunday or Monday. I noticed that if I initidb and run "make

Re: rename labels in heapam.c?

2019-03-22 Thread Andres Freund
Hi, On 2019-03-22 17:09:23 -0400, Tom Lane wrote: > Andres Freund writes: > > For the umpteenth time I was annoyed by the names of labels in > > heapam.c. It's really not useful to see a 'goto l1;' etc. > > Yeah, those label names are uninformative as can be. > > > How about renaming l1 to

Re: rename labels in heapam.c?

2019-03-22 Thread Tom Lane
Andres Freund writes: > For the umpteenth time I was annoyed by the names of labels in > heapam.c. It's really not useful to see a 'goto l1;' etc. Yeah, those label names are uninformative as can be. > How about renaming l1 to retry_delete_locked, l2 to retry_update_locked, > l3 to

Re: speeding up planning with partitions

2019-03-22 Thread Tom Lane
Amit Langote writes: > [ v34 patch set ] I had a bit of a look through this. I went ahead and pushed 0008 and 0009, as they seem straightforward and independent of the rest. (BTW, 0009 makes 0003's dubious optimization in set_relation_partition_info quite unnecessary.) As for the rest:

rename labels in heapam.c?

2019-03-22 Thread Andres Freund
Hi, For the umpteenth time I was annoyed by the names of labels in heapam.c. It's really not useful to see a 'goto l1;' etc. How about renaming l1 to retry_delete_locked, l2 to retry_update_locked, l3 to retry_lock_tuple_locked etc? Especially with the subsidiary functions for updates and

Re: [PATCH v20] GSSAPI encryption support

2019-03-22 Thread Robbie Harwood
Stephen Frost writes: > One of the things that I really didn't care for in this patch was the > use of the string buffers, without any real checks (except for "oh, > you tried to allocated over 1G"...) to make sure that the other side > of the connection wasn't feeding us ridiculous packets, and

Introduce MIN/MAX aggregate functions to pg_lsn

2019-03-22 Thread Fabrízio de Royes Mello
Hi all, Before we introduce pg_lsn datatype the LSN was expressed as a TEXT type, so a simple query using MIN/MAX functions works as expected. Query like: SELECT min(restart_lsn) FROM pg_replication_slots; SELECT min(sent_lsn) FROM pg_stat_replication ; So attached patch aims to introduce

Re: propagating replica identity to partitions

2019-03-22 Thread Peter Eisentraut
On 2019-03-22 17:52, Alvaro Herrera wrote: > To recap: my proposed change is to make > ALTER TABLE ... REPLICA IDENTITY > when applied on a partitioned table affect all of its partitions instead > of expecting the user to invoke the command for each partition. If you are operating on a

Re: shared-memory based stats collector

2019-03-22 Thread Andres Freund
Ping? Unless there's a new version pretty soon, we're going to have to move this to the next CF, I think.

Re: [HACKERS] CLUSTER command progress monitor

2019-03-22 Thread Robert Haas
On Tue, Mar 19, 2019 at 2:47 PM Robert Haas wrote: > how close you were getting to rewriting the entire heap. This is the > one thing I found but did not fix; any chance you could make this > change and update the documentation to match? Hi, is anybody working on this? -- Robert Haas

Re: Psql patch to show access methods info

2019-03-22 Thread Sergey Cherkashin
Taking into account the wishes of all the reviewers, the current position of the patch is as follows: The \dA command displays a list of access methods. # \dA List of access methods Name | Type | Handler +---+-- brin | index |

Re: Concurrency bug with vacuum full (cluster) and toast

2019-03-22 Thread Robert Haas
On Tue, Mar 19, 2019 at 1:37 PM Alexander Korotkov wrote: > Thank you for pointing, but none of the threads you pointed describe > this exact problem. Now I see this bug have a set of cute siblings :) Yeah. I really thought this precise issue -- the interlocking between the VACUUM of the main

Re: New vacuum option to do only freezing

2019-03-22 Thread Robert Haas
On Fri, Mar 8, 2019 at 12:14 AM Masahiko Sawada wrote: > IIUC we've discussed the field-and-value style vacuum option. I > suggested that since we have already the disable_page_skipping option > the disable_page_skipping option would be more natural style and > consistent. I think "VACUUM

Re: Ordered Partitioned Table Scans

2019-03-22 Thread Robert Haas
On Fri, Mar 22, 2019 at 12:40 PM Tom Lane wrote: > Robert Haas writes: > > On Fri, Mar 22, 2019 at 11:56 AM Tom Lane wrote: > >> In cases where, say, the first child requires no sort but also doesn't > >> emit very many rows, while the second child requires an expensive sort, > >> the planner

Re: Offline enabling/disabling of data checksums

2019-03-22 Thread Fabien COELHO
Hello Christoph, - pg_log(PG_PROGRESS, "syncing target data directory\n"); - syncTargetDirectory(); Doesn't the control file still need syncing? Indeed it does, and it is done in update_controlfile if the last argument is true. Basically update_controlfile latest version

Re: Problem with default partition pruning

2019-03-22 Thread Thibaut Madelaine
Le 22/03/2019 à 07:38, Amit Langote a écrit : > Hosoya-san, > > On 2019/03/22 15:02, Yuzuko Hosoya wrote: >> I understood Amit's proposal. But I think the issue Thibaut reported would >> occur regardless of whether clauses have OR clauses or not as follows. >> I tested a query which should

Error message inconsistency

2019-03-22 Thread Simon Riggs
As noted by a PostgreSQL user to me, error messages for NOT NULL constraints are inconsistent - they do not mention the relation name in the message, as all other variants of this message do. e.g. postgres=# create table nn (id integer not null); CREATE TABLE postgres=# insert into nn values

Re: Enable data checksums by default

2019-03-22 Thread Andres Freund
On 2019-03-22 18:01:32 +0100, Tomas Vondra wrote: > On 3/22/19 5:41 PM, Andres Freund wrote: > > Hi, > > > > On 2019-03-22 17:32:10 +0100, Tomas Vondra wrote: > >> On 3/22/19 5:10 PM, Andres Freund wrote: > >>> IDK, being able to verify in some form that backups aren't corrupted on > >>> an IO

Re: Enable data checksums by default

2019-03-22 Thread Tomas Vondra
On 3/22/19 5:41 PM, Andres Freund wrote: > Hi, > > On 2019-03-22 17:32:10 +0100, Tomas Vondra wrote: >> On 3/22/19 5:10 PM, Andres Freund wrote: >>> IDK, being able to verify in some form that backups aren't corrupted on >>> an IO level is mighty nice. That often does allow to detect the issue

Re: propagating replica identity to partitions

2019-03-22 Thread Alvaro Herrera
On 2019-Mar-22, Robert Haas wrote: > On Thu, Mar 21, 2019 at 5:01 PM Alvaro Herrera > wrote: > > I already argued that TABLESPACE and OWNER TO are documented to work > > that way, and have been for a long time, whereas REPLICA IDENTITY has > > never been. If you want to change long-standing

Re: Enable data checksums by default

2019-03-22 Thread Andres Freund
Hi, On 2019-03-22 17:32:10 +0100, Tomas Vondra wrote: > On 3/22/19 5:10 PM, Andres Freund wrote: > > IDK, being able to verify in some form that backups aren't corrupted on > > an IO level is mighty nice. That often does allow to detect the issue > > while one still has older backups around. > >

Re: Ordered Partitioned Table Scans

2019-03-22 Thread Tom Lane
Robert Haas writes: > On Fri, Mar 22, 2019 at 11:56 AM Tom Lane wrote: >> In cases where, say, the first child requires no sort but also doesn't >> emit very many rows, while the second child requires an expensive sort, >> the planner will have a ridiculously optimistic opinion of the cost of >>

Re: Ordered Partitioned Table Scans

2019-03-22 Thread Robert Haas
On Fri, Mar 22, 2019 at 12:21 PM Tom Lane wrote: > Once again: this objection is not a "death sentence for this patch". > I simply wish to suppress the option to generate an ordered Append > when some of the inputs would require an added sort step. As long > as we have pre-ordered paths for all

Re: propagating replica identity to partitions

2019-03-22 Thread Robert Haas
On Thu, Mar 21, 2019 at 5:01 PM Alvaro Herrera wrote: > I already argued that TABLESPACE and OWNER TO are documented to work > that way, and have been for a long time, whereas REPLICA IDENTITY has > never been. If you want to change long-standing behavior, be my guest, > but that's not my patch.

Re: Enable data checksums by default

2019-03-22 Thread Tomas Vondra
On 3/22/19 5:10 PM, Andres Freund wrote: > Hi, > > On 2019-03-22 12:07:22 -0400, Tom Lane wrote: >> Christoph Berg writes: >>> I think, the next step in that direction would be to enable data >>> checksums by default. They make sense in most setups, >> >> Well, that is exactly the point that

Re: Ordered Partitioned Table Scans

2019-03-22 Thread Tom Lane
Robert Haas writes: > On Fri, Mar 22, 2019 at 11:56 AM Tom Lane wrote: >> In cases where, say, the first child requires no sort but also doesn't >> emit very many rows, while the second child requires an expensive sort, >> the planner will have a ridiculously optimistic opinion of the cost of >>

Re: Ordered Partitioned Table Scans

2019-03-22 Thread Robert Haas
On Fri, Mar 22, 2019 at 11:56 AM Tom Lane wrote: > In cases where, say, the first child requires no sort but also doesn't > emit very many rows, while the second child requires an expensive sort, > the planner will have a ridiculously optimistic opinion of the cost of > fetching slightly more

Re: Enable data checksums by default

2019-03-22 Thread Andres Freund
Hi, On 2019-03-22 12:07:22 -0400, Tom Lane wrote: > Christoph Berg writes: > > I think, the next step in that direction would be to enable data > > checksums by default. They make sense in most setups, > > Well, that is exactly the point that needs some proof, not just > an unfounded assertion.

Re: Enable data checksums by default

2019-03-22 Thread Tom Lane
Christoph Berg writes: > I think, the next step in that direction would be to enable data > checksums by default. They make sense in most setups, Well, that is exactly the point that needs some proof, not just an unfounded assertion. IMO, the main value of checksums is that they allow the

Re: Ordered Partitioned Table Scans

2019-03-22 Thread Tom Lane
David Rowley writes: > Thanks for explaining. I see where you're coming from now. I think > this point would carry more weight if using the Append instead of the > MergeAppend were worse in some cases as we could end up using an > inferior plan accidentally. However, that's not the case. The

Re: Ordered Partitioned Table Scans

2019-03-22 Thread Simon Riggs
On Fri, 22 Mar 2019 at 11:39, Tom Lane wrote: > Simon Riggs writes: > > I agree that the issue of mixing sorts at various points will make > nonsense > > of the startup cost/total cost results. > > Right. > > > I don't see LIMIT costing being broken as a reason to restrict this > >

Re: Ordered Partitioned Table Scans

2019-03-22 Thread Tom Lane
Simon Riggs writes: > I agree that the issue of mixing sorts at various points will make nonsense > of the startup cost/total cost results. Right. > I don't see LIMIT costing being broken as a reason to restrict this > optimization. I would ask that we allow improvements to the important use >

Re: [proposal] Add an option for returning SQLSTATE in psql error message

2019-03-22 Thread Tom Lane
Peter Eisentraut writes: > But now that I read the patch again, I'm not sure why this needs to > touch libpq. The formatting of error messages in psql should be handled > in psql. Maybe in an ideal world that'd be the case, but psql has always just depended on PQerrorMessage(). I don't think

Re: Ordered Partitioned Table Scans

2019-03-22 Thread David Rowley
On Sat, 23 Mar 2019 at 04:12, Tom Lane wrote: > The reason why I'm skeptical about LIMIT with a plan of the form > append-some-sorts-together is that there are going to be large > discontinuities in the cost-vs-number-of-rows-returned graph, > ie, every time you finish one child plan and start

Re: Offline enabling/disabling of data checksums

2019-03-22 Thread Christoph Berg
Re: Fabien COELHO 2019-03-22 > Attached is a quick patch about "pg_rewind", so that the control file is > updated after everything else is committed to disk. > update_controlfile(datadir_target, progname, _new, do_sync); > > - pg_log(PG_PROGRESS, "syncing target data directory\n"); >

Re: Ordered Partitioned Table Scans

2019-03-22 Thread Simon Riggs
On Fri, 22 Mar 2019 at 11:12, Tom Lane wrote: > David Rowley writes: > > On Sat, 9 Mar 2019 at 10:52, Tom Lane wrote: > >>> This can be a huge win for queries of the form "ORDER BY partkey LIMIT > >>> x". Even if the first subpath(s) aren't natively ordered, not all of > >>> the sorts should

Re: [proposal] Add an option for returning SQLSTATE in psql error message

2019-03-22 Thread Peter Eisentraut
On 2019-03-21 19:01, David Steele wrote: > What do you think, Peter? Is the extra test valuable or will it cause > unpredictable outputs as Tom and Michael predict? Yes, I'm OK with that. But now that I read the patch again, I'm not sure why this needs to touch libpq. The formatting of error

Re: pg_basebackup ignores the existing data directory permissions

2019-03-22 Thread Peter Eisentraut
On 2019-03-22 05:00, Michael Paquier wrote: > On Fri, Mar 22, 2019 at 02:45:24PM +1100, Haribabu Kommi wrote: >> How about letting the pg_basebackup to decide group permissions of the >> standby directory irrespective of the primary directory permissions. >> >> Default - permissions are same as

Enable data checksums by default

2019-03-22 Thread Christoph Berg
Lately, PostgreSQL has moved many defaults from "bare minimum" more to the "user friendly by default" side, e.g. hot_standby & replication in the default configuration, parallelism, and generally higher defaults for resource knobs like *_mem, autovacuum_* and so on. I think, the next step in that

Re: Removing unneeded self joins

2019-03-22 Thread David Rowley
On Sat, 23 Mar 2019 at 03:39, Alexander Kuzmenkov wrote: > The bug you mention later is an implementation bug that can be fixed (I > will expand on that below). Besides this, do you think current self-join > detection algorithm has fundamental correctness problems? I am not aware > of such

Re: Ordered Partitioned Table Scans

2019-03-22 Thread Tom Lane
David Rowley writes: > On Sat, 9 Mar 2019 at 10:52, Tom Lane wrote: >>> This can be a huge win for queries of the form "ORDER BY partkey LIMIT >>> x". Even if the first subpath(s) aren't natively ordered, not all of >>> the sorts should actually be performed. >> [ shrug... ] We've got no

Re: Removing unneeded self joins

2019-03-22 Thread Alexander Kuzmenkov
On 3/21/19 01:54, David Rowley wrote: I really just don't think checking the unique indexes match is going to cut it. You should be looking for a unique index where the join clauses match on either side of the join, not looking independently and then checking the indexes are the same ones.

Re: Ordered Partitioned Table Scans

2019-03-22 Thread David Rowley
On Sat, 9 Mar 2019 at 10:52, Tom Lane wrote: > > Julien Rouhaud writes: > > On Fri, Mar 8, 2019 at 9:15 PM Tom Lane wrote: > >> I think you should remove all that > >> and restrict this optimization to the case where all the subpaths are > >> natively ordered --- if we have to insert Sorts,

Re: Offline enabling/disabling of data checksums

2019-03-22 Thread Fabien COELHO
Done the switch for this case. For pg_rewind actually I think that this is an area where its logic could be improved a bit. So first the data folder is synced, and then the control file is updated. Attached is a quick patch about "pg_rewind", so that the control file is updated after

Re: Offline enabling/disabling of data checksums

2019-03-22 Thread Fabien COELHO
Bonjour Michaël, Does that look fine to you? Mostly. Patch v9 part 1 applies cleanly, compiles, global and local check ok, doc build ok. On write(), the error message is not translatable whereas it is for all others. I agree that a BIG STRONG warning is needed about not to start the

Re: Performance issue in foreign-key-aware join estimation

2019-03-22 Thread Tom Lane
David Rowley writes: > On Fri, 22 Mar 2019 at 10:10, Tom Lane wrote: >> I'm unsure how hard we should push to get something like this into v12. >> I'm concerned that its dependency on list_nth might result in performance >> regressions in some cases; ... > However, there's always a danger we

Re: speeding up planning with partitions

2019-03-22 Thread David Rowley
On Fri, 22 Mar 2019 at 20:39, Amit Langote wrote: > The problem is that make_partitionedrel_pruneinfo() does some work which > involves allocating arrays containing nparts elements and then looping > over the part_rels array to fill values in those arrays that will be used > by run-time pruning.

Re: Progress reporting for pg_verify_checksums

2019-03-22 Thread Michael Banck
Hi, Am Dienstag, den 19.03.2019, 09:04 +0900 schrieb Kyotaro HORIGUCHI: > At Mon, 18 Mar 2019 23:14:01 +0100 (CET), Fabien COELHO > wrote in > > >>> + /* we handle SIGUSR1 only, and toggle the value of show_progress > > >>> */ > > >>> + if (signum == SIGUSR1) > > >>> + 

Re: partitioned tables referenced by FKs

2019-03-22 Thread Jesper Pedersen
Hi Alvaro, On 3/21/19 6:18 PM, Alvaro Herrera wrote: On 2019-Mar-21, Jesper Pedersen wrote: pgbench -M prepared -f select.sql I'm seeing 82.64% spent in GetCachedPlan(). plan_cache_mode is auto. Hmm, I can't reproduce this at all ... I don't even see GetCachedPlan in the profile. Do

Re: GiST VACUUM

2019-03-22 Thread Andrey Borodin
> 22 марта 2019 г., в 19:37, Heikki Linnakangas написал(а): > > On 21/03/2019 19:04, Heikki Linnakangas wrote: >> Attached is the latest patch version, to be applied on top of the >> IntegerSet patch. > > And committed. Thanks Andrey! > > - Heikki Cool! Thank you very much! At the

Re: insensitive collations

2019-03-22 Thread Peter Eisentraut
On 2019-03-18 00:19, Peter Eisentraut wrote: > On 2019-03-11 21:36, Peter Eisentraut wrote: >> Patches here. This will allow all the existing collation customization >> options as well as the ones being proposed in this thread to work in >> older ICU versions. > > This has been committed, and

Re: Transaction commits VS Transaction commits (with parallel) VS query mean time

2019-03-22 Thread Robert Haas
On Fri, Mar 22, 2019 at 12:34 AM Alvaro Herrera wrote: > > And then you have to decide what to do about other background > > transactions. > > Not count them if they're implementation details; otherwise count them. > For example, IMO autovacuum transactions should definitely be counted > (as one

Re: Special role for subscriptions

2019-03-22 Thread Andrey Borodin
Hi! > 22 марта 2019 г., в 19:17, Petr Jelinek > написал(а): > > I still don't like that we are running the subscription workers as > superuser even for subscriptions created by regular user. That has > plenty of privilege escalation issues in terms of how user functions are > run (we execute

Re: partitioned tables referenced by FKs

2019-03-22 Thread Jesper Pedersen
Hi Alvaro, On 3/21/19 4:49 PM, Alvaro Herrera wrote: I think the attached is a better solution, which I'll go push shortly. I see you pushed this, plus 0002 as well. Thanks ! Best regards, Jesper

Re: jsonpath

2019-03-22 Thread Alexander Korotkov
On Fri, Mar 22, 2019 at 5:38 AM John Naylor wrote: > On Thu, Mar 21, 2019 at 9:59 PM Alexander Korotkov > wrote: > > Attaches patches improving jsonpath parser. First one introduces > > cosmetic changes, while second gets rid of backtracking. I'm also > > planning to add high-level comment for

Re: speeding up planning with partitions

2019-03-22 Thread Jesper Pedersen
Hi Amit, On 3/22/19 3:39 AM, Amit Langote wrote: I took a stab at this. I think rearranging the code in make_partitionedrel_pruneinfo() slightly will take care of this. The problem is that make_partitionedrel_pruneinfo() does some work which involves allocating arrays containing nparts

Re: GiST VACUUM

2019-03-22 Thread Heikki Linnakangas
On 22/03/2019 13:37, Heikki Linnakangas wrote: On 21/03/2019 19:04, Heikki Linnakangas wrote: Attached is the latest patch version, to be applied on top of the IntegerSet patch. And committed. Thanks Andrey! This caused the buildfarm to go pink... I was able to reproduce it, by running the

Re: jsonpath

2019-03-22 Thread Nikita Glukhov
On 21.03.2019 16:58, Alexander Korotkov wrote: On Tue, Mar 19, 2019 at 8:10 PM Alexander Korotkov wrote: Attaches patches improving jsonpath parser. First one introduces cosmetic changes, while second gets rid of backtracking. I'm also planning to add high-level comment for both grammar and

Re: GiST VACUUM

2019-03-22 Thread Heikki Linnakangas
On 21/03/2019 19:04, Heikki Linnakangas wrote: Attached is the latest patch version, to be applied on top of the IntegerSet patch. And committed. Thanks Andrey! - Heikki

Re: Contribution to Perldoc for TestLib module in Postgres

2019-03-22 Thread Ramanarayana
Hi, Please find the first version of the patch for review. I was not sure what some of the functions are used for and marked them with TODO. Cheers Ram 4.0 v1_perldoc_testlib.patch Description: Binary data

Re: Special role for subscriptions

2019-03-22 Thread Petr Jelinek
Hi, On 22/03/2019 03:15, Andrey Borodin wrote: > >> 22 марта 2019 г., в 9:28, Michael Paquier написал(а): >> >> On Thu, Mar 21, 2019 at 10:06:03AM -0300, Euler Taveira wrote: >>> It will be really strange but I can live with that. Another idea is >>> CREATE bit to create subscriptions (without

Re: Offline enabling/disabling of data checksums

2019-03-22 Thread Michael Paquier
On Fri, Mar 22, 2019 at 10:04:02AM +0100, Michael Banck wrote: > How about this: > > + > + Notes > + > + When enabling checksums in a cluster, the operation can potentially take a > + long time if the data directory is large. During this operation, the > + cluster or other programs

Re: Performance issue in foreign-key-aware join estimation

2019-03-22 Thread David Rowley
Thanks for having a hack at this. On Fri, 22 Mar 2019 at 10:10, Tom Lane wrote: > I'm unsure how hard we should push to get something like this into v12. > I'm concerned that its dependency on list_nth might result in performance > regressions in some cases; it's a lot easier to believe that

Re: GiST VACUUM

2019-03-22 Thread Heikki Linnakangas
On 22/03/2019 10:00, Andrey Borodin wrote: 22 марта 2019 г., в 1:04, Heikki Linnakangas написал(а): PS. for Gist, we could almost use the LSN / NSN mechanism to detect the case that a deleted page is reused: Add a new field to the GiST page header, to store a new "deleteNSN" field. When a page

Re: pg_upgrade version checking questions

2019-03-22 Thread Christoph Berg
Re: Peter Eisentraut 2019-03-22 <57769959-8960-a9ca-fc9c-4dbb12629...@2ndquadrant.com> > I'm still in favor of defaulting --new-bindir appropriately. It seems > silly not to. We know where the directory is, we don't have to ask anyone. Fwiw I've been wondering why I have to pass that option

Re: [HACKERS] Can ICU be used for a database's default sort order?

2019-03-22 Thread Peter Eisentraut
On 2019-03-21 18:46, Marius Timmer wrote: > as I mentioned three weeks ago the patch from October 2018 did not apply > on the master. In the meantime I rebased it. Additionally I fixed some > Makefiles because a few icu-libs were missing. Now this patch applies > and compiles successfully on my

Re: pg_upgrade version checking questions

2019-03-22 Thread Peter Eisentraut
On 2019-03-19 16:51, Tom Lane wrote: > I'm not really getting your point here. Packagers ordinarily tie > those versions together anyway, I'd expect --- there's no upside > to not doing so, and plenty of risk if one doesn't, because of > exactly the sort of coordinated-changes hazard I'm talking

Re: Offline enabling/disabling of data checksums

2019-03-22 Thread Michael Banck
Hi, Am Freitag, den 22.03.2019, 17:37 +0900 schrieb Michael Paquier: > On Fri, Mar 22, 2019 at 09:13:43AM +0100, Michael Banck wrote: > > Don't we need a big warning that the cluster must not be started during > > operation of pg_checksums as well, now that we don't disallow it? > > The same

Re: Offline enabling/disabling of data checksums

2019-03-22 Thread Michael Paquier
On Fri, Mar 22, 2019 at 09:13:43AM +0100, Michael Banck wrote: > Don't we need a big warning that the cluster must not be started during > operation of pg_checksums as well, now that we don't disallow it? The same applies to pg_rewind and pg_basebackup, so I would classify that as a pilot error.

Re: Fwd: Add tablespace tap test to pg_rewind

2019-03-22 Thread Shaoqi Bai
On Fri, Mar 22, 2019 at 1:34 PM Michael Paquier wrote: > On Thu, Mar 21, 2019 at 11:41:01PM +0800, Shaoqi Bai wrote: > > Have updated the patch doing as you suggested > > + RewindTest::setup_cluster($test_mode, ['-g']); > + RewindTest::start_master(); > > There is no need to test for group

Re: selecting from partitions and constraint exclusion

2019-03-22 Thread Amit Langote
Hi David, Thanks for checking. On 2019/03/20 19:41, David Rowley wrote: > On Wed, 20 Mar 2019 at 17:37, Amit Langote > wrote: >> That's because get_relation_constraints() no longer (as of PG 11) includes >> the partition constraint for SELECT queries. But that's based on an >> assumption that

Re: Offline enabling/disabling of data checksums

2019-03-22 Thread Michael Banck
Hi, Am Freitag, den 22.03.2019, 09:27 +0900 schrieb Michael Paquier: > I have added in the docs a warning about a host crash while doing the > operation, with a recommendation to check the state of the checksums > on the data folder should it happen, and the previous portion of the > docs about

Re: Special role for subscriptions

2019-03-22 Thread Evgeniy Efimkin
Hi! > These are basically that the truncate, insert, delete and insert > rights for the role creating the subscription. Why would we actually > need that? It's for security reasons. Because possible to attack target server. If publication have system tables for instance pg_authid >

Re: GiST VACUUM

2019-03-22 Thread Andrey Borodin
> 22 марта 2019 г., в 1:04, Heikki Linnakangas написал(а): > ... > When I started testing this, I quickly noticed that empty pages were not > being deleted nearly as much as I expected. I tracked it to this check in > gistdeletepage: > >> + if (GistFollowRight(leafPage) >> +

Re: Connections hang indefinitely while taking a gin index's LWLock buffer_content lock

2019-03-22 Thread Alexander Korotkov
On Fri, Mar 22, 2019 at 12:06 AM Alvaro Herrera wrote: > On 2019-Mar-21, Alexander Korotkov wrote: > > > However, I think this still can be backpatched correctly. We can > > determine whether xlog record data contains deleteXid by its size. > > See the attached patch. I haven't test this yet.

Re: Libpq support to connect to standby server as priority

2019-03-22 Thread Haribabu Kommi
On Fri, Mar 22, 2019 at 7:32 AM Haribabu Kommi wrote: > > On Fri, Mar 22, 2019 at 6:57 AM Robert Haas wrote: > >> On Thu, Mar 21, 2019 at 2:26 AM Haribabu Kommi >> wrote: >> > Based on the above new options that can be added to >> target_session_attrs, >> > >> > primary - it is just an alias

  1   2   >