[HACKERS] Misleading comment in slru.h

Hi hackers, As mentioned in another thread[1], slru.h says: * Note: slru.c currently assumes that segment file names will be four hex * digits. This sets a lower bound on the segment size (64K transactions * for 32-bit TransactionIds). That comment is out of date: commit 638cf09e

[HACKERS] Incorrect mentions to pg_xlog in walmethods.c/h

Hi all, I have noticed $subject. A patch is attached. Those comments are not completely wrong either as pg_basebackup can generate pg_xlog as well, still I would recommend to just mention "pg_wal". Thanks, -- Michael walmethods-comments.patch Description: Binary data -- Sent via

Re: [HACKERS] Logical decoding on standby

On 21 June 2017 at 17:30, sanyam jain wrote: > Hi, > After changing > sendTimeLineIsHistoric = state->currTLI == ThisTimeLineID; > to > sendTimeLineIsHistoric = state->currTLI != ThisTimeLineID; > > I was facing another issue. > On promotion of a cascaded server

Re: [HACKERS] Logical decoding on standby

On 21 June 2017 at 13:28, sanyam jain wrote: > Hi, > > In this patch in walsender.c sendTimeLineIsHistoric is set to true when > current and ThisTimeLineID are equal. > > sendTimeLineIsHistoric = state->currTLI == ThisTimeLineID; > > > Shouldn't sendTimeLineIsHistoric is

[HACKERS] Out of date comment in predicate.c

Hi hackers, Commit ea9df812d8502fff74e7bc37d61bdc7d66d77a7f got rid of FirstPredicateLockMgrLock, but it's still referred to in a comment in predicate.c where the locking protocol is documented. I think it's probably best to use the name of the macro that's usually used to access the lock array

Re: [HACKERS] transition table behavior with inheritance appears broken

On Sat, Jun 24, 2017 at 10:57:49PM -0700, Noah Misch wrote: > On Fri, Jun 23, 2017 at 02:39:48PM +0100, Andrew Gierth wrote: > > > "Noah" == Noah Misch writes: > > > > Noah> This PostgreSQL 10 open item is past due for your status update. > > Noah> Kindly send a status

Re: [HACKERS] Logical decoding on standby

Hi, >After changing >sendTimeLineIsHistoric = state->currTLI == ThisTimeLineID; >to >sendTimeLineIsHistoric = state->currTLI != ThisTimeLineID; > >I was facing another issue. >On promotion of a cascaded server ThisTimeLineID in the standby server having >>logical slot becomes 0. >Then i added a

Re: [HACKERS] Modifing returning value of PQgetvalue.

On 6/24/17 06:31, Dmitry Igrishin wrote: > PQgetvalue returns a value of type char* (without const). But the > documentation says: > "The pointer returned by PQgetvalue points to storage that is part of > the PGresult structure. /One should not modify the data it points to/" > (my italics). Could

Re: [HACKERS] pg_basebackup fails on Windows when using tablespace mapping

On Tue, Jun 27, 2017 at 12:13 PM, Michael Paquier wrote: > At quick glance, I think that this should definitely be a client-only > fix. One reason is that pg_basebackup should be able to work with past > servers. A second is that this impacts as well the backend code,

Re: [HACKERS] distinct estimate of a hard-coded VALUES list

On Tue, Aug 23, 2016 at 5:28 AM, Tom Lane wrote: > Tomas Vondra writes: > > On 08/22/2016 07:42 PM, Alvaro Herrera wrote: > >> Also, if we patch it this way and somebody has a slow query because of a > >> lot of duplicate values, it's easy to

Re: [HACKERS] pg_basebackup fails on Windows when using tablespace mapping

On Tue, Jun 27, 2017 at 1:57 AM, nb wrote: > Trying to take a `pg_basebackup -T OLDDIR=NEWDIR [etc]` on master (server > running the cluster) fails on Windows with error "pg_basebackup: directory > "OLDDIR" exists but is not empty". Yes, I can see this error easily. > This

Re: [HACKERS] intermittent failures in Cygwin from select_parallel tests

On Mon, Jun 26, 2017 at 8:09 PM, Andrew Dunstan wrote: > > > On 06/26/2017 10:36 AM, Amit Kapila wrote: >> On Fri, Jun 23, 2017 at 9:12 AM, Andrew Dunstan >> wrote: >>> >>> On 06/22/2017 10:24 AM, Tom Lane wrote: Andrew Dunstan

Re: [HACKERS] Setting pd_lower in GIN metapage

On Mon, Jun 26, 2017 at 4:11 PM, Masahiko Sawada wrote: > Thank you for the patches! I checked additional patches for brin and > spgist. They look good to me. Last versions are still missing something: brin_mask() and spg_mask() can be updated so as mask_unused_space() is

Re: [HACKERS] Causal reads take II

On Sun, Jun 25, 2017 at 2:36 AM, Simon Riggs wrote: > I'm very happy that you are addressing this topic. > > I noticed you didn't put in links my earlier doubts about this > specific scheme, though I can see doubts from myself and Heikki at > least in the URLs. I maintain

Re: [HACKERS] Timing-sensitive case in src/test/recovery TAP tests

On Tue, Jun 27, 2017 at 3:44 AM, Tom Lane wrote: > Looks good as far as it goes, but I wonder whether any of the other > get_slot_xmins calls need polling too. Don't feel a need to add such > calls until someone exhibits a failure there, but I won't be very > surprised if

Re: [HACKERS] Reducing pg_ctl's reaction time

Andres Freund writes: > On 2017-06-26 17:38:03 -0400, Tom Lane wrote: >> Hm. Take that a bit further, and we could drop the connection probes >> altogether --- just put the whole responsibility on the postmaster to >> show in the pidfile whether it's ready for connections or

Re: [HACKERS] psql's \d and \dt are sending their complaints to different output files

> On 19 Jun 2017, at 17:32, Tom Lane wrote: > > So, if we're getting into enforcing consistency in describe.c, there's > lots to do. > > * listDbRoleSettings does this for a server too old to have the desired > feature: > > fprintf(pset.queryFout, >

Re: [HACKERS] Reducing pg_ctl's reaction time

On 2017-06-26 17:38:03 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2017-06-26 17:30:30 -0400, Tom Lane wrote: > >> No, I don't like that at all. Has race conditions against updates > >> coming from the startup process. > > > You'd obviously have to take the

Re: [HACKERS] Reducing pg_ctl's reaction time

Andres Freund writes: > On 2017-06-26 17:30:30 -0400, Tom Lane wrote: >> No, I don't like that at all. Has race conditions against updates >> coming from the startup process. > You'd obviously have to take the appropriate locks. I think the issue > here is less race

Re: [HACKERS] Reducing pg_ctl's reaction time

On 2017-06-26 17:30:30 -0400, Tom Lane wrote: > Andres Freund writes: > > It'd be quite possible to address the race-condition by moving the > > updating of the control file to postmaster, to the > > CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) block. That'd require > >

Re: [HACKERS] Reducing pg_ctl's reaction time

Andres Freund writes: > It'd be quite possible to address the race-condition by moving the > updating of the control file to postmaster, to the > CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) block. That'd require > updating the control file from postmaster, which'd be

Re: [HACKERS] Reducing pg_ctl's reaction time

Hi, On 2017-06-26 16:49:07 -0400, Tom Lane wrote: > Andres Freund writes: > > Arguably we could and should improve the logic when the server has > > started, right now it's pretty messy because we never treat a standby as > > up if hot_standby is disabled... > > True. If

Re: [HACKERS] Reducing pg_ctl's reaction time

Andres Freund writes: > Arguably we could and should improve the logic when the server has > started, right now it's pretty messy because we never treat a standby as > up if hot_standby is disabled... True. If you could tell the difference between "HS disabled" and "HS not

Re: [HACKERS] Pluggable storage

On Mon, Jun 26, 2017 at 10:55 PM, Tomas Vondra wrote: > On 06/26/2017 05:18 PM, Alexander Korotkov wrote: > >> I see that design question for PostgreSQL pluggable storages is very hard. >> > > IMHO it's mostly expected to be hard. > > Firstly, PostgreSQL is a

Re: [HACKERS] Reducing pg_ctl's reaction time

I wrote: > Andres Freund writes: >> It'd not be unreasonble to check pg_control first, and only after that >> indicates readyness check via the protocol. > Hm, that's a thought. The problem here isn't the frequency of checks, > but the log spam. Actually, that wouldn't help

Re: [HACKERS] Reducing pg_ctl's reaction time

On 2017-06-26 16:26:00 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2017-06-26 16:19:16 -0400, Tom Lane wrote: > >> Sure, what do you think an appropriate behavior would be? > > > It'd not be unreasonble to check pg_control first, and only after that > > indicates

Re: [HACKERS] Reducing pg_ctl's reaction time

Andres Freund writes: > On 2017-06-26 16:19:16 -0400, Tom Lane wrote: >> Sure, what do you think an appropriate behavior would be? > It'd not be unreasonble to check pg_control first, and only after that > indicates readyness check via the protocol. Hm, that's a thought.

Re: [HACKERS] \set AUTOROLLBACK ON

On Mon, Jun 26, 2017 at 12:35:47PM -0700, David G. Johnston wrote: > On Mon, Jun 26, 2017 at 12:19 PM, David Fetter wrote: > > > On Mon, Jun 26, 2017 at 04:00:55PM +0200, Joel Jacobson wrote: > > > Hi hackers, > > > > > > A colleague of mine wondered if there is a way to always

Re: [HACKERS] Reducing pg_ctl's reaction time

On 2017-06-26 16:19:16 -0400, Tom Lane wrote: > Jeff Janes writes: > > The 10 fold increase in log spam during long PITR recoveries is a bit > > unfortunate. > > > 9153 2017-06-26 12:55:40.243 PDT FATAL: the database system is starting up > > 9154 2017-06-26 12:55:40.345

Re: [HACKERS] Reducing pg_ctl's reaction time

Jeff Janes writes: > The 10 fold increase in log spam during long PITR recoveries is a bit > unfortunate. > 9153 2017-06-26 12:55:40.243 PDT FATAL: the database system is starting up > 9154 2017-06-26 12:55:40.345 PDT FATAL: the database system is starting up > 9156

Re: [HACKERS] Reducing pg_ctl's reaction time

On Mon, Jun 26, 2017 at 12:15 PM, Tom Lane wrote: > Michael Paquier writes: > > On Mon, Jun 26, 2017 at 7:13 AM, Tom Lane wrote: > >> The attached proposed patch adjusts pg_ctl to check every 100msec, > >> instead of every

Re: [HACKERS] Pluggable storage

Hi, On 06/26/2017 05:18 PM, Alexander Korotkov wrote: Hackers, I see that design question for PostgreSQL pluggable storages is very hard. IMHO it's mostly expected to be hard. Firstly, PostgreSQL is a mature product with many advanced features, and reworking a low-level feature without

Re: [HACKERS] \set AUTOROLLBACK ON

On Mon, Jun 26, 2017 at 12:19 PM, David Fetter wrote: > On Mon, Jun 26, 2017 at 04:00:55PM +0200, Joel Jacobson wrote: > > Hi hackers, > > > > A colleague of mine wondered if there is a way to always run > > everything you type into psql in a db txn and automatically rollback >

Re: [HACKERS] \set AUTOROLLBACK ON

On Mon, Jun 26, 2017 at 04:00:55PM +0200, Joel Jacobson wrote: > Hi hackers, > > A colleague of mine wondered if there is a way to always run > everything you type into psql in a db txn and automatically rollback > it as soon as it finish. > I couldn't think of any way to do so, but thought it

Re: [HACKERS] Reducing pg_ctl's reaction time

Michael Paquier writes: > On Mon, Jun 26, 2017 at 7:13 AM, Tom Lane wrote: >> The attached proposed patch adjusts pg_ctl to check every 100msec, >> instead of every second, for the postmaster to be done starting or >> stopping. >> +#define

Re: [HACKERS] Timing-sensitive case in src/test/recovery TAP tests

Michael Paquier writes: > On Mon, Jun 26, 2017 at 12:12 PM, Craig Ringer wrote: >> I'm not sure I understand this. > The patch attached may explain that better. Your patch added 3 poll > phases. I think that a 4th is needed. At the same time I

Re: [HACKERS] Another reason why the recovery tests take a long time

On 26 June 2017 at 19:06, Tom Lane wrote: > I wrote: >> So this looks like a pretty obvious race condition in the postmaster, >> which should be resolved by having it set a flag on receipt of >> PMSIGNAL_START_WALRECEIVER that's cleared only when it does start a >> new

Re: [HACKERS] Another reason why the recovery tests take a long time

I wrote: > So this looks like a pretty obvious race condition in the postmaster, > which should be resolved by having it set a flag on receipt of > PMSIGNAL_START_WALRECEIVER that's cleared only when it does start a > new walreceiver. Concretely, I propose the attached patch. Together with

Re: [HACKERS] Another reason why the recovery tests take a long time

On 2017-06-26 13:42:52 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2017-06-26 12:32:00 -0400, Tom Lane wrote: > >> ... But I wonder whether it's intentional that the old > >> walreceiver dies in the first place. That FATAL exit looks suspiciously > >> like it wasn't

Re: [HACKERS] Another reason why the recovery tests take a long time

Andres Freund writes: > On 2017-06-26 12:32:00 -0400, Tom Lane wrote: >> ... But I wonder whether it's intentional that the old >> walreceiver dies in the first place. That FATAL exit looks suspiciously >> like it wasn't originally-designed-in behavior. > It's quite

Re: [HACKERS] fix empty array expression in get_qual_for_list()

On Mon, Jun 26, 2017 at 8:14 PM, Tom Lane wrote: > Jeevan Ladhe writes: > > In case of list partitioned table: > > 1. If there is a partition accepting only null values and nothing else, > then > > currently the partition constraints for such a

[HACKERS] pg_basebackup fails on Windows when using tablespace mapping

Summary: Trying to take a `pg_basebackup -T OLDDIR=NEWDIR [etc]` on master (server running the cluster) fails on Windows with error "pg_basebackup: directory "OLDDIR" exists but is not empty". Version: 9.6.2, installed from Standard EDB package (built with MSVC). Repro steps: 1. Have a

Re: [HACKERS] Another reason why the recovery tests take a long time

Hi, On 2017-06-26 12:32:00 -0400, Tom Lane wrote: > I've found another edge-case bug through investigation of unexpectedly > slow recovery test runs. It goes like this: > > * While streaming from master to slave, test script shuts down master > while slave is left running. We soon restart the

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

On Mon, Jun 26, 2017 at 2:26 AM, Mark Rofail wrote: > *What I did:* > > > >- read into the old patch but couldn't apply it since it's quite old. >It needs to be rebased and that's what I am working on. It's a lot of >work. > - incomplete patch can be

[HACKERS] Another reason why the recovery tests take a long time

I've found another edge-case bug through investigation of unexpectedly slow recovery test runs. It goes like this: * While streaming from master to slave, test script shuts down master while slave is left running. We soon restart the master, but meanwhile: * slave's walreceiver process fails,

Re: [HACKERS] Pluggable storage

Hackers, I see that design question for PostgreSQL pluggable storages is very hard. BTW, I think it worth analyzing existing use-cases of pluggable storages. I think that the most famous DBMS with pluggable storage API is MySQL. This why I decided to start with it. I've added MySQL/MariaDB

Re: [HACKERS] intermittent failures in Cygwin from select_parallel tests

On 06/26/2017 10:45 AM, Tom Lane wrote: > Andrew Dunstan writes: >> On 06/23/2017 07:47 AM, Andrew Dunstan wrote: >>> Rerunning with some different settings to see if I can get separate cores. >> Numerous attempts to get core dumps following methods suggested in

Re: [HACKERS] intermittent failures in Cygwin from select_parallel tests

Andrew Dunstan writes: > On 06/23/2017 07:47 AM, Andrew Dunstan wrote: >> Rerunning with some different settings to see if I can get separate cores. > Numerous attempts to get core dumps following methods suggested in > Google searches have failed. The latest one

Re: [HACKERS] fix empty array expression in get_qual_for_list()

Jeevan Ladhe writes: > In case of list partitioned table: > 1. If there is a partition accepting only null values and nothing else, then > currently the partition constraints for such a partition are constructed as > "((a IS NULL) OR (a = ANY

Re: [HACKERS] intermittent failures in Cygwin from select_parallel tests

On 06/26/2017 10:36 AM, Amit Kapila wrote: > On Fri, Jun 23, 2017 at 9:12 AM, Andrew Dunstan > wrote: >> >> On 06/22/2017 10:24 AM, Tom Lane wrote: >>> Andrew Dunstan writes: Please let me know if there are tests I can run. I

Re: [HACKERS] Optional message to user when terminating/cancelling backend

On 06/26/2017 07:15 AM, Joel Jacobson wrote: +1 On Tue, Jun 20, 2017 at 8:54 PM, Alvaro Herrera wrote: Unless you have a lot of users running psql manually, I don't see how this is actually very useful or actionable. What would the user do with the information?

Re: [HACKERS] intermittent failures in Cygwin from select_parallel tests

On Fri, Jun 23, 2017 at 9:12 AM, Andrew Dunstan wrote: > > > On 06/22/2017 10:24 AM, Tom Lane wrote: >> Andrew Dunstan writes: >>> Please let me know if there are tests I can run. I missed your earlier >>> request in this thread,

Re: [HACKERS] Optional message to user when terminating/cancelling backend

+1 On Tue, Jun 20, 2017 at 8:54 PM, Alvaro Herrera wrote: > Unless you have a lot of users running psql manually, I don't see how > this is actually very useful or actionable. What would the user do with > the information? Hopefully your users already trust that you'd

Re: [HACKERS] intermittent failures in Cygwin from select_parallel tests

On 06/23/2017 07:47 AM, Andrew Dunstan wrote: > > On 06/23/2017 12:11 AM, Tom Lane wrote: >> Andrew Dunstan writes: >>> On 06/22/2017 10:24 AM, Tom Lane wrote: That earlier request is still valid. Also, if you can reproduce the symptom that lorikeet

[HACKERS] \set AUTOROLLBACK ON

Hi hackers, A colleague of mine wondered if there is a way to always run everything you type into psql in a db txn and automatically rollback it as soon as it finish. I couldn't think of any way to do so, but thought it would be a nice feature and probably quite easy to add to psql, so I thought

Re: [HACKERS] A mistake in a comment

Victor Drobny writes: > I believe that I have found a mistake in a comment to > parse_phrase_operator function. The comment has the following line: > a b (distance is no greater than X) > which is not. According to documentation and practical results, this > line

Re: [HACKERS] fix empty array expression in get_qual_for_list()

Here is an example: create table t1 ( a int) partition by list (a); create table t11 partition of t1 for values in (null); *Current constraints:* postgres=# \d+ t11; Table "public.t11" Column | Type | Collation | Nullable | Default | Storage | Stats target

[HACKERS] fix empty array expression in get_qual_for_list()

Hi, In case of list partitioned table: 1. If there is a partition accepting only null values and nothing else, then currently the partition constraints for such a partition are constructed as "((a IS NULL) OR (a = ANY (ARRAY[]::integer[])))". I think there it is better to avoid constructing an

Re: [HACKERS] Race between SELECT and ALTER TABLE NO INHERIT

Hello, At Mon, 26 Jun 2017 18:16:42 +0900, Amit Langote wrote in <7f5fe522-f328-3c40-565f-5e1ce3745...@lab.ntt.co.jp> > Hello, > > On 2017/06/26 17:46, Kyotaro HORIGUCHI wrote: > > Hello. > > > > I had a case of unexpected error caused by ALTER TABLE NO > >

Re: [HACKERS] Race between SELECT and ALTER TABLE NO INHERIT

Hello, On 2017/06/26 17:46, Kyotaro HORIGUCHI wrote: > Hello. > > I had a case of unexpected error caused by ALTER TABLE NO > INHERIT. > > =# CREATE TABLE p (a int); > =# CREATE TABLE c1 () INHERITS (p); > > session A=# BEGIN; > session A=# ALTER TABLE c1 NO INHERIT p; > > session B=# EXPLAIN

Re: [HACKERS] shift_sjis_2004 related autority files are remaining

Hi, At Sun, 25 Jun 2017 09:20:10 +0900 (JST), Tatsuo Ishii wrote in <20170625.092010.542143642647288693.t-is...@sraoss.co.jp> > > I don't have a clear opinion on this particular issue, but I think we > > should have clarity on why particular files or code exist. So why do >

[HACKERS] Race between SELECT and ALTER TABLE NO INHERIT

Hello. I had a case of unexpected error caused by ALTER TABLE NO INHERIT. =# CREATE TABLE p (a int); =# CREATE TABLE c1 () INHERITS (p); session A=# BEGIN; session A=# ALTER TABLE c1 NO INHERIT p; session B=# EXPLAIN ANALYZE SELECT * FROM p; (blocked) session A=# COMMIT; session B: ERROR:

[HACKERS] A mistake in a comment

Hello, I believe that I have found a mistake in a comment to parse_phrase_operator function. The comment has the following line: a b (distance is no greater than X) which is not. According to documentation and practical results, this line should me changed on something like: a b (distance

Re: [HACKERS] Optional message to user when terminating/cancelling backend

> On 22 Jun 2017, at 17:52, Dilip Kumar wrote: > > On Thu, Jun 22, 2017 at 7:18 PM, Daniel Gustafsson wrote: >> Good point. I’ve attached a new version which issues a NOTICE on truncation >> and also addresses all other comments so far in this thread.

[HACKERS] Remove old comments in dependencies.c and README.dependencies

Hi, I found some comments which are not implemented. As far as I have examined, these comments refer to min_group_size, but min_group_size was decided not to adopt and removed[1], so it seems these comments also should be removed. [1]

Re: [HACKERS] Setting pd_lower in GIN metapage

On Mon, Jun 26, 2017 at 3:54 PM, Amit Langote wrote: > On 2017/06/26 10:54, Michael Paquier wrote: >> On Fri, Jun 23, 2017 at 11:17 AM, Amit Langote >> wrote: >>> That was it, thanks for the pointer. >> >> GinInitMetabuffer() sets up

Re: [HACKERS] pg_filedump doesn't compile with v10 sources

Hi, On Mon, Jun 26, 2017 at 12:25 PM, tushar wrote: > Hi, > > While trying to do - make of pg_filedump against v10 sources , getting an > errors > > [centos@centos-cpula pg_filedump]$ make > cc -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions

[HACKERS] pg_filedump doesn't compile with v10 sources

Hi, While trying to do - make of pg_filedump against v10 sources , getting an errors [centos@centos-cpula pg_filedump]$ make cc -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -DLINUX_OOM_ADJ=0 -Wall

Re: [HACKERS] Setting pd_lower in GIN metapage

On 2017/06/26 10:54, Michael Paquier wrote: > On Fri, Jun 23, 2017 at 11:17 AM, Amit Langote > wrote: >> That was it, thanks for the pointer. > > GinInitMetabuffer() sets up pd_lower and pd_upper anyway using > PageInit so the check of PageIsVerified is guaranteed

Re: [HACKERS] Reducing pg_ctl's reaction time

On Mon, Jun 26, 2017 at 7:13 AM, Tom Lane wrote: > The attached proposed patch adjusts pg_ctl to check every 100msec, > instead of every second, for the postmaster to be done starting or > stopping. This cuts the runtime of the recovery TAP tests from around > 4m30s to around