Re: [HACKERS] Parallel pg_dump's error reporting doesn't work worth squat

2016-05-31 Thread Kyotaro HORIGUCHI
At Tue, 31 May 2016 12:29:50 -0400, Tom Lane wrote in <7445.1464712...@sss.pgh.pa.us> > Kyotaro HORIGUCHI writes: > > At Fri, 27 May 2016 13:20:20 -0400, Tom Lane wrote in > > <14603.1464369...@sss.pgh.pa.us> > >> Kyotaro HORIGUCHI writes: > >>> By the way, the reason of the "invalid snapshot

Re: [HACKERS] foreign table batch inserts

2016-05-31 Thread Etsuro Fujita
On 2016/05/31 14:53, Amit Langote wrote: On 2016/05/30 22:59, Craig Ringer wrote: On 30 May 2016 at 16:17, Etsuro Fujita wrote: That's a good point, but the basic idea is to send the local query almost-as-is to the remote server if possible. For example, if the local query is "INSERT INTO f

Re: [HACKERS] Question and suggestion about application binary compatibility policy

2016-05-31 Thread Craig Ringer
On 1 June 2016 at 13:09, Tsunakawa, Takayuki wrote: > From: pgsql-hackers-ow...@postgresql.org [mailto: > pgsql-hackers-ow...@postgresql.org] On Behalf Of Craig Ringer > > While that's probably OK, it's not especially desirable. The typical > Windows deployment model involves the application bund

Re: [HACKERS] Question and suggestion about application binary compatibility policy

2016-05-31 Thread Tsunakawa, Takayuki
From: pgsql-hackers-ow...@postgresql.org [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Craig Ringer While that's probably OK, it's not especially desirable. The typical Windows deployment model involves the application bundling all its direct dependencies except when those are provide

Re: [HACKERS] Statement timeout

2016-05-31 Thread Craig Ringer
On 1 June 2016 at 08:33, Tatsuo Ishii wrote: > > FWIW, I think the existing behavior is just fine. It corresponds to what > > PQexec has always done with multi-statement query strings; that is, > > statement_timeout governs the total time to execute the transaction (the > > whole query string, u

Re: [HACKERS] Question and suggestion about application binary compatibility policy

2016-05-31 Thread Craig Ringer
On 30 May 2016 at 11:04, Tsunakawa, Takayuki < tsunakawa.ta...@jp.fujitsu.com> wrote: > Hello, > > I'd like to ask you about the policy of application binary compatibility. > And have a suggestion as well. > > QUESTION > == > > My customer asked me

[HACKERS] Change in order of criteria - reg

2016-05-31 Thread sri harsha
Hi, In PostgreSQL , does the order in which the criteria is given matter ?? For example Query 1 : Select * from TABLE where a > 5 and b < 10; Query 2 : Select * from TABLE where b <10 and a > 5; Are query 1 and query 2 the same in PostgreSQL or different ?? If its different , WHY ?? Than

Re: [HACKERS] PostmasterPid not marked with PGDLLIMPORT

2016-05-31 Thread Craig Ringer
On 1 June 2016 at 11:48, Michael Paquier wrote: > Could it be possible to mark PostmasterPid with PGDLLIMPORT on HEAD > and back-branches? > Sounds sensible to me. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

[HACKERS] PostmasterPid not marked with PGDLLIMPORT

2016-05-31 Thread Michael Paquier
Hi all, While hacking a background worker for Windows/Linux that is sending signals to the Postmaster depending on the state of the server where Postgres is running (particularly after a certain size threshold is reached on the partition of PGDATA SIGINT is sent to PostmasterPid to have it stop cl

Re: [HACKERS] Question and suggestion about application binary compatibility policy

2016-05-31 Thread Tsunakawa, Takayuki
> From: Michael Meskes [mailto:mes...@postgresql.org] > e.g. a random hit from google:=C2=A0https://www.bottomupcs.com/libra > ries_and_the_linker.xhtml > > There even is a wikipedia page about > it:=C2=A0https://en.wikipedia.org/wiki/ > Soname Thank you for good pointers. The former is particul

Re: [HACKERS] Performance degradation in commit ac1d794

2016-05-31 Thread Robert Haas
On Tue, May 31, 2016 at 10:23 PM, Tom Lane wrote: > Noah Misch writes: >> On Tue, May 31, 2016 at 10:09:05PM -0400, Robert Haas wrote: >>> What I *think* is going on here is: >>> - ac1d794 lowered performance >>> - backend_flush_after with a non-zero default lowered performance with >>> a vengean

Re: [HACKERS] parallel.c is not marked as test covered

2016-05-31 Thread Noah Misch
On Sun, May 29, 2016 at 01:31:24AM -0400, Noah Misch wrote: > On Sun, May 15, 2016 at 12:53:13PM +, Clément Prévost wrote: > > On Mon, May 9, 2016 at 4:50 PM Andres Freund wrote: > > > I think it's a good idea to run a force-parallel run on some buildfarm > > > members. But I'm rather convince

Re: [HACKERS] Reviewing freeze map code

2016-05-31 Thread Robert Haas
On Sun, May 29, 2016 at 1:44 AM, Noah Misch wrote: > On Fri, May 06, 2016 at 04:42:48PM -0400, Robert Haas wrote: >> On Thu, May 5, 2016 at 2:20 PM, Andres Freund wrote: >> > On 2016-05-02 14:48:18 -0700, Andres Freund wrote: >> > + charnew_vmbuf[BLCKSZ]; >> > +

Re: [HACKERS] Performance degradation in commit ac1d794

2016-05-31 Thread Tom Lane
Noah Misch writes: > On Tue, May 31, 2016 at 10:09:05PM -0400, Robert Haas wrote: >> What I *think* is going on here is: >> - ac1d794 lowered performance >> - backend_flush_after with a non-zero default lowered performance with >> a vengeance >> - 98a64d0 repaired the damage done by ac1d794, or mu

Re: [HACKERS] Performance degradation in commit ac1d794

2016-05-31 Thread Noah Misch
On Tue, May 31, 2016 at 10:09:05PM -0400, Robert Haas wrote: > On Sun, May 29, 2016 at 1:40 AM, Noah Misch wrote: > > On Fri, Dec 25, 2015 at 08:08:15PM +0300, Васильев Дмитрий wrote: > >> I suddenly found commit ac1d794 gives up to 3 times performance > >> degradation. > >> > >> I tried to run p

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Robert Haas
On Tue, May 31, 2016 at 10:13 PM, Tom Lane wrote: > Robert Haas writes: >> On Tue, May 31, 2016 at 10:02 PM, Tom Lane wrote: >>> The reloption does not set an exact value, according to the code: > >> True, max_parallel_degree is an overriding limit. But the point is >> that, without the relopti

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Tom Lane
Robert Haas writes: > On Tue, May 31, 2016 at 10:02 PM, Tom Lane wrote: >> The reloption does not set an exact value, according to the code: > True, max_parallel_degree is an overriding limit. But the point is > that, without the reloption, you can't get lots of workers on a small > table. The

Re: [HACKERS] Hard to maintain duplication in contain_volatile_functions_not_nextval_walker

2016-05-31 Thread Robert Haas
On Fri, May 27, 2016 at 10:36 PM, Amit Kapila wrote: > On Sat, May 28, 2016 at 12:28 AM, Andres Freund wrote: >> contain_volatile_functions_walker is duplicated, near entirely, in >> contain_volatile_functions_not_nextval_walker. > > Previously, I also had same observation. > >> Wouldn't it have

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Robert Haas
On Tue, May 31, 2016 at 10:02 PM, Tom Lane wrote: > Robert Haas writes: >> Now, this case is a little trickier. If we called it simply >> parallel_degree rather than max_parallel_degree, then it would have >> the same name as the reloption. But the reloption sets an exact >> value, and the GUC

Re: [HACKERS] Performance degradation in commit ac1d794

2016-05-31 Thread Robert Haas
On Sun, May 29, 2016 at 1:40 AM, Noah Misch wrote: > On Fri, Dec 25, 2015 at 08:08:15PM +0300, Васильев Дмитрий wrote: >> I suddenly found commit ac1d794 gives up to 3 times performance degradation. >> >> I tried to run pgbench -s 1000 -j 48 -c 48 -S -M prepared on 70 CPU-core >> machine: >> commi

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Tom Lane
Robert Haas writes: > Now, this case is a little trickier. If we called it simply > parallel_degree rather than max_parallel_degree, then it would have > the same name as the reloption. But the reloption sets an exact > value, and the GUC sets a cap, which is a significant difference. The relop

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Robert Haas
On Tue, May 31, 2016 at 8:51 PM, Peter Eisentraut wrote: > On 5/31/16 4:04 PM, Tom Lane wrote: >> The name should be closely related to what we use for #3. I could go for >> max_total_parallel_workers for #2 and max_parallel_workers for #3. >> Or maybe max_parallel_workers_total? > > Most cluster

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Robert Haas
On Tue, May 31, 2016 at 5:58 PM, Tom Lane wrote: > Alvaro Herrera writes: >> Robert Haas wrote: >>> I just want to point out that if we change #1, we're breaking >>> postgresql.conf compatibility for, IMHO, not a whole lot of benefit. >>> I'd just leave it alone. > >> We can add the old name as a

Re: [HACKERS] Statement timeout

2016-05-31 Thread Amit Kapila
On Wed, Jun 1, 2016 at 6:03 AM, Tatsuo Ishii wrote: > > From the document about statement_timeout (config.sgml): > > Abort any statement that takes more than the specified number of > milliseconds, starting from the time the command arrives at the server > from the client.

Re: [HACKERS] Question and suggestion about application binary compatibility policy

2016-05-31 Thread Tsunakawa, Takayuki
> From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Marco Atzeri > on cygwin the postgresql binary package already include the library > versions: > >/usr/bin/cygecpg-6.dll >/usr/bin/cygecpg_compat-3.dll >/usr/bin/cygpgtypes-3.dll >

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Peter Eisentraut
On 5/31/16 4:04 PM, Tom Lane wrote: The name should be closely related to what we use for #3. I could go for max_total_parallel_workers for #2 and max_parallel_workers for #3. Or maybe max_parallel_workers_total? Most cluster-wide settings like this are named max_something (max_connections, m

Re: [HACKERS] Rename synchronous_standby_names?

2016-05-31 Thread Michael Paquier
On Wed, Jun 1, 2016 at 3:56 AM, David G. Johnston wrote: > On Tue, May 31, 2016 at 2:19 PM, Peter Eisentraut > wrote: >> >> On 5/31/16 1:47 PM, Jaime Casanova wrote: >>> >>> Are we going to change synchronous_standby_names? Certainly the GUC is >>> not *only* a list of names anymore. >>> >>> sync

[HACKERS] User demand, and idea, for C-code conversions from JSON arrays to PostgreSQL arrays

2016-05-31 Thread David G. Johnston
All, Oven in the "JSON[B] arrays are second-class citizens" thread [1] I made the observation that the only way to get a PostgreSQL array from a JSON array is via the "elements->cast->array_agg" chain. For JSON arrays that are homogeneous in nature the capability to go "directly" from JSON to jso

Re: [HACKERS] Statement timeout

2016-05-31 Thread Tatsuo Ishii
> FWIW, I think the existing behavior is just fine. It corresponds to what > PQexec has always done with multi-statement query strings; that is, > statement_timeout governs the total time to execute the transaction (the > whole query string, unless you put transaction control commands in there). >

Re: [HACKERS] COMMENT ON, psql and access methods

2016-05-31 Thread Michael Paquier
On Wed, Jun 1, 2016 at 4:52 AM, Robert Haas wrote: > On Fri, May 27, 2016 at 7:53 AM, Michael Paquier > wrote: >> As far as I can see, COMMENT ON has no support for access methods. >> Wouldn't we want to add it as it is created by a command? On top of >> that, perhaps we could have a backslash co

Re: [HACKERS] JSON[B] arrays are second-class citizens

2016-05-31 Thread Peter van Hardenberg
The idea of converting a JSONB array to a PG array is appealing and would potentially be more general-purpose than adding a new unnest. I'm not sure how feasible either suggestion is. I will say that I think the current state of affairs is gratuitously verbose and expects users to memorize a subst

Re: [HACKERS] Parallel safety tagging of extension functions

2016-05-31 Thread Andreas Karlsson
On 05/31/2016 06:47 PM, Tom Lane wrote: Given that, your original approach of manually updating proargtypes in the existing pg_proc row for the functions may be the best way. Anything else is going to be more complicated and ultimately will still require at least one direct catalog update. It

Re: [HACKERS] JSON[B] arrays are second-class citizens

2016-05-31 Thread Tom Lane
David Fetter writes: > On Tue, May 31, 2016 at 05:06:00PM -0400, David G. Johnston wrote: >> While likely not that common the introduction of an ambiguity makes >> raises the bar considerably. > What ambiguity? My first thought about it was that select unnest('{1,2,3}'); would start failing.

Re: [HACKERS] JSON[B] arrays are second-class citizens

2016-05-31 Thread David G. Johnston
On Tue, May 31, 2016 at 5:46 PM, David Fetter wrote: > On Tue, May 31, 2016 at 05:06:00PM -0400, David G. Johnston wrote: > > On Tue, May 31, 2016 at 4:34 PM, David Fetter wrote: > > > > > Folks, > > > > > > While querying some JSONB blobs at work in preparation for a massive > > > rework of the

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Tom Lane
Alvaro Herrera writes: > Robert Haas wrote: >> I just want to point out that if we change #1, we're breaking >> postgresql.conf compatibility for, IMHO, not a whole lot of benefit. >> I'd just leave it alone. > We can add the old name as a synonym in guc.c to maintain compatibility. I doubt this

Re: [HACKERS] JSON[B] arrays are second-class citizens

2016-05-31 Thread David Fetter
On Tue, May 31, 2016 at 05:06:00PM -0400, David G. Johnston wrote: > On Tue, May 31, 2016 at 4:34 PM, David Fetter wrote: > > > Folks, > > > > While querying some JSONB blobs at work in preparation for a massive > > rework of the data infrastructure, I ran into things that really > > puzzled me,

Re: [HACKERS] JSON[B] arrays are second-class citizens

2016-05-31 Thread Corey Huinker
On Tue, May 31, 2016 at 5:06 PM, David G. Johnston < david.g.johns...@gmail.com> wrote: > On Tue, May 31, 2016 at 4:34 PM, David Fetter wrote: > >> Folks, >> >> While querying some JSONB blobs at work in preparation for a massive >> rework of the data infrastructure, I ran into things that really

Re: [HACKERS] JSON[B] arrays are second-class citizens

2016-05-31 Thread David G. Johnston
On Tue, May 31, 2016 at 4:34 PM, David Fetter wrote: > Folks, > > While querying some JSONB blobs at work in preparation for a massive > rework of the data infrastructure, I ran into things that really > puzzled me, to wit: > > SELECT * FROM unnest('["a","b","c"]'::jsonb); > ERROR: function unne

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread David G. Johnston
On Tue, May 31, 2016 at 4:12 PM, Robert Haas wrote: > On Tue, May 31, 2016 at 4:04 PM, Tom Lane wrote: > > Alvaro Herrera writes: > >> Robert Haas wrote: > >>> So I think in the long run we should have three limits: > >>> > >>> 1. Cluster-wide limit on number of worker processes for all purpose

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Peter Geoghegan
On Tue, May 31, 2016 at 1:27 PM, Alvaro Herrera wrote: > This is a very good point. > > I think parallel maintenance commands are going to require different > tuning than different queries, and I'd rather have separate parameters > for those things rather than force the same parameter being change

[HACKERS] JSON[B] arrays are second-class citizens

2016-05-31 Thread David Fetter
Folks, While querying some JSONB blobs at work in preparation for a massive rework of the data infrastructure, I ran into things that really puzzled me, to wit: SELECT * FROM unnest('["a","b","c"]'::jsonb); ERROR: function unnest(jsonb) does not exist SELECT * FROM jsonb_array_elements('["a","b

Re: [HACKERS] [GENERAL] Permission Denied Error on pg_xlog/RECOVERYXLOG file

2016-05-31 Thread Jeffrey.Marshall
Hi Folks! The permissions on the RECOVERYXLOG file at the time of the error are 0400: -r. 1 postgres postgres 16777216 May 31 09:51 RECOVERYXLOG I sent that info to Tom earlier this afternoon (still learning the posting protocols - sorry) - his response is below: From Tom: Ah, that con

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Alvaro Herrera
Robert Haas wrote: > I just want to point out that if we change #1, we're breaking > postgresql.conf compatibility for, IMHO, not a whole lot of benefit. > I'd just leave it alone. We can add the old name as a synonym in guc.c to maintain compatibility. > I would propose to call #2 max_parallel_

Re: [HACKERS] [PATCH][Documination] Add optional USING keyword before opclass name in INSERT statemet

2016-05-31 Thread David G. Johnston
On Tue, May 31, 2016 at 3:55 PM, Nikolay Shaplov wrote: > В письме от 31 мая 2016 15:38:38 пользователь Robert Haas написал: > > > >>> 99% of the time, you'd be right. But this is an unusual case, for > the > > >>> reasons I mentioned before. > > >> > > >> I tend to agree with Nikolay. I can't

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Robert Haas
On Tue, May 31, 2016 at 4:04 PM, Tom Lane wrote: > Alvaro Herrera writes: >> Robert Haas wrote: >>> So I think in the long run we should have three limits: >>> >>> 1. Cluster-wide limit on number of worker processes for all purposes >>> (currently, max_worker_processes). >>> >>> 2. Cluster-wide l

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Josh berkus
On 05/31/2016 01:04 PM, Tom Lane wrote: > The name should be closely related to what we use for #3. I could go for > max_total_parallel_workers for #2 and max_parallel_workers for #3. > Or maybe max_parallel_workers_total? How about parallel_worker_pool? -- -- Josh Berkus Red Hat OSAS (any opin

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Tom Lane
Alvaro Herrera writes: > Robert Haas wrote: >> So I think in the long run we should have three limits: >> >> 1. Cluster-wide limit on number of worker processes for all purposes >> (currently, max_worker_processes). >> >> 2. Cluster-wide limit on number of worker processes for parallelism >> (do

Re: [HACKERS] Perf Benchmarking and regression.

2016-05-31 Thread Robert Haas
On Fri, May 27, 2016 at 12:37 AM, Andres Freund wrote: > I don't think the situation is quite that simple. By *disabling* backend > flushing it's also easy to see massive performance regressions. In > situations where shared buffers was configured appropriately for the workload > (not the case

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Peter Geoghegan
On Tue, May 31, 2016 at 12:55 PM, Alvaro Herrera wrote: > Agreed -- things like pglogical and BDR rely on background workers to do > their jobs. Many other users of bgworkers have popped up, so I think > it'd be a bad idea if parallel queries are able to monopolize all the > available slots. Tha

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Alvaro Herrera
Robert Haas wrote: > Also, I think that we might actually want to add an > additional GUC to prevent the parallel query system from consuming the > entire pool of processes established by max_worker_processes. If > you're doing anything else with worker processes on your system, you > might well

Re: [HACKERS] [PATCH][Documination] Add optional USING keyword before opclass name in INSERT statemet

2016-05-31 Thread Nikolay Shaplov
В письме от 31 мая 2016 15:38:38 пользователь Robert Haas написал: > >>> 99% of the time, you'd be right. But this is an unusual case, for the > >>> reasons I mentioned before. > >> > >> I tend to agree with Nikolay. I can't see much upside in making this > >> change. At best, nothing will bre

Re: [HACKERS] COMMENT ON, psql and access methods

2016-05-31 Thread Robert Haas
On Fri, May 27, 2016 at 7:53 AM, Michael Paquier wrote: > As far as I can see, COMMENT ON has no support for access methods. > Wouldn't we want to add it as it is created by a command? On top of > that, perhaps we could have a backslash command in psql to list the > supported access methods, like

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread David G. Johnston
On Tue, May 31, 2016 at 3:35 PM, Robert Haas wrote: > On Tue, May 31, 2016 at 3:19 PM, Tom Lane wrote: > > I wrote: > >> At the risk of opening another can of worms, what about renaming > >> max_worker_processes as well? It would be a good thing if that > >> had "cluster" in it somewhere, or so

Re: [HACKERS] [PATCH][Documination] Add optional USING keyword before opclass name in INSERT statemet

2016-05-31 Thread Robert Haas
On Tue, May 31, 2016 at 12:34 PM, Tom Lane wrote: > Robert Haas writes: >> On Thu, May 26, 2016 at 3:28 PM, Tom Lane wrote: >>> 99% of the time, you'd be right. But this is an unusual case, for the >>> reasons I mentioned before. > >> I tend to agree with Nikolay. I can't see much upside in ma

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread David G. Johnston
On Tue, May 31, 2016 at 3:19 PM, Tom Lane wrote: > I wrote: > > At the risk of opening another can of worms, what about renaming > > max_worker_processes as well? It would be a good thing if that > > had "cluster" in it somewhere, or something that indicates it's a > > system-wide value not a pe

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Robert Haas
On Tue, May 31, 2016 at 3:19 PM, Tom Lane wrote: > I wrote: >> At the risk of opening another can of worms, what about renaming >> max_worker_processes as well? It would be a good thing if that >> had "cluster" in it somewhere, or something that indicates it's a >> system-wide value not a per-ses

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Tom Lane
I wrote: > At the risk of opening another can of worms, what about renaming > max_worker_processes as well? It would be a good thing if that > had "cluster" in it somewhere, or something that indicates it's a > system-wide value not a per-session value. "max_workers_per_cluster" > would answer, t

Re: [HACKERS] Re: PATCH: Split stats file per database WAS: autovacuum stress-testing our system

2016-05-31 Thread Tom Lane
Tomas Vondra writes: > I've checked how this worked in 9.2 (before the 9.3 patch that split the > file per db), and back then last_statsrequest (transformed to > request_time) was used to decide whether we need to write something. But > now we do that by simply checking whether the list is empt

Re: [HACKERS] Logic behind parallel default? WAS: Rename max_parallel_degree?

2016-05-31 Thread Tom Lane
Josh berkus writes: > On 05/31/2016 11:10 AM, Tom Lane wrote: >> The 9.6 open-items list cites >> https://www.postgresql.org/message-id/flat/20160420174631.3qjjhpwsvvx5b...@alap3.anarazel.de > Looks like we didn't decide for the release, just the beta. Indeed. I think it's premature to have thi

Re: [HACKERS] Rename synchronous_standby_names?

2016-05-31 Thread David G. Johnston
On Tue, May 31, 2016 at 2:19 PM, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 5/31/16 1:47 PM, Jaime Casanova wrote: > >> Are we going to change synchronous_standby_names? Certainly the GUC is >> not *only* a list of names anymore. >> >> synchronous_standby_config? >> synchrono

Re: [HACKERS] Re: PATCH: Split stats file per database WAS: autovacuum stress-testing our system

2016-05-31 Thread Tomas Vondra
On 05/31/2016 07:24 PM, Tom Lane wrote: Tomas Vondra writes: On 05/31/2016 06:59 PM, Tom Lane wrote: I'm confused here --- are you speaking of having removed if (msg->cutoff_time > req->request_time) req->request_time = msg->cutoff_time; ? That is not a check for clock

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread David G. Johnston
On Tue, May 31, 2016 at 2:37 PM, Josh berkus wrote: > On 05/31/2016 11:27 AM, Peter Geoghegan wrote: > > On Tue, May 31, 2016 at 11:22 AM, Josh berkus wrote: > >>> I think we can hope that developers are going to be less confused about > >>> that than users. > >> > >> Makes sense. > > > > Maybe

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Tom Lane
Josh berkus writes: > On 05/31/2016 11:29 AM, Tom Lane wrote: >> Josh berkus writes: >>> One more consistency question: what's the effect of running out of >>> max_parallel_workers? >> ITYM max_worker_processes (ie, the cluster-wide pool size)? > Yes. Sorry for contributing to the confusion.

Re: [HACKERS] Logic behind parallel default? WAS: Rename max_parallel_degree?

2016-05-31 Thread Josh berkus
On 05/31/2016 11:10 AM, Tom Lane wrote: > Josh berkus writes: >> Is there a thread on how we determined this default of 2? I can't find >> one under likely search terms. > > The 9.6 open-items list cites > > https://www.postgresql.org/message-id/flat/20160420174631.3qjjhpwsvvx5b...@alap3.anaraz

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Josh berkus
On 05/31/2016 11:29 AM, Tom Lane wrote: > Josh berkus writes: >> One more consistency question: what's the effect of running out of >> max_parallel_workers? > > ITYM max_worker_processes (ie, the cluster-wide pool size)? Yes. Sorry for contributing to the confusion. Too many similar-sounding p

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Josh berkus
On 05/31/2016 11:27 AM, Peter Geoghegan wrote: > On Tue, May 31, 2016 at 11:22 AM, Josh berkus wrote: >>> I think we can hope that developers are going to be less confused about >>> that than users. >> >> Makes sense. > > Maybe EXPLAIN doesn't have to use the term parallel worker at all. It > can

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Tom Lane
Josh berkus writes: > One more consistency question: what's the effect of running out of > max_parallel_workers? ITYM max_worker_processes (ie, the cluster-wide pool size)? > That is, say max_parallel_workers is set to 10, and 8 are already > allocated. If I ask for max_parallel_X = 4, how many

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread David G. Johnston
On Tue, May 31, 2016 at 2:22 PM, Josh berkus wrote: > On 05/31/2016 11:17 AM, Peter Eisentraut wrote: > > On 5/31/16 2:02 PM, Josh berkus wrote: > >> I get where you're coming from, but I think Haas's query plan output is > >> going to show us the confusion we're going to get. So we need to eith

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Peter Geoghegan
On Tue, May 31, 2016 at 11:22 AM, Josh berkus wrote: >> I think we can hope that developers are going to be less confused about >> that than users. > > Makes sense. Maybe EXPLAIN doesn't have to use the term parallel worker at all. It can instead use a slightly broader terminology, possibly inclu

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Josh berkus
On 05/31/2016 11:17 AM, Peter Eisentraut wrote: > On 5/31/16 2:02 PM, Josh berkus wrote: >> I get where you're coming from, but I think Haas's query plan output is >> going to show us the confusion we're going to get. So we need to either >> change the parameter, the explain output, or brace ourse

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Tom Lane
"David G. Johnston" writes: > On Tuesday, May 31, 2016, Tom Lane wrote: >> I really think that a GUC named "max_parallel_workers", which in fact >> limits the number of workers and not something else, is the way to go. > What is your opinion on the internal name for this? Leave it as "degree"?

Re: [HACKERS] Rename synchronous_standby_names?

2016-05-31 Thread Peter Eisentraut
On 5/31/16 1:47 PM, Jaime Casanova wrote: Are we going to change synchronous_standby_names? Certainly the GUC is not *only* a list of names anymore. synchronous_standby_config? synchronous_standbys (adjust to correct english if necesary)? If the existing values are still going to be accepted,

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Peter Geoghegan
On Tue, May 31, 2016 at 11:17 AM, Peter Eisentraut wrote: > Changing the explain output doesn't sound so bad to me. > > The users' problem is that the parameter setting ought to match the EXPLAIN > output. > > The developers' problem is that the EXPLAIN output actually corresponds to > leader + (N

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Tom Lane
"David G. Johnston" writes: > On Tuesday, May 31, 2016, Tom Lane wrote: >>> I really think that a GUC named "max_parallel_workers", which in fact >>> limits the number of workers and not something else, is the way to go. > If going this route I'd still rather add the word "assisting" > or "addit

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread David G. Johnston
On Tuesday, May 31, 2016, Peter Geoghegan wrote: > On Tue, May 31, 2016 at 11:02 AM, Josh berkus > wrote: > > I get where you're coming from, but I think Haas's query plan output is > > going to show us the confusion we're going to get. So we need to either > > change the parameter, the explain

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Peter Eisentraut
On 5/31/16 2:02 PM, Josh berkus wrote: I get where you're coming from, but I think Haas's query plan output is going to show us the confusion we're going to get. So we need to either change the parameter, the explain output, or brace ourselves for endless repeated questions. Changing the expla

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread David G. Johnston
On Tuesday, May 31, 2016, Tom Lane wrote: > I wrote: > > I really think that a GUC named "max_parallel_workers", which in fact > > limits the number of workers and not something else, is the way to go. > > To be concrete, I suggest comparing the attached documentation patch > with Robert's. Whic

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Peter Geoghegan
On Tue, May 31, 2016 at 11:09 AM, Peter Geoghegan wrote: > The only reason I favor defining parallel_degree = 1 I meant "redefining max_parallel_degree =1 to effectively disable parallel query", of course. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.o

Re: [HACKERS] Logic behind parallel default? WAS: Rename max_parallel_degree?

2016-05-31 Thread Joshua D. Drake
On 05/31/2016 11:05 AM, Josh berkus wrote: On 05/31/2016 11:00 AM, Tom Lane wrote: ! If this occurs, the plan will run with fewer workers than expected, ! which may be inefficient. The default value is 2. Setting this ! value to 0 disables parallel query execution.

Re: [HACKERS] Logic behind parallel default? WAS: Rename max_parallel_degree?

2016-05-31 Thread Tom Lane
Josh berkus writes: > Is there a thread on how we determined this default of 2? I can't find > one under likely search terms. The 9.6 open-items list cites https://www.postgresql.org/message-id/flat/20160420174631.3qjjhpwsvvx5b...@alap3.anarazel.de regards, tom lane -

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Peter Geoghegan
On Tue, May 31, 2016 at 11:02 AM, Josh berkus wrote: > I get where you're coming from, but I think Haas's query plan output is > going to show us the confusion we're going to get. So we need to either > change the parameter, the explain output, or brace ourselves for endless > repeated questions.

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Tom Lane
Peter Geoghegan writes: > Even when the leader is consuming input from workers, that's still perhaps > pegging one CPU core. So, it doesn't really invalidate what I said about > the number of cores being the primary consideration. Agreed, but if we think that people need to be thinking in those t

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread David G. Johnston
On Tuesday, May 31, 2016, Tom Lane wrote: > Josh berkus > writes: > > On 05/31/2016 10:16 AM, Peter Geoghegan wrote: > >> But the distinction between parallel workers and backends that can > >> participate in parallel query does need to be user-visible. Worker > >> processes are a commodity (i.e.

Re: [HACKERS] Logic behind parallel default? WAS: Rename max_parallel_degree?

2016-05-31 Thread Josh berkus
On 05/31/2016 11:00 AM, Tom Lane wrote: > ! If this occurs, the plan will run with fewer workers than expected, > ! which may be inefficient. The default value is 2. Setting this > ! value to 0 disables parallel query execution. Is there a thread on how we determined t

Re: [HACKERS] Rename synchronous_standby_names?

2016-05-31 Thread Tom Lane
Jaime Casanova writes: > Are we going to change synchronous_standby_names? Certainly the GUC is > not *only* a list of names anymore. > synchronous_standby_config? > synchronous_standbys (adjust to correct english if necesary)? I could get behind renaming it to synchronous_standby_config ...

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Josh berkus
On 05/31/2016 10:51 AM, Peter Geoghegan wrote: > On Tue, May 31, 2016 at 10:46 AM, Josh berkus wrote: >> In parallel seq scan and join, do the "masters" behave as workers as well? > > It depends. They will if they can. If the parallel seq scan leader > isn't getting enough work to do from workers

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Tom Lane
I wrote: > I really think that a GUC named "max_parallel_workers", which in fact > limits the number of workers and not something else, is the way to go. To be concrete, I suggest comparing the attached documentation patch with Robert's. Which one is more understandable? (I have not bothered pre

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Peter Geoghegan
On Tue, May 31, 2016 at 10:46 AM, Josh berkus wrote: > In parallel seq scan and join, do the "masters" behave as workers as well? It depends. They will if they can. If the parallel seq scan leader isn't getting enough work to do from workers (enough tuples to process from the shared memory queue)

[HACKERS] Rename synchronous_standby_names?

2016-05-31 Thread Jaime Casanova
Hi, Are we going to change synchronous_standby_names? Certainly the GUC is not *only* a list of names anymore. synchronous_standby_config? synchronous_standbys (adjust to correct english if necesary)? -- Jaime Casanova www.2ndQuadrant.com PostgreSQL Development, 24x7 Suppor

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Josh berkus
On 05/31/2016 10:38 AM, Peter Geoghegan wrote: > On Tue, May 31, 2016 at 10:23 AM, Josh berkus wrote: >> It's still WAY simpler to understand "max_parallel is the number of >> parallel workers I requested". > > (Sorry Josh, somehow hit reply, not reply-all) > > Yes, it is. But as long as paralle

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Joshua D. Drake
On 05/31/2016 10:30 AM, Tom Lane wrote: Josh berkus writes: On 05/31/2016 10:16 AM, Peter Geoghegan wrote: But the distinction between parallel workers and backends that can participate in parallel query does need to be user-visible. Worker processes are a commodity (i.e. the user must conside

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Peter Geoghegan
On Tue, May 31, 2016 at 10:23 AM, Josh berkus wrote: > It's still WAY simpler to understand "max_parallel is the number of > parallel workers I requested". (Sorry Josh, somehow hit reply, not reply-all) Yes, it is. But as long as parallel workers are not really that distinct to the leader-as-wor

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Tom Lane
Josh berkus writes: > On 05/31/2016 10:16 AM, Peter Geoghegan wrote: >> But the distinction between parallel workers and backends that can >> participate in parallel query does need to be user-visible. Worker >> processes are a commodity (i.e. the user must consider >> max_worker_processes). > It

Re: [HACKERS] Re: PATCH: Split stats file per database WAS: autovacuum stress-testing our system

2016-05-31 Thread Tom Lane
Tomas Vondra writes: > On 05/31/2016 06:59 PM, Tom Lane wrote: >> I'm confused here --- are you speaking of having removed >> if (msg->cutoff_time > req->request_time) >> req->request_time = msg->cutoff_time; >> ? That is not a check for clock skew, it's intending to be sure that

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Josh berkus
On 05/31/2016 10:16 AM, Peter Geoghegan wrote: > On Tue, May 31, 2016 at 10:10 AM, Josh berkus wrote: >> "max_parallel_degree is the amount of parallelism in the query, with the >> understanding that the original parent process counts as 1, which means >> that if you set it to 1 you get no paralle

Re: [HACKERS] Re: PATCH: Split stats file per database WAS: autovacuum stress-testing our system

2016-05-31 Thread Tomas Vondra
On 05/31/2016 06:59 PM, Tom Lane wrote: Tomas Vondra writes: On 05/26/2016 10:10 PM, Tom Lane wrote: I posted a patch at https://www.postgresql.org/message-id/13023.1464213...@sss.pgh.pa.us which I think is functionally equivalent to what you have here, but it goes to some lengths to make the

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Peter Geoghegan
On Tue, May 31, 2016 at 10:10 AM, Josh berkus wrote: > "max_parallel_degree is the amount of parallelism in the query, with the > understanding that the original parent process counts as 1, which means > that if you set it to 1 you get no parallelism, and if you want 4 > parallel workers you need

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Joshua D. Drake
On 05/31/2016 10:10 AM, Josh berkus wrote: Compare this: "max_parallel is the maximum number of parallel workers which will work on each stage of the query which is parallizable. If you set it to 4, you get up to 4 workers." with this: "max_parallel_degree is the amount of parallelism in the

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Josh berkus
On 05/31/2016 10:03 AM, Tom Lane wrote: > Josh berkus writes: >> I realize there's a lot of water under the bridge here, but I think >> we're going to get 1000 questions on -general of the type: "I asked for >> 8 parallel workers, why did I only get 7?". I believe we will regret >> this change.

Re: [HACKERS] Rename max_parallel_degree?

2016-05-31 Thread Tom Lane
Josh berkus writes: > I realize there's a lot of water under the bridge here, but I think > we're going to get 1000 questions on -general of the type: "I asked for > 8 parallel workers, why did I only get 7?". I believe we will regret > this change. > So, one vote from me to revert. Well, that

  1   2   >