[HACKERS] Fix performance degradation of contended LWLock on NUMA

2017-06-05 Thread Sokolov Yura
Good day, everyone. This patch improves performance of contended LWLock. It was tested on 4 socket 72 core x86 server (144 HT) Centos 7.1 gcc 4.8.5 Results: pgbench -i -s 300 + pgbench --skip-some-updates Clients | master | patched +=+=== 30 |32k |32k 50

Re: retry shm attach for windows (WAS: Re: [HACKERS] OK, so culicidae is *still* broken)

2017-06-05 Thread Tom Lane
Amit Kapila writes: > Sure. I think it is slightly tricky because specs don't say clearly > how ASLR can impact the behavior of any API and in my last attempt I > could not reproduce the issue. > I can try to do basic verification with the patch you have proposed, > but

Re: [HACKERS] PG10 transition tables, wCTEs and multiple operations on the same table

2017-06-05 Thread Robert Haas
On Sat, Jun 3, 2017 at 10:39 PM, Thomas Munro wrote: > In the meantime, it seems like you agree that rejecting wCTEs that > affect tables with triggers with transition tables is the best > response to this bug report? Do you think that parse analysis is the > right

Re: [HACKERS] Bug in ExecModifyTable function and trigger issues for foreign tables

2017-06-05 Thread Etsuro Fujita
On 2017/06/05 17:39, Ashutosh Bapat wrote: On Fri, Jun 2, 2017 at 2:40 PM, Etsuro Fujita While updating the patch, I noticed the patch rewrites the UPDATE targetList incorrectly in some cases; rewrite_inherited_tlist I added to adjust_appendrel_attrs (1) removes all junk items from the

Re: [HACKERS] make check false success

2017-06-05 Thread Tom Lane
Sandro Santilli writes: > On Fri, Jun 02, 2017 at 08:20:25AM -0400, Robert Haas wrote: >> On Thu, Jun 1, 2017 at 10:18 AM, Sandro Santilli wrote: >>> I noticed that the `check` Makefile rule imported by PGXS is giving >>> a success exit code even when it is unsupported.

Re: [HACKERS] Fix a typo in README.dependencies

2017-06-05 Thread Ashutosh Bapat
On Mon, Jun 5, 2017 at 8:22 AM, atorikoshi wrote: > Hi, > > I found below formula to compute selectivities, but > I think the last Probability 'P(b=?)' should be 'P(c=?)'. > >> P(a=?,b=?,c=?) = P(a=?,b=?) * (d + (1-d)*P(b=?)) > > > Attached patch fixes it, and

[HACKERS] Challenges preventing us moving to 64 bit transaction id (XID)?

2017-06-05 Thread Tianzhou Chen
Hi Pg Hackers, XID wraparound seems to be quite a big concern and we introduce changes like “adding another frozen bit to each page” [http://rhaas.blogspot.com/2016/03/no-more-full-table-vacuums.html to tackle this. I am

[HACKERS] Update comments in nodeModifyTable.c

2017-06-05 Thread Etsuro Fujita
While working on [1], I noticed that the comment in ExecModifyTable: * Foreign table updates have a wholerow attribute when the * relation has an AFTER ROW trigger. is not 100% correct because a foreign table has a wholerow attrubute when the relation has an

Re: retry shm attach for windows (WAS: Re: [HACKERS] OK, so culicidae is *still* broken)

2017-06-05 Thread Amit Kapila
On Mon, Jun 5, 2017 at 4:58 PM, Magnus Hagander wrote: > On Mon, Jun 5, 2017 at 1:16 PM, Amit Kapila wrote: >> >> On Mon, Jun 5, 2017 at 9:15 AM, Tom Lane wrote: >> > Amit Kapila writes: >> > >> >> I

Re: [HACKERS] Proposal : For Auto-Prewarm.

2017-06-05 Thread Rafia Sabih
On Sun, Jun 4, 2017 at 12:45 PM, Mithun Cy wrote: > On Wed, May 31, 2017 at 10:18 PM, Robert Haas wrote: >> + *contrib/autoprewarm.c >> >> Wrong. > > -- Oops Sorry fixed. > >> +Oiddatabase;/* database */ >> +

Re: [HACKERS] simplehash.h typo

2017-06-05 Thread Heikki Linnakangas
On 06/05/2017 11:26 AM, Andres Freund wrote: On 2017-06-05 11:10:12 +0300, Heikki Linnakangas wrote: Fixed, thanks. I also fixed and clarified some other comments in the file that seemed wrong or confusing to me. Thanks for looking - I don't see any commit though? Pushed now. Note to self:

Re: [HACKERS] Bug in ExecModifyTable function and trigger issues for foreign tables

2017-06-05 Thread Ashutosh Bapat
On Fri, Jun 2, 2017 at 2:40 PM, Etsuro Fujita wrote: > On 2017/05/16 21:36, Etsuro Fujita wrote: >> >> One approach I came up with to fix this issue is to rewrite the targetList >> entries of an inherited UPDATE/DELETE properly using rewriteTargetListUD, >> when

Re: [HACKERS] make check false success

2017-06-05 Thread Sandro Santilli
On Fri, Jun 02, 2017 at 08:20:25AM -0400, Robert Haas wrote: > On Thu, Jun 1, 2017 at 10:18 AM, Sandro Santilli wrote: > > I noticed that the `check` Makefile rule imported by PGXS is giving > > a success exit code even when it is unsupported. > > > > The attached patch fixes that. >

Re: retry shm attach for windows (WAS: Re: [HACKERS] OK, so culicidae is *still* broken)

2017-06-05 Thread Magnus Hagander
On Mon, Jun 5, 2017 at 1:16 PM, Amit Kapila wrote: > On Mon, Jun 5, 2017 at 9:15 AM, Tom Lane wrote: > > Amit Kapila writes: > > > >> I think the same problem can happen during reattach as well. > >> Basically,

Re: retry shm attach for windows (WAS: Re: [HACKERS] OK, so culicidae is *still* broken)

2017-06-05 Thread Magnus Hagander
On Mon, Jun 5, 2017 at 1:35 PM, Amit Kapila wrote: > On Mon, Jun 5, 2017 at 4:58 PM, Magnus Hagander > wrote: > > On Mon, Jun 5, 2017 at 1:16 PM, Amit Kapila > wrote: > >> > >> On Mon, Jun 5, 2017 at 9:15 AM, Tom Lane

Re: [HACKERS] Bug in ExecModifyTable function and trigger issues for foreign tables

2017-06-05 Thread Etsuro Fujita
On 2017/06/02 18:10, Etsuro Fujita wrote: On 2017/05/16 21:36, Etsuro Fujita wrote: One approach I came up with to fix this issue is to rewrite the targetList entries of an inherited UPDATE/DELETE properly using rewriteTargetListUD, when generating a plan for each child table in

Re: [HACKERS] Challenges preventing us moving to 64 bit transaction id (XID)?

2017-06-05 Thread Heikki Linnakangas
On 06/05/2017 11:49 AM, Tianzhou Chen wrote: Hi Pg Hackers, XID wraparound seems to be quite a big concern and we introduce changes like “adding another frozen bit to each page” [http://rhaas.blogspot.com/2016/03/no-more-full-table-vacuums.html

Re: retry shm attach for windows (WAS: Re: [HACKERS] OK, so culicidae is *still* broken)

2017-06-05 Thread Amit Kapila
On Mon, Jun 5, 2017 at 9:15 AM, Tom Lane wrote: > Amit Kapila writes: > >> I think the same problem can happen during reattach as well. >> Basically, MapViewOfFileEx can fail to load image at predefined >> address (UsedShmemSegAddr). > > Once we've

Re: [HACKERS] PG10 transition tables, wCTEs and multiple operations on the same table

2017-06-05 Thread Tom Lane
Robert Haas writes: > On Sat, Jun 3, 2017 at 10:39 PM, Thomas Munro > wrote: >> In the meantime, it seems like you agree that rejecting wCTEs that >> affect tables with triggers with transition tables is the best >> response to this bug

[HACKERS] Re: [BUGS] BUG #14682: row level security not work with partitioned table

2017-06-05 Thread Joe Conway
On 06/04/2017 03:33 PM, Noah Misch wrote: > On Fri, Jun 02, 2017 at 09:28:16AM +0900, Michael Paquier wrote: >> On Thu, Jun 1, 2017 at 11:13 AM, Mike Palmiotto >> wrote: >> > This is indeed a bug. fireRIRrules is currently skipping the RLS >> > policy check when

Re: [HACKERS] intermittent failures in Cygwin from select_parallel tests

2017-06-05 Thread Tom Lane
Andrew Dunstan writes: > Buildfarm member lorikeet is failing occasionally with a failed > assertion during the select_parallel regression tests like this: > 2017-06-03 05:12:37.382 EDT [59327d84.1160:38] LOG: statement: select > count(*) from tenk1, tenk2

[HACKERS] shm_toc_lookup API

2017-06-05 Thread Tom Lane
shm_toc_lookup() returns NULL if it fails to find the desired key. Out of the 30 or so call sites, there is exactly one that has any use for that. Some of the rest have Asserts that they get back a non-null result, but the majority just blithely dereference the pointer. I do not find this cool

Re: [HACKERS] PG10 transition tables, wCTEs and multiple operations on the same table

2017-06-05 Thread Robert Haas
On Mon, Jun 5, 2017 at 10:15 AM, Tom Lane wrote: >> I'm starting to like the approach of reverting the entire transition >> tables patch. Failing to consider the possibility of a plan with >> multiple ModifyTable nodes seems like a pretty fundamental design >> mistake, and

Re: [HACKERS] [BUGS] BUG #14682: row level security not work with partitioned table

2017-06-05 Thread Robert Haas
On Mon, Jun 5, 2017 at 10:20 AM, Joe Conway wrote: > Unless Robert objects, I'll work with Mike to get a fix posted and > committed in the next day or two. That would be great. Thanks. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL

Re: [HACKERS] Proposal : For Auto-Prewarm.

2017-06-05 Thread Robert Haas
On Mon, Jun 5, 2017 at 7:58 AM, Rafia Sabih wrote: > I had a look at the patch from stylistic/formatting point of view, > please find the attached patch for the suggested modifications. Many of these seem worse, like these ones: - * Quit if we've reached

Re: [HACKERS] Default Partition for Range

2017-06-05 Thread Dilip Kumar
On Mon, Jun 5, 2017 at 1:43 AM, Beena Emerson wrote: > The new patch is rebased over default_partition_v18.patch > [http://www.mail-archive.com/pgsql-hackers@postgresql.org/msg315831.html] I have done the initial review of the patch, I have few comments. + if

Re: [HACKERS] make check false success

2017-06-05 Thread Robert Haas
On Mon, Jun 5, 2017 at 5:23 AM, Sandro Santilli wrote: > Why not ? The caller is attempting to make an unsupported target, > how's that different from calling `make unexistent` ? That's a good point, but what Tom wrote is along the lines of my concerns also, especially his last

[HACKERS] intermittent failures in Cygwin from select_parallel tests

2017-06-05 Thread Andrew Dunstan
Buildfarm member lorikeet is failing occasionally with a failed assertion during the select_parallel regression tests like this: 2017-06-03 05:12:37.382 EDT [59327d84.1160:38] LOG: statement: select count(*) from tenk1, tenk2 where tenk1.hundred > 1 and tenk2.thousand=0; TRAP:

Re: [HACKERS] shm_toc_lookup API

2017-06-05 Thread Robert Haas
On Mon, Jun 5, 2017 at 11:19 AM, Tom Lane wrote: > shm_toc_lookup() returns NULL if it fails to find the desired key. > Out of the 30 or so call sites, there is exactly one that has any > use for that. Some of the rest have Asserts that they get back > a non-null result, but

Re: [HACKERS] Make ANALYZE more selective about what is a "most common value"?

2017-06-05 Thread Robert Haas
On Sun, Jun 4, 2017 at 5:30 PM, Tom Lane wrote: > I've been thinking about the behavior discussed in > https://www.postgresql.org/message-id/flat/20170522132017.29944.48391%40wrigleys.postgresql.org > and it seems to me that there are a couple of things we ought to do about >

Re: [HACKERS] shm_toc_lookup API

2017-06-05 Thread Robert Haas
On Mon, Jun 5, 2017 at 12:19 PM, Tom Lane wrote: > While I'm looking at this ... seems like there's a pretty basic coding- > rule violation here, namely that shm_toc_lookup() thinks it can read > toc->toc_nentry without any sort of locking. Since that field is declared >

Re: [HACKERS] shm_toc_lookup API

2017-06-05 Thread Tom Lane
Robert Haas writes: > On Mon, Jun 5, 2017 at 11:19 AM, Tom Lane wrote: >> I think that shm_toc_lookup() ought to be made to throw elog(ERROR) on an >> unexpected failure. To satisfy the one caller that doesn't want an error, >> we could either add a

[HACKERS] GSoC 2017 weekly progress reports ("Explicitly support predicate locks in index access methods besides b-tree")

2017-06-05 Thread Shubham Barai
GSoC (week 1) Hi, Here is the list of things I have done during this week. 1. read documentation on how to set up development environment 2. installed PostgreSQL on Ubuntu from source code 3. read documentation on gist index (http://www.sai.msu.su/~ megera/postgres/gist/) 4. went through

Re: [HACKERS] Challenges preventing us moving to 64 bit transaction id (XID)?

2017-06-05 Thread Ashutosh Bapat
On Mon, Jun 5, 2017 at 2:38 PM, Heikki Linnakangas wrote: > On 06/05/2017 11:49 AM, Tianzhou Chen wrote: >> >> Hi Pg Hackers, >> >> XID wraparound seems to be quite a big concern and we introduce >> changes like “adding another frozen bit to each page” >>

[HACKERS] Minor fix for EventCacheLookup()

2017-06-05 Thread Amit Langote
It should return NIL when no entry is found in the cache, not NULL. Attached patch fixes that. Thanks, Amit diff --git a/src/backend/utils/cache/evtcache.c b/src/backend/utils/cache/evtcache.c index 54ddc55f76..6faf4ae354 100644 --- a/src/backend/utils/cache/evtcache.c +++

Re: [HACKERS] Challenges preventing us moving to 64 bit transaction id (XID)?

2017-06-05 Thread Ashutosh Bapat
On Tue, Jun 6, 2017 at 9:48 AM, Craig Ringer wrote: > On 6 June 2017 at 12:13, Ashutosh Bapat > wrote: > >> What happens when the epoch is so low that the rest of the XID does >> not fit in 32bits of tuple header? Or such a case should

Re: [HACKERS] Challenges preventing us moving to 64 bit transaction id (XID)?

2017-06-05 Thread Ashutosh Bapat
On Tue, Jun 6, 2017 at 10:00 AM, Tom Lane wrote: > Ashutosh Bapat writes: >> On Tue, Jun 6, 2017 at 9:48 AM, Craig Ringer wrote: >>> Storing an epoch implies that rows can't have (xmin,xmax) different by >>> more than

Re: [HACKERS] Challenges preventing us moving to 64 bit transaction id (XID)?

2017-06-05 Thread Craig Ringer
On 6 June 2017 at 12:13, Ashutosh Bapat wrote: > What happens when the epoch is so low that the rest of the XID does > not fit in 32bits of tuple header? Or such a case should never arise? Storing an epoch implies that rows can't have (xmin,xmax) different by

Re: [HACKERS] pg_dump issues

2017-06-05 Thread Craig Ringer
On 6 June 2017 at 11:37, Дмитрий Воронин wrote: > Hello, > > We are working on one project with postgres as engeneer. > > Bellow is list of inconveniences that we are having with postgresql. We > would like to solve them as possible. > > We can't create any schema dump

Re: [HACKERS] Challenges preventing us moving to 64 bit transaction id (XID)?

2017-06-05 Thread Tom Lane
Ashutosh Bapat writes: > On Tue, Jun 6, 2017 at 9:48 AM, Craig Ringer wrote: >> Storing an epoch implies that rows can't have (xmin,xmax) different by >> more than one epoch. So if you're updating/deleting an extremely old >> tuple you'll

Re: [HACKERS] postgres_fdw cost estimation defaults and documentation

2017-06-05 Thread Ashutosh Bapat
On Tue, Jun 6, 2017 at 12:07 AM, Jeff Janes wrote: > > The default value for fdw_tuple_cost is 0.01, which seems way too low. If I > set up a loop-back foreign server with a large fetch_size, then tests like: > > select * from pgbench_accounts except select * from >

[HACKERS] postgres_fdw cost estimation defaults and documentation

2017-06-05 Thread Jeff Janes
The default value for fdw_tuple_cost is 0.01, which seems way too low. If I set up a loop-back foreign server with a large fetch_size, then tests like: select * from pgbench_accounts except select * from loopback.pgbench_accounts vs select * from pgbench_accounts except select * from

Re: [HACKERS] shm_toc_lookup API

2017-06-05 Thread Tom Lane
Robert Haas writes: > On Mon, Jun 5, 2017 at 12:19 PM, Tom Lane wrote: >> In practice it probably can't fail even if 64-bit reads aren't atomic, >> simply because we'll never have enough entries in a shm_toc to make the >> high-order half ever change.

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-06-05 Thread Michael Paquier
On Mon, Jun 5, 2017 at 10:29 AM, Andres Freund wrote: > On 2017-06-02 17:20:23 -0700, Andres Freund wrote: >> Attached is a *preliminary* patch series implementing this. I've first >> reverted the previous patch, as otherwise backpatchable versions of the >> necessary patches

Re: [HACKERS] Server ignores contents of SASLInitialResponse

2017-06-05 Thread Michael Paquier
On Mon, Jun 5, 2017 at 7:49 AM, Noah Misch wrote: > IMMEDIATE ATTENTION REQUIRED. This PostgreSQL 10 open item is long past due > for your status update. Please reacquaint yourself with the policy on open > item ownership[1] and then reply immediately. If I do not hear from

Re: [HACKERS] UPDATE of partition key

2017-06-05 Thread Amit Kapila
On Thu, Jun 1, 2017 at 3:25 AM, Robert Haas wrote: > On Mon, May 29, 2017 at 5:26 AM, Amit Kapila wrote: >>> But I think, we can also take step-by-step approach even for v11. If >>> we agree that it is ok to silently do the updates as long as we

Re: [HACKERS] UPDATE of partition key

2017-06-05 Thread Amit Khandekar
On 5 June 2017 at 11:27, Amit Kapila wrote: > On Fri, Jun 2, 2017 at 4:37 PM, Amit Khandekar wrote: >> On 2 June 2017 at 01:17, Robert Haas wrote: >>> On Thu, Jun 1, 2017 at 7:41 AM, Amit Khandekar

Re: [HACKERS] proposal psql \gdesc

2017-06-05 Thread Fabien COELHO
new update - rebase, changed message Thanks. New patch applies cleanly, make check still ok. -- Fabien. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] simplehash.h typo

2017-06-05 Thread Heikki Linnakangas
On 05/28/2017 04:50 AM, Jeff Janes wrote: /* round up size to the next power of 2, that's the bucketing works */ That should probably be "that's the **way** bucketing works". Or maybe it is an idiom I don't grok. Fixed, thanks. I also fixed and clarified some other comments in the

Re: [HACKERS] Server ignores contents of SASLInitialResponse

2017-06-05 Thread Heikki Linnakangas
On 06/05/2017 09:34 AM, Michael Paquier wrote: On Mon, Jun 5, 2017 at 7:49 AM, Noah Misch wrote: IMMEDIATE ATTENTION REQUIRED. This PostgreSQL 10 open item is long past due for your status update. Please reacquaint yourself with the policy on open item ownership[1] and

Re: [HACKERS] simplehash.h typo

2017-06-05 Thread Andres Freund
On 2017-06-05 11:10:12 +0300, Heikki Linnakangas wrote: > On 05/28/2017 04:50 AM, Jeff Janes wrote: > > /* round up size to the next power of 2, that's the bucketing works */ > > > > > > That should probably be "that's the **way** bucketing works". Or maybe it > > is an idiom I don't grok.

Re: [HACKERS] Make ANALYZE more selective about what is a "most common value"?

2017-06-05 Thread Gavin Flower
On 06/06/17 10:12, Gavin Flower wrote: On 06/06/17 09:41, Mark Kirkwood wrote: On 05/06/17 09:30, Tom Lane wrote: I've been thinking about the behavior discussed in https://www.postgresql.org/message-id/flat/20170522132017.29944.48391%40wrigleys.postgresql.org and it seems to me that there

Re: [HACKERS] Make ANALYZE more selective about what is a "most common value"?

2017-06-05 Thread Robert Haas
On Mon, Jun 5, 2017 at 3:34 PM, Tom Lane wrote: > I think that a single count in a 30K-row sample is noise by definition. Not if the table only *has* 30K rows. Or even 100K. And anyway we're talking about what to do with a value we hit at least twice, which is not quite the

Re: [HACKERS] Use of non-restart-safe storage by temp_tablespaces

2017-06-05 Thread Bruce Momjian
On Mon, Jun 5, 2017 at 07:38:43PM -0700, Andres Freund wrote: > On 2017-06-05 22:34:17 -0400, Bruce Momjian wrote: > > On Mon, Jun 5, 2017 at 04:38:32PM -0500, Jerry Sievers wrote: > > > The SAN snaps capture the entire pgdata and WAL pg_xlog area but there > > > is no attempt to copy the NVME

Re: [HACKERS] Do we need the gcc feature "__builtin_expect" to promote the branches prediction?

2017-06-05 Thread Hao Lee
and another things come into my mind, in some scenario, as we know that the native library is not the most effective way to do that, such as, allocation a large amount of memories by using "alloc()"... and "memmove()", so on. As the SIMD instruction became the standard in CPU, therefore, we can

Re: [HACKERS] Do we need the gcc feature "__builtin_expect" to promote the branches prediction?

2017-06-05 Thread Craig Ringer
On 6 June 2017 at 10:44, Hao Lee wrote: > and another things come into my mind, in some scenario, as we know that the > native library is not the most effective way to do that, such as, allocation > a large amount of memories by using "alloc()"... and "memmove()", so on. As >

[HACKERS] pg_dump issues

2017-06-05 Thread Дмитрий Воронин
Hello,We are working on one project with postgres as engeneer.Bellow is list of inconveniences that we are having with postgresql. We would like to solve them as possible.We can't create any schema dump with another (user defined) name. E.g. we dump schema test and we want to save it's dump with

Re: [HACKERS] PROVE_FLAGS

2017-06-05 Thread Andrew Dunstan
On 05/23/2017 06:59 AM, Andrew Dunstan wrote: > On 17 May 2017 at 14:30, Robert Haas wrote: >> On Tue, May 16, 2017 at 9:20 PM, Andrew Dunstan >> wrote: >>> Inheriting variables from the environment is a part of make by design. >>> We have

Re: [HACKERS] shm_toc_lookup API

2017-06-05 Thread Tom Lane
Andres Freund writes: > On 2017-06-05 14:57:10 -0400, Tom Lane wrote: >> Meh. Without volatile, I think that the compiler would be within its >> rights to elide the nentry local variable and re-fetch toc->toc_nentry >> each time through the loop. > I don't think that's true.

Re: [HACKERS] Make ANALYZE more selective about what is a "most common value"?

2017-06-05 Thread Tom Lane
Robert Haas writes: > On Sun, Jun 4, 2017 at 5:30 PM, Tom Lane wrote: >> First, I think we need a larger hard floor on the number of occurrences >> of a value that're required to make ANALYZE decide it is a "most common >> value". > This kind of math

Re: [HACKERS] Why does logical replication launcher set application_name?

2017-06-05 Thread Petr Jelinek
On 03/06/17 05:18, Peter Eisentraut wrote: > On 6/2/17 16:44, Petr Jelinek wrote: >> However, I am not sure about the bgw_name_extra. I think I would have >> preferred keeping full bgw_name field which would be used where full >> name is needed and bgw_type where only the worker type is used. The

[HACKERS] HACKERS[PROPOSAL] split ProcArrayLock into multiple parts

2017-06-05 Thread Jim Van Fleet
Hi, I have been experimenting with splitting the ProcArrayLock into parts. That is, to Acquire the ProcArrayLock in shared mode, it is only necessary to acquire one of the parts in shared mode; to acquire the lock in exclusive mode, all of the parts must be acquired in exclusive mode. For

Re: [HACKERS] HACKERS[PROPOSAL] split ProcArrayLock into multiple parts

2017-06-05 Thread Sokolov Yura
Excuse me, Jim.I was tired and misunderstand proposal: I thought of ProcArray sharding, but proposal is about ProcArrayLock sharding.BTW, I just posted improvement to LWLock:https://www.postgresql.org/message-id/2968c0be065baab8865c4c95de3f435c%40postgrespro.ruWould you mind to test against that

Re: [HACKERS] PG10 transition tables, wCTEs and multiple operations on the same table

2017-06-05 Thread Thomas Munro
On Tue, Jun 6, 2017 at 2:15 AM, Tom Lane wrote: > Robert Haas writes: >> On Sat, Jun 3, 2017 at 10:39 PM, Thomas Munro >> wrote: >>> In the meantime, it seems like you agree that rejecting wCTEs that >>> affect tables

Re: [HACKERS] shm_toc_lookup API

2017-06-05 Thread Andres Freund
On 2017-06-05 14:57:10 -0400, Tom Lane wrote: > > If it doesn't prevent both the hardware and the compiler from > > reordering, it's broken. See the comments for pg_read_barrier() in > > atomics.h. > > Meh. Without volatile, I think that the compiler would be within its > rights to elide the

Re: [HACKERS] shm_toc_lookup API

2017-06-05 Thread Andres Freund
On 2017-06-05 15:06:06 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2017-06-05 14:57:10 -0400, Tom Lane wrote: > >> Meh. Without volatile, I think that the compiler would be within its > >> rights to elide the nentry local variable and re-fetch toc->toc_nentry > >>

Re: [HACKERS] Use of non-restart-safe storage by temp_tablespaces

2017-06-05 Thread Jerry Sievers
Bruce Momjian writes: > On Wed, May 31, 2017 at 08:28:51AM -0700, Mark Dilger wrote: > >> > Uh, I thought only the sessions that created the temporary objects could >> > see them, and since they are not in WAL and autovacuum can't see them, >> > their non-existence in a

Re: [HACKERS] Make ANALYZE more selective about what is a "most common value"?

2017-06-05 Thread Mark Kirkwood
On 05/06/17 09:30, Tom Lane wrote: I've been thinking about the behavior discussed in https://www.postgresql.org/message-id/flat/20170522132017.29944.48391%40wrigleys.postgresql.org and it seems to me that there are a couple of things we ought to do about it. First, I think we need a larger

Re: [HACKERS] PG10 transition tables, wCTEs and multiple operations on the same table

2017-06-05 Thread Tom Lane
Thomas Munro writes: > On Tue, Jun 6, 2017 at 2:15 AM, Tom Lane wrote: >> FWIW, parse analysis is surely NOT the time for such a check. Triggers >> might get added to a table between analysis and execution. I think you >> might have to do it

Re: [HACKERS] Make ANALYZE more selective about what is a "most common value"?

2017-06-05 Thread Gavin Flower
On 06/06/17 09:41, Mark Kirkwood wrote: On 05/06/17 09:30, Tom Lane wrote: I've been thinking about the behavior discussed in https://www.postgresql.org/message-id/flat/20170522132017.29944.48391%40wrigleys.postgresql.org and it seems to me that there are a couple of things we ought to do

Re: [HACKERS] HACKERS[PROPOSAL] split ProcArrayLock into multiple parts

2017-06-05 Thread Sokolov Yura
Hi, Jim.How do you ensure of transaction order?Example:- you lock shard A and gather info. You find transaction T1 in-progress.- Then you unlock shard A.- T1 completes. T2, that depends on T1, also completes. But T2 was on shard B.- you lock shard B, and gather info from.- You didn't saw T2 as in

Re: [HACKERS] HACKERS[PROPOSAL] split ProcArrayLock into multiple parts

2017-06-05 Thread Jim Van Fleet
NP, Sokolov -- pgsql-hackers-ow...@postgresql.org wrote on 06/05/2017 03:26:46 PM: > From: Sokolov Yura > To: Jim Van Fleet > Cc: pgsql-hackers@postgresql.org > Date: 06/05/2017 03:28 PM > Subject: Re: [HACKERS] HACKERS[PROPOSAL] split

Re: [HACKERS] Why does logical replication launcher set application_name?

2017-06-05 Thread Peter Eisentraut
On 6/2/17 23:23, Peter Eisentraut wrote: > On 6/2/17 15:08, Peter Eisentraut wrote: >> On 5/30/17 23:10, Peter Eisentraut wrote: >>> Here is a proposed solution that splits bgw_name into bgw_type and >>> bgw_name_extra. bgw_type shows up in pg_stat_activity.backend_type. >>> Uses of

Re: [HACKERS] PG10 transition tables, wCTEs and multiple operations on the same table

2017-06-05 Thread Thomas Munro
On Tue, Jun 6, 2017 at 10:11 AM, Tom Lane wrote: > Thomas Munro writes: >> On Tue, Jun 6, 2017 at 2:15 AM, Tom Lane wrote: >>> FWIW, parse analysis is surely NOT the time for such a check. Triggers >>> might get added to a

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-06-05 Thread Michael Paquier
On Tue, Jun 6, 2017 at 9:47 AM, Andres Freund wrote: > On 2017-06-05 15:30:38 +0900, Michael Paquier wrote: >> I think that it would be interesting to be able to >> trigger a feedback message using SIGHUP in WAL receivers, refactoring >> at the same time SIGHUP handling for

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-06-05 Thread Andres Freund
On 2017-06-05 15:30:38 +0900, Michael Paquier wrote: > I have looked at all those patches. The set looks solid to me. Thanks! > Here are some comments about 0003. > + /* > +* Have WalSndLoop() terminate the connection in an orderly > +* manner, after writing

Re: [HACKERS] PG10 transition tables, wCTEs and multiple operations on the same table

2017-06-05 Thread Thomas Munro
On Tue, Jun 6, 2017 at 12:58 PM, Craig Ringer wrote: > On 4 June 2017 at 06:41, Kevin Grittner wrote: >> On Fri, Jun 2, 2017 at 8:46 PM, Thomas Munro >> wrote: >> >>> So, afterTriggers.query_stack is used to handle the

Re: retry shm attach for windows (WAS: Re: [HACKERS] OK, so culicidae is *still* broken)

2017-06-05 Thread Amit Kapila
On Mon, Jun 5, 2017 at 7:26 PM, Tom Lane wrote: > Amit Kapila writes: >> Sure. I think it is slightly tricky because specs don't say clearly >> how ASLR can impact the behavior of any API and in my last attempt I >> could not reproduce the issue. >

Re: [HACKERS] Use of non-restart-safe storage by temp_tablespaces

2017-06-05 Thread Bruce Momjian
On Mon, Jun 5, 2017 at 04:38:32PM -0500, Jerry Sievers wrote: > The SAN snaps capture the entire pgdata and WAL pg_xlog area but there > is no attempt to copy the NVME device when the snaps are made. > > There's an event trigger plus batch job now running tou avoid this risk. > > We realize too

Re: [HACKERS] Error while creating subscription when server is running in single user mode

2017-06-05 Thread Peter Eisentraut
On 6/2/17 23:06, Peter Eisentraut wrote: > On 6/2/17 15:41, Tom Lane wrote: >> It's certainly plausible that we could have the latch code just ignore >> WL_POSTMASTER_DEATH if not IsUnderPostmaster. I think that the original >> reasoning for not doing that was that the calling code should know

Re: [HACKERS] ALTER SUBSCRIPTION ..SET PUBLICATION refresh is not throwing error.

2017-06-05 Thread Peter Eisentraut
On 6/2/17 22:13, Peter Eisentraut wrote: > On 5/27/17 06:54, Petr Jelinek wrote: >> On 27/05/17 04:00, Euler Taveira wrote: >>> 2017-05-26 21:29 GMT-03:00 Petr Jelinek >> >: >>> >>> >>> Actually another possibility would be to

[HACKERS] inconsistent application_name use in logical workers

2017-06-05 Thread Peter Eisentraut
The logical replication code is supposed to use the subscription name as the fallback_application_name, but in some cases it uses the slot name, which could be different. See attached patch to correct this. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development,

Re: [HACKERS] Use of non-restart-safe storage by temp_tablespaces

2017-06-05 Thread Andres Freund
On 2017-06-05 22:34:17 -0400, Bruce Momjian wrote: > On Mon, Jun 5, 2017 at 04:38:32PM -0500, Jerry Sievers wrote: > > The SAN snaps capture the entire pgdata and WAL pg_xlog area but there > > is no attempt to copy the NVME device when the snaps are made. > > > > There's an event trigger plus

Re: [HACKERS] PG10 transition tables, wCTEs and multiple operations on the same table

2017-06-05 Thread Craig Ringer
On 4 June 2017 at 06:41, Kevin Grittner wrote: > On Fri, Jun 2, 2017 at 8:46 PM, Thomas Munro > wrote: > >> So, afterTriggers.query_stack is used to handle the reentrancy that >> results from triggers running further statements that might fire >>

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-06-05 Thread Andres Freund
Hi, On 2017-06-05 15:30:38 +0900, Michael Paquier wrote: > + * This will trigger walsenders to send the remaining WAL, prevent them from > + * accepting further commands. After that they'll wait till the last WAL is > + * written. > s/prevent/preventing/? > I would rephrase the last sentence a

Re: [HACKERS] Server ignores contents of SASLInitialResponse

2017-06-05 Thread Michael Paquier
On Thu, Jun 1, 2017 at 4:58 AM, Heikki Linnakangas wrote: > I bisected that; the culprit was commit 61bf96cab0, where I refactored the > libpq authentication code in preparation for SCRAM. The logic around that > free() was always a bit wonky, but the refactoring made it outright

Re: [HACKERS] Is ECPG's SET CONNECTION really not thread-aware?

2017-06-05 Thread Tsunakawa, Takayuki
From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Tsunakawa, > Takayuki > The following page says: > > https://www.postgresql.org/docs/devel/static/ecpg-connect.html#ecpg-se > t-connection > > -- >