Re: [HACKERS] drop/truncate table sucks for large values of shared buffers

2015-06-29 Thread Simon Riggs
On 28 June 2015 at 17:17, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: On 27 June 2015 at 15:10, Tom Lane t...@sss.pgh.pa.us wrote: I don't like this too much because it will fail badly if the caller is wrong about the maximum possible page number for the

Re: [HACKERS] proposal: condition blocks in psql

2015-06-29 Thread Merlin Moncure
On Sun, Jun 28, 2015 at 1:59 AM, Fabien COELHO coe...@cri.ensmp.fr wrote: \if_ver_eq 9.2 What do you thinking about it? Couldn't this kind of thing be done directly with PL/pgSQL? you can use PL/pgSQL - but there are some limits * maintenance large plpgsql functions I agree with

[HACKERS] Re: In-core regression tests for replication, cascading, archiving, PITR, etc.

2015-06-29 Thread Michael Paquier
On Wed, Mar 18, 2015 at 1:59 PM, Michael Paquier michael.paqu...@gmail.com wrote: Feedback is of course welcome, but note that I am not seriously expecting any until we get into 9.6 development cycle and I am adding this patch to the next CF. I have moved this patch to CF 2015-09, as I have

Re: [HACKERS] anole: assorted stability problems

2015-06-29 Thread Andres Freund
On 2015-06-29 12:11:08 +0200, Andres Freund wrote: On 2015-06-29 00:42:53 -0400, Tom Lane wrote: #define S_UNLOCK(lock) \ do { _Asm_sched_fence(); (*(lock)) = 0; } while (0) Robert, how did you choose that? Isn't _Asm_sched_fence just a compiler barrier? Shouldn't this be a

Re: [HACKERS] UPSERT on partition

2015-06-29 Thread Amit Langote
On 2015-06-25 AM 09:51, Amit Langote wrote: Peter, On 2015-06-25 AM 02:35, Peter Geoghegan wrote: Inheritance with triggers is a leaky abstraction, so this kind of thing is always awkward. Still, UPSERT has full support for *inheritance* -- that just doesn't help in this case. Could

Re: [HACKERS] anole: assorted stability problems

2015-06-29 Thread Andres Freund
On 2015-06-29 00:42:53 -0400, Tom Lane wrote: #define S_UNLOCK(lock)\ do { _Asm_sched_fence(); (*(lock)) = 0; } while (0) Robert, how did you choose that? Isn't _Asm_sched_fence just a compiler barrier? Shouldn't this be a _Asm_mf()? which immediately raises the question of why

Re: [HACKERS] Rework the way multixact truncations work

2015-06-29 Thread Andres Freund
On 2015-06-29 23:54:40 +1200, Thomas Munro wrote: On Mon, Jun 22, 2015 at 7:24 AM, Andres Freund and...@anarazel.de wrote: It'd be very welcome to see some wider testing and review on this. I started looking at this and doing some testing. Here is some initial feedback: Perhaps

Re: [HACKERS] pg_trgm version 1.2

2015-06-29 Thread Merlin Moncure
On Sat, Jun 27, 2015 at 5:17 PM, Jeff Janes jeff.ja...@gmail.com wrote: This patch implements version 1.2 of contrib module pg_trgm. This supports the triconsistent function, introduced in version 9.4 of the server, to make it faster to implement indexed queries where some keys are common and

Re: [HACKERS] Rework the way multixact truncations work

2015-06-29 Thread Thomas Munro
On Mon, Jun 22, 2015 at 7:24 AM, Andres Freund and...@anarazel.de wrote: It'd be very welcome to see some wider testing and review on this. I started looking at this and doing some testing. Here is some initial feedback: Perhaps vac_truncate_clog needs a new name now that it does more, maybe

Re: [HACKERS] pg_file_settings view vs. Windows

2015-06-29 Thread Tom Lane
Sawada Masahiko sawada.m...@gmail.com writes: On Mon, Jun 29, 2015 at 12:20 AM, Tom Lane t...@sss.pgh.pa.us wrote: So let me make a radical proposal that both gets rid of the portability problem and, arguably, makes the view more useful than it is today. I think we should define the view as

[HACKERS] Reduce ProcArrayLock contention

2015-06-29 Thread Amit Kapila
I have been working on to analyze different ways to reduce the contention around ProcArrayLock. I have evaluated mainly 2 ideas, first one is to partition the ProcArrayLock (the basic idea is to allow multiple clients (equal to number of ProcArrayLock partitions) to perform

Re: [HACKERS] Oh, this is embarrassing: init file logic is still broken

2015-06-29 Thread Tatsuo Ishii
F.Y.I. here is the results I did on my laptop (Ubuntu 14, i7-4600U, 16GB mem, 512GB SSD). Unlike Josh, I used Unix domain sockets. In summary: 9.4.3: 943.439840 9.4.4: 429.953953 9.4 stable as of June 30: 929.804491 So comparing with 9.4.3, 9.4.4 is 54% slow, and 9.4-stable is 1.4% slow. I

Re: [HACKERS] Reduce ProcArrayLock contention

2015-06-29 Thread Andres Freund
On June 29, 2015 7:02:10 PM GMT+02:00, Simon Riggs si...@2ndquadrant.com wrote: On 29 June 2015 at 16:27, Amit Kapila amit.kapil...@gmail.com wrote: Thanks to Robert Haas for having discussion (offlist) about the idea and suggestions to improve it and also Andres Freund for having discussion

Re: [HACKERS] Reduce ProcArrayLock contention

2015-06-29 Thread Simon Riggs
On 29 June 2015 at 16:27, Amit Kapila amit.kapil...@gmail.com wrote: Thanks to Robert Haas for having discussion (offlist) about the idea and suggestions to improve it and also Andres Freund for having discussion and sharing thoughts about this idea at PGCon. Weird. This patch is

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-06-29 Thread Josh Berkus
On 06/29/2015 01:01 AM, Michael Paquier wrote: On Mon, Jun 29, 2015 at 4:20 AM, Josh Berkus j...@agliodbs.com wrote: Right. Well, another reason we should be using a system catalog and not a single GUC ... I assume that this takes into account the fact that you will still need a SIGHUP to

Re: [HACKERS] Reduce ProcArrayLock contention

2015-06-29 Thread Simon Riggs
On 29 June 2015 at 18:11, Andres Freund and...@anarazel.de wrote: On June 29, 2015 7:02:10 PM GMT+02:00, Simon Riggs si...@2ndquadrant.com wrote: On 29 June 2015 at 16:27, Amit Kapila amit.kapil...@gmail.com wrote: Thanks to Robert Haas for having discussion (offlist) about the idea

[HACKERS] Bug in bttext_abbrev_convert()

2015-06-29 Thread Peter Geoghegan
Commits b181a919 and arguably c79b6413 added bugs to bttext_abbrev_convert() in the process of fixing some others. In the master branch, bttext_abbrev_convert() can leak memory when the C locale happens to be used and we must detoast, which is unacceptable for about the same reason that it's

Re: [HACKERS] 9.5 release notes

2015-06-29 Thread Tom Lane
Andres Freund and...@anarazel.de writes: I'm working on integrating the suggestions I made last week to the release notes. Would anybody mind if I start to add commit ids in comments, similar to what Tom has done for minor releases, to news items? +1. Helps confirm which items are meant to

Re: [HACKERS] PANIC in GIN code

2015-06-29 Thread Heikki Linnakangas
On 06/29/2015 07:20 PM, Jeff Janes wrote: On Mon, Jun 29, 2015 at 1:37 AM, Heikki Linnakangas hlinn...@iki.fi wrote: Another piece of info here that might be relevant. Almost all UPDATE_META_PAGE xlog records other than the last one have two backup blocks. The last UPDATE_META_PAGE record

Re: [HACKERS] Solaris testers wanted for strxfrm() behavior

2015-06-29 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: Joyent confirms that the bug is fixed on SmartOS: The more interesting bit of information would be *when* it was fixed. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to

Re: [HACKERS] 9.5 release notes

2015-06-29 Thread Tom Lane
Andres Freund and...@anarazel.de writes: Haven't yet thought much about the format, maybe in the style of git log --pretty='format:[%h] %aN [%ci]: %s' upstream/master I'd personally like to see the hash and the time, the rest isn't particularly important to me. [ ... plays with that ... ]

Re: [HACKERS] Solaris testers wanted for strxfrm() behavior

2015-06-29 Thread Josh Berkus
On 06/29/2015 02:08 PM, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: Joyent confirms that the bug is fixed on SmartOS: The more interesting bit of information would be *when* it was fixed. Answer: not certain, but fixed at least 2 years ago. -- Josh Berkus PostgreSQL Experts Inc.

Re: [HACKERS] 9.5 release notes

2015-06-29 Thread Andres Freund
On 2015-06-29 17:30:57 -0400, Tom Lane wrote: Andres Freund and...@anarazel.de writes: Haven't yet thought much about the format, maybe in the style of git log --pretty='format:[%h] %aN [%ci]: %s' upstream/master I'd personally like to see the hash and the time, the rest isn't

Re: [HACKERS] 9.5 release notes

2015-06-29 Thread Andres Freund
Hi, On 2015-06-11 00:15:21 -0400, Bruce Momjian wrote: I have committed the first draft of the 9.5 release notes. I'm working on integrating the suggestions I made last week to the release notes. Would anybody mind if I start to add commit ids in comments, similar to what Tom has done for minor

Re: [HACKERS] pg_stat_*_columns?

2015-06-29 Thread Jim Nasby
On 6/26/15 6:09 PM, Joel Jacobson wrote: Can't we just use the infrastructure of PostgreSQL to handle the few megabytes of data we are talking about here? Why not just store the data in a regular table? Why bother with special files and special data structures? If it's just a table we want to

Re: [HACKERS] 9.5 release notes

2015-06-29 Thread Tom Lane
Andres Freund and...@anarazel.de writes: How about: git log --pretty='format:%cd [%h] %cN: %s' --date=short upstream/master Ah yes, didn't see that option. That's definitely better. I'd still vote for restricting it to fit in an 80-column window though. regards, tom

Re: [HACKERS] Solaris testers wanted for strxfrm() behavior

2015-06-29 Thread Josh Berkus
All, Joyent confirms that the bug is fixed on SmartOS: This is what I see: SunOS pkgsrc-pbulk-2014Q4-1.local 5.11 joyent_20141030T081701Z i86pc i386 i86pc locale is_IS.ISO8859-1: strxfrm returned 212; last modified byte at 58 (0x0) locale is_IS.ISO8859-1: strxfrm returned 212; last

Re: [HACKERS] 9.5 release notes

2015-06-29 Thread Andres Freund
On 2015-06-29 17:58:54 -0400, Tom Lane wrote: Yeah we are. The only places you'll find where we aren't formatting to 77 or 78 columns or so are where it would require breaking SGML tags in weird places. Which isn't exactly infrequent... Anyway, how about: format:%cd [%h] %(8,trunc)%cN:

Re: [HACKERS] 9.5 release notes

2015-06-29 Thread Andres Freund
On 2015-06-29 17:44:06 -0400, Tom Lane wrote: Andres Freund and...@anarazel.de writes: How about: git log --pretty='format:%cd [%h] %cN: %s' --date=short upstream/master Ah yes, didn't see that option. That's definitely better. I'd still vote for restricting it to fit in an 80-column

Re: [HACKERS] 9.5 release notes

2015-06-29 Thread Tom Lane
Andres Freund and...@anarazel.de writes: On 2015-06-29 17:44:06 -0400, Tom Lane wrote: Ah yes, didn't see that option. That's definitely better. I'd still vote for restricting it to fit in an 80-column window though. I don't see the benefit of truncating away the end of the commit message

Re: [HACKERS] 9.5 release notes

2015-06-29 Thread Andres Freund
On 2015-06-29 17:05:59 -0400, Tom Lane wrote: +1. Helps confirm which items are meant to correspond to which commits. That's what made me think of it. In case you didn't realize it already, the stuff I put into the minor release notes is from src/tools/git_changelog. Dunno whether we need

Re: [HACKERS] Parallel Seq Scan

2015-06-29 Thread Jeff Davis
[Jumping in without catching up on entire thread. Please let me know if these questions have already been covered.] 1. Can you change the name to something like ParallelHeapScan? Parallel Sequential is a contradiction. (I know this is bikeshedding and I won't protest further if you keep the

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-06-29 Thread Michael Paquier
On Mon, Jun 29, 2015 at 4:20 AM, Josh Berkus j...@agliodbs.com wrote: On 06/28/2015 04:36 AM, Sawada Masahiko wrote: On Sat, Jun 27, 2015 at 3:53 AM, Josh Berkus j...@agliodbs.com wrote: On 06/26/2015 11:32 AM, Robert Haas wrote: I think your proposal is worth considering, but you would need

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-06-29 Thread Etsuro Fujita
Hi KaiGai-san, On 2015/06/27 21:09, Kouhei Kaigai wrote: BTW, if you try newer version of postgres_fdw foreign join patch, please provide me to reproduce the problem/ OK Did you forget to attach the patch, or v17 is in use? Sorry, I made a mistake. The problem was produced using v16

Re: [HACKERS] pg_rewind failure by file deletion in source server

2015-06-29 Thread Michael Paquier
On Mon, Jun 29, 2015 at 3:46 AM, Heikki Linnakangas hlinn...@iki.fi wrote: On 06/24/2015 09:43 AM, Michael Paquier wrote: Attached is a new set of patches. Except for the last ones that addresses one issue of pg_rewind (symlink management when streaming PGDATA), all the others introduce

Re: [HACKERS] PANIC in GIN code

2015-06-29 Thread Heikki Linnakangas
On 06/29/2015 01:12 AM, Jeff Janes wrote: Now I'm getting a different error, with or without checksums. ERROR: invalid page in block 0 of relation base/16384/16420 CONTEXT: automatic vacuum of table jjanes.public.foo 16420 is the gin index. I can't even get the page with pageinspect:

Re: [HACKERS] pg_rewind failure by file deletion in source server

2015-06-29 Thread Michael Paquier
On Mon, Jun 29, 2015 at 3:53 PM, Vladimir Borodin r...@simply.name wrote: 28 июня 2015 г., в 21:46, Heikki Linnakangas hlinn...@iki.fi wrote: I've committed the additional option to the functions in genfile.c (I renamed it to missing_ok, for clarity), and the pg_rewind changes to use that

Re: [HACKERS] anole: assorted stability problems

2015-06-29 Thread Robert Haas
On Mon, Jun 29, 2015 at 6:11 AM, Andres Freund and...@anarazel.de wrote: On 2015-06-29 00:42:53 -0400, Tom Lane wrote: #define S_UNLOCK(lock)\ do { _Asm_sched_fence(); (*(lock)) = 0; } while (0) Robert, how did you choose that? Isn't _Asm_sched_fence just a compiler barrier?

Re: [HACKERS] Reduce ProcArrayLock contention

2015-06-29 Thread Robert Haas
On Mon, Jun 29, 2015 at 1:22 PM, Simon Riggs si...@2ndquadrant.com wrote: Yes, I know. And we all had a long conversation about how to do it without waking up the other procs. Forming a list, like we use for sync rep and having just a single process walk the queue was the way I suggested then

Re: [HACKERS] Solaris testers wanted for strxfrm() behavior

2015-06-29 Thread Robert Haas
On Mon, Jun 29, 2015 at 6:07 PM, Josh Berkus j...@agliodbs.com wrote: On 06/29/2015 02:08 PM, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: Joyent confirms that the bug is fixed on SmartOS: The more interesting bit of information would be *when* it was fixed. Answer: not certain,

Re: [HACKERS] anole: assorted stability problems

2015-06-29 Thread Andres Freund
On 2015-06-29 22:45:49 -0400, Robert Haas wrote: On Mon, Jun 29, 2015 at 10:32 PM, Andres Freund and...@anarazel.de wrote: On 2015-06-29 22:11:33 -0400, Robert Haas wrote: On Mon, Jun 29, 2015 at 6:11 AM, Andres Freund and...@anarazel.de wrote: On 2015-06-29 00:42:53 -0400, Tom Lane wrote:

[HACKERS] LWLock deadlock and gdb advice

2015-06-29 Thread Jeff Janes
I have a 9.5alpha1 cluster which is locked up. All the user back ends seem to be waiting on semop, eventually on WALInsertLockAcquire. Is there a way to use gdb to figure out who holds the lock they are waiting for? It is compiled with both debug and cassert. I am hoping someone can give me

Re: [HACKERS] anole: assorted stability problems

2015-06-29 Thread Andres Freund
On 2015-06-29 22:11:33 -0400, Robert Haas wrote: On Mon, Jun 29, 2015 at 6:11 AM, Andres Freund and...@anarazel.de wrote: On 2015-06-29 00:42:53 -0400, Tom Lane wrote: #define S_UNLOCK(lock)\ do { _Asm_sched_fence(); (*(lock)) = 0; } while (0) Robert, how did you choose

[HACKERS] Refactor to split nodeAgg.c?

2015-06-29 Thread Jeff Davis
I was going to rebase my HashAgg patch, and got some conflicts related to the grouping sets patch. I could probably sort them out, but I think that may be the tipping point where we want to break up nodeAgg.c into nodeSortedAgg.c and nodeHashAgg.c, and probably a common file as well. This would

Re: [HACKERS] anole: assorted stability problems

2015-06-29 Thread Robert Haas
On Mon, Jun 29, 2015 at 10:32 PM, Andres Freund and...@anarazel.de wrote: On 2015-06-29 22:11:33 -0400, Robert Haas wrote: On Mon, Jun 29, 2015 at 6:11 AM, Andres Freund and...@anarazel.de wrote: On 2015-06-29 00:42:53 -0400, Tom Lane wrote: #define S_UNLOCK(lock)\ do {

Re: [HACKERS] anole: assorted stability problems

2015-06-29 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Mon, Jun 29, 2015 at 10:32 PM, Andres Freund and...@anarazel.de wrote: You removed a volatile at the same time, and volatile on IA64 has acquire/release semantics. Can you explain what you mean by volatile having acquire/release semantics? I

Re: [HACKERS] PANIC in GIN code

2015-06-29 Thread Jeff Janes
On Mon, Jun 29, 2015 at 2:08 PM, Heikki Linnakangas hlinn...@iki.fi wrote: Now kill -9 postmaster, and restart. Voila, the page headers are all zeros: postgres=# select * from page_header(get_raw_page('i_foo', 0)); lsn| checksum | flags | lower | upper | special | pagesize |

Re: [HACKERS] anole: assorted stability problems

2015-06-29 Thread Robert Haas
On Mon, Jun 29, 2015 at 10:53 PM, Andres Freund and...@anarazel.de wrote: On 2015-06-29 22:45:49 -0400, Robert Haas wrote: On Mon, Jun 29, 2015 at 10:32 PM, Andres Freund and...@anarazel.de wrote: On 2015-06-29 22:11:33 -0400, Robert Haas wrote: On Mon, Jun 29, 2015 at 6:11 AM, Andres Freund

Re: [HACKERS] LWLock deadlock and gdb advice

2015-06-29 Thread Peter Geoghegan
On Mon, Jun 29, 2015 at 5:37 PM, Jeff Janes jeff.ja...@gmail.com wrote: Is there a way to use gdb to figure out who holds the lock they are waiting for? Have you considered building with LWLOCK_STATS defined, and LOCK_DEBUG defined? That might do it. Otherwise, I suggest dereferencing the l

Re: [HACKERS] Reduce ProcArrayLock contention

2015-06-29 Thread Amit Kapila
On Mon, Jun 29, 2015 at 10:52 PM, Simon Riggs si...@2ndquadrant.com wrote: On 29 June 2015 at 18:11, Andres Freund and...@anarazel.de wrote: On June 29, 2015 7:02:10 PM GMT+02:00, Simon Riggs si...@2ndquadrant.com wrote: On 29 June 2015 at 16:27, Amit Kapila amit.kapil...@gmail.com wrote:

Re: [HACKERS] Refactor to split nodeAgg.c?

2015-06-29 Thread Tom Lane
Jeff Davis pg...@j-davis.com writes: I was going to rebase my HashAgg patch, and got some conflicts related to the grouping sets patch. I could probably sort them out, but I think that may be the tipping point where we want to break up nodeAgg.c into nodeSortedAgg.c and nodeHashAgg.c, and

Re: [HACKERS] Bug in bttext_abbrev_convert()

2015-06-29 Thread Robert Haas
On Mon, Jun 29, 2015 at 7:47 PM, Peter Geoghegan p...@heroku.com wrote: Commits b181a919 and arguably c79b6413 added bugs to bttext_abbrev_convert() in the process of fixing some others. In the master branch, bttext_abbrev_convert() can leak memory when the C locale happens to be used and we

Re: [HACKERS] drop/truncate table sucks for large values of shared buffers

2015-06-29 Thread Amit Kapila
On Mon, Jun 29, 2015 at 7:18 PM, Simon Riggs si...@2ndquadrant.com wrote: On 28 June 2015 at 17:17, Tom Lane t...@sss.pgh.pa.us wrote: I'm not sure what you consider dire, but missing a dirty buffer belonging to the to-be-destroyed table would result in the system being permanently unable to

Re: [HACKERS] drop/truncate table sucks for large values of shared buffers

2015-06-29 Thread Amit Kapila
On Mon, Jun 29, 2015 at 5:41 AM, Amit Kapila amit.kapil...@gmail.com wrote: On Sun, Jun 28, 2015 at 9:05 PM, Tom Lane t...@sss.pgh.pa.us wrote: Amit Kapila amit.kapil...@gmail.com writes: On Sat, Jun 27, 2015 at 7:40 PM, Tom Lane t...@sss.pgh.pa.us wrote: I don't like this too much

Re: [HACKERS] LWLock deadlock and gdb advice

2015-06-29 Thread Amit Kapila
On Tue, Jun 30, 2015 at 6:25 AM, Peter Geoghegan p...@heroku.com wrote: On Mon, Jun 29, 2015 at 5:37 PM, Jeff Janes jeff.ja...@gmail.com wrote: Is there a way to use gdb to figure out who holds the lock they are waiting for? Have you considered building with LWLOCK_STATS defined, and

Re: [HACKERS] drop/truncate table sucks for large values of shared buffers

2015-06-29 Thread Simon Riggs
On 30 June 2015 at 05:02, Amit Kapila amit.kapil...@gmail.com wrote: On Mon, Jun 29, 2015 at 7:18 PM, Simon Riggs si...@2ndquadrant.com wrote: On 28 June 2015 at 17:17, Tom Lane t...@sss.pgh.pa.us wrote: I'm not sure what you consider dire, but missing a dirty buffer belonging to the

Re: [HACKERS] Solaris testers wanted for strxfrm() behavior

2015-06-29 Thread Noah Misch
On Mon, Jun 29, 2015 at 11:52:26AM +1200, Thomas Munro wrote: On Mon, Jun 29, 2015 at 10:57 AM, Tom Lane t...@sss.pgh.pa.us wrote: Thomas Munro thomas.mu...@enterprisedb.com writes: Just by the way, I wonder if this was that bug: https://illumos.org/issues/1594 Oooh. Might or might not

Re: [HACKERS] pg_trgm version 1.2

2015-06-29 Thread Merlin Moncure
On Mon, Jun 29, 2015 at 7:23 AM, Merlin Moncure mmonc...@gmail.com wrote: On Sat, Jun 27, 2015 at 5:17 PM, Jeff Janes jeff.ja...@gmail.com wrote: V1.1: Time: 1743.691 ms --- after repeated execution to warm the cache V1.2: Time: 2.839 ms --- after repeated execution to warm the cache