Re: [HACKERS] Speedup twophase transactions

2016-09-19 Thread Michael Paquier
On Sat, Sep 17, 2016 at 2:45 AM, Stas Kelvich wrote: > Looking through old version i’ve noted few things that were bothering me: > > * In case of crash replay PREPARE redo accesses SUBTRANS, but > StartupSUBTRANS() isn’t called yet > in StartupXLOG(). > * Several functions in twophase.c have to w

[HACKERS] [RFC] Should we fix postmaster to avoid slow shutdown?

2016-09-19 Thread Tsunakawa, Takayuki
Hello, Please let me ask you about possible causes of a certain problem, slow shutdown of postmaster when a backend crashes, and whether to fix PostgreSQL. Our customer is using 64-bit PostgreSQL 9.2.8 on RHEL 6.4. Yes, the PostgreSQL version is rather old but there's no relevant bug fix in la

[HACKERS] Calculation of param_source_rels in add_paths_to_joinrel

2016-09-19 Thread Ashutosh Bapat
Hi, There's code in add_paths_to_joinrel() which computes the set of target relations that should overlap parameterization of any proposed join path. 120 /* 121 * Decide whether it's sensible to generate parameterized paths for this 122 * joinrel, and if so, which relations such p

Re: [HACKERS] WIP: Covering + unique indexes.

2016-09-19 Thread Amit Kapila
On Tue, Sep 6, 2016 at 10:18 PM, Anastasia Lubennikova wrote: > 28.08.2016 09:13, Amit Kapila: > > On Mon, Aug 15, 2016 at 8:15 PM, Anastasia Lubennikova > wrote: > > > So the patch is correct. > We can go further and remove this index_truncate_tuple() call, because > the first key of any inner (

Re: [HACKERS] pgbench - minor fix for meta command only scripts

2016-09-19 Thread Fabien COELHO
Hello Heikki, Yeah, it really is quite a mess. I tried to review your patch, and I think it's correct, but I couldn't totally convince myself, because of the existing messiness of the logic. Alas:-( So I bit the bullet and started refactoring. Wow! I came up with the attached. It refact

Re: [HACKERS] Improvements in psql hooks for variables

2016-09-19 Thread Ashish Tyagi
> Sorry about that, I forgot to make clean, here's an updated patch. Ongoing CMake changes will help to avoid such things, "out of source build". On Mon, Sep 19, 2016 at 6:20 PM, Daniel Verite wrote: > Rahila Syed wrote: > > > > I am beginning to review this patch. Initial comment. I got

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-09-19 Thread Dilip Kumar
On Tue, Sep 20, 2016 at 8:37 AM, Amit Kapila wrote: > I think it is former (8 socket machine). I confirm this is 8 sockets machine(cthulhu) > > > You point related to high-client count is valid and I am sure it won't > give noticeable benefit at lower client-count as the the > CLOGControlLock co

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-09-19 Thread Amit Kapila
On Tue, Sep 20, 2016 at 12:40 AM, Robert Haas wrote: > On Sun, Sep 18, 2016 at 5:11 PM, Tomas Vondra > wrote: >> IMHO in the ideal case the first message in this thread would provide a test >> case, demonstrating the effect of the patch. Then we wouldn't have the issue >> of looking for a good wo

[HACKERS] Remove the comment on the countereffectiveness of large shared_buffers on Windows

2016-09-19 Thread Tsunakawa, Takayuki
Hello, > From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Magnus Hagander > On Wed, Aug 24, 2016 at 4:35 AM, Tsunakawa, Takayuki > wrote: > As a similar topic, I wonder whether the following still holds true, > after many improvements on sh

Re: [HACKERS] Hash Indexes

2016-09-19 Thread AP
On Mon, Sep 19, 2016 at 05:50:13PM +1200, Mark Kirkwood wrote: > >I'm rather unenthused about having a hash index implementation that's > >mildly better in some corner cases, but otherwise doesn't have much > >benefit. That'll mean we'll have to step up our user education a lot, > >and we'll have t

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2016-09-19 Thread Craig Ringer
On 16 September 2016 at 21:28, Robert Haas wrote: > On Thu, Sep 15, 2016 at 8:52 PM, Craig Ringer wrote: >> On 2 September 2016 at 23:29, Petr Jelinek wrote: >> >>> You could put it to txid.c where all the other txid stuff is in? >> >> Yeah, even though it's in adt/ I think it'll do. >> >> I tho

Re: [HACKERS] PATCH: Keep one postmaster monitoring pipe per process

2016-09-19 Thread Andres Freund
On 2016-09-20 11:07:03 +1200, Thomas Munro wrote: > Yeah, I wondered why that was different than the pattern established > elsewhere when I was hacking on replication code. There are actually > several places where we call PostmasterIsAlive() unconditionally in a > loop that waits for WL_POSTMASTE

Re: [HACKERS] PATCH: Keep one postmaster monitoring pipe per process

2016-09-19 Thread Thomas Munro
On Sat, Sep 17, 2016 at 6:23 AM, Andres Freund wrote: > On 2016-09-16 09:55:48 +0200, Marco Pfatschbacher wrote: >> On Thu, Sep 15, 2016 at 12:26:16PM -0700, Andres Freund wrote: >> > Yikes, that's a pretty absurd implementation. >> >> Not when you take into account that it's been written over 20

Re: [HACKERS] [bug fix] pg_recvlogical is missing in the Windows installation

2016-09-19 Thread MauMau
From: Robert Haas On Sat, Sep 17, 2016 at 7:44 PM, Michael Paquier wrote: > On Sun, Sep 18, 2016 at 7:01 AM, MauMau wrote: >> pg_recvlogical is not included in the Windows client installation, >> which is performed by running "install client". The >> attached patch based on HEAD fixes this. I

Re: [HACKERS] pgbench - fix stats when using \sleep

2016-09-19 Thread Heikki Linnakangas
On 08/23/2016 05:47 PM, Fabien COELHO wrote: When \sleep is used within a pgbench script it resets txn_scheduled which is used for computing stats about the transaction, resulting in absurd statistics: latency average = 0.649 ms *** ??? *** ... script statistics: - statement latencies i

Re: [HACKERS] Declarative partitioning - another take

2016-09-19 Thread Robert Haas
On Mon, Aug 15, 2016 at 7:21 AM, Amit Langote wrote: >> +if (partexprs) >> +recordDependencyOnSingleRelExpr(&myself, >> +(Node *) partexprs, >> +RelationGetRelid(rel), >> +

Re: [HACKERS] Hash Indexes

2016-09-19 Thread Robert Haas
On Fri, Sep 16, 2016 at 2:38 PM, Andres Freund wrote: >> I think that exploring it well requires good code. If the code is good, >> why not commit it? > > Because getting there requires a lot of effort, debugging it afterwards > would take effort, and maintaining it would also takes a fair amount

Re: [HACKERS] PoC: Make it possible to disallow WHERE-less UPDATE and DELETE

2016-09-19 Thread David Fetter
On Mon, Sep 19, 2016 at 03:00:51PM -0400, Peter Eisentraut wrote: > On 9/19/16 12:02 AM, David Fetter wrote: > >> - The claim in the documentation that only superusers can do things > >> > with this module is not generally correct. > > I think that the claims are fixed. This is SUSET, at least i

Re: [HACKERS] Exclude schema during pg_restore

2016-09-19 Thread Michael Banck
Hi, sorry, it took me a while to find time to look at this. On Thu, Sep 01, 2016 at 09:39:56PM -0400, Peter Eisentraut wrote: > On 8/31/16 4:10 AM, Michael Banck wrote: > > attached is a small patch that adds an -N option to pg_restore, in order > > to exclude a schema, in addition to -n for the

Re: [HACKERS] Fix for PL/Python slow input arrays traversal issue

2016-09-19 Thread Dave Cramer
Yes, this should be closed as it is contained in https://commitfest.postgresql.org/10/697/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-09-19 Thread Andres Freund
On 2016-09-19 15:10:58 -0400, Robert Haas wrote: > Personally, I find the results so far posted on this thread thoroughly > unimpressive. I acknowledge that Dilip's results appear to show that > in a best-case scenario these patches produce a rather large gain. > However, that gain seems to happen

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-09-19 Thread Robert Haas
On Sun, Sep 18, 2016 at 5:11 PM, Tomas Vondra wrote: > IMHO in the ideal case the first message in this thread would provide a test > case, demonstrating the effect of the patch. Then we wouldn't have the issue > of looking for a good workload two years later. > > But now that I look at the first

Re: [HACKERS] PoC: Make it possible to disallow WHERE-less UPDATE and DELETE

2016-09-19 Thread Peter Eisentraut
On 9/19/16 12:02 AM, David Fetter wrote: >> - The claim in the documentation that only superusers can do things >> > with this module is not generally correct. > I think that the claims are fixed. This is SUSET, at least in this > patch, because anything short of that that changes query behavior

Re: [HACKERS] PoC: Make it possible to disallow WHERE-less UPDATE and DELETE

2016-09-19 Thread Robert Haas
On Mon, Sep 19, 2016 at 12:02 AM, David Fetter wrote: >> - The claim in the documentation that only superusers can do things >> with this module is not generally correct. > > I think that the claims are fixed. This is SUSET, at least in this > patch, because anything short of that that changes

Re: [HACKERS] [bug fix] pg_recvlogical is missing in the Windows installation

2016-09-19 Thread Robert Haas
On Sat, Sep 17, 2016 at 7:44 PM, Michael Paquier wrote: > On Sun, Sep 18, 2016 at 7:01 AM, MauMau wrote: >> pg_recvlogical is not included in the Windows client installation, >> which is performed by running "install client". The >> attached patch based on HEAD fixes this. I confirmed nothing

Re: [HACKERS] Rename max_parallel_degree?

2016-09-19 Thread Robert Haas
On Sat, Sep 17, 2016 at 2:40 AM, Amit Kapila wrote: > On Fri, Sep 16, 2016 at 11:54 PM, Robert Haas wrote: >> >> + /* >> +* We need a write barrier to make sure the update of >> +* parallel_terminate_count is done before the store to in_use >> +*/ >> >> Does the orde

Re: [HACKERS] Rename max_parallel_degree?

2016-09-19 Thread Robert Haas
On Fri, Sep 16, 2016 at 3:53 PM, Julien Rouhaud wrote: > That's fine by me. Should this be done (if there's no objection) in the > same patch, or in another one? I'd say "same patch". >> I'd suggest renaming the "parallel" flag to BackgroundWorkerSlot to >> "is_parallel_worker". Or, actually,

Re: [HACKERS] Parallel sec scan in plpgsql

2016-09-19 Thread Robert Haas
On Sat, Sep 17, 2016 at 11:54 PM, Amit Kapila wrote: > In general, I think we should support the cases as required (or > written) by you from plpgsql or sql functions. We need more work to > support such cases. There are probably two ways of supporting such > cases, we can build some intelligence

Re: [HACKERS] recovery_min_apply-delay and remote_apply

2016-09-19 Thread Robert Haas
On Fri, Sep 16, 2016 at 5:55 PM, Thomas Munro wrote: > On Sat, Sep 17, 2016 at 8:45 AM, Bernd Helmle wrote: >> Current PostgreSQL Documentation on recovery.conf has this about >> recovery_min_apply_delay[1]: >> >> ---<--- >> >> This parameter is intended for use with streaming replication deploym

Re: [HACKERS] more parallel query documentation

2016-09-19 Thread Robert Haas
On Fri, Sep 16, 2016 at 4:28 PM, Alvaro Herrera wrote: > I agree it should be added. I suggest that it could even be added to > the 9.6 docs, if you can make it. Here's a patch. I intend to commit this pretty quickly unless somebody objects, and also to backpatch it into 9.6. I'm sure it's not

Re: [HACKERS] Hash Indexes

2016-09-19 Thread Jeff Janes
On Sun, Sep 18, 2016 at 11:44 PM, Amit Kapila wrote: > On Mon, Sep 19, 2016 at 11:20 AM, Mark Kirkwood > wrote: > > On 17/09/16 06:38, Andres Freund wrote: > > > > While I see the point of what you are saying here, I recall all previous > > discussions about has indexes tended to go a bit like t

Re: [HACKERS] pageinspect: Hash index support

2016-09-19 Thread Jesper Pedersen
On 09/14/2016 04:21 PM, Jeff Janes wrote: I suggest that pageinspect functions are more convenient to use via the get_raw_page interface, that is, instead of reading the buffer themselves, the buffer is handed over from elsewhere and they receive it as bytea. This enables other use cases such as

Re: [HACKERS] Hash Indexes

2016-09-19 Thread Kenneth Marshall
On Mon, Sep 19, 2016 at 12:14:26PM +0530, Amit Kapila wrote: > On Mon, Sep 19, 2016 at 11:20 AM, Mark Kirkwood > wrote: > > > > > > On 17/09/16 06:38, Andres Freund wrote: > >> > >> On 2016-09-16 09:12:22 -0700, Jeff Janes wrote: > >>> > >>> On Thu, Sep 15, 2016 at 7:23 AM, Andres Freund > >>> wr

Re: [HACKERS] "Some tests to cover hash_index"

2016-09-19 Thread Mithun Cy
On Sat, Aug 6, 2016 at 9:41 AM, Amit Kapila wrote: > I wonder why you have included a new file for these tests, why can't be these added to existing hash_index.sql. tests in hash_index.sql did not cover overflow pages, above tests were for mainly for them. So I thought having a separate test file

Re: [HACKERS] PATCH: Keep one postmaster monitoring pipe per process

2016-09-19 Thread Robert Haas
On Fri, Sep 16, 2016 at 2:23 PM, Andres Freund wrote: >> Every read is an event, and that's what PostmasterIsAlive does. > > But in most places we only do a PostmasterIsAlive if WaitLatch returns > WL_POSTMASTER_DEATH. The only walreceiver related place that doesn't is > WalRcvWaitForStartPositio

Re: [HACKERS] Re: [COMMITTERS] pgsql: Change the way that LWLocks for extensions are allocated.

2016-09-19 Thread Amit Kapila
On Sat, Sep 10, 2016 at 5:02 PM, Amit Kapila wrote: > On Tue, Aug 30, 2016 at 8:01 AM, Andres Freund wrote: >> On 2016-08-30 07:57:19 +0530, Amit Kapila wrote: >>> I will write such a test case either in this week or early next week. >> >> Great. >> > > Okay, attached patch adds some simple tests

Re: [HACKERS] Improvements in psql hooks for variables

2016-09-19 Thread Daniel Verite
Rahila Syed wrote: > I am beginning to review this patch. Initial comment. I got following > compilation error when I applied the patch on latest sources and run make. Sorry about that, I forgot to make clean, here's an updated patch. Best regards, -- Daniel Vérité PostgreSQL-powered m

Re: [HACKERS] Possibly too stringent Assert() in b-tree code

2016-09-19 Thread Robert Haas
On Mon, Sep 19, 2016 at 7:07 AM, Amit Kapila wrote: >> Of course, the database could have been corrupted after having encountered >> many crashes during my experiments. Neverthelesss, even without in-depth >> knowledge of the b-tree code I suspect that this stack trace might reflect a >> legal sit

Re: [HACKERS] Possibly too stringent Assert() in b-tree code

2016-09-19 Thread Amit Kapila
On Mon, Sep 19, 2016 at 1:32 PM, Antonin Houska wrote: > I've recently seen this when using 9.6: > > #0 0x7f147892f0c7 in raise () from /lib64/libc.so.6 > #1 0x7f1478930478 in abort () from /lib64/libc.so.6 > #2 0x009683a1 in ExceptionalCondition (conditionName=0x9f2ef8 > "!(((

Re: [HACKERS] IF (NOT) EXISTS in psql-completion

2016-09-19 Thread Pavel Stehule
Hi > Pavel > > >> 2. Make keywords' case follow to input >> >> Allow the keywords suggested along with databse objects to >> follow the input letter case. The core part of this patch is a >> new function additional_kw_query(), which dynamically generates >> additional query string with s

[HACKERS] Possibly too stringent Assert() in b-tree code

2016-09-19 Thread Antonin Houska
I've recently seen this when using 9.6: #0 0x7f147892f0c7 in raise () from /lib64/libc.so.6 #1 0x7f1478930478 in abort () from /lib64/libc.so.6 #2 0x009683a1 in ExceptionalCondition (conditionName=0x9f2ef8 "!(((PageHeader) (page))->pd_special >= (__builtin_offsetof (PageHeaderD

Re: [HACKERS] Improvements in psql hooks for variables

2016-09-19 Thread Rahila Syed
Hello, I am beginning to review this patch. Initial comment. I got following compilation error when I applied the patch on latest sources and run make. command.c: In function ‘exec_command’: *command.c:257:5*: error: too few arguments to function ‘ParseVariableBool’ ParseVariableBool(opt1 +