Re: [HACKERS] Idea: closing the loop for pg_ctl reload

2015-07-23 Thread Heikki Linnakangas
On 07/04/2015 04:24 AM, Jan de Visser wrote: On July 3, 2015 06:21:09 PM Tom Lane wrote: Jan de Visser j...@de-visser.net writes: Attached a new patch, rebased against the current head. Errors in pg_hba.conf and pg_ident.conf are now also noticed. I checked the documentation for pg_ctl

Re: [HACKERS] extend pgbench expressions with functions

2015-07-23 Thread Heikki Linnakangas
On 03/06/2015 11:41 AM, Fabien COELHO wrote: This patch extends pgbench expression with functions. Currently only one abs function is added. The point is rather to bootstrap the infrastructure for other functions (such as hash, random variants...) to be added later. I think it would actually

Re: [HACKERS] WAL logging problem in 9.4.3?

2015-07-22 Thread Heikki Linnakangas
On 07/22/2015 11:18 AM, Simon Riggs wrote: On 10 July 2015 at 00:06, Tom Lane t...@sss.pgh.pa.us wrote: Andres Freund and...@anarazel.de writes: On 2015-07-06 11:49:54 -0400, Tom Lane wrote: Rather than reverting cab9a0656c36739f, which would re-introduce a different performance problem,

Re: [HACKERS] polymorphic types - enforce casting to most common type automatically

2015-07-22 Thread Heikki Linnakangas
On 07/11/2015 12:19 AM, Pavel Stehule wrote: 2015-07-10 18:43 GMT+02:00 Tom Lane t...@sss.pgh.pa.us: An example of what would presumably happen if we adopted this sort of rule (I've not checked whether the patch as written does this, but it would logically follow) is that appending a float to

Re: [HACKERS] could not truncate directory pg_subtrans: apparent wraparound

2015-07-22 Thread Heikki Linnakangas
On 06/13/2015 05:02 AM, Thomas Munro wrote: Since the multixact equivalent of this problem[1] fell through the cracks on the multixact mega-thread, here is an updated patch that addresses this problem for both pg_subtrans and pg_multixact/offsets using the same approach: always step back one

Re: [HACKERS] Selectivity estimation for intarray with @@

2015-07-21 Thread Heikki Linnakangas
On 07/21/2015 03:44 PM, Alexander Korotkov wrote: While Uriy is on vacation, I've revised this patch a bit. I whacked this around quite a bit, and I think it's in a committable state now. But if you could run whatever tests you were using before on this, to make sure it still produces the

Re: [HACKERS] Fillfactor for GIN indexes

2015-07-21 Thread Heikki Linnakangas
On 07/21/2015 04:14 PM, Alexander Korotkov wrote: On Tue, Jul 21, 2015 at 3:52 PM, Heikki Linnakangas hlinn...@iki.fi wrote: On 07/21/2015 02:56 PM, Alexander Korotkov wrote: Probably, but currently we are in quite unlogical situation. We have default fillfactor = 90 for GiST where it has

Re: [HACKERS] Selectivity estimation for intarray with @@

2015-07-21 Thread Heikki Linnakangas
On 07/21/2015 07:28 PM, Alexander Korotkov wrote: On Tue, Jul 21, 2015 at 6:49 PM, Heikki Linnakangas hlinn...@iki.fi wrote: In this version of patch it's not checked if variable is actually and int[] not query_int. See following test case. # create table test2 as (select '1'::query_int val

Re: [HACKERS] PL/pgSQL, RAISE and error context

2015-07-21 Thread Heikki Linnakangas
On 07/21/2015 10:38 AM, Pavel Stehule wrote: where we are with this patch? Can I do some for it? I still feel this approach is misguided, and we should be tweaking psql and/or libpq instead. I don't feel strongly though, and if some other committer wants to pick this up in its current form,

Re: [HACKERS] Fillfactor for GIN indexes

2015-07-21 Thread Heikki Linnakangas
On 07/21/2015 02:00 PM, Alexander Korotkov wrote: On Tue, Jul 21, 2015 at 12:40 PM, Heikki Linnakangas hlinn...@iki.fi wrote: Has anyone done any performance testing of this? The purpose of a fillfactor is to avoid the storm of page splits right after building the index, when there are some

Re: [HACKERS] Fillfactor for GIN indexes

2015-07-21 Thread Heikki Linnakangas
Has anyone done any performance testing of this? The purpose of a fillfactor is to avoid the storm of page splits right after building the index, when there are some random updates to the table. It causes the index to bloat, as every full page is split to two half-full pages, and also adds

Re: [HACKERS] Fillfactor for GIN indexes

2015-07-21 Thread Heikki Linnakangas
On 07/21/2015 02:56 PM, Alexander Korotkov wrote: Probably, but currently we are in quite unlogical situation. We have default fillfactor = 90 for GiST where it has no use cases at all and effectively is just a waste of space. Why is it useless for GiST? - Heikki -- Sent via pgsql-hackers

[HACKERS] Re: [COMMITTERS] pgsql: Retain comments on indexes and constraints at ALTER TABLE ... TY

2015-07-20 Thread Heikki Linnakangas
On 07/18/2015 04:15 PM, Alvaro Herrera wrote: Heikki Linnakangas wrote: If it's there just to so you can run the regression tests that come with it, it might make sense to just add a default case to that switch to handle any unrecognized commands, and perhaps even remove the cases

[HACKERS] All-zero page in GIN index causes assertion failure

2015-07-20 Thread Heikki Linnakangas
...@gmail.com wrote: On Wed, Jul 15, 2015 at 8:44 AM, Heikki Linnakangas hlinn...@iki.fi wrote: I don't see how this is related to the LWLock issue, but I didn't see it without your patch. Perhaps the system just didn't survive long enough to uncover it without the patch (although it shows up pretty

Re: [HACKERS] All-zero page in GIN index causes assertion failure

2015-07-20 Thread Heikki Linnakangas
On 07/20/2015 11:14 AM, Heikki Linnakangas wrote: ISTM ginvacuumcleanup should check for PageIsNew, and put the page to the FSM. That's what btvacuumpage() gistvacuumcleanup() do. spgvacuumpage() seems to also check for PageIsNew(), but it seems broken in a different way: it initializes the page

Re: [HACKERS] 9.6 First Commitfest Begins

2015-07-17 Thread Heikki Linnakangas
ahead with the do the review you intend to do as soon as possible. Please also feel free to just review a patch without putting your name in, drive-by reviews are welcome. On 07/07/2015 02:31 PM, Heikki Linnakangas wrote: The first week of the July commitfest has passed. A lot of progress has

[HACKERS] Re: [COMMITTERS] pgsql: Retain comments on indexes and constraints at ALTER TABLE ... TY

2015-07-17 Thread Heikki Linnakangas
On 07/17/2015 05:40 PM, Michael Paquier wrote: On Fri, Jul 17, 2015 at 11:16 PM, Kevin Grittner kgri...@ymail.com wrote: Heikki Linnakangas heikki.linnakan...@iki.fi wrote: This fixes bug #13126, reported by Kirill Simonov. It looks like you missed something with the addition

Re: [HACKERS] LWLock deadlock and gdb advice

2015-07-15 Thread Heikki Linnakangas
On 06/30/2015 11:24 PM, Andres Freund wrote: On 2015-06-30 22:19:02 +0300, Heikki Linnakangas wrote: Hm. Right. A recheck of the value after the queuing should be sufficient to fix? That's how we deal with the exact same scenarios for the normal lock state, so that shouldn't be very hard to add

Re: [HACKERS] [BUGS] BUG #13126: table constraint loses its comment

2015-07-14 Thread Heikki Linnakangas
On 07/14/2015 10:29 AM, Michael Paquier wrote: On Tue, Jul 14, 2015 at 1:42 AM, Heikki Linnakangas wrote: I plan to commit the attached patches later today or tomorrow. But how do you feel about back-patching this? It should be possible to backpatch, although at a quick test it seems

Re: [HACKERS] WIP: Enhanced ALTER OPERATOR

2015-07-14 Thread Heikki Linnakangas
On 07/13/2015 03:43 PM, Uriy Zhuravlev wrote: Hello hackers. Attached is a new version of patch: * port syntax from NULL to truth NONE * fix documentation (thanks Heikki) * rebase to master Thanks, committed after some editing. I put all the logic into AlterOperator function, in

Re: [HACKERS] WIP: Enhanced ALTER OPERATOR

2015-07-14 Thread Heikki Linnakangas
On 07/14/2015 07:28 PM, Jeff Janes wrote: I'm getting some compiler warnings now: operatorcmds.c: In function 'AlterOperator': operatorcmds.c:504: warning: 'address.objectSubId' may be used uninitialized in this function operatorcmds.c:365: note: 'address.objectSubId' was declared here

Re: [HACKERS] dead assignment src/bin/scripts/print.c line 421

2015-07-13 Thread Heikki Linnakangas
On 07/13/2015 04:56 PM, Laurent Laborde wrote: Friendly greetings ! in file src/bin/scripts/print.c line 421 : need_recordsep = false; then set to true line 424. Now i'm pretty sure it's a meaningless bug without any consequence (the commit that introduced it is 15 years old). There is a lot

Re: [HACKERS] [BUGS] BUG #13126: table constraint loses its comment

2015-07-13 Thread Heikki Linnakangas
fix, on the grounds that it's not a very serious bug and there's always some risk in backpatching. - Heikki From c4865eb873a9cafb7e247cc69b7030243b74f3be Mon Sep 17 00:00:00 2001 From: Heikki Linnakangas heikki.linnakan...@iki.fi Date: Mon, 13 Jul 2015 19:22:31 +0300 Subject: [PATCH 1/2] Reformat

Re: [HACKERS] WAL logging problem in 9.4.3?

2015-07-10 Thread Heikki Linnakangas
On 07/10/2015 02:06 AM, Tom Lane wrote: Andres Freund and...@anarazel.de writes: On 2015-07-06 11:49:54 -0400, Tom Lane wrote: Rather than reverting cab9a0656c36739f, which would re-introduce a different performance problem, perhaps we could have COPY create a new relfilenode when it does

Re: [HACKERS] configure can't detect proper pthread flags

2015-07-10 Thread Heikki Linnakangas
On 07/08/2015 08:50 PM, Tom Lane wrote: Heikki Linnakangas hlinn...@iki.fi writes: The only scenario where you might now get warnings if we switch to upstream version, and didn't before, is if one of the flags makes pthreads to work, but also creates compiler warnings, while another flag later

Re: [HACKERS] Fillfactor for GIN indexes

2015-07-10 Thread Heikki Linnakangas
On 07/10/2015 01:13 PM, Alexander Korotkov wrote: On Fri, Jul 10, 2015 at 4:54 AM, Michael Paquier michael.paqu...@gmail.com wrote: + #define GIN_MIN_FILLFACTOR20 + #define GIN_DEFAULT_FILLFACTOR90 I am still worrying about using a default fillfactor at 90, as we did a lot

Re: [HACKERS] WIP: Enhanced ALTER OPERATOR

2015-07-10 Thread Heikki Linnakangas
On 07/06/2015 07:21 PM, Uriy Zhuravlev wrote: Hello hackers. This is the fifth version of the patch (the fourth was unsuccessful :)). I added documentation and was held a small refactoring. I edited the formatting of the syntax page a bit, and came up with this: ALTER OPERATOR name ( {

Re: [HACKERS] creating extension including dependencies

2015-07-10 Thread Heikki Linnakangas
On 07/09/2015 07:05 PM, Petr Jelinek wrote: On 2015-07-07 15:41, Andres Freund wrote: On 2015-07-07 22:36:29 +0900, Fujii Masao wrote: On Mon, Jun 15, 2015 at 7:50 AM, Petr Jelinek p...@2ndquadrant.com wrote: Hi, I am getting tired installing manually required extensions manually. I was

Re: ctidscan as an example of custom-scan (Re: [HACKERS] [v9.5] Custom Plan API)

2015-07-10 Thread Heikki Linnakangas
On 07/02/2015 08:21 AM, Kouhei Kaigai wrote: Folks, Moved this patch to next CF 2015-02 because of lack of review(ers). Do we still need this patch as contrib module? It was originally required it as example of custom-scan interface last summer, however, here was no strong requirement

Re: [HACKERS] Additional role attributes superuser review

2015-07-10 Thread Heikki Linnakangas
On 05/08/2015 07:35 AM, Stephen Frost wrote: Gavin, * Gavin Flower (gavinflo...@archidevsys.co.nz) wrote: What if I had a company with several subsidiaries using the same database, and want to prefix roles and other things with the subsidiary's initials? (I am not saying this would be a good

Re: [HACKERS] Fix to expose a GUC variable, Log_disconnections, to outside of postgres.c

2015-07-09 Thread Heikki Linnakangas
On 07/09/2015 07:19 AM, Satoshi Nagayasu wrote: On 2015/07/09 13:06, Tom Lane wrote: Satoshi Nagayasu sn...@uptime.jp writes: I just found that Log_disconnections value has not been exposed to outside of postgres.c. Despite that, Log_connections has already been exposed. Why would an

Re: [HACKERS] Improving log capture of TAP tests with IPC::Run

2015-07-09 Thread Heikki Linnakangas
On 07/09/2015 04:50 AM, Michael Paquier wrote: On Thu, Jul 9, 2015 at 12:49 AM, Heikki Linnakangas wrote: Looking at the manual page of Test::More, it looks like you could change where the perl script's STDOUT and STDERR point to, because Test::More takes a copy of them (when? at program

Re: [HACKERS] Improving log capture of TAP tests with IPC::Run

2015-07-09 Thread Heikki Linnakangas
On 07/09/2015 04:09 PM, Michael Paquier wrote: On Thu, Jul 9, 2015 at 7:29 PM, Heikki Linnakangas hlinn...@iki.fi wrote: The missing --debug is a separate issue. I won't argue against that, but is it worth creating a new thread just for this one-liner? The local mode of pg_rewind uses

Re: [HACKERS] configure can't detect proper pthread flags

2015-07-08 Thread Heikki Linnakangas
this should be committed to master and REL9_5_STABLE only. Thoughts? - Heikki From 711e5c7a1762f0c12e7b7e457c84484827246086 Mon Sep 17 00:00:00 2001 From: Heikki Linnakangas heikki.linnakan...@iki.fi Date: Wed, 8 Jul 2015 11:11:18 +0300 Subject: [PATCH 1/1] Replace our hacked version

Re: [HACKERS] Improving log capture of TAP tests with IPC::Run

2015-07-08 Thread Heikki Linnakangas
On 07/08/2015 11:26 AM, Michael Paquier wrote: On Wed, Jul 8, 2015 at 6:10 AM, Heikki Linnakangas hlinn...@iki.fi wrote: * whenever a test case is reported as success/fail. Just to be sure, does this concern the ok/not ok messages printed out by each test run? Or is it a custom message

Re: [HACKERS] configure can't detect proper pthread flags

2015-07-08 Thread Heikki Linnakangas
On 07/08/2015 04:38 PM, Tom Lane wrote: Heikki Linnakangas hlinn...@iki.fi writes: I suggest that we revert that work-around for that GCC bug, and stop testing the pthread flags as soon as we find one that works. OK ... Then we can also remove the test for whether the compiler produces any

Re: [HACKERS] Fillfactor for GIN indexes

2015-07-08 Thread Heikki Linnakangas
In dataPlaceToPageLeaf-function: if (append) { /* * Even when appending, trying to append more items than will fit is * not completely free, because we will merge the new items and old * items into an array

Re: [HACKERS] FPW compression leaks information

2015-07-07 Thread Heikki Linnakangas
On 07/07/2015 07:31 PM, Fujii Masao wrote: Or another crazy idea is to append random length dummy data into compressed FPW. Which would make it really hard for an attacker to guess the information from WAL location. It makes the signal more noisy, but you can still mount the same attack if

Re: [HACKERS] Improving log capture of TAP tests with IPC::Run

2015-07-07 Thread Heikki Linnakangas
On 06/25/2015 07:14 AM, Michael Paquier wrote: After looking at the issues with the TAP test suite that hamster faced a couple of days ago, which is what has been discussed on this thread: http://www.postgresql.org/message-id/13002.1434307...@sss.pgh.pa.us I have developed a patch to improve

Re: [HACKERS] FPW compression leaks information

2015-07-07 Thread Heikki Linnakangas
On 07/07/2015 04:31 PM, Stephen Frost wrote: * Heikki Linnakangas (hlinn...@iki.fi) wrote: On 07/07/2015 04:15 PM, Stephen Frost wrote: * Fujii Masao (masao.fu...@gmail.com) wrote: On Thu, Apr 16, 2015 at 4:26 PM, Michael Paquier michael.paqu...@gmail.com wrote: + the compression

Re: [HACKERS] Providing catalog view to pg_hba.conf file - Patch submission

2015-07-07 Thread Heikki Linnakangas
On 05/16/2015 06:00 AM, Haribabu Kommi wrote: Regarding next version- are you referring to 9.6 and therefore we should go ahead and bounce this to the next CF, or were you planning to post a next version of the patch today? Yes, for 9.6 version. No new patch emerged that could be reviewed in

Re: [HACKERS] 9.6 First Commitfest Begins

2015-07-07 Thread Heikki Linnakangas
The first week of the July commitfest has passed. A lot of progress has been made, but there's still a lot to do. There are still 53 patches in Needs Review state. Please pick a patch, and review it. Any patch. Don't be afraid of reviewing a patch that someone else has signed up for. If you

Re: [HACKERS] Transactions involving multiple postgres foreign servers

2015-07-07 Thread Heikki Linnakangas
On 02/17/2015 11:26 AM, Ashutosh Bapat wrote: Hi All, Here are the steps and infrastructure for achieving atomic commits across multiple foreign servers. I have tried to address most of the concerns raised in this mail thread before. Let me know, if I have left something. Attached is a WIP

Re: [HACKERS] Fix broken Install.bat when target directory contains a space

2015-07-07 Thread Heikki Linnakangas
On 07/07/2015 02:56 AM, Tom Lane wrote: Michael Paquier michael.paqu...@gmail.com writes: On Tue, Jul 7, 2015 at 4:19 AM, Heikki Linnakangas hlinn...@iki.fi wrote: Ok, committed that way. Shoudn't this patch be backpatched? In the backbranches install.bat does not work correctly with paths

Re: [HACKERS] PL/pgSQL, RAISE and error context

2015-07-07 Thread Heikki Linnakangas
On 01/26/2015 05:14 PM, Tom Lane wrote: Pavel Stehule pavel.steh...@gmail.com writes: 2015-01-26 14:02 GMT+01:00 Marko Tiikkaja ma...@joh.to: I am thinking, so solution /* if we are doing RAISE, don't report its location */ if (estate-err_text == raise_skip_msg) return;

Re: [HACKERS] FPW compression leaks information

2015-07-07 Thread Heikki Linnakangas
On 07/07/2015 04:15 PM, Stephen Frost wrote: * Fujii Masao (masao.fu...@gmail.com) wrote: On Thu, Apr 16, 2015 at 4:26 PM, Michael Paquier michael.paqu...@gmail.com wrote: + the compression ratio of a full page image gives a hint of what is + the existing data of this page.

Re: [HACKERS] PL/pgSQL, RAISE and error context

2015-07-07 Thread Heikki Linnakangas
On 07/07/2015 04:56 PM, Merlin Moncure wrote: On Tue, Jul 7, 2015 at 8:13 AM, Heikki Linnakangas hlinn...@iki.fi wrote: On 01/26/2015 05:14 PM, Tom Lane wrote: Pavel Stehule pavel.steh...@gmail.com writes: 2015-01-26 14:02 GMT+01:00 Marko Tiikkaja ma...@joh.to: I am thinking, so solution

Re: [HACKERS] PATCH:do not set Win32 server-side socket buffer size on windows 2012

2015-07-06 Thread Heikki Linnakangas
On 07/04/2015 02:03 PM, chenhj wrote: 2015-07-03 16:49:44,David Rowley david.row...@2ndquadrant.com wrote: I'm wondering what the original test setup was. I'm assuming psql and postgres both running on separate windows machines? I've tested the patch just connecting to a database running

Re: [HACKERS] PATCH:do not set Win32 server-side socket buffer size on windows 2012

2015-07-06 Thread Heikki Linnakangas
On 07/06/2015 04:30 PM, Tom Lane wrote: Heikki Linnakangas hlinn...@iki.fi writes: I was able to reproduce huge gains from this, after I introduced an artificial latency to all network packets with: tc qdisc add dev eth2 root netem delay 100ms With that, and with the client on different

Re: [HACKERS] Fix broken Install.bat when target directory contains a space

2015-07-06 Thread Heikki Linnakangas
On 07/06/2015 04:38 AM, Michael Paquier wrote: On Sat, Jul 4, 2015 at 3:57 AM, Heikki Linnakangas wrote: Hmm. Why is install.bat not like build.bat, i.e. just a thin wrapper that just calls install.pl, passing all arguments? I guess we just haven't noticed it. And indeed it makes everything

Re: [HACKERS] [RFC] LSN Map

2015-07-06 Thread Heikki Linnakangas
On 02/24/2015 04:55 AM, Robert Haas wrote: On Mon, Feb 23, 2015 at 12:52 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Dunno, but Jim's got a point. This is a maintenance burden to all indexams, if they all have to remember to update the LSN map separately. It needs to be done in some

Re: [HACKERS] Let PostgreSQL's On Schedule checkpoint write buffer smooth spread cycle by tuning IsCheckpointOnSchedule?

2015-07-05 Thread Heikki Linnakangas
On 07/04/2015 07:34 PM, Fabien COELHO wrote: In summary, the X^1.5 correction seems to work pretty well. It doesn't completely eliminate the problem, but it makes it a lot better. I've looked at the maths. I think that the load is distributed as the derivative of this function, that is (1.5

Re: [HACKERS] Let PostgreSQL's On Schedule checkpoint write buffer smooth spread cycle by tuning IsCheckpointOnSchedule?

2015-07-05 Thread Heikki Linnakangas
On 07/05/2015 08:19 AM, Fabien COELHO wrote: I am a bit skeptical about this. We need test scenarios that clearly show the benefit of having and of not having this behavior. It might be that doing this always is fine for everyone. Do you mean I have to proove that there is an actual problem

Re: [HACKERS] Fix broken Install.bat when target directory contains a space

2015-07-03 Thread Heikki Linnakangas
On 04/21/2015 04:02 PM, Michael Paquier wrote: On Tue, Apr 21, 2015 at 4:33 PM, Asif Naeem anaeem...@gmail.com wrote: The v2 patch looks good to me, just a minor concern on usage message i.e. C:\PG\postgresql\src\tools\msvcinstall Invalid command line options. Usage: install.bat targetdir

Re: [HACKERS] Asynchronous execution on FDW

2015-07-03 Thread Heikki Linnakangas
On 07/02/2015 08:48 AM, Kyotaro HORIGUCHI wrote: - It was a problem when to give the first kick for async exec. It is not in ExecInit phase, and ExecProc phase does not fit, too. An extra phase ExecPreProc or something is too invasive. So I tried pre-exec callback. Any init-node can

Re: [HACKERS] PATCH: remove nclients/nthreads constraint from pgbench

2015-07-03 Thread Heikki Linnakangas
On 07/03/2015 07:50 AM, Fabien COELHO wrote: This doesn't behave correctly if you set -j to greater than -c, and also use the rate limit option: Indeed. v3 attached fixed the case when nthreads nclients. Ok, committed. Thanks! - Heikki -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Fix pgbench --progress report under (very) low rate

2015-07-03 Thread Heikki Linnakangas
On 06/15/2015 09:12 PM, Fabien COELHO wrote: v3 rebase (after pgbench moved to src/bin) and minor style tweaking. v4 adds a fix to another progress timing issue: Currently if pgbench/postgres get stuck somewhere, the report catches up by repeating progresses several time in a row, which

Re: [HACKERS] Determine operator from it's function

2015-07-03 Thread Heikki Linnakangas
On 07/03/2015 01:20 AM, Jim Nasby wrote: Is there a way to determine the operator that resulted in calling the operator function? I thought fcinfo-flinfo-fn_expr might get set to the OpExpr, but seems it can be a FuncExpr even when called via an operator... Don't think there is. Why do you

Re: [HACKERS] PATCH: pgbench - remove thread fork-emulation

2015-07-03 Thread Heikki Linnakangas
On 04/28/2015 02:18 AM, Fabien COELHO wrote: This patch removes the pgbench thread fork-emulation code and simplifies things where possible, especially around pthread_create and pthread_join. The stats collection for the report is done directly instead of using an intermediate structure. As a

Re: [HACKERS] [PATCH] Generalized JSON output functions

2015-07-03 Thread Heikki Linnakangas
On 05/27/2015 09:51 PM, Andrew Dunstan wrote: On 05/27/2015 02:37 PM, Robert Haas wrote: On Tue, May 26, 2015 at 2:50 AM, Shulgin, Oleksandr oleksandr.shul...@zalando.de wrote: Is it reasonable to add this patch to CommitFest now? It's always reasonable to add a patch to the CommitFest if

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-07-03 Thread Heikki Linnakangas
On 06/21/2015 11:12 AM, Fabien COELHO wrote: Hello Josh, Add backslash continuations to pgbench custom scripts. [...] IMHO this approach is the best compromise. I don't personally agree. I believe that it it worth breaking backwards compatibility to support line breaks in pgbench

Re: [HACKERS] Patch to improve a few appendStringInfo* calls

2015-07-02 Thread Heikki Linnakangas
On 06/15/2015 03:56 AM, David Rowley wrote: On 29 May 2015 at 12:51, Peter Eisentraut pete...@gmx.net wrote: On 5/12/15 4:33 AM, David Rowley wrote: Shortly after I sent the previous patch I did a few more searches and also found some more things that are not quite right. Most of these are to

Re: [HACKERS] PATCH: remove nclients/nthreads constraint from pgbench

2015-07-02 Thread Heikki Linnakangas
On 05/08/2015 03:02 PM, Fabien COELHO wrote: Remove pgbench constraint that the number of clients must be a multiple of the number of threads, by sharing clients among threads as evenly as possible. Rational: allows to test the database load with any number of client without unrelated

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2015-07-02 Thread Heikki Linnakangas
On 07/01/2015 09:19 PM, Peter Geoghegan wrote: On Wed, Jul 1, 2015 at 10:45 AM, Peter Geoghegan p...@heroku.com wrote: On Tue, Jun 30, 2015 at 11:19 AM, Heikki Linnakangas hlinn...@iki.fi wrote: I agree it would be cleaner to have a separate CHECK_UNIQUE_XXX code for speculative insertions

[HACKERS] 9.6 First Commitfest Begins

2015-07-02 Thread Heikki Linnakangas
Hello everyone! It is time to start reviewing patches: 1. Pick a patch from the list at: https://commitfest.postgresql.org/5/ 2. Review it. Test it. Try to break it. Do we want the feature? Any weird interactions in corner-cases? 3. Reply on the thread on pgsql-hackers with your findings.

Re: [HACKERS] psql tabcomplete - minor bugfix - tabcomplete for SET ROLE TO xxx

2015-07-02 Thread Heikki Linnakangas
On 05/29/2015 10:41 AM, Pavel Stehule wrote: 2015-05-29 9:28 GMT+02:00 Jeevan Chalke jeevan.cha...@gmail.com: I agree with Peter that We don't tab-complete everything we possibly could, but using tabs after SET ROLE TO provides DEFAULT as an option which seems wrong. This patch adds list of

Re: [HACKERS] drop/truncate table sucks for large values of shared buffers

2015-07-02 Thread Heikki Linnakangas
On 07/02/2015 04:33 PM, Simon Riggs wrote: On 2 July 2015 at 14:08, Heikki Linnakangas hlinn...@iki.fi wrote: On 06/27/2015 07:45 AM, Amit Kapila wrote: Sometime back on one of the PostgreSQL blog [1], there was discussion about the performance of drop/truncate table for large values

Re: [HACKERS] drop/truncate table sucks for large values of shared buffers

2015-07-02 Thread Heikki Linnakangas
On 07/02/2015 04:18 PM, Amit Kapila wrote: On Thu, Jul 2, 2015 at 6:38 PM, Heikki Linnakangas hlinn...@iki.fi wrote: On 06/27/2015 07:45 AM, Amit Kapila wrote: Sometime back on one of the PostgreSQL blog [1], there was discussion about the performance of drop/truncate table for large values

Re: [HACKERS] drop/truncate table sucks for large values of shared buffers

2015-07-02 Thread Heikki Linnakangas
On 06/27/2015 07:45 AM, Amit Kapila wrote: Sometime back on one of the PostgreSQL blog [1], there was discussion about the performance of drop/truncate table for large values of shared_buffers and it seems that as the value of shared_buffers increase the performance of drop/truncate table

Re: [HACKERS] Add checksums without --initdb

2015-07-02 Thread Heikki Linnakangas
On 07/02/2015 11:28 PM, Andres Freund wrote: On 2015-07-02 22:53:40 +0300, Heikki Linnakangas wrote: Add a enabling-checksums mode to the server where it calculates checksums for anything it writes, but doesn't check or complain about incorrect checksums on reads. Put the server into that mode

Re: [HACKERS] PATCH:do not set Win32 server-side socket buffer size on windows 2012

2015-07-02 Thread Heikki Linnakangas
difference. It may be because the case is hobbled by other inefficiencies, in the virtualization or somewhere else, but I wonder if others can reproduce the speedup? - Heikki From 2a8b9c5fe10d5b8dc4e4bcf00ed0025bc6d24a23 Mon Sep 17 00:00:00 2001 From: Heikki Linnakangas heikki.linnakan...@iki.fi

Re: [HACKERS] Memory leak fixes for pg_dump, pg_dumpall, initdb and pg_upgrade

2015-07-02 Thread Heikki Linnakangas
On 06/08/2015 09:48 AM, Michael Paquier wrote: Hi all, Please find attached a set of fixes for a couple of things in src/bin: - pg_dump/pg_dumpall: -- getFormattedTypeName, convertTSFunction and myFormatType return strdup'd results that are never free'd. -- convertTSFunction returns const char.

Re: [HACKERS] Configurable location for extension .control files

2015-07-02 Thread Heikki Linnakangas
On 03/04/2015 09:41 AM, Oskari Saarenmaa wrote: 18.02.2015, 01:49, Jim Nasby kirjoitti: On 2/17/15 4:39 PM, Oskari Saarenmaa wrote: 10.06.2013, 17:51, Dimitri Fontaine kirjoitti: Andres Freund and...@2ndquadrant.com writes: In any case, no packager is going to ship an insecure-by-default

Re: [HACKERS] Fix autoconf deprecation warnings

2015-07-02 Thread Heikki Linnakangas
On 05/31/2015 05:22 AM, Andreas Karlsson wrote: I have attached new versions which apply on the current master. Thanks, applied. --- a/configure.in +++ b/configure.in @@ -1473,7 +1473,7 @@ if test x$pgac_cv_func_sigsetjmp = xyes; then AC_DEFINE(HAVE_SIGSETJMP, 1, [Define to 1 if you have

Re: [HACKERS] Time to fully remove heap_formtuple() and friends?

2015-07-02 Thread Heikki Linnakangas
On 06/13/2015 07:10 AM, Peter Geoghegan wrote: On Fri, Jun 12, 2015 at 8:47 PM, Tom Lane t...@sss.pgh.pa.us wrote: Peter Geoghegan p...@heroku.com writes: Attached patch actually removes heap_formtuple() and friends. Does this seem worthwhile? Seems reasonable, but at this point I would say

Re: [HACKERS] Add checksums without --initdb

2015-07-02 Thread Heikki Linnakangas
On 07/02/2015 10:39 PM, David Christensen wrote: Possible concerns here are whether checksums are included in WAL full_page_writes or if they are independently calculated; if the latter I think we’d be fine. If checksums are all handled at the layer below WAL than any streamed/processed changes

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2015-06-30 Thread Heikki Linnakangas
On 06/11/2015 02:19 AM, Peter Geoghegan wrote: Currently, speculative insertion (the INSERT ... ON CONFLICT DO UPDATE executor/storage infrastructure) uses checkUnique == UNIQUE_CHECK_PARTIAL for unique indexes, which is a constant originally only used by deferred unique constraints. It occurred

Re: [HACKERS] LWLock deadlock and gdb advice

2015-06-30 Thread Heikki Linnakangas
On 06/30/2015 07:37 PM, Alvaro Herrera wrote: Jeff Janes wrote: I've gotten the LWLock deadlock again. User backend 24841 holds the WALInsertLocks 7 and is blocked attempting to acquire 6 . So it seems to be violating the lock ordering rules (although I don't see that rule spelled out in

Re: [HACKERS] pg_rewind failure by file deletion in source server

2015-06-30 Thread Heikki Linnakangas
On 06/29/2015 09:44 AM, Michael Paquier wrote: On Mon, Jun 29, 2015 at 4:55 AM, Heikki Linnakangas wrote: But we'll still need to handle the pg_xlog symlink case somehow. Perhaps it would be enough to special-case pg_xlog for now. Well, sure, pg_rewind does not copy the soft links either way

Re: [HACKERS] LWLock deadlock and gdb advice

2015-06-30 Thread Heikki Linnakangas
On 06/30/2015 07:05 PM, Jeff Janes wrote: On Mon, Jun 29, 2015 at 11:28 PM, Jeff Janes jeff.ja...@gmail.com wrote: On Mon, Jun 29, 2015 at 5:55 PM, Peter Geoghegan p...@heroku.com wrote: On Mon, Jun 29, 2015 at 5:37 PM, Jeff Janes jeff.ja...@gmail.com wrote: Is there a way to use gdb to

Re: [HACKERS] LWLock deadlock and gdb advice

2015-06-30 Thread Heikki Linnakangas
On 06/30/2015 10:09 PM, Andres Freund wrote: On 2015-06-30 21:08:53 +0300, Heikki Linnakangas wrote: /* * XXX: We can significantly optimize this on platforms with 64bit * atomics

Re: [HACKERS] PANIC in GIN code

2015-06-30 Thread Heikki Linnakangas
On 06/30/2015 02:18 AM, Jeff Janes wrote: On Mon, Jun 29, 2015 at 2:08 PM, Heikki Linnakangas hlinn...@iki.fi wrote: I just pushed a fix for this, but unfortunately it didn't make it 9.5alpha1. Thanks. I think that that fixed it. It survived for over an hour this time. Thanks. I grepped

Re: [HACKERS] PANIC in GIN code

2015-06-29 Thread Heikki Linnakangas
On 06/29/2015 07:20 PM, Jeff Janes wrote: On Mon, Jun 29, 2015 at 1:37 AM, Heikki Linnakangas hlinn...@iki.fi wrote: Another piece of info here that might be relevant. Almost all UPDATE_META_PAGE xlog records other than the last one have two backup blocks. The last UPDATE_META_PAGE record

Re: [HACKERS] PANIC in GIN code

2015-06-29 Thread Heikki Linnakangas
On 06/29/2015 01:12 AM, Jeff Janes wrote: Now I'm getting a different error, with or without checksums. ERROR: invalid page in block 0 of relation base/16384/16420 CONTEXT: automatic vacuum of table jjanes.public.foo 16420 is the gin index. I can't even get the page with pageinspect:

Re: [HACKERS] pg_rewind failure by file deletion in source server

2015-06-28 Thread Heikki Linnakangas
On 06/26/2015 10:10 PM, Andres Freund wrote: On 2015-06-26 15:07:59 -0400, Robert Haas wrote: I realize that the recent fsync fiasco demonstrated that people keep files not readable by PG in the data directory It wasn't unreadable files that were the primary problem, it was files with read

Re: [HACKERS] Redesigning checkpoint_segments

2015-06-28 Thread Heikki Linnakangas
On 06/26/2015 02:08 PM, Heikki Linnakangas wrote: I'm not sure what to do about this. With the attached patch, you get the same leisurely pacing with restartpoints as you get with checkpoints, but you exceed max_wal_size during recovery, by the amount determined by checkpoint_completion_target

Re: [HACKERS] pg_rewind failure by file deletion in source server

2015-06-28 Thread Heikki Linnakangas
On 06/24/2015 09:43 AM, Michael Paquier wrote: Attached is a new set of patches. Except for the last ones that addresses one issue of pg_rewind (symlink management when streaming PGDATA), all the others introduce if_not_exists options for the functions of genfile.c. The pg_rewind stuff could be

Re: [HACKERS] PANIC in GIN code

2015-06-28 Thread Heikki Linnakangas
On 06/26/2015 10:53 PM, Jeff Janes wrote: On Fri, Jun 26, 2015 at 11:40 AM, Heikki Linnakangas hlinn...@iki.fi wrote: The page is being split (that's evident from info=48 above). ginPlaceToPage calls GinNewBuffer, which calls GetFreeIndexPage(). That finds a page that can be recycled

Re: [HACKERS] GIN: Implementing triConsistent and strategy number

2015-06-26 Thread Heikki Linnakangas
On 06/24/2015 11:11 PM, Jeff Janes wrote: Is there a way to implement triConsistent for only some of the strategy numbers? No. It looks like I would have to internally re-implement something like shimTriConsistentFn for each strategy number for which I don't want to implement the ternary

Re: [HACKERS] Should we back-patch SSL renegotiation fixes?

2015-06-26 Thread Heikki Linnakangas
On 06/26/2015 04:53 PM, Robert Haas wrote: On Thu, Jun 25, 2015 at 8:03 AM, Andres Freund and...@anarazel.de wrote: I don't accept the argument that there are not ways to tell users about things they might want to do. We probably could do that. But why would we want to? It's just as much

Re: [HACKERS] PANIC in GIN code

2015-06-26 Thread Heikki Linnakangas
On 06/26/2015 08:02 PM, Jeff Janes wrote: Under high load against f7bb7f0625771bc71869cda, I occasionally get: PANIC: XLogBeginInsert was not called It seems to only come from vacuuming. Here is an example back-trace: #0 0x003dcb632625 in raise () from /lib64/libc.so.6 #1

Re: [HACKERS] Redesigning checkpoint_segments

2015-06-26 Thread Heikki Linnakangas
On 05/27/2015 12:26 AM, Jeff Janes wrote: On Thu, May 21, 2015 at 8:40 AM, Fujii Masao masao.fu...@gmail.com wrote: On Thu, May 21, 2015 at 3:53 PM, Jeff Janes jeff.ja...@gmail.com wrote: One of the points of max_wal_size and its predecessor is to limit how big pg_xlog can grow. But running

Re: [HACKERS] Redesigning checkpoint_segments

2015-06-26 Thread Heikki Linnakangas
On 06/26/2015 03:40 PM, Robert Haas wrote: Actually, I've seen a number of presentations indicating that the pacing of checkpoints is already too aggressive near the beginning, because as soon as we initiate the checkpoint we have a storm of full page writes. I'm sure we can come up with

[HACKERS] Re: Let PostgreSQL's On Schedule checkpoint write buffer smooth spread cycle by tuning IsCheckpointOnSchedule?

2015-06-26 Thread Heikki Linnakangas
This needs more performance testing. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Should we back-patch SSL renegotiation fixes?

2015-06-25 Thread Heikki Linnakangas
On 06/25/2015 03:03 PM, Andres Freund wrote: The situation is this: We have broken code using broken code. I think we either got to apply, darn nontrivial, fixes from http://archives.postgresql.org/message-id/54DE6FAF.6050005%40vmware.com or we got to cripple the options. It's also not the

Re: [HACKERS] pg_rewind failure by file deletion in source server

2015-06-23 Thread Heikki Linnakangas
On 06/23/2015 07:51 AM, Michael Paquier wrote: So... Attached are a set of patches dedicated at fixing this issue: Thanks for working on this! - 0001, add if_not_exists to pg_tablespace_location, returning NULL if path does not exist - 0002, same with pg_stat_file, returning NULL if file

Re: [HACKERS] pg_rewind failure by file deletion in source server

2015-06-23 Thread Heikki Linnakangas
On 06/23/2015 05:03 PM, Fujii Masao wrote: On Tue, Jun 23, 2015 at 9:19 PM, Heikki Linnakangas hlinn...@iki.fi wrote: On 06/23/2015 07:51 AM, Michael Paquier wrote: So... Attached are a set of patches dedicated at fixing this issue: Thanks for working on this! - 0001, add if_not_exists

Re: [HACKERS] A couple of newlines missing in pg_rewind log entries

2015-06-23 Thread Heikki Linnakangas
On 06/23/2015 07:39 AM, Michael Paquier wrote: Hi all, Some grepping is showing up that a couple of newlines are missing in pg_rewind, leading to unreadable log entries: libpq_fetch.c:pg_log(PG_DEBUG, getting file chunks); Fixed. logging.c:pg_log(PG_PROGRESS, %*s/%s kB (%d%%)

Re: [HACKERS] user space function is_power_user

2015-06-22 Thread Heikki Linnakangas
On 06/22/2015 09:51 AM, Pavel Stehule wrote: Hi I often need a function for identification if current user is database owner or is superuser. It can be pretty simply implemented in C level. Do you think it should be available in core? current_setting('is_superuser'); - Heikki -- Sent

Re: [HACKERS] pg_stat_*_columns?

2015-06-20 Thread Heikki Linnakangas
On 06/20/2015 11:32 AM, Tom Lane wrote: Magnus Hagander mag...@hagander.net writes: On Sat, Jun 20, 2015 at 10:55 AM, Tom Lane t...@sss.pgh.pa.us wrote: I dunno that tweaking the format would accomplish much. Where I'd love to get to is to not have to write the data to disk at all (except at

<    1   2   3   4   5   6   7   8   9   10   >