Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2014-12-30 Thread Michael Paquier
On Tue, Dec 23, 2014 at 5:54 PM, Oskari Saarenmaa o...@ohmu.fi wrote: If the short-lived lock is the only blocker for this feature at the moment could we just require an additional qualifier for CONCURRENTLY (FORCE?) until the lock can be removed, something like: =# [blah] FWIW, I'd just

Re: [HACKERS] Maximum number of WAL files in the pg_xlog directory

2014-12-30 Thread Guillaume Lelarge
Sorry for my very late answer. It's been a tough month. 2014-11-27 0:00 GMT+01:00 Bruce Momjian br...@momjian.us: On Mon, Nov 3, 2014 at 12:39:26PM -0800, Jeff Janes wrote: It looked to me that the formula, when descending from a previously stressed state, would be: greatest(1 +

Re: [HACKERS] Compression of full-page-writes

2014-12-30 Thread Jeff Davis
On Fri, 2013-08-30 at 09:57 +0300, Heikki Linnakangas wrote: Speeding up the CRC calculation obviously won't help with the WAL volume per se, ie. you still generate the same amount of WAL that needs to be shipped in replication. But then again, if all you want to do is to reduce the volume,

Re: [HACKERS] Coverity and pgbench

2014-12-30 Thread Tatsuo Ishii
Anybody looks into problems in pgbench pointed out by Coverity? If no, I would like to work on fixing them because I need to write patches for -f option related issues anyway. Done. Best regards, -- Tatsuo Ishii SRA OSS, Inc. Japan English: http://www.sraoss.co.jp/index_en.php

Re: [HACKERS] Coverity and pgbench

2014-12-30 Thread Michael Paquier
On Tue, Dec 30, 2014 at 8:39 PM, Tatsuo Ishii is...@postgresql.org wrote: Anybody looks into problems in pgbench pointed out by Coverity? If no, I would like to work on fixing them because I need to write patches for -f option related issues anyway. Done. Just wondering, where are those

Re: [HACKERS] Coverity and pgbench

2014-12-30 Thread Tatsuo Ishii
Just wondering, where are those reports located? It would have been good to point out where the leaks actually were.. The report is a closed one. You need to be a member of PostgreSQL project of Coverity to look into the report. If want to join the project, you need to contact to one of admins

[HACKERS] Re: [HACKERS] Patch: add recovery_timeout option to control timeout of restore_command nonzero status code

2014-12-30 Thread Alexey Vasiliev
Hello. Thanks, I understand, what look in another part of code. Hope right now I did right changes. To not modify current pg_usleep calculation, I changed  restore_command_retry_interval value to seconds (not milliseconds). In this case, min value - 1 second. Mon, 29 Dec 2014 00:15:03 +0900

Re: [HACKERS] Compression of full-page-writes

2014-12-30 Thread Michael Paquier
On Tue, Dec 30, 2014 at 6:21 PM, Jeff Davis pg...@j-davis.com wrote: On Fri, 2013-08-30 at 09:57 +0300, Heikki Linnakangas wrote: Speeding up the CRC calculation obviously won't help with the WAL volume per se, ie. you still generate the same amount of WAL that needs to be shipped in

Re: [HACKERS] Compression of full-page-writes

2014-12-30 Thread Andres Freund
On 2014-12-30 21:23:38 +0900, Michael Paquier wrote: On Tue, Dec 30, 2014 at 6:21 PM, Jeff Davis pg...@j-davis.com wrote: On Fri, 2013-08-30 at 09:57 +0300, Heikki Linnakangas wrote: Speeding up the CRC calculation obviously won't help with the WAL volume per se, ie. you still generate the

Re: [HACKERS] [GENERAL] ON_ERROR_ROLLBACK

2014-12-30 Thread Bernd Helmle
--On 29. Dezember 2014 12:55:11 -0500 Tom Lane t...@sss.pgh.pa.us wrote: Given the lack of previous complaints, this probably isn't backpatching material, but it sure seems like a bit of attention to consistency would be warranted here. Now that i read it i remember a client complaining

Re: [HACKERS] Patch: add recovery_timeout option to control timeout of restore_command nonzero status code

2014-12-30 Thread Michael Paquier
On Tue, Dec 30, 2014 at 9:33 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Tue, Dec 30, 2014 at 9:10 PM, Alexey Vasiliev leopard...@inbox.ru wrote: To not modify current pg_usleep calculation, I changed restore_command_retry_interval value to seconds (not milliseconds). In this case,

Re: [HACKERS] Additional role attributes superuser review

2014-12-30 Thread Stephen Frost
* Amit Kapila (amit.kapil...@gmail.com) wrote: On Tue, Dec 30, 2014 at 6:52 AM, Stephen Frost sfr...@snowman.net wrote: There is one issue that occurs to me, however. We're talking about pg_dump, but what about pg_dumpall? In particular, I don't think the DUMP privilege should provide

Re: [HACKERS] What exactly is our CRC algorithm?

2014-12-30 Thread Heikki Linnakangas
On 12/30/2014 09:40 AM, Abhijit Menon-Sen wrote: Hi. I'm re-attaching the two patches as produced by format-patch. I haven't listed any reviewers. It's either just Andres, or maybe a lot of people. Is anyone in a position to try out the patches on MSVC and see if they build and work sensibly,

Re: [HACKERS] [GENERAL] ON_ERROR_ROLLBACK

2014-12-30 Thread Tom Lane
Bernd Helmle maili...@oopsware.de writes: --On 29. Dezember 2014 12:55:11 -0500 Tom Lane t...@sss.pgh.pa.us wrote: Given the lack of previous complaints, this probably isn't backpatching material, but it sure seems like a bit of attention to consistency would be warranted here. Now that i

Re: [HACKERS] Additional role attributes superuser review

2014-12-30 Thread Magnus Hagander
On Mon, Dec 29, 2014 at 11:01 PM, Stephen Frost sfr...@snowman.net wrote: * Adam Brightwell (adam.brightw...@crunchydatasolutions.com) wrote: I'd suggest it's called DUMP if that's what it allows, to keep it separate from the backup parts. Makes sense to me. I'm fine calling it

Re: [HACKERS] Serialization exception : Who else was involved?

2014-12-30 Thread Olivier MATROT
Indeed, NOTICE is wrong because it would doom the transaction that sets the flag if it should be later PREPARED. I think that reporting the PIDs and the current activity of each process would be nice. DeadLoackReport() is using pgstat_get_backend_current_activity() to get the process activity.

Re: [HACKERS] Additional role attributes superuser review

2014-12-30 Thread Stephen Frost
* Magnus Hagander (mag...@hagander.net) wrote: On Mon, Dec 29, 2014 at 11:01 PM, Stephen Frost sfr...@snowman.net wrote: That said, a 'DUMP' privilege which allows the user to dump the contents of the entire database is entirely reasonable. We need to be clear in the documentation though-

[HACKERS] Re[2]: [HACKERS] Patch: add recovery_timeout option to control timeout of restore_command nonzero status code

2014-12-30 Thread Alexey Vasiliev
Tue, 30 Dec 2014 21:39:33 +0900 от Michael Paquier michael.paqu...@gmail.com: On Tue, Dec 30, 2014 at 9:33 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Tue, Dec 30, 2014 at 9:10 PM, Alexey Vasiliev leopard...@inbox.ru wrote: To not modify current pg_usleep calculation, I

Re: [HACKERS] Detecting backend failures via libpq / DBD::Pg

2014-12-30 Thread Greg Sabino Mullane
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 Andrew Gierth asked: this is to send a simple SELECT via PQexec Why not PQexec(conn, ) ? Because I want to leave a good clue for debugging; so DBAs are better able to figure out where a mystery slew of queries is coming from. The query

Re: [HACKERS] What exactly is our CRC algorithm?

2014-12-30 Thread Abhijit Menon-Sen
At 2014-12-30 16:05:50 +0200, hlinnakan...@vmware.com wrote: A couple of quick comments: bswap32 is unused on on little-endian systems. That will give a compiler warning. Huh. I don't get a warning, even when I add -Wunused to the build flags. But since you mention it, it would be better to

Re: [HACKERS] BUG #12330: ACID is broken for unique constraints

2014-12-30 Thread Merlin Moncure
On Mon, Dec 29, 2014 at 3:53 PM, Kevin Grittner kgri...@ymail.com wrote: I tend to build out applications on top of functions and the inability to set isolation mode inside a function confounds me from using anything but 'read committed'. Hey, no problem -- just set

Re: [HACKERS] [GENERAL] ON_ERROR_ROLLBACK

2014-12-30 Thread David Johnston
On Tue, Dec 30, 2014 at 8:54 AM, Adrian Klaver adrian.kla...@aklaver.com wrote: On 12/30/2014 07:43 AM, David G Johnston wrote: Tom Lane-2 wrote Bernd Helmle lt; mailings@ gt; writes: --On 29. Dezember 2014 12:55:11 -0500 Tom Lane lt; tgl@.pa gt; wrote: Given the lack of

[HACKERS] pg_ctl {start,restart,reload} bad handling of stdout file descriptor

2014-12-30 Thread Luis Menina
Hi, I've been trying to run some pg_ctl command inside a python script, and saw that some of them where deadlocking. It seems that the commands that start postgres handle stdout in a way that that block the caller. Redirecting stdout to /dev/null or to a file using the -l option allow me to

Re: [HACKERS] pg_ctl {start,restart,reload} bad handling of stdout file descriptor

2014-12-30 Thread Luis Menina
I knew I'd forget something... I'm running Postgres 9.3.5 on Debian/Linux. Regards, -- Luis MENINA / Software Engineer Web: www.anevia.com Anevia: 1 rue René Anjolvy, 94250 Gentilly, France -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] 9.5: Better memory accounting, towards memory-bounded HashAgg

2014-12-30 Thread Jeff Janes
On Sun, Dec 28, 2014 at 12:45 PM, Peter Geoghegan p...@heroku.com wrote: On Sun, Dec 28, 2014 at 12:37 PM, Jeff Davis pg...@j-davis.com wrote: Do others have similar numbers? I'm quite surprised at how little work_mem seems to matter for these plans (HashJoin might be a different story

Re: [HACKERS] Documentation of bt_page_items()'s ctid field

2014-12-30 Thread Heikki Linnakangas
On 12/30/2014 04:08 AM, Peter Geoghegan wrote: Attached documentation patch describes the purpose of bt_page_items()'s ctid field. This has come up enough times in disaster recovery or testing scenarios that I feel it's worth drawing particular attention to. How much detail on the b-tree

Re: [HACKERS] BUG #12330: ACID is broken for unique constraints

2014-12-30 Thread Greg Stark
On Mon, Dec 29, 2014 at 4:17 PM, Merlin Moncure mmonc...@gmail.com wrote: Serialization errors only exist as a concession to concurrency and performance. Again, they should be returned as sparsely as possible I think this is fuzzy thinking. Serialization *errors* themselves are a concession

Re: [HACKERS] What exactly is our CRC algorithm?

2014-12-30 Thread Andres Freund
On 2014-12-30 21:36:19 +0530, Abhijit Menon-Sen wrote: Thanks for spotting that. I had meant to change the test to 7. But again, now that you mention it, I'm not sure it's necessary. The CRC32* instructions don't have the usual SSE alignment requirements, and I see that the Linux kernel

Re: [HACKERS] Maximum number of WAL files in the pg_xlog directory

2014-12-30 Thread Jeff Janes
On Tue, Dec 30, 2014 at 12:35 AM, Guillaume Lelarge guilla...@lelarge.info wrote: Sorry for my very late answer. It's been a tough month. 2014-11-27 0:00 GMT+01:00 Bruce Momjian br...@momjian.us: On Mon, Nov 3, 2014 at 12:39:26PM -0800, Jeff Janes wrote: It looked to me that the formula,

Re: [HACKERS] Documentation of bt_page_items()'s ctid field

2014-12-30 Thread Peter Geoghegan
On Tue, Dec 30, 2014 at 8:59 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: How much detail on the b-tree internals do we want to put in the pageinspect documentation? I can see that being useful, but should we also explain e.g. that the first item on each (non-rightmost) page is the

Re: [HACKERS] Documentation of bt_page_items()'s ctid field

2014-12-30 Thread Heikki Linnakangas
On 12/30/2014 10:07 PM, Peter Geoghegan wrote: On Tue, Dec 30, 2014 at 8:59 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: How much detail on the b-tree internals do we want to put in the pageinspect documentation? I can see that being useful, but should we also explain e.g. that the

Re: [HACKERS] Documentation of bt_page_items()'s ctid field

2014-12-30 Thread Peter Geoghegan
On Tue, Dec 30, 2014 at 12:21 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: In practice, we never store any actual key value for the minus infinity key. I guess the code would ignore it if it was there, but it would make more sense to explain that the first data key on an internal page

Re: [HACKERS] pg_ctl {start,restart,reload} bad handling of stdout file descriptor

2014-12-30 Thread Tom Lane
Luis Menina lmen...@anevia.com writes: I've been trying to run some pg_ctl command inside a python script, and saw that some of them where deadlocking. It seems that the commands that start postgres handle stdout in a way that that block the caller. Redirecting stdout to /dev/null or to a file

[HACKERS] Re: [COMMITTERS] pgsql: pg_event_trigger_dropped_objects: Add name/args output columns

2014-12-30 Thread Alvaro Herrera
Alvaro Herrera wrote: pg_event_trigger_dropped_objects: Add name/args output columns These columns can be passed to pg_get_object_address() and used to reconstruct the dropped objects identities in a remote server containing similar objects, so that the drop can be replicated. This is

Re: [HACKERS] [GENERAL] ON_ERROR_ROLLBACK

2014-12-30 Thread Andrew Dunstan
On 12/30/2014 09:20 AM, Tom Lane wrote: Bernd Helmle maili...@oopsware.de writes: --On 29. Dezember 2014 12:55:11 -0500 Tom Lane t...@sss.pgh.pa.us wrote: Given the lack of previous complaints, this probably isn't backpatching material, but it sure seems like a bit of attention to consistency

Re: [HACKERS] Re: [COMMITTERS] pgsql: pg_event_trigger_dropped_objects: Add name/args output columns

2014-12-30 Thread Tom Lane
Alvaro Herrera alvhe...@2ndquadrant.com writes: Alvaro Herrera wrote: pg_event_trigger_dropped_objects: Add name/args output columns This is causing buildfarm member dunlin to fail: ... No other member so far has reported a problem here. Not sure if that's the strangest bit, or the fact

[HACKERS] Failure on markhor with CLOBBER_CACHE_ALWAYS for test brin

2014-12-30 Thread Michael Paquier
HI all. markhor has run for the first time in 8 days, and there is something in range e703261..72dd233 making the regression test of brin crashing. See here: http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=markhordt=2014-12-30%2020%3A58%3A49 Regards, -- Michael -- Sent via

Re: [HACKERS] POLA violation with \c service=

2014-12-30 Thread David Fetter
On Wed, Dec 17, 2014 at 08:14:04AM -0500, Andrew Dunstan wrote: On 12/17/2014 04:11 AM, Heikki Linnakangas wrote: On 12/17/2014 10:03 AM, Albe Laurenz wrote: David Fetter wrote: I've noticed that psql's \c function handles service= requests in a way that I can only characterize as broken.

Re: [HACKERS] Failure on markhor with CLOBBER_CACHE_ALWAYS for test brin

2014-12-30 Thread Alvaro Herrera
Michael Paquier wrote: HI all. markhor has run for the first time in 8 days, and there is something in range e703261..72dd233 making the regression test of brin crashing. See here: http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=markhordt=2014-12-30%2020%3A58%3A49 This shows that

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-12-30 Thread Peter Geoghegan
On Mon, Dec 29, 2014 at 11:52 PM, Jeff Janes jeff.ja...@gmail.com wrote: Correct, I haven't seen any problems with approach #1 That helps with debugging #2, then. That's very helpful. Generally the problem will occur early on in the process, and if not then it will not occur at all. I think

Re: [HACKERS] Additional role attributes superuser review

2014-12-30 Thread Amit Kapila
On Tue, Dec 30, 2014 at 6:35 PM, Stephen Frost sfr...@snowman.net wrote: * Amit Kapila (amit.kapil...@gmail.com) wrote: On Tue, Dec 30, 2014 at 6:52 AM, Stephen Frost sfr...@snowman.net wrote: another could be have a separate privilege (DBA) or a role which is not a superuser, however can

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

2014-12-30 Thread Robert Haas
On Fri, Dec 26, 2014 at 7:57 AM, David Rowley dgrowle...@gmail.com wrote: 1. Do we need to keep the 128 byte aggregate state size for machines without 128 bit ints? This has been reduced to 48 bytes in the patch, which is in favour code being compiled with a compiler which has 128 bit ints. I

Re: [HACKERS] orangutan seizes up during isolation-check

2014-12-30 Thread Robert Haas
On Sun, Dec 28, 2014 at 4:58 PM, Noah Misch n...@leadboat.com wrote: I wondered whether to downgrade FATAL to LOG in back branches. Introducing a new reason to block startup is disruptive for a minor release, but having the postmaster deadlock at an unpredictable later time is even more

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-12-30 Thread Guillaume Lelarge
2014-12-12 14:58 GMT+01:00 Heikki Linnakangas hlinnakan...@vmware.com: On 12/10/2014 04:32 PM, Dennis Kögel wrote: Hi, Am 04.09.2014 um 17:50 schrieb Jehan-Guillaume de Rorthais j...@dalibo.com: Since few months, we occasionally see .ready files appearing on some slave instances from

Re: [HACKERS] Maximum number of WAL files in the pg_xlog directory

2014-12-30 Thread Guillaume Lelarge
2014-12-30 18:45 GMT+01:00 Jeff Janes jeff.ja...@gmail.com: On Tue, Dec 30, 2014 at 12:35 AM, Guillaume Lelarge guilla...@lelarge.info wrote: Sorry for my very late answer. It's been a tough month. 2014-11-27 0:00 GMT+01:00 Bruce Momjian br...@momjian.us: On Mon, Nov 3, 2014 at