[HACKERS] Re: [COMMITTERS] pgsql: Do all-visible handling in lazy_vacuum_page() outside its critic

2014-06-23 Thread Jeff Davis
On Fri, 2014-06-20 at 09:10 +, Andres Freund wrote: To fix, simply move all the all-visible handling outside the critical section. Doing so means that the PD_ALL_VISIBLE on the page won't be included in the full page image of the HEAP2_CLEAN record anymore. But that's fine, the flag will

Re: [HACKERS] [BUGS] BUG #10728: json_to_recordset with nested json objects NULLs columns

2014-06-23 Thread Andrew Dunstan
On 06/23/2014 11:43 AM, Tom Lane wrote: Michael Paquier michael.paqu...@gmail.com writes: Digging more into that, I have found the issue and a fix for it. It happens that populate_recordset_object_start, which is used to initialize the process for the population of the record, is taken *each*

Re: [HACKERS] New functions in sslinfo module

2014-06-23 Thread Andreas Karlsson
On 04/21/2014 07:51 AM, Воронин Дмитрий wrote: I put patch generated on git diffs to this letter. I make an a thread in postgresql commit fest: https://commitfest.postgresql.org/action/patch_view?id=1438 Thanks for the patch, it seems like a useful feature. === General === - Applies cleanly

Re: [HACKERS] idle_in_transaction_timeout

2014-06-23 Thread Kevin Grittner
Andres Freund and...@2ndquadrant.com wrote: On 2014-06-23 20:29:17 +0900, Fujii Masao wrote: On Mon, Jun 23, 2014 at 7:48 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-06-22 19:03:32 -0700, Kevin Grittner wrote: Andres Freund and...@2ndquadrant.com wrote: I think we'll want a

Re: [HACKERS] idle_in_transaction_timeout

2014-06-23 Thread David G Johnston
I think that'd be rather confusing. For one it'd need to be idle_in_transaction_timeout Why? We're cancelling an idle transaction, not an idle in transaction, whatever that is. ​The confusion derives from the fact we are affecting a session whose state is idle in transaction, not one

Re: [HACKERS] idle_in_transaction_timeout

2014-06-23 Thread David G Johnston
A long idle in transaction state pretty much always indicates a problematic interaction with postgres. True. Which makes me wonder whether we shouldn't default this to something non-zero -- even if it is 5 or 10 days. ​I guess it depends on how parental we want to be. But if we go that

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2014-06-23 Thread Stephen Frost
* Abhijit Menon-Sen (a...@2ndquadrant.com) wrote: At 2014-06-23 08:50:33 -0400, sfr...@snowman.net wrote: I'm not a huge fan of adding this as a contrib module I added it to the CF because we're interested in auditing functionality for 9.5, and as far as I can tell, there's nothing better

Re: [HACKERS] ALTER TABLESPACE MOVE command tag tweak

2014-06-23 Thread Stephen Frost
All, * Stephen Frost (sfr...@snowman.net) wrote: * Tom Lane (t...@sss.pgh.pa.us) wrote: Alvaro Herrera alvhe...@2ndquadrant.com writes: ALTER TABLESPACE MOVE is a glorified ALTER TABLE. If ALTER TABLESPACE MOVE returned ALTER TABLE as a tag, I think it'd work well too; but not ALTER

Re: [HACKERS] Extended Prefetching using Asynchronous IO - proposal and patch

2014-06-23 Thread Jim Nasby
On 6/20/14, 5:12 PM, John Lumby wrote: I also appreciate it is not easy to review the patch. There are really 4 (or maybe 5) parts : . async io (librt functions) . buffer management (allocating, locking and pinning etc) . scanners making prefetch calls .

[HACKERS] Re: [REVIEW] psql tab completion for DROP TRIGGER/RULE and ALTER TABLE ... DISABLE/ENABLE

2014-06-23 Thread Heikki Linnakangas
On 06/18/2014 03:39 AM, Andreas Karlsson wrote: On 06/18/2014 02:34 AM, Ian Barwick wrote: On 14/06/18 7:51, Andreas Karlsson wrote: On 06/17/2014 01:36 PM, Ian Barwick wrote: One issue - the table's internal triggers will also be listed. which can result in something like this: This is a

Re: [HACKERS] Sending out a request for more buildfarm animals?

2014-06-23 Thread Tomas Vondra
On 15.5.2014 00:41, Tomas Vondra wrote: On 13.5.2014 20:42, Tomas Vondra wrote: On 10.5.2014 20:21, Tomas Vondra wrote: On 9.5.2014 00:47, Tomas Vondra wrote: And I've requested 6 more animals - two for each compiler. One set for tests with basic CLOBBER, one set for recursive CLOBBER. Can

Re: [HACKERS] Extended Prefetching using Asynchronous IO - proposal and patch

2014-06-23 Thread John Lumby
Date: Thu, 19 Jun 2014 15:43:44 -0300 Subject: Re: Extended Prefetching using Asynchronous IO - proposal and patch From: klaussfre...@gmail.com To: st...@mit.edu CC: hlinnakan...@vmware.com; johnlu...@hotmail.com; pgsql-hackers@postgresql.org On

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2014-06-23 Thread Jeff Janes
On Fri, Mar 21, 2014 at 12:48 AM, Dilip kumar dilip.ku...@huawei.com wrote: On 16 January 2014 19:53, Euler Taveira Wrote, For the case where you have tables of varying size this would lead to a reduced overall processing time as it prevents large (read: long processing time) tables to be

[HACKERS] PostgreSQL for VAX on NetBSD/OpenBSD

2014-06-23 Thread Robert Haas
Hello VAX Enthusiasts: PostgreSQL currently has some very minimal code to support the VAX architecture. We have never supported OpenVMS, so this code would only be used if someone were to compile PostgreSQL for VAX on an operating system that we *do* support, such as NetBSD or OpenBSD. However,

Re: [HACKERS] autovacuum scheduling starvation and frenzy

2014-06-23 Thread Jeff Janes
On Thu, May 15, 2014 at 4:06 PM, Jeff Janes jeff.ja...@gmail.com wrote: On Thu, May 15, 2014 at 12:55 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Jeff Janes wrote: If you have a database with a large table in it that has just passed autovacuum_freeze_max_age, all future workers

[HACKERS] Re: [COMMITTERS] pgsql: Do all-visible handling in lazy_vacuum_page() outside its critic

2014-06-23 Thread Andres Freund
Hi, On 2014-06-23 13:00:11 -0700, Jeff Davis wrote: On Fri, 2014-06-20 at 09:10 +, Andres Freund wrote: To fix, simply move all the all-visible handling outside the critical section. Doing so means that the PD_ALL_VISIBLE on the page won't be included in the full page image of the

Re: [HACKERS] ALTER TABLESPACE MOVE command tag tweak

2014-06-23 Thread Alvaro Herrera
Stephen Frost wrote: Any further thoughts on this? I haven't tried to go implement anything yet but I'm definitely concerned that we may run into a keyword issue with ALTER TABLE, but I don't really want to add 'TABLES' either. Anyone have any other suggestions or ideas? I thought that the

Re: [HACKERS] idle_in_transaction_timeout

2014-06-23 Thread Andres Freund
On 2014-06-23 13:19:47 -0700, Kevin Grittner wrote: which already seems less clear (because the transaction belongs to idle) I have no idea what that means. It's idle_in_transaction_session_timeout. Not idle_in_transaction_session_timeout. and for another that distinction seems to be to

Re: [HACKERS] tab completion for setting search_path

2014-06-23 Thread Andres Freund
On 2014-06-23 13:10:34 -0400, Robert Haas wrote: On Mon, Jun 23, 2014 at 9:10 AM, Kevin Grittner kgri...@ymail.com wrote: I would be for excluding the pg_toast, pg_toast_temp_n, and pg_temp_n schemas, and including public and pg_catalog. +1. Jeff, are you willing to update the patch that

Re: [HACKERS] PostgreSQL for VAX on NetBSD/OpenBSD

2014-06-23 Thread Greg Stark
On Mon, Jun 23, 2014 at 3:09 PM, Robert Haas robertmh...@gmail.com wrote: However, we don't know of anyone who has tried to do this in a very long time, and are therefore considering removing the remaining support for the VAX platform. Has anyone tried to build PostgreSQL for VAX lately?

Re: [HACKERS] Extended Prefetching using Asynchronous IO - proposal and patch

2014-06-23 Thread Greg Stark
On Mon, Jun 23, 2014 at 2:43 PM, John Lumby johnlu...@hotmail.com wrote: It is when some *other* backend gets there first with the ReadBuffer that things are a bit trickier.The current version of the patch did polling for that case but that drew criticism,and so an imminent new

Re: [HACKERS] How about a proper TEMPORARY TABLESPACE?

2014-06-23 Thread Fabrízio de Royes Mello
On Mon, Jun 23, 2014 at 1:17 PM, Robert Haas robertmh...@gmail.com wrote: On Wed, Jun 18, 2014 at 9:39 PM, Matheus de Oliveira matioli.math...@gmail.com wrote: Then, to summarize Matheus must do: * use an option instead of change the syntax and catalog to indicate that a tablespace is

Re: [HACKERS] autovacuum scheduling starvation and frenzy

2014-06-23 Thread Tom Lane
Jeff Janes jeff.ja...@gmail.com writes: I didn't add this patch to the commitfest, because it was just a point for discussion and not actually proposed for application. But It doesn't seem to have provoked much discussion either. Should I go add this to the next commitfest? I do see it

Re: [HACKERS] [BUGS] BUG #10728: json_to_recordset with nested json objects NULLs columns

2014-06-23 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: On 06/23/2014 11:43 AM, Tom Lane wrote: However, it seems to me that these functions (json[b]_to_record[set]) are handling the nested-json-objects case in a fairly brain-dead fashion to start with. I would like to propose that we should think about

Re: [HACKERS] [BUGS] BUG #10728: json_to_recordset with nested json objects NULLs columns

2014-06-23 Thread Andrew Dunstan
On 06/23/2014 07:34 PM, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: On 06/23/2014 11:43 AM, Tom Lane wrote: However, it seems to me that these functions (json[b]_to_record[set]) are handling the nested-json-objects case in a fairly brain-dead fashion to start with. I would

Re: [HACKERS] ALTER TABLESPACE MOVE command tag tweak

2014-06-23 Thread Stephen Frost
* Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: Stephen Frost wrote: Any further thoughts on this? I haven't tried to go implement anything yet but I'm definitely concerned that we may run into a keyword issue with ALTER TABLE, but I don't really want to add 'TABLES' either. Anyone

Re: [HACKERS] PostgreSQL for VAX on NetBSD/OpenBSD

2014-06-23 Thread Robert Haas
On Mon, Jun 23, 2014 at 6:58 PM, Greg Stark st...@mit.edu wrote: On Mon, Jun 23, 2014 at 3:09 PM, Robert Haas robertmh...@gmail.com wrote: However, we don't know of anyone who has tried to do this in a very long time, and are therefore considering removing the remaining support for the VAX

Re: [HACKERS] autovacuum scheduling starvation and frenzy

2014-06-23 Thread Robert Haas
On Mon, Jun 23, 2014 at 7:19 PM, Tom Lane t...@sss.pgh.pa.us wrote: Jeff Janes jeff.ja...@gmail.com writes: I didn't add this patch to the commitfest, because it was just a point for discussion and not actually proposed for application. But It doesn't seem to have provoked much discussion

[HACKERS] Supporting Windows SChannel as OpenSSL replacement

2014-06-23 Thread Jeff Janes
On Wed, Jun 11, 2014 at 7:51 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: I did again the refactoring you did back in 2006, patch attached. One thing I did differently: I moved the raw, non-encrypted, read/write functions to separate functions: pqsecure_raw_read and

Re: [HACKERS] ALTER TABLESPACE MOVE command tag tweak

2014-06-23 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: That it's more-or-less a bulk 'ALTER TABLE' operation is why I had been trying to think of a way to put it under that command. What if we had a more general way to reference 'all objects in a tablespace'? tablespace.* or ALL:TABLESAPCE? Are there

[HACKERS] tab completion for setting search_path

2014-06-23 Thread Jeff Janes
On Mon, Jun 23, 2014 at 3:53 PM, Andres Freund and...@2ndquadrant.com javascript:; wrote: On 2014-06-23 13:10:34 -0400, Robert Haas wrote: On Mon, Jun 23, 2014 at 9:10 AM, Kevin Grittner kgri...@ymail.com javascript:; wrote: I would be for excluding the pg_toast, pg_toast_temp_n, and

Re: [HACKERS] [BUGS] BUG #10728: json_to_recordset with nested json objects NULLs columns

2014-06-23 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: On 06/23/2014 07:34 PM, Tom Lane wrote: I'm not following your comment about 9.3. The json[b]_to_record[set] functions are new in 9.4, which is what makes me feel it's not too late to redefine their behavior. But changing behavior of stuff that was

Re: [HACKERS] checking for interrupts during heap insertion

2014-06-23 Thread Robert Haas
On Mon, Jun 23, 2014 at 3:30 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: load, then it might take quite a long time. I'm thinking we might want a CHECK_FOR_INTERRUPTS() in the following two places: 1. Inside toast_save_datum, at the top of the loop that starts with while (data_todo

Re: [HACKERS] [BUGS] BUG #10728: json_to_recordset with nested json objects NULLs columns

2014-06-23 Thread Michael Paquier
On Tue, Jun 24, 2014 at 10:43 AM, Tom Lane t...@sss.pgh.pa.us wrote: Andrew Dunstan and...@dunslane.net writes: On 06/23/2014 07:34 PM, Tom Lane wrote: I'm not following your comment about 9.3. The json[b]_to_record[set] functions are new in 9.4, which is what makes me feel it's not too

Re: [HACKERS] ALTER TABLESPACE MOVE command tag tweak

2014-06-23 Thread Alvaro Herrera
Tom Lane wrote: Stephen Frost sfr...@snowman.net writes: That it's more-or-less a bulk 'ALTER TABLE' operation is why I had been trying to think of a way to put it under that command. What if we had a more general way to reference 'all objects in a tablespace'? tablespace.* or

Re: [HACKERS] [BUGS] BUG #10728: json_to_recordset with nested json objects NULLs columns

2014-06-23 Thread Michael Paquier
On Tue, Jun 24, 2014 at 9:34 AM, Andrew Dunstan and...@dunslane.net wrote: On 06/23/2014 07:34 PM, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: On 06/23/2014 11:43 AM, Tom Lane wrote: However, it seems to me that these functions (json[b]_to_record[set]) are handling the

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2014-06-23 Thread Abhijit Menon-Sen
At 2014-06-23 16:51:55 -0400, sfr...@snowman.net wrote: We can't control what gets audited through the catalog and have to manage that outside of PG, right? Right. I understand now. Are both the connected user and the current role that the command is running under logged? Yes, they are.

Re: [HACKERS] tab completion for setting search_path

2014-06-23 Thread Jeff Janes
On Mon, Jun 23, 2014 at 6:25 PM, Jeff Janes jeff.ja...@gmail.com wrote: On Mon, Jun 23, 2014 at 3:53 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-06-23 13:10:34 -0400, Robert Haas wrote: On Mon, Jun 23, 2014 at 9:10 AM, Kevin Grittner kgri...@ymail.com wrote: I would be for

[HACKERS] New developer TODO suggestions

2014-06-23 Thread Craig Ringer
Hi all Someone recently mentioned that there's no generate_series(numeric, numeric, numeric) . That strikes me as a great candidate for a new-developer-learning-PostgreSQL TODO. A couple of other things I occasionally run into that'd fit the bill: * A user-level elog(...) / ereport(...)

Re: [HACKERS] ALTER TABLESPACE MOVE command tag tweak

2014-06-23 Thread Noah Misch
On Mon, Jun 23, 2014 at 05:10:02PM -0400, Stephen Frost wrote: * Stephen Frost (sfr...@snowman.net) wrote: * Tom Lane (t...@sss.pgh.pa.us) wrote: I'm not against changing it- doing operations on a whole tablespace felt like it would make sense under 'ALTER TABLESPACE' to me (hence the

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-06-23 Thread Amit Kapila
On Mon, Jun 23, 2014 at 9:12 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-06-23 19:59:10 +0530, Amit Kapila wrote: On Tue, Jun 17, 2014 at 8:56 PM, Andres Freund and...@2ndquadrant.com wrote: 2. LWLockAcquireCommon() { .. if (!LWLockDequeueSelf(l)) { /* * Somebody

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2014-06-23 Thread Dilip kumar
On 24 June 2014 03:31, Jeff Wrote, Attached the latest updated patch 1. Rebased the patch to current GIT head. 2. Doc is updated. 3. Supported parallel execution for all db option also. This patch needs to be rebased after the analyze-in-stages patch,

Re: [HACKERS] ALTER TABLESPACE MOVE command tag tweak

2014-06-23 Thread Alvaro Herrera
Noah Misch wrote: Concerning the problem that started this thread, I would raise one ALTER TABLE event per table and not fire an event for the bulk request as a whole. Yes, that's how it already works. Essentially, ALTER TABLESPACE MOVE calls AlterTableInternal, and that function calls one

Re: [HACKERS] releaseOk and LWLockWaitForVar

2014-06-23 Thread Amit Kapila
On Mon, Jun 23, 2014 at 10:10 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 06/23/2014 05:38 PM, Amit Kapila wrote: While looking at function LWLockWaitForVar(), espacially below code: TRACE_POSTGRESQL_LWLOCK_ACQUIRE(T_NAME(l), T_ID(l), LW_EXCLUSIVE); I think in this function

Re: [HACKERS] Use a signal to trigger a memory context dump?

2014-06-23 Thread Noah Misch
+1 for having an API better than GDB to make a process emit a memory usage dump. This is my top non-crash cause for use of GDB in production. On Mon, Jun 23, 2014 at 07:21:22PM +0200, Andres Freund wrote: On 2014-06-23 10:07:36 -0700, Tom Lane wrote: Andres Freund and...@2ndquadrant.com

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2014-06-23 Thread Jeff Janes
On Monday, June 23, 2014, Dilip kumar dilip.ku...@huawei.com wrote: On 24 June 2014 03:31, Jeff Wrote, Attached the latest updated patch 1. Rebased the patch to current GIT head. 2. Doc is updated. 3. Supported parallel execution for all db option also. This patch needs to

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-06-23 Thread Amit Kapila
On Mon, Jun 23, 2014 at 9:12 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-06-23 19:59:10 +0530, Amit Kapila wrote: 12. #ifdef LWLOCK_DEBUG lock-owner = MyProc; #endif Shouldn't it be reset in LWLockRelease? That's actually intentional. It's quite useful to know the last

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2014-06-23 Thread Dilip kumar
On 24 June 2014 11:02 Jeff Wrote, I mean that the other commit, the one conflicting with your patch, is still not finished. It probably would not have been committed if we realized the problem at the time. That other patch runs analyze in stages at different settings of

<    1   2