Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-16 Thread David Rowley
On Wed, Jan 15, 2014 at 5:39 AM, Florian Pflug f...@phlo.org wrote: The notnullcount machinery seems to apply to both STRICT and non-STRICT transfer function pairs. Shouldn't that be constrained to STRICT transfer function pairs? For non-STRICT pairs, it's up to the transfer functions to deal

Re: [Lsf-pc] [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-16 Thread knizhnik
I wonder if kernel can sometimes provide weaker version of fsync() which is not enforcing all pending data to be written immediately but just servers as write barrier, guaranteeing that all write operations preceding fsync() will be completed before any of subsequent operations. It will allow

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2014-01-16 Thread Heikki Linnakangas
On 01/16/2014 03:25 AM, Peter Geoghegan wrote: I think you should consider breaking off the relcache parts of my patch and committing them, because they're independently useful. If we are going to have a lot of conflicts that need to be handled by a heap_delete(), there is no point in inserting

Re: [HACKERS] Display oprcode and its volatility in \do+

2014-01-16 Thread Rushabh Lathia
Hi, I have reviewed you patch. -- Patch got applied cleanly (using patch -p1) -- Make Make install works fine -- make check looks good I done code-walk and it looks good. Also did some manual testing and haven't found any issue with the implementation. Even I personally felt the Function and

Re: [HACKERS] PoC: Partial sort

2014-01-16 Thread Jeremy Harris
On 22/12/13 20:26, Alexander Korotkov wrote: On Sat, Dec 14, 2013 at 6:30 PM, Jeremy Harris j...@wizmail.org wrote: On 14/12/13 12:54, Andres Freund wrote: Is that actually all that beneficial when sorting with a bog standard qsort() since that doesn't generally benefit from data being

Re: [HACKERS] patch: option --if-exists for pg_dump

2014-01-16 Thread Jeevan Chalke
Hi Pavel, I have reviewed the patch and here are my concerns and notes: POSITIVES: --- 1. Patch applies with some white-space errors. 2. make / make install / make check is smooth. No issues as such. 3. Feature looks good as well. 4. NO concern on overall design. 5. Good work. NEGATIVES: ---

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2014-01-16 Thread Peter Geoghegan
On Thu, Jan 16, 2014 at 12:35 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Makes sense. Can you extract that into a separate patch, please? Okay. On an unrelated note, here are results for a benchmark that compares the two patches for an insert heavy workload:

Re: [HACKERS] Display oprcode and its volatility in \do+

2014-01-16 Thread Marko Tiikkaja
On 1/16/14 9:53 AM, Rushabh Lathia wrote: I have reviewed you patch. -- Patch got applied cleanly (using patch -p1) -- Make Make install works fine -- make check looks good I done code-walk and it looks good. Also did some manual testing and haven't found any issue with the implementation.

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-16 Thread David Rowley
On Thu, Jan 16, 2014 at 9:45 AM, Florian Pflug f...@phlo.org wrote: I'm currently thinking the best way forward is to get a basic patch without any NUMERIC stuff committed, and to revisit this after that's done. Attached is a version to that effect. The number of inverse transition functions

Re: [HACKERS] [RFC] overflow checks optimized away

2014-01-16 Thread Heikki Linnakangas
On 11/29/2013 04:04 AM, Greg Stark wrote: Attached is what I have so far. I have to say I'm starting to come around to Tom's point of view. This is a lot of hassle for not much gain. i've noticed a number of other overflow checks that the llvm optimizer is not picking up on so even after this

Re: [HACKERS] drop duplicate buffers in OS

2014-01-16 Thread Aidan Van Dyk
Can we just get the backend that dirties the page to the posix_fadvice DONTNEED? Or have another helper that sweeps the shared buffers and does this post-first-dirty? a. On Wed, Jan 15, 2014 at 1:34 PM, Robert Haas robertmh...@gmail.com wrote: On Wed, Jan 15, 2014 at 1:53 AM, KONDO Mitsumasa

Re: [HACKERS] Deprecations in authentication

2014-01-16 Thread Magnus Hagander
On Wed, Jan 15, 2014 at 6:57 PM, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: One thing I noticed - in MSVC, the config parameter krb5 (equivalent of the removed --with-krb5) enabled *both* krb5 and gssapi, and there is no separate config parameter for

Re: [HACKERS] Why conf.d should be default, and auto.conf and recovery.conf should be in it

2014-01-16 Thread Peter Eisentraut
On 1/15/14, 11:23 PM, Stephen Frost wrote: * Peter Eisentraut (pete...@gmx.net) wrote: In my mind, a conf.d directory is an extension of a single-file configuration, and so it should be handled that way. I'm apparently out on some funny limb with this thought, but I'll throw it out there

Re: [HACKERS] Why conf.d should be default, and auto.conf and recovery.conf should be in it

2014-01-16 Thread Alvaro Herrera
Peter Eisentraut wrote: In my mind, a conf.d directory is an extension of a single-file configuration, and so it should be handled that way. +1 on this. This means 1. it is to be read automatically by the server without need for an include_dir conf.d option in the main postgresql.conf.

Re: [HACKERS] [PATCH] Filter error log statements by sqlstate

2014-01-16 Thread Jeevan Chalke
Hi Oskari, Patch looks good to me now. I have found no issues too. It is good to go in now. However, few small suggestions: 1. Whenever we know that a variable is containing only 32 bits, better define it as uint32 and not just int (m_sqlstate in get_sqlstate_error_level() function). int size

Re: [HACKERS] Why conf.d should be default, and auto.conf and recovery.conf should be in it

2014-01-16 Thread Stephen Frost
Peter, * Peter Eisentraut (pete...@gmx.net) wrote: Then one might as well argue that the pg_db_role_setting table be relocated to /etc. It's the same facility, only on a slightly different level. The fact that postgresql.auto.conf looks the same as a plain-text configuration file is an

Re: [HACKERS] Why conf.d should be default, and auto.conf and recovery.conf should be in it

2014-01-16 Thread Andres Freund
On 2014-01-16 08:34:23 -0500, Stephen Frost wrote: Peter, * Peter Eisentraut (pete...@gmx.net) wrote: Then one might as well argue that the pg_db_role_setting table be relocated to /etc. It's the same facility, only on a slightly different level. The fact that postgresql.auto.conf

Re: [HACKERS] WAL Rate Limiting

2014-01-16 Thread Robert Haas
On Wed, Jan 15, 2014 at 7:54 AM, Magnus Hagander mag...@hagander.net wrote: On Wed, Jan 15, 2014 at 3:20 AM, Simon Riggs si...@2ndquadrant.com wrote: We've discussed previously the negative impact of large bulk operations, especially wrt WAL writes. Patch here allows maintenance operations to

Re: [HACKERS] [PATCH] Relocation of tablespaces in pg_basebackup

2014-01-16 Thread Alvaro Herrera
Please keep the --help and the options in the SGML table in alphabetical order within their respective sections. -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list

Re: [HACKERS] WAL Rate Limiting

2014-01-16 Thread Andres Freund
On 2014-01-16 09:06:30 -0500, Robert Haas wrote: Seems like a really bad name if we are only slowing down some commands - that seems to indicate we're slowing down all of them. I think it should be something that indicates that it only affects the maintenance commands. And why should it

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2014-01-16 Thread Euler Taveira
On 08-11-2013 06:20, Dilip kumar wrote: On 08 November 2013 13:38, Jan Lentfer For this use case, would it make sense to queue work (tables) in order of their size, starting on the largest one? For the case where you have tables of varying size this would lead to a reduced overall

Re: [HACKERS] [PATCH] Relocation of tablespaces in pg_basebackup

2014-01-16 Thread Alvaro Herrera
Eyeballing this patch, three thoughts: 1. I wonder whether ilist.c/h should be moved to src/common so that frontend code could use it. 2. I wonder whether ilist.c should gain the ability to have singly-linked lists with a pointer to the tail node for appending to the end. This code would use

Re: [HACKERS] [PATCH] Relocation of tablespaces in pg_basebackup

2014-01-16 Thread Andres Freund
Hi, On 2014-01-16 11:25:59 -0300, Alvaro Herrera wrote: Eyeballing this patch, three thoughts: 1. I wonder whether ilist.c/h should be moved to src/common so that frontend code could use it. Sounds like a good idea. There's some debugging checks that elog, but that should be fixable easily.

Re: [HACKERS] Changeset Extraction v7.0 (was logical changeset generation)

2014-01-16 Thread Robert Haas
On Wed, Jan 15, 2014 at 3:39 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-01-15 13:28:25 -0500, Robert Haas wrote: - I think you should just regard ReplicationSlotCtlLock as protecting the name and active flags of every slot. ReplicationSlotCreate() would then not need to mess with

Re: [HACKERS] Performance Improvement by reducing WAL for Update Operation

2014-01-16 Thread Robert Haas
On Thu, Jan 16, 2014 at 12:07 AM, Amit Kapila amit.kapil...@gmail.com wrote: On Thu, Jan 16, 2014 at 12:49 AM, Robert Haas robertmh...@gmail.com wrote: On Wed, Jan 15, 2014 at 7:28 AM, Amit Kapila amit.kapil...@gmail.com wrote: Unpatched --- testname

Re: [HACKERS] Changeset Extraction v7.0 (was logical changeset generation)

2014-01-16 Thread Andres Freund
Hi, On 2014-01-16 09:34:51 -0500, Robert Haas wrote: - ReplicationSlotAcquire probably needs to ignore slots that are not active. Not sure what you mean? If the slot isn't in_use we'll skip it in the loop. active != in_use. I suppose your point is that the slot can't be in_use if

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2014-01-16 Thread Alvaro Herrera
Euler Taveira wrote: On 08-11-2013 06:20, Dilip kumar wrote: On 08 November 2013 13:38, Jan Lentfer For this use case, would it make sense to queue work (tables) in order of their size, starting on the largest one? For the case where you have tables of varying size this would

Re: [HACKERS] Display oprcode and its volatility in \do+

2014-01-16 Thread Tom Lane
Marko Tiikkaja ma...@joh.to writes: On 1/16/14 9:53 AM, Rushabh Lathia wrote: Even I personally felt the Function and Volatility is nice to have info into \do+. FWIW, I'm on board with the idea of printing the oprcode, but adding volatility here seems like probably a waste of valuable terminal

Re: [HACKERS] Display oprcode and its volatility in \do+

2014-01-16 Thread Marko Tiikkaja
On 1/16/14 4:22 PM, Tom Lane wrote: Marko Tiikkaja ma...@joh.to writes: On 1/16/14 9:53 AM, Rushabh Lathia wrote: Even I personally felt the Function and Volatility is nice to have info into \do+. FWIW, I'm on board with the idea of printing the oprcode, but adding volatility here seems like

Re: [HACKERS] WAL Rate Limiting

2014-01-16 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: I don't really see much difficulty in determining what's a utility command and what not for the purpose of this? All utility commands which create WAL in O(table_size) or worse. By that definition, INSERT, UPDATE, and DELETE can all be utility

Re: [HACKERS] Why conf.d should be default, and auto.conf and recovery.conf should be in it

2014-01-16 Thread Christian Kruse
Hi Alvaro, On 16/01/14 10:21, Alvaro Herrera wrote: 1. it is to be read automatically by the server without need for an include_dir conf.d option in the main postgresql.conf. +1 4. there is no point in disabling it, and thus we offer no mechanism to do that. Not only there is „no

Re: [HACKERS] Display oprcode and its volatility in \do+

2014-01-16 Thread Tom Lane
Marko Tiikkaja ma...@joh.to writes: On 1/16/14 4:22 PM, Tom Lane wrote: FWIW, I'm on board with the idea of printing the oprcode, but adding volatility here seems like probably a waste of valuable terminal width. I think that the vast majority of operators have immutable or at worst stable

Re: [HACKERS] WAL Rate Limiting

2014-01-16 Thread Tom Lane
Heikki Linnakangas hlinnakan...@vmware.com writes: On 01/16/2014 05:39 PM, Andres Freund wrote: Do you see a reasonable way to implement this generically for all commands? I don't. In suitable safe places, check if you've written too much WAL, and sleep if so. Call it

Re: [HACKERS] Why conf.d should be default, and auto.conf and recovery.conf should be in it

2014-01-16 Thread Stephen Frost
* Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: Peter Eisentraut wrote: In my mind, a conf.d directory is an extension of a single-file configuration, and so it should be handled that way. +1 on this. This means 1. it is to be read automatically by the server without need for an

Re: [HACKERS] Why conf.d should be default, and auto.conf and recovery.conf should be in it

2014-01-16 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: Given that the majority didn't seem to be convinced by this and that the feature was written differently this isn't a convincing argument for the location of the file given the current feature, no? I'll start by pointing out that the only reason

Re: [HACKERS] Add force option to dropdb

2014-01-16 Thread salah jubeh
If the user owns objects, that will prevent this from working also.  I have the feeling that adding DROP OWNED BY and/or REASSIGNED OWNED BY calls to this utility would be a bit excessive, but who knows. Please find attached the first attempt to drop drop the client connections. I have added an

Re: [Lsf-pc] [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-16 Thread Jeremy Harris
On 14/01/14 22:23, Dave Chinner wrote: On Tue, Jan 14, 2014 at 11:40:38AM -0800, Kevin Grittner wrote: To quantify that, in a production setting we were seeing pauses of up to two minutes with shared_buffers set to 8GB and default dirty

Re: [HACKERS] Why conf.d should be default, and auto.conf and recovery.conf should be in it

2014-01-16 Thread Alvaro Herrera
Stephen Frost wrote: * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: Peter Eisentraut wrote: In my mind, a conf.d directory is an extension of a single-file configuration, and so it should be handled that way. +1 on this. This means 1. it is to be read automatically by the

Re: [HACKERS] WAL Rate Limiting

2014-01-16 Thread Andres Freund
On 2014-01-16 10:35:20 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: I don't really see much difficulty in determining what's a utility command and what not for the purpose of this? All utility commands which create WAL in O(table_size) or worse. By that definition,

Re: [HACKERS] WAL Rate Limiting

2014-01-16 Thread Simon Riggs
On 16 January 2014 16:56, Andres Freund and...@2ndquadrant.com wrote: On 2014-01-16 17:47:51 +0200, Heikki Linnakangas wrote: On 01/16/2014 05:39 PM, Andres Freund wrote: On 2014-01-16 10:35:20 -0500, Tom Lane wrote: I think possibly a more productive approach to this would be to treat it as

Re: [HACKERS] WAL Rate Limiting

2014-01-16 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-01-16 17:47:51 +0200, Heikki Linnakangas wrote: In suitable safe places, check if you've written too much WAL, and sleep if so. That'd be most places doing XLogInsert() if you want to be consistent. See my other response. There's no need

Re: [HACKERS] Why conf.d should be default, and auto.conf and recovery.conf should be in it

2014-01-16 Thread Andres Freund
On 2014-01-16 11:12:55 -0500, Tom Lane wrote: If we were to do this, I'd argue that the location of this hard-wired config directory ought to be selected by a configure option. in fact, I'd argue that the default behavior with no such option be that there's no such hard-wired directory. That

Re: [HACKERS] WAL Rate Limiting

2014-01-16 Thread Andres Freund
On 2014-01-16 17:47:51 +0200, Heikki Linnakangas wrote: On 01/16/2014 05:39 PM, Andres Freund wrote: On 2014-01-16 10:35:20 -0500, Tom Lane wrote: I think possibly a more productive approach to this would be to treat it as a session-level GUC setting, rather than hard-wiring it to affect

Re: [HACKERS] Why conf.d should be default, and auto.conf and recovery.conf should be in it

2014-01-16 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: 1. it is to be read automatically by the server without need for an include_dir conf.d option in the main postgresql.conf. I am not thrilled with the idea that we're claiming ownership of the

Re: [HACKERS] WAL Rate Limiting

2014-01-16 Thread Andres Freund
On 2014-01-16 11:19:28 -0500, Tom Lane wrote: I think the usecases that would want this for DML probably also wan this to work for unlogged, temp tables. Huh? Unlogged tables generate *zero* WAL, by definition. Yes. That's my point. If we provide it as a generic resource control - which

Re: [HACKERS] WAL Rate Limiting

2014-01-16 Thread Simon Riggs
On 16 January 2014 17:22, Andres Freund and...@2ndquadrant.com wrote: On 2014-01-16 11:19:28 -0500, Tom Lane wrote: I think the usecases that would want this for DML probably also wan this to work for unlogged, temp tables. Huh? Unlogged tables generate *zero* WAL, by definition. Yes.

Re: [HACKERS] Why conf.d should be default, and auto.conf and recovery.conf should be in it

2014-01-16 Thread Stephen Frost
Avlaro, * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: Please explain. I don't see a reason not to. Are you saying you expect to put, say, Apache configuration files in the same directory as PostgreSQL's? That doesn't sound really tenable to me, and it doesn't sounds like what any sane

Re: [HACKERS] WAL Rate Limiting

2014-01-16 Thread Andres Freund
On 2014-01-16 17:20:19 +0100, Simon Riggs wrote: I'm comfortable with a session level parameter. I was mulling over a wal_rate_limit_scope parameter but I think that is too much. I will follow Craig's proposal to define this in terms of MB/s, adding that I would calc from beginning of

Re: [HACKERS] Why conf.d should be default, and auto.conf and recovery.conf should be in it

2014-01-16 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: So, if we want to support antything but a) relative to pgdata b) relative to postgresql.conf it needs to be configurable at startup time. Possibly with an added initdb switch to set the location. How would an initdb switch be better than an

Re: [HACKERS] Why conf.d should be default, and auto.conf and recovery.conf should be in it

2014-01-16 Thread Andres Freund
On 2014-01-16 11:35:00 -0500, Stephen Frost wrote: * Andres Freund (and...@2ndquadrant.com) wrote: So, if we want to support antything but a) relative to pgdata b) relative to postgresql.conf it needs to be configurable at startup time. Possibly with an added initdb switch to set the

Re: [HACKERS] Why conf.d should be default, and auto.conf and recovery.conf should be in it

2014-01-16 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: On 2014-01-16 11:35:00 -0500, Stephen Frost wrote: * Andres Freund (and...@2ndquadrant.com) wrote: So, if we want to support antything but a) relative to pgdata b) relative to postgresql.conf it needs to be configurable at startup time.

Re: [HACKERS] WAL Rate Limiting

2014-01-16 Thread Simon Riggs
On 16 January 2014 17:29, Andres Freund and...@2ndquadrant.com wrote: Please let me know if I missed something (rather than debating what is or is not a maintenance command). If we're going to do this for DML - which I am far from convinced of - we also should do it for SELECT, since that

Re: [HACKERS] WAL Rate Limiting

2014-01-16 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: Agreed, but it won't happen in this release. I/O resource control to follow in later releases. This release? TBH, I think this patch has arrived too late to be considered for 9.4. It's a fairly major feature and clearly not without controversy, so I

Re: [HACKERS] [PATCH] Relocation of tablespaces in pg_basebackup

2014-01-16 Thread Steeve Lennmark
Alvaro, On Thu, Jan 16, 2014 at 3:20 PM, Alvaro Herrera alvhe...@2ndquadrant.comwrote: Please keep the --help and the options in the SGML table in alphabetical order within their respective sections. Ah, I failed to see that there was sub groups and thought the options where not

Re: [HACKERS] Heavily modified big table bloat even in auto vacuum is running

2014-01-16 Thread tirtho
Is this now fixed? If so, where do I find the patch for postgres 9.2.2. Thanks - Tirthankar -- View this message in context: http://postgresql.1045698.n5.nabble.com/Heavily-modified-big-table-bloat-even-in-auto-vacuum-is-running-tp5774274p5787477.html Sent from the PostgreSQL - hackers

Re: [HACKERS] WAL Rate Limiting

2014-01-16 Thread Simon Riggs
On 16 January 2014 17:49, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: Agreed, but it won't happen in this release. I/O resource control to follow in later releases. This release? TBH, I think this patch has arrived too late to be considered for 9.4. It's a

Re: [HACKERS] [PATCH] Relocation of tablespaces in pg_basebackup

2014-01-16 Thread Steeve Lennmark
Alvaro, On Thu, Jan 16, 2014 at 3:25 PM, Alvaro Herrera alvhe...@2ndquadrant.comwrote: Eyeballing this patch, three thoughts: 1. I wonder whether ilist.c/h should be moved to src/common so that frontend code could use it. That would be nice, I guess lack of helpers is why a lot of stuff is

Re: [HACKERS] WAL Rate Limiting

2014-01-16 Thread Heikki Linnakangas
On 01/16/2014 05:39 PM, Andres Freund wrote: On 2014-01-16 10:35:20 -0500, Tom Lane wrote: I think possibly a more productive approach to this would be to treat it as a session-level GUC setting, rather than hard-wiring it to affect certain commands and not others. Do you see a reasonable way

Re: [HACKERS] Patch for fail-back without fresh backup

2014-01-16 Thread Jeff Janes
On Thu, Nov 21, 2013 at 2:43 PM, Andres Freund and...@2ndquadrant.comwrote: On 2013-11-21 14:40:36 -0800, Jeff Janes wrote: But if the transaction would not have otherwise generated WAL (i.e. a select that did not have to do any HOT pruning, or an update with zero rows matching the where

Re: [HACKERS] Patch for fail-back without fresh backup

2014-01-16 Thread Andres Freund
On 2014-01-16 09:25:51 -0800, Jeff Janes wrote: On Thu, Nov 21, 2013 at 2:43 PM, Andres Freund and...@2ndquadrant.comwrote: On 2013-11-21 14:40:36 -0800, Jeff Janes wrote: But if the transaction would not have otherwise generated WAL (i.e. a select that did not have to do any HOT

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-16 Thread Florian Pflug
On Jan16, 2014, at 09:07 , David Rowley dgrowle...@gmail.com wrote: On Wed, Jan 15, 2014 at 5:39 AM, Florian Pflug f...@phlo.org wrote: The notnullcount machinery seems to apply to both STRICT and non-STRICT transfer function pairs. Shouldn't that be constrained to STRICT transfer function

Re: [HACKERS] WAL Rate Limiting

2014-01-16 Thread Jeff Janes
On Thu, Jan 16, 2014 at 8:19 AM, Tom Lane t...@sss.pgh.pa.us wrote: I think the usecases that would want this for DML probably also wan this to work for unlogged, temp tables. Huh? Unlogged tables generate *zero* WAL, by definition. Transactions that only change unlogged tables still

Re: [HACKERS] Why conf.d should be default, and auto.conf and recovery.conf should be in it

2014-01-16 Thread Alvaro Herrera
Tom Lane wrote: Stephen Frost sfr...@snowman.net writes: * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: 1. it is to be read automatically by the server without need for an include_dir conf.d option in the main postgresql.conf. I am not thrilled with the idea that we're claiming

Re: [HACKERS] Why conf.d should be default, and auto.conf and recovery.conf should be in it

2014-01-16 Thread Stephen Frost
* Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: Stephen Frost wrote: I missed the part of this where you point out that Apache on Debian has some kind of problem because it's possible for an admin to remove the 'IncludeDir sites-enabled' line from apache2.conf. I don't see that this

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-16 Thread Heikki Linnakangas
On 01/16/2014 01:02 PM, David Rowley wrote: sum(bigint) became a bit weird as it uses numeric types internally, so I had to keep the do_numeric_discard() function to support it. It's pretty weird that we have implemented sum(bigint) that way. I understand that the result is a numeric so that

Re: [HACKERS] Why conf.d should be default, and auto.conf and recovery.conf should be in it

2014-01-16 Thread Tom Lane
Alvaro Herrera alvhe...@2ndquadrant.com writes: Tom Lane wrote: Another point here is that hard-wiring a config directory location into the executables completely breaks many scenarios for running multiple clusters with the same executables. Therefore the proposal is not to hardwire the

Re: [HACKERS] Why conf.d should be default, and auto.conf and recovery.conf should be in it

2014-01-16 Thread Josh Berkus
On 01/16/2014 10:46 AM, Tom Lane wrote: I'm fine if the proposal is that postgresql.conf include include_dir conf.d by default (where that's read as relative to postgresql.conf's own directory). Even better if it's not terribly difficult for a packager to change that, because I think some

Re: [HACKERS] Why conf.d should be default, and auto.conf and recovery.conf should be in it

2014-01-16 Thread Stephen Frost
* Josh Berkus (j...@agliodbs.com) wrote: On 01/16/2014 10:46 AM, Tom Lane wrote: I'm fine if the proposal is that postgresql.conf include include_dir conf.d by default (where that's read as relative to postgresql.conf's own directory). Even better if it's not terribly difficult for a

Re: [HACKERS] new json funcs

2014-01-16 Thread Peter Eisentraut
On 1/3/14, 9:00 PM, Andrew Dunstan wrote: Here is a patch for the new json functions I mentioned a couple of months ago. These are: json_to_record json_to_recordset json_object json_build_array json_build_object json_object_agg So far there are no docs, but

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-16 Thread Tom Lane
Heikki Linnakangas hlinnakan...@vmware.com writes: I propose that we reimplement sum(bigint) in a more efficient way: For the internal state, let's use an int8 and a numeric overflow field. The transition function adds to the int8 variable, and checks for overflow. On overflow, increment

Re: [HACKERS] Why conf.d should be default, and auto.conf and recovery.conf should be in it

2014-01-16 Thread Josh Berkus
On 01/16/2014 07:32 AM, Christian Kruse wrote: Hi Alvaro, On 16/01/14 10:21, Alvaro Herrera wrote: 1. it is to be read automatically by the server without need for an include_dir conf.d option in the main postgresql.conf. +1 4. there is no point in disabling it, and thus we offer no

Re: [HACKERS] Patch for fail-back without fresh backup

2014-01-16 Thread Jeff Janes
On Thu, Jan 16, 2014 at 9:37 AM, Andres Freund and...@2ndquadrant.comwrote: On 2014-01-16 09:25:51 -0800, Jeff Janes wrote: On Thu, Nov 21, 2013 at 2:43 PM, Andres Freund and...@2ndquadrant.com wrote: On 2013-11-21 14:40:36 -0800, Jeff Janes wrote: But if the transaction would not

Re: [HACKERS] WAL Rate Limiting

2014-01-16 Thread Peter Eisentraut
The documentation doesn't build: openjade:config.sgml:1416:11:E: end tag for VARIABLELIST omitted, but OMITTAG NO was specified openjade:config.sgml:1390:5: start tag was here -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Patch for fail-back without fresh backup

2014-01-16 Thread Andres Freund
On 2014-01-16 11:01:29 -0800, Jeff Janes wrote: I think the argument is that drawing the next value from a sequence can generate xlog that needs to be flushed, but doesn't assign an xid. Then that should assign an xid. Which would yield correct behaviour with async commit where it's currently

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-16 Thread Heikki Linnakangas
On 01/16/2014 08:59 PM, Tom Lane wrote: Heikki Linnakangas hlinnakan...@vmware.com writes: I propose that we reimplement sum(bigint) in a more efficient way: For the internal state, let's use an int8 and a numeric overflow field. The transition function adds to the int8 variable, and checks for

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-16 Thread Andres Freund
On 2014-01-16 21:07:33 +0200, Heikki Linnakangas wrote: On 01/16/2014 08:59 PM, Tom Lane wrote: Heikki Linnakangas hlinnakan...@vmware.com writes: I propose that we reimplement sum(bigint) in a more efficient way: For the internal state, let's use an int8 and a numeric overflow field. The

Re: [HACKERS] Patch for fail-back without fresh backup

2014-01-16 Thread Tom Lane
Jeff Janes jeff.ja...@gmail.com writes: I think the argument is that drawing the next value from a sequence can generate xlog that needs to be flushed, but doesn't assign an xid. I would think the sequence should flush that record before it hands out the value, not before the commit, but...

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-16 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: You'll have to handle adding negative values and underflow as well. Right. Maybe it's instead sufficient to just have flag indicating that you're working with a state that hasn't overflowed so far and just plain int8 math as long as that's the

Re: [HACKERS] dblink performance regression

2014-01-16 Thread Josh Berkus
On 12/07/2013 05:50 PM, Joe Conway wrote: On 12/07/2013 05:41 PM, Fabrízio de Royes Mello wrote: On Sat, Dec 7, 2013 at 11:20 PM, Michael Paquier michael.paqu...@gmail.com mailto:michael.paqu...@gmail.com wrote: IMHO is more elegant create a procedure to encapsulate the code to avoid

Re: [HACKERS] [PATCH] Relocation of tablespaces in pg_basebackup

2014-01-16 Thread Peter Eisentraut
You appear to be generating your patches with git diff --no-prefix. Don't do that, leave the a/ and b/ in. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-16 Thread Andres Freund
On 2014-01-16 14:23:47 -0500, Tom Lane wrote: Maybe it's instead sufficient to just have flag indicating that you're working with a state that hasn't overflowed so far and just plain int8 math as long as that's the case, and entirely fall back to the current path once overflowed. That will

Re: Review: [HACKERS] ECPG infrastructure changes part 1, was: Re: ECPG fixes

2014-01-16 Thread Alvaro Herrera
Boszormenyi Zoltan escribió: All patches are attached again for completeness. Thanks. I pushed a commit comprising patches 09 through 14. -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing

Re: [HACKERS] Backup throttling

2014-01-16 Thread Andres Freund
Hi, On 2014-01-15 18:52:32 -0300, Alvaro Herrera wrote: Another thing I found a bit strange was the use of the latch. What this patch does is create a separate latch which is used for the throttling. This means that if the walsender process receives a signal, it will not wake up if it's

Re: [HACKERS] Display oprcode and its volatility in \do+

2014-01-16 Thread Tom Lane
I wrote: Anybody else have an opinion? Given the lack of other votes, I pushed this without a volatility column, and with some other changes --- mostly, I kept the Description column last, because that's how all the other \d commands do it. regards, tom lane -- Sent

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-16 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-01-16 14:23:47 -0500, Tom Lane wrote: Dunno, I think that a transition state containing both an int64 and a (presumably separately palloc'd) numeric will be a real PITA. Yea, not sure myself. I just dislike the idea of having a good part of

Re: [HACKERS] Display oprcode and its volatility in \do+

2014-01-16 Thread Marko Tiikkaja
On 1/16/14, 9:32 PM, Tom Lane wrote: Given the lack of other votes, I pushed this without a volatility column, and with some other changes --- mostly, I kept the Description column last, because that's how all the other \d commands do it. Thanks! And looks like I missed the documentation as

Re: [HACKERS] dblink performance regression

2014-01-16 Thread Joe Conway
yes Joe Sent with AquaMail for Android http://www.aqua-mail.com On January 16, 2014 2:32:55 PM Josh Berkus j...@agliodbs.com wrote: On 12/07/2013 05:50 PM, Joe Conway wrote: On 12/07/2013 05:41 PM, Fabrízio de Royes Mello wrote: On Sat, Dec 7, 2013 at 11:20 PM, Michael Paquier

Re: Review: [HACKERS] ECPG infrastructure changes part 1, was: Re: ECPG fixes

2014-01-16 Thread Alvaro Herrera
Alvaro Herrera escribió: Boszormenyi Zoltan escribió: All patches are attached again for completeness. Thanks. I pushed a commit comprising patches 09 through 14. Now also pushed 15 to 17. -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7

Re: [HACKERS] Backup throttling

2014-01-16 Thread Peter Geoghegan
On Thu, Jan 16, 2014 at 12:03 PM, Andres Freund and...@2ndquadrant.com wrote: slightly related: we should start to reuse procLatch for walsenders instead of having a separate latch someday. +1. The potential for bugs from failing to account for this within signal handlers seems like a concern.

[HACKERS] ALTER TABLE ... SET TABLESPACE pg_default

2014-01-16 Thread Stephen Frost
Greetings, Harking back to 10 years ago when tablespaces were added, it looks like we originally figured that users didn't need permissions to create tables in the database default, per 2467394e. That strikes me as perfectly fair. Unfortunately, the later addition of ALTER TABLE ...

[HACKERS] ALTER TABLESPACE ... MOVE ALL TO ...

2014-01-16 Thread Stephen Frost
Greetings, It's a day late and I'm a dollar short, but attached is a (very) minor patch to allow users to more easily move their various objects from one tablespace to another. Included are docs and a regression test; I'm happy to improve on both should folks send me suggestions. As

Re: [Lsf-pc] [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-16 Thread Jan Kara
On Wed 15-01-14 21:37:16, Robert Haas wrote: On Wed, Jan 15, 2014 at 8:41 PM, Jan Kara j...@suse.cz wrote: On Wed 15-01-14 10:12:38, Robert Haas wrote: On Wed, Jan 15, 2014 at 4:35 AM, Jan Kara j...@suse.cz wrote: Filesystems could in theory provide facility like atomic write (at least

Re: [Lsf-pc] [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-16 Thread Jan Kara
On Wed 15-01-14 10:12:38, Robert Haas wrote: On Wed, Jan 15, 2014 at 4:35 AM, Jan Kara j...@suse.cz wrote: Filesystems could in theory provide facility like atomic write (at least up to a certain size say in MB range) but it's not so easy and when there are no strong usecases fs people are

Re: [Lsf-pc] [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-16 Thread Dave Chinner
On Wed, Jan 15, 2014 at 07:31:15PM -0500, Tom Lane wrote: Dave Chinner da...@fromorbit.com writes: On Wed, Jan 15, 2014 at 07:08:18PM -0500, Tom Lane wrote: No, we'd be happy to re-request it during each checkpoint cycle, as long as that wasn't an unduly expensive call to make. I'm not

Re: [Lsf-pc] [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-16 Thread Jeff Layton
On Wed, 15 Jan 2014 21:37:16 -0500 Robert Haas robertmh...@gmail.com wrote: On Wed, Jan 15, 2014 at 8:41 PM, Jan Kara j...@suse.cz wrote: On Wed 15-01-14 10:12:38, Robert Haas wrote: On Wed, Jan 15, 2014 at 4:35 AM, Jan Kara j...@suse.cz wrote: Filesystems could in theory provide facility

Re: [Lsf-pc] [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-16 Thread Theodore Ts'o
On Wed, Jan 15, 2014 at 10:35:44AM +0100, Jan Kara wrote: Filesystems could in theory provide facility like atomic write (at least up to a certain size say in MB range) but it's not so easy and when there are no strong usecases fs people are reluctant to make their code more complex

[HACKERS] 9.3.2 Client-only installation per docs fails creating directory.

2014-01-16 Thread Mike Blackwell
Per the docs (15.4 Installation Procedure, Client-only installation), after running make, the following should work: gmake -C src/bin installgmake -C src/include installgmake -C src/interfaces installgmake -C doc install However, it fails on the first command: [postgresql-9.3.2]$ sudo gmake -C

Re: [HACKERS] [PATCH] Filter error log statements by sqlstate

2014-01-16 Thread Tom Lane
Oskari Saarenmaa o...@ohmu.fi writes: [ 0001-Filter-error-log-statements-by-sqlstate.patch ] I looked at this patch. It took me some time to understand that what it actually does has got approximately nothing to do with what one might first expect: rather than suppressing the entire log message

Re: [HACKERS] [PATCH] Filter error log statements by sqlstate

2014-01-16 Thread Peter Eisentraut
Please fix the compiler warning: elog.c: In function ‘check_log_sqlstate_error’: elog.c:3789:41: warning: ‘new_newval_end’ may be used uninitialized in this function [-Wuninitialized] -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] ECPG FETCH readahead, was: Re: ECPG fixes

2014-01-16 Thread Alvaro Herrera
Boszormenyi Zoltan escribió: Rebased patches after the regression test and other details were fixed in the infrastructure part. This thread started in 2010, and various pieces have been applied already and some others have changed in nature. Would you please post a new patchset, containing

  1   2   >