Re: [HACKERS] pg_receivexlog add synchronous mode

2014-06-10 Thread furuyao
No. IIUC walreceiver does flush *less* frequently than what you implemented on pg_receivexlog. Your version of pg_receivexlog tries to do flush every time when it receives one WAL chunk. OTOH, walreceiver does flush only when there is no extra WAL chunk in receive buffer. IOW, after writing

Re: [HACKERS] RETURNING PRIMARY KEY syntax extension

2014-06-10 Thread Hannu Krosing
On 06/10/2014 03:19 AM, Tom Dunstan wrote: A definite +1 on this feature. A while ago I got partway through hacking the hibernate postgres dialect to make it issue a RETURNING clause to spit out the primary key before I realised that the driver was already doing a RETURNING * internally. On

Re: [HACKERS] RETURNING PRIMARY KEY syntax extension

2014-06-10 Thread Tom Dunstan
On 10 June 2014 17:49, Hannu Krosing ha...@2ndquadrant.com wrote: RETURNING GENERATED KEYS perhaps, but then how do we determine that? What about RETURNING CHANGED FIELDS ? Might be quite complicated technically, but this is what is probably wanted. Seems to be getting further away from

Re: [HACKERS] pg_xlogdump --stats

2014-06-10 Thread furuyao
-Original Message- From: pgsql-hackers-ow...@postgresql.org [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Abhijit Menon-Sen Sent: Wednesday, June 04, 2014 7:47 PM To: pgsql-hackers@postgresql.org Subject: [HACKERS] pg_xlogdump --stats Hi. Here's a patch to make

Re: [HACKERS] [bug fix] Memory leak in dblink

2014-06-10 Thread MauMau
From: Amit Kapila amit.kapil...@gmail.com Is there a need to free memory context in PG_CATCH()? In error path the memory due to this temporary context will get freed as it will delete the transaction context and this temporary context will definitely be in the hierarchy of it, so it should also

Re: [HACKERS] RETURNING PRIMARY KEY syntax extension

2014-06-10 Thread Hannu Krosing
On 06/10/2014 11:02 AM, Tom Dunstan wrote: On 10 June 2014 17:49, Hannu Krosing ha...@2ndquadrant.com mailto:ha...@2ndquadrant.com wrote: RETURNING GENERATED KEYS perhaps, but then how do we determine that? What about RETURNING CHANGED FIELDS ? Might be quite complicated

Re: [HACKERS] /proc/self/oom_adj is deprecated in newer Linux kernels

2014-06-10 Thread Gurjeet Singh
On Mon, Sep 19, 2011 at 10:18 AM, Tom Lane t...@sss.pgh.pa.us wrote: Peter Eisentraut pete...@gmx.net writes: On sön, 2011-09-18 at 12:21 -0400, Tom Lane wrote: But having said that, it wouldn't be very hard to arrange things so that if you did have both symbols defined, the code would only

Re: [HACKERS] Re: why postgresql define NTUP_PER_BUCKET as 10, not other numbers smaller

2014-06-10 Thread Robert Haas
On Tue, Jun 10, 2014 at 1:13 AM, b8flowerfire b8flowerf...@gmail.com wrote: Thanks for the explanation. But i don't think it is very convincible. Simply reduce the value of NTUP_PER_BUCKET will enlarge the pointer array and reduce the tuples in one batch. But is that effect significant to the

Re: [HACKERS] updated emacs configuration

2014-06-10 Thread Noah Misch
On Mon, Jun 09, 2014 at 09:04:02PM -0400, Peter Eisentraut wrote: On Sun, 2014-06-08 at 21:55 -0400, Noah Misch wrote: After upgrading to GNU Emacs 23.4.1 from a version predating directory-local variables, I saw switch/case indentation go on the fritz. My hooks were issuing (c-set-style

Re: [HACKERS] why postgresql define NTUP_PER_BUCKET as 10, not other numbers smaller

2014-06-10 Thread Robert Haas
On Mon, Jun 9, 2014 at 11:09 AM, Tom Lane t...@sss.pgh.pa.us wrote: Keep in mind that that standard advice is meant for all-in-memory cases, not for cases where the alternative to running with longer hash chains is dumping tuples out to disk and reading them back. Sure, but that doesn't help

Re: [HACKERS] [bug fix] Memory leak in dblink

2014-06-10 Thread Robert Haas
On Tue, Jun 10, 2014 at 12:27 AM, Amit Kapila amit.kapil...@gmail.com wrote: Is there a need to free memory context in PG_CATCH()? In error path the memory due to this temporary context will get freed as it will delete the transaction context and this temporary context will definitely be in

Re: [HACKERS] /proc/self/oom_adj is deprecated in newer Linux kernels

2014-06-10 Thread Tom Lane
Gurjeet Singh gurj...@singh.im writes: Startup scripts are not solely in the domain of packagers. End users can also be expected to develop/edit their own startup scripts. Providing it as GUC would have given end users both the peices, but with a compile-time option they have only one half of

Re: [HACKERS] cancelling statement due to user request error occurs but the transaction has committed.

2014-06-10 Thread Robert Haas
On Sun, Jun 8, 2014 at 2:52 AM, Amit Kapila amit.kapil...@gmail.com wrote: I think this cancellation request must not interrupt the internal commited transaction. This is because clients may misunderstand the transaction has rollbacked. There can be similar observation if the server goes

Re: [HACKERS] cancelling statement due to user request error occurs but the transaction has committed.

2014-06-10 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: I don't agree with this analysis. If the connection is closed after the client sends a COMMIT and before it gets a response, then the client must indeed be smart enough to figure out whether or not the commit happened. But if the server sends a

Re: [HACKERS] cancelling statement due to user request error occurs but the transaction has committed.

2014-06-10 Thread Kevin Grittner
Robert Haas robertmh...@gmail.com wrote: If the connection is closed after the client sends a COMMIT and before it gets a response, then the client must indeed be smart enough to figure out whether or not the commit happened.  But if the server sends a response, the client should be able to

Re: [HACKERS] /proc/self/oom_adj is deprecated in newer Linux kernels

2014-06-10 Thread Gurjeet Singh
On Tue, Jun 10, 2014 at 10:02 AM, Tom Lane t...@sss.pgh.pa.us wrote: Gurjeet Singh gurj...@singh.im writes: Startup scripts are not solely in the domain of packagers. End users can also be expected to develop/edit their own startup scripts. Providing it as GUC would have given end users both

Re: [HACKERS] /proc/self/oom_adj is deprecated in newer Linux kernels

2014-06-10 Thread Robert Haas
On Tue, Jun 10, 2014 at 10:02 AM, Tom Lane t...@sss.pgh.pa.us wrote: Gurjeet Singh gurj...@singh.im writes: Startup scripts are not solely in the domain of packagers. End users can also be expected to develop/edit their own startup scripts. Providing it as GUC would have given end users both

Re: [HACKERS] cancelling statement due to user request error occurs but the transaction has committed.

2014-06-10 Thread Robert Haas
On Tue, Jun 10, 2014 at 10:18 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: I don't agree with this analysis. If the connection is closed after the client sends a COMMIT and before it gets a response, then the client must indeed be smart enough to figure out

Re: [HACKERS] /proc/self/oom_adj is deprecated in newer Linux kernels

2014-06-10 Thread Andres Freund
On 2014-06-10 07:56:01 -0400, Gurjeet Singh wrote: Providing it as GUC would have given end users both the peices, but with a compile-time option they have only one half of the solution; except if they go compile their own binaries, which forces them into being packagers. I am not alone in

Re: [HACKERS] updated emacs configuration

2014-06-10 Thread Tom Lane
Noah Misch n...@leadboat.com writes: On Mon, Jun 09, 2014 at 09:04:02PM -0400, Peter Eisentraut wrote: On Sun, 2014-06-08 at 21:55 -0400, Noah Misch wrote: After upgrading to GNU Emacs 23.4.1 from a version predating directory-local variables, I saw switch/case indentation go on the fritz. My

Re: [HACKERS] why postgresql define NTUP_PER_BUCKET as 10, not other numbers smaller

2014-06-10 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Mon, Jun 9, 2014 at 11:09 AM, Tom Lane t...@sss.pgh.pa.us wrote: I'm quite prepared to believe that we should change NTUP_PER_BUCKET ... but appealing to standard advice isn't a good basis for arguing that. Actual performance measurements (in both

Re: [HACKERS] why postgresql define NTUP_PER_BUCKET as 10, not other numbers smaller

2014-06-10 Thread Robert Haas
On Tue, Jun 10, 2014 at 10:27 AM, Tom Lane t...@sss.pgh.pa.us wrote: Well, it's all in what scenario you test, right? If you test the case where something overflows work_mem as a result of the increased size of the bucket array, it's always going to suck. And if you test the case where that

Re: [HACKERS] cancelling statement due to user request error occurs but the transaction has committed.

2014-06-10 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Jun 10, 2014 at 10:18 AM, Tom Lane t...@sss.pgh.pa.us wrote: ... So this whole area is a minefield, and the only attractive thing we can do is to try to reduce the number of errors that can get thrown post-commit. We already, for example, do

Re: [HACKERS] /proc/self/oom_adj is deprecated in newer Linux kernels

2014-06-10 Thread Robert Haas
On Tue, Jun 10, 2014 at 10:32 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-06-10 07:56:01 -0400, Gurjeet Singh wrote: Providing it as GUC would have given end users both the peices, but with a compile-time option they have only one half of the solution; except if they go compile

Re: [HACKERS] cancelling statement due to user request error occurs but the transaction has committed.

2014-06-10 Thread Robert Haas
On Tue, Jun 10, 2014 at 10:42 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Jun 10, 2014 at 10:18 AM, Tom Lane t...@sss.pgh.pa.us wrote: ... So this whole area is a minefield, and the only attractive thing we can do is to try to reduce the number of

Re: [HACKERS] why postgresql define NTUP_PER_BUCKET as 10, not other numbers smaller

2014-06-10 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Jun 10, 2014 at 10:27 AM, Tom Lane t...@sss.pgh.pa.us wrote: I don't really recall any hard numbers being provided. I think if we looked at some results that said here's the average gain, and here's the worst-case loss, and here's an estimate

Re: [HACKERS] /proc/self/oom_adj is deprecated in newer Linux kernels

2014-06-10 Thread Gurjeet Singh
On Tue, Jun 10, 2014 at 10:32 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-06-10 07:56:01 -0400, Gurjeet Singh wrote: Providing it as GUC would have given end users both the peices, but with a compile-time option they have only one half of the solution; except if they go compile

Re: [HACKERS] /proc/self/oom_adj is deprecated in newer Linux kernels

2014-06-10 Thread Andres Freund
On 2014-06-10 10:42:41 -0400, Robert Haas wrote: On Tue, Jun 10, 2014 at 10:32 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-06-10 07:56:01 -0400, Gurjeet Singh wrote: Providing it as GUC would have given end users both the peices, but with a compile-time option they have only

Re: [HACKERS] Compression of full-page-writes

2014-06-10 Thread Rahila Syed
Hello , In order to facilitate changing of compression algorithms and to be able to recover using WAL records compressed with different compression algorithms, information about compression algorithm can be stored in WAL record. XLOG record header has 2 to 4 padding bytes in order to align the

Re: [HACKERS] /proc/self/oom_adj is deprecated in newer Linux kernels

2014-06-10 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Jun 10, 2014 at 10:02 AM, Tom Lane t...@sss.pgh.pa.us wrote: I don't find that this argument holds any water at all. Anyone who's developing their own start script can be expected to manage recompiling Postgres. Huh? Lots of people install

Re: [HACKERS] NUMA packaging and patch

2014-06-10 Thread Robert Haas
On Mon, Jun 9, 2014 at 1:00 PM, Kevin Grittner kgri...@ymail.com wrote: Andres Freund and...@2ndquadrant.com wrote: On 2014-06-09 08:59:03 -0700, Kevin Grittner wrote: *) There is a lot of advice floating around (for example here:

Re: [HACKERS] /proc/self/oom_adj is deprecated in newer Linux kernels

2014-06-10 Thread Robert Haas
On Tue, Jun 10, 2014 at 10:51 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Jun 10, 2014 at 10:02 AM, Tom Lane t...@sss.pgh.pa.us wrote: I don't find that this argument holds any water at all. Anyone who's developing their own start script can be

Re: [HACKERS] why postgresql define NTUP_PER_BUCKET as 10, not other numbers smaller

2014-06-10 Thread Robert Haas
On Tue, Jun 10, 2014 at 10:46 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Jun 10, 2014 at 10:27 AM, Tom Lane t...@sss.pgh.pa.us wrote: I don't really recall any hard numbers being provided. I think if we looked at some results that said here's the

Re: [HACKERS] /proc/self/oom_adj is deprecated in newer Linux kernels

2014-06-10 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: Independent of the rest of the discussion, I think there's one more point: Trying to keep your system stable by *increasing* the priority of normal backends is a bad idea. If you system gets into OOM land you need to fix that, not whack who gets

Re: [HACKERS] /proc/self/oom_adj is deprecated in newer Linux kernels

2014-06-10 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Jun 10, 2014 at 10:51 AM, Tom Lane t...@sss.pgh.pa.us wrote: So? The RPM packager could probably be expected to have compiled with the oom-adjust-reset option enabled. If not, why aren't these people lobbying the packager to meet their

Re: [HACKERS] /proc/self/oom_adj is deprecated in newer Linux kernels

2014-06-10 Thread Andres Freund
Andres Freund and...@2ndquadrant.com writes: Independent of the rest of the discussion, I think there's one more point: Trying to keep your system stable by *increasing* the priority of normal backends is a bad idea. If you system gets into OOM land you need to fix that, not whack who

Re: [HACKERS] /proc/self/oom_adj is deprecated in newer Linux kernels

2014-06-10 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-06-10 11:04:52 -0400, Tom Lane wrote: But anyway, yeah, the point of this feature is that the OOM priority of the postmaster, and *only* the postmaster, should be raised. Allowing unprivileged people to break that is not attractive on any

Re: [HACKERS] /proc/self/oom_adj is deprecated in newer Linux kernels

2014-06-10 Thread Andres Freund
On 2014-06-10 11:14:43 -0400, Tom Lane wrote: Sure, but what's that have to do with this? Any Red Hat or PGDG RPM will come with this code already enabled in the build, so there is no need for anyone to have a GUC to play around with the behavior. That's imo a fair point. Unless I

Re: [HACKERS] /proc/self/oom_adj is deprecated in newer Linux kernels

2014-06-10 Thread Tom Lane
Gurjeet Singh gurj...@singh.im writes: As it stands right now, a user can decrease the likelyhood of Postmaster being killed by adjusting the start script, but that decreases the likelyhood of al the child processes, too, making the Postmaster just as likely as a kill-candidate, maybe even

Re: [HACKERS] /proc/self/oom_adj is deprecated in newer Linux kernels

2014-06-10 Thread Andres Freund
On 2014-06-10 11:20:28 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-06-10 11:04:52 -0400, Tom Lane wrote: But anyway, yeah, the point of this feature is that the OOM priority of the postmaster, and *only* the postmaster, should be raised. Allowing

Re: [HACKERS] /proc/self/oom_adj is deprecated in newer Linux kernels

2014-06-10 Thread Robert Haas
On Tue, Jun 10, 2014 at 11:14 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Jun 10, 2014 at 10:51 AM, Tom Lane t...@sss.pgh.pa.us wrote: So? The RPM packager could probably be expected to have compiled with the oom-adjust-reset option enabled. If

Re: [HACKERS] /proc/self/oom_adj is deprecated in newer Linux kernels

2014-06-10 Thread Robert Haas
On Tue, Jun 10, 2014 at 11:29 AM, Tom Lane t...@sss.pgh.pa.us wrote: This patch gives the user a control to let the backend's likelyhood of being killed be different/higher than that of the postmaster. If you think your users might want to give the postmaster OOM-exemption, why don't you just

Re: [HACKERS] /proc/self/oom_adj is deprecated in newer Linux kernels

2014-06-10 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-06-10 11:14:43 -0400, Tom Lane wrote: Because it would convert the intended behavior (postmaster and only postmaster is exempt from OOM kill) into a situation where possibly all of the database processes are exempt from OOM kill, at the whim

Re: [HACKERS] /proc/self/oom_adj is deprecated in newer Linux kernels

2014-06-10 Thread Andres Freund
On 2014-06-10 11:35:23 -0400, Robert Haas wrote: Because it would convert the intended behavior (postmaster and only postmaster is exempt from OOM kill) into a situation where possibly all of the database processes are exempt from OOM kill, at the whim of somebody who should not have the

Re: [HACKERS] /proc/self/oom_adj is deprecated in newer Linux kernels

2014-06-10 Thread Gurjeet Singh
On Tue, Jun 10, 2014 at 11:14 AM, Tom Lane t...@sss.pgh.pa.us wrote: In my view, the root-owned startup script grants OOM exemption to the postmaster because it *knows* that the postmaster's children will drop the exemption. If that trust can be violated because some clueless DBA decided to

Re: [HACKERS] /proc/self/oom_adj is deprecated in newer Linux kernels

2014-06-10 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Well, clearly, somebody hasn't got it right, or there wouldn't be this complaint. I'll grant you that somebody may be EnterpriseDB's own packaging in this instance, but I wouldn't like to bet that no one else has ever got this wrong nor ever will.

Re: [HACKERS] Scaling shared buffer eviction

2014-06-10 Thread Robert Haas
On Sun, Jun 8, 2014 at 9:51 AM, Kevin Grittner kgri...@ymail.com wrote: Amit Kapila amit.kapil...@gmail.com wrote: I have improved the patch by making following changes: a. Improved the bgwriter logic to log for xl_running_xacts info and removed the hibernate logic as bgwriter will now

Re: [HACKERS] /proc/self/oom_adj is deprecated in newer Linux kernels

2014-06-10 Thread Andres Freund
On 2014-06-10 11:40:25 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-06-10 11:14:43 -0400, Tom Lane wrote: Because it would convert the intended behavior (postmaster and only postmaster is exempt from OOM kill) into a situation where possibly all of the

Re: [HACKERS] /proc/self/oom_adj is deprecated in newer Linux kernels

2014-06-10 Thread Robert Haas
On Tue, Jun 10, 2014 at 11:46 AM, Tom Lane t...@sss.pgh.pa.us wrote: I'd be okay with a configure option, if you think that would make this issue more visible to packagers. It's delegating the responsibility to the DBA level that I'm unhappy about. [...] How about using an environment

Re: [HACKERS] /proc/self/oom_adj is deprecated in newer Linux kernels

2014-06-10 Thread Tom Lane
and...@anarazel.de (Andres Freund) writes: On 2014-06-10 11:20:28 -0400, Tom Lane wrote: Maybe I'm mistaken, but I thought once the fork_process code has reset our process's setting to zero it's not possible to lower it again (without privileges we'd not have). No, doesn't look that way.

Re: [HACKERS] /proc/self/oom_adj is deprecated in newer Linux kernels

2014-06-10 Thread Gurjeet Singh
On Tue, Jun 10, 2014 at 11:24 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-06-10 11:14:43 -0400, Tom Lane wrote: Sure, but what's that have to do with this? Any Red Hat or PGDG RPM will come with this code already enabled in the build, so there is no need for anyone to have a GUC

Re: [HACKERS] /proc/self/oom_adj is deprecated in newer Linux kernels

2014-06-10 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Jun 10, 2014 at 11:29 AM, Tom Lane t...@sss.pgh.pa.us wrote: If you think your users might want to give the postmaster OOM-exemption, why don't you just activate the existing code when you build? Resetting the OOM setting to zero is safe

Re: [HACKERS] /proc/self/oom_adj is deprecated in newer Linux kernels

2014-06-10 Thread Andres Freund
On 2014-06-10 11:52:17 -0400, Tom Lane wrote: and...@anarazel.de (Andres Freund) writes: On 2014-06-10 11:20:28 -0400, Tom Lane wrote: Maybe I'm mistaken, but I thought once the fork_process code has reset our process's setting to zero it's not possible to lower it again (without

Re: [HACKERS] /proc/self/oom_adj is deprecated in newer Linux kernels

2014-06-10 Thread Gurjeet Singh
On Tue, Jun 10, 2014 at 11:52 AM, Tom Lane t...@sss.pgh.pa.us wrote: and...@anarazel.de (Andres Freund) writes: On 2014-06-10 11:20:28 -0400, Tom Lane wrote: Maybe I'm mistaken, but I thought once the fork_process code has reset our process's setting to zero it's not possible to lower it again

Re: [HACKERS] why postgresql define NTUP_PER_BUCKET as 10, not other numbers smaller

2014-06-10 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: If we could allow NTUP_PER_BUCKET to drop when the hashtable is expected to fit in memory either way, perhaps with some safety margin (e.g. we expect to use less than 75% of work_mem), I bet that would make the people who have been complaining about

Re: [HACKERS] Proposing pg_hibernate

2014-06-10 Thread Robert Haas
On Thu, Jun 5, 2014 at 8:32 AM, Gurjeet Singh gurj...@singh.im wrote: On Wed, Jun 4, 2014 at 2:50 PM, Robert Haas robertmh...@gmail.com wrote: The thing I was concerned about is that the system might have been in recovery for months. What was hot at the time the base backup was taken seems

Re: [HACKERS] /proc/self/oom_adj is deprecated in newer Linux kernels

2014-06-10 Thread Tom Lane
Gurjeet Singh gurj...@singh.im writes: On Tue, Jun 10, 2014 at 11:52 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert's idea of having the start script set an environment variable to control the OOM adjustment reset seems like it would satisfy my concern. I'm fine with this solution. Should this

Re: [HACKERS] /proc/self/oom_adj is deprecated in newer Linux kernels

2014-06-10 Thread Robert Haas
On Tue, Jun 10, 2014 at 11:56 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Jun 10, 2014 at 11:29 AM, Tom Lane t...@sss.pgh.pa.us wrote: If you think your users might want to give the postmaster OOM-exemption, why don't you just activate the existing

[HACKERS] branch for 9.5?

2014-06-10 Thread Robert Haas
As 9.5CF1 is due to start in about 5 days, it seems we'd better branch pretty soon. Anyone planning to take care of that? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] branch for 9.5?

2014-06-10 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: As 9.5CF1 is due to start in about 5 days, it seems we'd better branch pretty soon. Anyone planning to take care of that? I think I've done it the last couple times, and am happy to do so again. Anyone want to wait longer? As Robert says, we need to

Re: [HACKERS] /proc/self/oom_adj is deprecated in newer Linux kernels

2014-06-10 Thread David G Johnston
In short: I can accept the idea that picking reasonable specific values is impossible so let's just ensure that children are always killed before the parent (basically the default behavior). If you then say that any system that is capable of implementing that rule should have it set then leaving

Re: [HACKERS] /proc/self/oom_adj is deprecated in newer Linux kernels

2014-06-10 Thread Joshua D. Drake
On 06/10/2014 07:02 AM, Tom Lane wrote: Gurjeet Singh gurj...@singh.im writes: Startup scripts are not solely in the domain of packagers. End users can also be expected to develop/edit their own startup scripts. Providing it as GUC would have given end users both the peices, but with a

Re: [HACKERS] NUMA packaging and patch

2014-06-10 Thread Josh Berkus
On 06/08/2014 03:45 PM, Kevin Grittner wrote: By default, the OS cache and buffers are allocated in the memory node with the shortest distance from the CPU a process is running on. Note that this will stop being the default in future Linux kernels. However, we'll have to deal with the old

Re: [HACKERS] /proc/self/oom_adj is deprecated in newer Linux kernels

2014-06-10 Thread David G Johnston
Gurjeet Singh-4 wrote Even if the clueless DBA tries to set the oom_score_adj below that of Postmaster, Linux kernel prevents that from being done. I demonstrate that in the below screen share. I used GUC as well as plain command line to try and set a child's badness (oom_score_adj) to be

Re: [HACKERS] /proc/self/oom_adj is deprecated in newer Linux kernels

2014-06-10 Thread Magnus Hagander
On Jun 10, 2014 7:05 PM, Joshua D. Drake j...@commandprompt.com wrote: On 06/10/2014 07:02 AM, Tom Lane wrote: Gurjeet Singh gurj...@singh.im writes: Startup scripts are not solely in the domain of packagers. End users can also be expected to develop/edit their own startup scripts.

Re: [HACKERS] /proc/self/oom_adj is deprecated in newer Linux kernels

2014-06-10 Thread Gurjeet Singh
On Tue, Jun 10, 2014 at 1:13 PM, David G Johnston david.g.johns...@gmail.com wrote: Gurjeet Singh-4 wrote So the argument that this GUC is a security concern, can be ignored. Root user (one with control of start script) still controls the lowest badness setting of all Postgres processes. If

Re: [HACKERS] NUMA packaging and patch

2014-06-10 Thread Kevin Grittner
Josh Berkus j...@agliodbs.com wrote: On 06/08/2014 03:45 PM, Kevin Grittner wrote: By default, the OS cache and buffers are allocated in the memory node with the shortest distance from the CPU a process is running on. Note that this will stop being the default in future Linux kernels.

[HACKERS] Getting “make: Entering an unknown directory” error while building postgresql on MIPS platform

2014-06-10 Thread shreesha21
make[4]: Leaving directory `/home/shreesha/platform/utils/postgresql-9.3.4/src/port' /home/shreesha/platform/tools/bin/make -C timezone all make[4]: Entering directory `/home/shreesha/platform/utils/postgresql-9.3.4/src/timezone' I am trying to install postgresql on MIPS platform and I am getting

Re: [HACKERS] Re: why postgresql define NTUP_PER_BUCKET as 10, not other numbers smaller

2014-06-10 Thread Jeff Janes
On Tue, Jun 10, 2014 at 5:17 AM, Robert Haas robertmh...@gmail.com wrote: The problem case is when you have 1 batch and the increased memory consumption causes you to switch to 2 batches. That's expensive. It seems clear based on previous testing that *on the average* NTUP_PER_BUCKET = 1

Re: [HACKERS] API change advice: Passing plan invalidation info from the rewriter into the planner?

2014-06-10 Thread Adam Brightwell
Hi all, This is my first post to the mailing list and I am looking forward to working with everyone in the community. With that said... I'll take a look at changing the cache key to include user ID and ripping out the plan invalidation logic from the current patch tomorrow but I seriously

Re: [HACKERS] Getting “make: Entering an unknown directory” error while building postgresql on MIPS platform

2014-06-10 Thread Tom Lane
shreesha21 shreesha1...@gmail.com writes: make[4]: Leaving directory `/home/shreesha/platform/utils/postgresql-9.3.4/src/port' /home/shreesha/platform/tools/bin/make -C timezone all make[4]: Entering directory `/home/shreesha/platform/utils/postgresql-9.3.4/src/timezone' I am trying to

[HACKERS] make check For Extensions

2014-06-10 Thread David E. Wheeler
Hackers, Andres said during the unconference last month that there was a way to get `make check` to work with PGXS. The idea is that it would initialize a temporary cluster, start it on an open port, install an extension, and run the extension's test suite. I think the pg_regress

Re: [HACKERS] Why is it JSQuery?

2014-06-10 Thread David E. Wheeler
On Jun 6, 2014, at 3:50 PM, Josh Berkus j...@agliodbs.com wrote: Maybe we should call it jsonesque ;-) I propose JOQL: JSON Object Query Language. Best, David PS: JAQL sounds better, but [already exists](http://code.google.com/p/jaql/). signature.asc Description: Message signed with

[HACKERS] Re: Getting “make: Entering an unknown directory” error while building postgresql on MIPS platform

2014-06-10 Thread shreesha21
bash-4.1$ make --version GNU Make 3.81 Copyright (C) 2006 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. This program built for x86_64-redhat-linux-gnu Also, the

Re: [HACKERS] Re: Getting “make: Entering an unknown directory” error while building postgresql on MIPS platform

2014-06-10 Thread Tom Lane
shreesha21 shreesha1...@gmail.com writes: bash-4.1$ make --version GNU Make 3.81 Copyright (C) 2006 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. This

Re: [HACKERS] API change advice: Passing plan invalidation info from the rewriter into the planner?

2014-06-10 Thread Tom Lane
Adam Brightwell adam.brightw...@crunchydatasolutions.com writes: Through this effort, we have concluded that for RLS the case of invalidating a plan is only necessary when switching between a superuser and a non-superuser. Obviously, re-planning on every role change would be too costly, but

[HACKERS] Re: Getting “make: Entering an unknown directory” error while building postgresql on MIPS platform

2014-06-10 Thread shreesha21
Sorry for the confusion. Let me clarify what I am trying to do here. I am trying to configure and build postgresql for MIPS platform and I am doing this on a linux system - (Linux dev-shreesharao.arubanetworks.com 2.6.32-431.11.2.el6.x86_64 GNU/Linux) so that the libraries generated from this

Re: [HACKERS] Re: Getting “make: Entering an unknown directory” error while building postgresql on MIPS platform

2014-06-10 Thread Tom Lane
Perhaps you've got SUBDIRS defined in your environment? The following example seems to match the behavior you're reporting: $ cat test1/Makefile SUBDIRS = foo bar all: make -C test2 all $ cat test1/test2/Makefile all: echo SUBDIRS = $(SUBDIRS) $ make -C test1 all make:

Re: [HACKERS] Why is it JSQuery?

2014-06-10 Thread Oleg Bartunov
People, we have many other tasks than guessing the language name. jsquery is just an extension, which we invent to test our indexing stuff. Eventually, it grew out. I think we'll think on better name if developers agree to have it in core. For now, jsquery is good enough to us. jsquery name

Re: [HACKERS] Why is it JSQuery?

2014-06-10 Thread Oleg Bartunov
The closest problem we have is jsonb statistics (lack of, actually) , which prevents use of all the power of jsquery. I hope Jan Urbański could work on this. Oleg On Tue, Jun 10, 2014 at 11:06 PM, Oleg Bartunov obartu...@gmail.com wrote: People, we have many other tasks than guessing the

Re: [HACKERS] API change advice: Passing plan invalidation info from the rewriter into the planner?

2014-06-10 Thread Brightwell, Adam
Hey Tom, Hm ... I'm not following why we'd need a special case for superusers and not anyone else? Seems like any useful RLS scheme is going to require more privilege levels than just superuser and not-superuser. As it stands right now, superuser is the only case where RLS policies should

Re: [HACKERS] Why is it JSQuery?

2014-06-10 Thread David E. Wheeler
On Jun 10, 2014, at 12:06 PM, Oleg Bartunov obartu...@gmail.com wrote: we have many other tasks than guessing the language name. jsquery is just an extension, which we invent to test our indexing stuff. Eventually, it grew out. I think we'll think on better name if developers agree to have

Re: [HACKERS] RETURNING PRIMARY KEY syntax extension

2014-06-10 Thread Vik Fearing
On 06/09/2014 07:13 PM, Andres Freund wrote: On 2014-06-09 13:42:22 +0200, Vik Fearing wrote: On 06/09/2014 09:06 AM, Gavin Flower wrote: From memory all unique keys can be considered 'candidate primary keys', but only one can be designated 'the PRIMARY KEY'. Almost. Candidate keys are also

Re: [HACKERS] API change advice: Passing plan invalidation info from the rewriter into the planner?

2014-06-10 Thread Craig Ringer
On 06/11/2014 02:19 AM, Tom Lane wrote: Hm ... I'm not following why we'd need a special case for superusers and not anyone else? Seems like any useful RLS scheme is going to require more privilege levels than just superuser and not-superuser. What it really needs is to invalidate plans when

Re: [HACKERS] [GENERAL] Question about partial functional indexes and the query planner

2014-06-10 Thread Tom Lane
Brian Dunavant br...@omniti.com writes: I am using a partial functional index on a table where F(a) = a. Querying whre F(a) = a hits the index as expected. However the reverse statement a = F(a) does not. I have verified this in 9.3.4. Is this a deficiency with the query planner, or are

Re: [HACKERS] API change advice: Passing plan invalidation info from the rewriter into the planner?

2014-06-10 Thread Tom Lane
Craig Ringer cr...@2ndquadrant.com writes: On 06/11/2014 02:19 AM, Tom Lane wrote: Could we put the if superuser then ok test into the RLS condition test and thereby not need more than one plan at all? Only if we put it in another level of security barrier subquery, because otherwise the

Re: [HACKERS] API change advice: Passing plan invalidation info from the rewriter into the planner?

2014-06-10 Thread Craig Ringer
On 06/11/2014 07:24 AM, Tom Lane wrote: Is the point of that that the table owner might have put trojan-horse functions into the RLS qual? If so, why are we only concerned about defending the superuser and not other users? Seems like the right fix would be to insist that functions in the RLS

Re: [HACKERS] API change advice: Passing plan invalidation info from the rewriter into the planner?

2014-06-10 Thread Tom Lane
Craig Ringer cr...@2ndquadrant.com writes: On 06/11/2014 07:24 AM, Tom Lane wrote: Is the point of that that the table owner might have put trojan-horse functions into the RLS qual? If so, why are we only concerned about defending the superuser and not other users? Seems like the right fix

Re: [HACKERS] Re: why postgresql define NTUP_PER_BUCKET as 10, not other numbers smaller

2014-06-10 Thread Robert Haas
On Tue, Jun 10, 2014 at 1:43 PM, Jeff Janes jeff.ja...@gmail.com wrote: On Tue, Jun 10, 2014 at 5:17 AM, Robert Haas robertmh...@gmail.com wrote: The problem case is when you have 1 batch and the increased memory consumption causes you to switch to 2 batches. That's expensive. It seems clear

Re: [HACKERS] RETURNING PRIMARY KEY syntax extension

2014-06-10 Thread Andres Freund
On 2014-06-11 00:21:58 +0200, Vik Fearing wrote: On 06/09/2014 07:13 PM, Andres Freund wrote: On 2014-06-09 13:42:22 +0200, Vik Fearing wrote: On 06/09/2014 09:06 AM, Gavin Flower wrote: From memory all unique keys can be considered 'candidate primary keys', but only one can be designated

Re: [HACKERS] RETURNING PRIMARY KEY syntax extension

2014-06-10 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-06-11 00:21:58 +0200, Vik Fearing wrote: What? AFAIK, that only applies to an index. How can the data itself be partial? I don't follow? Gavin above talked about unique keys - which in postgres you can create using CREATE UNIQUE INDEX.

Re: [HACKERS] Compression of full-page-writes

2014-06-10 Thread Michael Paquier
On Tue, Jun 10, 2014 at 11:49 PM, Rahila Syed rahilasye...@gmail.com wrote: Hello , In order to facilitate changing of compression algorithms and to be able to recover using WAL records compressed with different compression algorithms, information about compression algorithm can be stored

Re: [HACKERS] branch for 9.5?

2014-06-10 Thread Tom Lane
I wrote: Robert Haas robertmh...@gmail.com writes: As 9.5CF1 is due to start in about 5 days, it seems we'd better branch pretty soon. Anyone planning to take care of that? I think I've done it the last couple times, and am happy to do so again. Anyone want to wait longer? As Robert says,

Re: [HACKERS] RETURNING PRIMARY KEY syntax extension

2014-06-10 Thread Tom Dunstan
On 11 June 2014 10:09, Tom Lane t...@sss.pgh.pa.us wrote: I'm not even 100% sold that automatically returning the primary key is going to save any application logic. Could somebody point out *exactly* where an app is going to save effort with this type of syntax, compared to requesting the

Re: [HACKERS] Proposing pg_hibernate

2014-06-10 Thread Gurjeet Singh
On Tue, Jun 10, 2014 at 12:02 PM, Robert Haas robertmh...@gmail.com wrote: If recovery has been running for a long time, then restoring buffers from some save file created before that is probably a bad idea, regardless of whether the buffers already loaded were read in by recovery itself or by

Re: [HACKERS] RETURNING PRIMARY KEY syntax extension

2014-06-10 Thread Tom Dunstan
Is it going to save enough to justify depending on a syntax that won't be universal for a long time to come? Oh, and on the won't-be-universal-for-a-while point - the status quo works fine, it's just less efficient than it should be. Once someone upgrades to 9.5 or whatever, and upgrades to

Re: [HACKERS] Proposing pg_hibernate

2014-06-10 Thread Gurjeet Singh
On Sun, Jun 8, 2014 at 3:24 AM, Amit Kapila amit.kapil...@gmail.com wrote: On Fri, Jun 6, 2014 at 5:31 PM, Gurjeet Singh gurj...@singh.im wrote: On Thu, Jun 5, 2014 at 11:32 PM, Amit Kapila amit.kapil...@gmail.com wrote: Buffer saver process itself can crash while saving or restoring

Re: [HACKERS] Proposing pg_hibernate

2014-06-10 Thread Amit Kapila
On Wed, Jun 11, 2014 at 7:59 AM, Gurjeet Singh gurj...@singh.im wrote: On Sun, Jun 8, 2014 at 3:24 AM, Amit Kapila amit.kapil...@gmail.com wrote: IIUC on shutdown request, postmaster will send signal to BG Saver and BG Saver will save the buffers and then postmaster will send signal to

Re: [HACKERS] 9.4 release notes

2014-06-10 Thread Noah Misch
I propose the attached, miscellaneous edits. The limit on tuplesort.c internal sort size is a limit on the number of tuples, not on memory usage. Specifically, the cap increased from 44739242 tuples to 2147483647 tuples. I didn't include those numbers, though. -- Noah Misch EnterpriseDB

Re: [HACKERS] [bug fix] Memory leak in dblink

2014-06-10 Thread Amit Kapila
On Tue, Jun 10, 2014 at 7:30 PM, Robert Haas robertmh...@gmail.com wrote: On Tue, Jun 10, 2014 at 12:27 AM, Amit Kapila amit.kapil...@gmail.com wrote: Is there a need to free memory context in PG_CATCH()? In error path the memory due to this temporary context will get freed as it will

  1   2   >