Re: [HACKERS] Create function prototype as part of PG_FUNCTION_INFO_V1

2014-04-16 Thread Craig Ringer
On 04/15/2014 10:17 PM, Tom Lane wrote: I actually think we should *add* a LIBPQEXPORT that handles this for libpq, much like PGDLLEXPORT does for postgres(.exe). And in the process, rename PGDLLEXPORT to POSTGRESEXPORT or PGSERVEREXPORT or something. My reaction to that is not bloody

[HACKERS] BGWorkers, shared memory pointers, and postmaster restart

2014-04-16 Thread Craig Ringer
Hi all I've been using the dynamic BGWorker support for some recent work, and I think I've found an issue with how postmaster restarts are handled. TL;DR: I don't think there's a safe way to use a BGWorker (static or dynamic) with bgw_restart_time != BGW_NEVER_RESTART and a bgw_main_arg Datum

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Peter Geoghegan
On Tue, Apr 15, 2014 at 9:44 PM, Robert Haas robertmh...@gmail.com wrote: On Mon, Apr 14, 2014 at 1:11 PM, Peter Geoghegan p...@heroku.com wrote: In the past, various hackers have noted problems they've observed with this scheme. A common pathology is to see frantic searching for a victim

Re: [HACKERS] Proposal: variant of regclass

2014-04-16 Thread Tatsuo Ishii
Well, I noticed that, too, but I didn't think it was my job to tell the patch author what functions he should have wanted. A follow-on patch to add to_regprocedure and to_regoperator wouldn't be much work, if you want that. And here is a patch for that. Looks good to me except duplicate

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Andres Freund
Hi, It's good to see focus on this - some improvements around s_b are sorely needed. On 2014-04-14 10:11:53 -0700, Peter Geoghegan wrote: 1) Throttles incrementation of usage_count temporally. It becomes impossible to increment usage_count for any given buffer more frequently than every 3

Re: [HACKERS] The question about the type numeric

2014-04-16 Thread amulsul
But the sign is 0. So is there anything wrong? have look in src/backend/utils/adt/numeric.c @ 154 155 for POS NEG defination given as 154 #define NUMERIC_POS 0x 155 #define NUMERIC_NEG 0x4000 Regards, Amul Sul -- View this message in context:

Re: [HACKERS] ECPG FETCH readahead

2014-04-16 Thread Boszormenyi Zoltan
2014-04-16 10:54 keltezéssel, Boszormenyi Zoltan írta: 2014-01-18 18:08 keltezéssel, Boszormenyi Zoltan írta: Hi, Alvaro Herrera wrote: Boszormenyi Zoltan escribió: Rebased patches after the regression test and other details were fixed in the infrastructure part. This thread started in

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Peter Geoghegan
On Wed, Apr 16, 2014 at 12:53 AM, Andres Freund and...@2ndquadrant.com wrote: I think this is unfortunately completely out of question. For one a gettimeofday() for every uffer pin will become a significant performance problem. Even the computation of the xact/stm start/stop timestamps shows

Re: [HACKERS] Patch: iff - if

2014-04-16 Thread Andreas 'ads' Scherbaum
On 04/16/2014 12:19 AM, Andreas 'ads' Scherbaum wrote: stumbled over a number of iff in the source where if is meant - not sure what the real story behind this is, but attached is a patch to fix the about 80 occurrences. Looks like I missed something in my math lessons ... --

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Andres Freund
On 2014-04-16 01:58:23 -0700, Peter Geoghegan wrote: On Wed, Apr 16, 2014 at 12:53 AM, Andres Freund and...@2ndquadrant.com wrote: I think this is unfortunately completely out of question. For one a gettimeofday() for every uffer pin will become a significant performance problem. Even the

[HACKERS] Dynamic Background Workers and clean exit

2014-04-16 Thread Petr Jelinek
Hello, I've been recently doing some work with dynamic bgworkers and noticed that I have no way of saying I am done now and want to exit cleanly because bgworkers get restarted automatically on exit code 0 no matter what is the restart interval set to. I understand the rationale for this

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Peter Geoghegan
On Wed, Apr 16, 2014 at 2:18 AM, Andres Freund and...@2ndquadrant.com wrote: *I* don't think any scheme that involves measuring the time around buffer pins is going to be acceptable. It's better than I say that now rather than when you've invested significant time into the approach, no? Well,

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Andres Freund
Hi, On 2014-04-16 02:57:54 -0700, Peter Geoghegan wrote: Why should I be the one with all the answers? Who said you need to be? The only thing I am saying is that I don't agree with some of your suggestions? I only responded to the thread now because downthread (in

Re: [HACKERS] BGWorkers, shared memory pointers, and postmaster restart

2014-04-16 Thread Craig Ringer
On 04/16/2014 02:37 PM, Craig Ringer wrote: Hi all I've been using the dynamic BGWorker support for some recent work, and I think I've found an issue with how postmaster restarts are handled. TL;DR: I don't think there's a safe way to use a BGWorker (static or dynamic) with

Re: [HACKERS] BGWorkers, shared memory pointers, and postmaster restart

2014-04-16 Thread Andres Freund
On 2014-04-16 19:11:37 +0800, Craig Ringer wrote: On 04/16/2014 02:37 PM, Craig Ringer wrote: Hi all I've been using the dynamic BGWorker support for some recent work, and I think I've found an issue with how postmaster restarts are handled. TL;DR: I don't think there's a safe way

Re: [HACKERS] Question about optimising (Postgres_)FDW

2014-04-16 Thread Etsuro Fujita
(2014/04/16 6:55), Hannu Krosing wrote: -- CREATE EXTENSION postgres_fdw; CREATE SERVER loop foreign data wrapper postgres_fdw OPTIONS (port '5432', dbname 'testdb'); CREATE USER MAPPING FOR PUBLIC SERVER loop; create table onemillion ( id serial

Re: [HACKERS] Request for Patch Feedback: Lag Lead Window Functions Can Ignore Nulls

2014-04-16 Thread Nicholas White
Thanks for the detailed feedback, I'm sorry it took so long to incorporate it. I've attached the latest version of the patch, fixing in particular: We have this block: I've re-written this so it only does a single pass through the window definitions (my patch originally added a second pass), and

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Robert Haas
On Wed, Apr 16, 2014 at 3:22 AM, Peter Geoghegan p...@heroku.com wrote: It's possible that I've misunderstood what you mean here, but do you really think it's likely that everything will be hot, in the event of using something like what I've sketched here? I think it's an important measure

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Ants Aasma
On Wed, Apr 16, 2014 at 7:44 AM, Robert Haas robertmh...@gmail.com wrote: I think that the basic problem here is that usage counts increase when buffers are referenced, but they decrease when buffers are evicted, and those two things are not in any necessary way connected to each other. In

Re: [HACKERS] BGWorkers, shared memory pointers, and postmaster restart

2014-04-16 Thread Craig Ringer
On 04/16/2014 07:21 PM, Andres Freund wrote: On 2014-04-16 19:11:37 +0800, Craig Ringer wrote: On 04/16/2014 02:37 PM, Craig Ringer wrote: Hi all I've been using the dynamic BGWorker support for some recent work, and I think I've found an issue with how postmaster restarts are handled.

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Merlin Moncure
On Tue, Apr 15, 2014 at 11:27 PM, Amit Kapila amit.kapil...@gmail.com wrote: On Wed, Apr 16, 2014 at 5:00 AM, Peter Geoghegan p...@heroku.com wrote: On Tue, Apr 15, 2014 at 3:59 PM, Ants Aasma a...@cybertec.at wrote: There's a paper on a non blocking GCLOCK algorithm, that does lock free clock

Re: [HACKERS] bgworker crashed or not?

2014-04-16 Thread Andres Freund
On 2014-02-03 11:29:22 -0500, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Mon, Feb 3, 2014 at 10:20 AM, Tom Lane t...@sss.pgh.pa.us wrote: So exit(0) - done, permanently exit(1) - done until restart interval exit(other) - crash and there's no way to obtain the restart

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Andres Freund
On 2014-04-16 07:55:44 -0500, Merlin Moncure wrote: 1. Bgwriter needs to be improved so that it can help in reducing usage count and finding next victim buffer (run the clock sweep and add buffers to the free list). 2. SetLatch for bgwriter (wakeup bgwriter) when elements in

Re: [HACKERS] Question about optimising (Postgres_)FDW

2014-04-16 Thread Hannu Krosing
On 04/16/2014 01:35 PM, Etsuro Fujita wrote: (2014/04/16 6:55), Hannu Krosing wrote: ... Maybe I'm missing something, but I think that you can do what I think you'd like to do by the following procedure: No, what I'd like PostgreSQL to do is to 1. select the id+set from local table 2. select

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Merlin Moncure
On Tue, Apr 15, 2014 at 11:44 PM, Robert Haas robertmh...@gmail.com wrote: I think that the basic problem here is that usage counts increase when buffers are referenced, but they decrease when buffers are evicted, and those two things are not in any necessary way connected to each other. In

Re: [HACKERS] Need Multixact Freezing Docs

2014-04-16 Thread Alvaro Herrera
Josh Berkus wrote: On 04/15/2014 02:25 PM, Josh Berkus wrote: Hackers, We need documentation on how users should intelligently set the multixact freeze settings. I'm happy to write the actual text, but I definitely don't have any idea how to set these myself. Under what

Re: [HACKERS] [BUG FIX] Compare returned value by socket() against PGINVALID_SOCKET instead of 0

2014-04-16 Thread Alvaro Herrera
Bruce Momjian wrote: Yes, I saw that yesterday and fixed it. I also did a dry run of backpatching and only 8.4 had conflicts, so I think we are good there. (This is like the readdir() fix all over again.) Once this is applied I will work on changing the libpq socket type to use portable

Re: [HACKERS] bgworker crashed or not?

2014-04-16 Thread Petr Jelinek
On 16/04/14 15:10, Andres Freund wrote: I think we really should bite the bullet and change this before 9.4 comes out. The current static bgworker facility has only been out there for one release, and dynamic bgworkers aren't released yet at all. If we wait with this for 9.5, we'll annoy many

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Andres Freund
On 2014-04-16 08:25:23 -0500, Merlin Moncure wrote: The downside of this approach was complexity and difficult to test for edge case complexity. I would like to point out though that while i/o efficiency gains are nice, I think contention issues are the bigger fish to fry. That's my feeling

Re: [HACKERS] [BUG FIX] Compare returned value by socket() against PGINVALID_SOCKET instead of 0

2014-04-16 Thread Bruce Momjian
On Wed, Apr 16, 2014 at 10:34:55AM -0300, Alvaro Herrera wrote: Bruce Momjian wrote: Yes, I saw that yesterday and fixed it. I also did a dry run of backpatching and only 8.4 had conflicts, so I think we are good there. (This is like the readdir() fix all over again.) Once this is

Re: [HACKERS] test failure on latest source

2014-04-16 Thread Marco Atzeri
On 13/04/2014 18:09, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-04-12 16:35:48 -0400, Tom Lane wrote: In principle, that commit shouldn't have affected behavior for pg_hba entries with numeric address fields ... Hm. getaddrinfo.c has this bit: /*

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Robert Haas
On Wed, Apr 16, 2014 at 9:35 AM, Andres Freund and...@2ndquadrant.com wrote: I think this is the wrong level to optimize things. Imo there's two possible solutions (that don't exclude each other): * perform the clock sweep in one process so there's a very fast way to get to a free buffer.

Re: [HACKERS] bgworker crashed or not?

2014-04-16 Thread Robert Haas
On Wed, Apr 16, 2014 at 9:10 AM, Andres Freund and...@2ndquadrant.com wrote: Agreed, but after further reflection it seems like if you've declared a restart interval, then done until restart interval is probably the common case. So how about exit(0) - done until restart interval, or

Re: [HACKERS] Archive recovery won't be completed on some situation.

2014-04-16 Thread Robert Haas
On Tue, Apr 15, 2014 at 2:52 AM, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp wrote: Hello, thank you for the discussion. At Tue, 1 Apr 2014 11:41:20 -0400, Robert Haas wrote I don't find that very radical at all. The backup_label file is *supposed* to be removed on the master if it

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Andres Freund
On 2014-04-16 10:29:29 -0400, Robert Haas wrote: On Wed, Apr 16, 2014 at 9:35 AM, Andres Freund and...@2ndquadrant.com wrote: I think this is the wrong level to optimize things. Imo there's two possible solutions (that don't exclude each other): * perform the clock sweep in one process so

Re: [HACKERS] [BUG FIX] Compare returned value by socket() against PGINVALID_SOCKET instead of 0

2014-04-16 Thread Bruce Momjian
On Fri, Apr 11, 2014 at 08:28:55AM -0400, Bruce Momjian wrote: On Fri, Apr 11, 2014 at 10:03:08AM +0530, Amit Kapila wrote: On Fri, Apr 11, 2014 at 10:00 AM, Amit Kapila amit.kapil...@gmail.com wrote: On Thu, Apr 10, 2014 at 5:21 PM, Bruce Momjian br...@momjian.us wrote: On Thu, Apr

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Andres Freund
Hi, Stephen flagged a ENOPARSE: On 2014-04-16 16:49:55 +0200, Andres Freund wrote: But I also agree with Merlin's that comment at the moment that the scalability issues (concurrency and size of shared buffers). That should have been: But I also agree with Merlin's comment that at the moment

Re: [HACKERS] The question about the type numeric

2014-04-16 Thread Robert Haas
On Tue, Apr 15, 2014 at 5:37 AM, sure.postgres sure.postg...@gmail.com wrote: Hi hackers, I am learning about numeric . The comment of NumericShort format is: * In the NumericShort format, the remaining 14 bits of the header word * (n_short.n_header) are allocated as follows: 1 for sign

Re: [HACKERS] test failure on latest source

2014-04-16 Thread Alvaro Herrera
Marco Atzeri wrote: On 13/04/2014 18:09, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-04-12 16:35:48 -0400, Tom Lane wrote: In principle, that commit shouldn't have affected behavior for pg_hba entries with numeric address fields ... Hm. getaddrinfo.c has this bit:

Re: [HACKERS] bgworker crashed or not?

2014-04-16 Thread Andres Freund
On 2014-04-16 10:37:12 -0400, Robert Haas wrote: On Wed, Apr 16, 2014 at 9:10 AM, Andres Freund and...@2ndquadrant.com wrote: Agreed, but after further reflection it seems like if you've declared a restart interval, then done until restart interval is probably the common case. So how about

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Robert Haas
On Wed, Apr 16, 2014 at 10:49 AM, Andres Freund and...@2ndquadrant.com wrote: 1. Improving the rate at which we can evict buffers, which is what you're talking about here. 2. Improving the choice of which buffers we evict, which is what Peter's talking about, or at least what I think he's

Re: [HACKERS] [BUG FIX] Compare returned value by socket() against PGINVALID_SOCKET instead of 0

2014-04-16 Thread Bruce Momjian
On Fri, Apr 11, 2014 at 08:28:55AM -0400, Bruce Momjian wrote: Once this is applied I will work on changing the libpq socket type to use portable pgsocket, but I am not planning to backpatch that unless we find a bug. Attached is a follow up patch which stores socket values in libpq as

Re: [HACKERS] test failure on latest source

2014-04-16 Thread Marco Atzeri
On 16/04/2014 17:14, Alvaro Herrera wrote: Marco Atzeri wrote: On 13/04/2014 18:09, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-04-12 16:35:48 -0400, Tom Lane wrote: In principle, that commit shouldn't have affected behavior for pg_hba entries with numeric address

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Andres Freund
On 2014-04-16 11:28:04 -0400, Robert Haas wrote: On Wed, Apr 16, 2014 at 10:49 AM, Andres Freund and...@2ndquadrant.com wrote: 1. Improving the rate at which we can evict buffers, which is what you're talking about here. 2. Improving the choice of which buffers we evict, which is what

Re: [HACKERS] bgworker crashed or not?

2014-04-16 Thread Robert Haas
On Wed, Apr 16, 2014 at 11:28 AM, Andres Freund and...@2ndquadrant.com wrote: I actually think the right answer here might be to give background workers a way to change their configured restart interval. We've already got a shared memory area that the postmaster reads to know how what to do

Re: [HACKERS] test failure on latest source

2014-04-16 Thread Tom Lane
Alvaro Herrera alvhe...@2ndquadrant.com writes: I don't know if this is relevant, but perhaps we're defining the constants in a way that conflicts with the values defined by cygwin. Hm, that's a thought, though I still don't see how it's relevant to the reported failure. Perhaps Cygwin is

Re: [HACKERS] bgworker crashed or not?

2014-04-16 Thread Andres Freund
On 2014-04-16 11:37:47 -0400, Robert Haas wrote: I think we probably also need a way to exit that's treated as an error, but doesn't lead to a PANIC restart. Why can't that be handled through ereport(ERROR/FATAL) rather than through the choice of exit status? It seems to me that the only

Re: [HACKERS] bgworker crashed or not?

2014-04-16 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-04-16 11:37:47 -0400, Robert Haas wrote: Why can't that be handled through ereport(ERROR/FATAL) rather than through the choice of exit status? I dislike that because it essentially requires the bgworker to have a full error catching

Re: [HACKERS] bgworker crashed or not?

2014-04-16 Thread Andres Freund
On 2014-04-16 11:54:25 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-04-16 11:37:47 -0400, Robert Haas wrote: Why can't that be handled through ereport(ERROR/FATAL) rather than through the choice of exit status? I dislike that because it essentially

Re: [HACKERS] PostgreSQL in Windows console and Ctrl-C

2014-04-16 Thread Robert Haas
On Tue, Apr 15, 2014 at 2:23 PM, Christian Ullrich ch...@chrullrich.net wrote: * From: Robert Haas On Mon, Apr 14, 2014 at 2:16 AM, Christian Ullrich ch...@chrullrich.net wrote: I meant creating a new one, yes. If, say, PGSQL_BACKGROUND_JOB was set, the postmaster etc. would ignore the

Re: [HACKERS] bgworker crashed or not?

2014-04-16 Thread Robert Haas
On Wed, Apr 16, 2014 at 12:00 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-04-16 11:54:25 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-04-16 11:37:47 -0400, Robert Haas wrote: Why can't that be handled through ereport(ERROR/FATAL) rather than

Re: [HACKERS] Dynamic Shared Memory stuff

2014-04-16 Thread Robert Haas
On Tue, Apr 15, 2014 at 10:46 PM, Amit Kapila amit.kapil...@gmail.com wrote: On Wed, Apr 16, 2014 at 3:01 AM, Robert Haas robertmh...@gmail.com wrote: On Tue, Apr 15, 2014 at 12:33 AM, Amit Kapila amit.kapil...@gmail.com wrote: On Mon, Apr 14, 2014 at 10:03 PM, Robert Haas

Re: [HACKERS] Question about optimising (Postgres_)FDW

2014-04-16 Thread Hannu Krosing
On 04/16/2014 03:16 PM, Hannu Krosing wrote: On 04/16/2014 01:35 PM, Etsuro Fujita wrote: (2014/04/16 6:55), Hannu Krosing wrote: ... Maybe I'm missing something, but I think that you can do what I think you'd like to do by the following procedure: No, what I'd like PostgreSQL to do is to

Re: [HACKERS] bgworker crashed or not?

2014-04-16 Thread Andres Freund
On 2014-04-16 12:04:26 -0400, Robert Haas wrote: And... so what's the problem? You seemed to be saying that the background worker would need to a more developed error-handling environment in order to do proper logging, but here you're saying (rightly, I believe) that it doesn't. Even if it

Re: [HACKERS] BGWorkers, shared memory pointers, and postmaster restart

2014-04-16 Thread Robert Haas
On Wed, Apr 16, 2014 at 7:11 AM, Craig Ringer cr...@2ndquadrant.com wrote: TL;DR: I don't think there's a safe way to use a BGWorker (static or dynamic) with bgw_restart_time != BGW_NEVER_RESTART and a bgw_main_arg Datum that points into shared memory, and think we might need a API change to

Re: [HACKERS] bgworker crashed or not?

2014-04-16 Thread Robert Haas
On Wed, Apr 16, 2014 at 12:11 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-04-16 12:04:26 -0400, Robert Haas wrote: And... so what's the problem? You seemed to be saying that the background worker would need to a more developed error-handling environment in order to do proper

Re: [HACKERS] Proposal: variant of regclass

2014-04-16 Thread Robert Haas
On Wed, Apr 16, 2014 at 3:27 AM, Tatsuo Ishii is...@postgresql.org wrote: Well, I noticed that, too, but I didn't think it was my job to tell the patch author what functions he should have wanted. A follow-on patch to add to_regprocedure and to_regoperator wouldn't be much work, if you want

Re: [HACKERS] Dynamic Background Workers and clean exit

2014-04-16 Thread Robert Haas
On Wed, Apr 16, 2014 at 5:27 AM, Petr Jelinek p...@2ndquadrant.com wrote: Hello, I've been recently doing some work with dynamic bgworkers and noticed that I have no way of saying I am done now and want to exit cleanly because bgworkers get restarted automatically on exit code 0 no matter

[HACKERS] AF_UNSPEC vs PF_UNSPEC

2014-04-16 Thread Tom Lane
While wondering what the heck is going on in http://www.postgresql.org/message-id/534e8fbb.9060...@gmail.com I chanced to notice that pgstat.c and a couple of other places set up arguments for getaddrinfo() like this: hints.ai_family = PF_UNSPEC; whereas the Single Unix Spec says clearly

Re: [HACKERS] bgworker crashed or not?

2014-04-16 Thread Andres Freund
On 2014-04-16 12:20:01 -0400, Robert Haas wrote: On Wed, Apr 16, 2014 at 12:11 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-04-16 12:04:26 -0400, Robert Haas wrote: And... so what's the problem? You seemed to be saying that the background worker would need to a more developed

Re: [HACKERS] Ctrl+C from sh can shut down daemonized PostgreSQL cluster

2014-04-16 Thread Robert Haas
On Mon, Feb 17, 2014 at 8:26 PM, Tom Lane t...@sss.pgh.pa.us wrote: Bruce Momjian br...@momjian.us writes: It certainly might be --- I have no idea. What surprised me is that we are relying solely on system() to block signals to pg_ctl-spawned servers. The question is whether that is

Re: [HACKERS] bgworker crashed or not?

2014-04-16 Thread Robert Haas
On Wed, Apr 16, 2014 at 12:28 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-04-16 12:20:01 -0400, Robert Haas wrote: On Wed, Apr 16, 2014 at 12:11 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-04-16 12:04:26 -0400, Robert Haas wrote: And... so what's the problem? You

Re: [HACKERS] Ctrl+C from sh can shut down daemonized PostgreSQL cluster

2014-04-16 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Mon, Feb 17, 2014 at 8:26 PM, Tom Lane t...@sss.pgh.pa.us wrote: Alternatively, we could do what the comments in pg_ctl have long thought desirable, namely get rid of use of system() in favor of fork()/exec(). With that, pg_ctl could do a setsid()

Re: [HACKERS] Ctrl+C from sh can shut down daemonized PostgreSQL cluster

2014-04-16 Thread Robert Haas
On Wed, Apr 16, 2014 at 12:35 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Mon, Feb 17, 2014 at 8:26 PM, Tom Lane t...@sss.pgh.pa.us wrote: Alternatively, we could do what the comments in pg_ctl have long thought desirable, namely get rid of use of

[HACKERS] Tracking replication slot blockings

2014-04-16 Thread Magnus Hagander
I'm thinking it could be interesting to know how many times (or in some other useful unit than times - how often) a specific replication slot has blocked xlog rotation. Since this AFAIK only happens during checkpoints, it seems it should be reasonably cheap to track? It would serve as an indicator

Re: [HACKERS] test failure on latest source

2014-04-16 Thread Marco Atzeri
On 16/04/2014 17:40, Tom Lane wrote: The bigger picture though is that this code isn't failing on the buildfarm. So what we need to ask is what's different about Marco's machine. good question. I checked again and I found that the fault is only on the cygwin 64 bit build but not on the

Re: [HACKERS] Tracking replication slot blockings

2014-04-16 Thread Andres Freund
Hi, On 2014-04-16 18:51:41 +0200, Magnus Hagander wrote: I'm thinking it could be interesting to know how many times (or in some other useful unit than times - how often) a specific replication slot has blocked xlog rotation. Since this AFAIK only happens during checkpoints, it seems it

Re: [HACKERS] Tracking replication slot blockings

2014-04-16 Thread Magnus Hagander
On Wed, Apr 16, 2014 at 6:56 PM, Andres Freund and...@2ndquadrant.comwrote: Hi, On 2014-04-16 18:51:41 +0200, Magnus Hagander wrote: I'm thinking it could be interesting to know how many times (or in some other useful unit than times - how often) a specific replication slot has blocked

Re: [HACKERS] LDAP: bugfix and deprecated OpenLDAP API

2014-04-16 Thread Magnus Hagander
On Mon, Oct 21, 2013 at 3:31 PM, Albe Laurenz laurenz.a...@wien.gv.atwrote: Peter Eisentraut wrote: --- 3511,3534 } /* ! * Perform an explicit anonymous bind. ! * This is not necessary in principle, but we want to set a timeout ! * of PGLDAP_TIMEOUT

Re: [HACKERS] Tracking replication slot blockings

2014-04-16 Thread Andres Freund
On 2014-04-16 19:09:09 +0200, Magnus Hagander wrote: On Wed, Apr 16, 2014 at 6:56 PM, Andres Freund and...@2ndquadrant.comwrote: The xlog removal code just check the global minimum required LSN - it doesn't check the individual slots. So you'd need to add a bit more code to that location.

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Claudio Freire
On Wed, Apr 16, 2014 at 4:22 AM, Peter Geoghegan p...@heroku.com wrote: I don't want to dismiss what you're saying about heating and cooling being unrelated, but I don't find the conclusion that not everything can be hot obvious. Maybe heat should be relative rather than absolute, and maybe

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Peter Geoghegan
On Wed, Apr 16, 2014 at 4:01 AM, Andres Freund and...@2ndquadrant.com wrote: Aren't you interested in the significance of the patch, and the test case? Not particularly in the specifics to be honest. The tradeoffs of the techniques you used in there seem prohibitive to me. It's easy to make

Re: [HACKERS] [GENERAL] pg_upgrade tablespaces

2014-04-16 Thread Bruce Momjian
On Sun, Jan 12, 2014 at 11:04:41PM -0500, Bruce Momjian wrote: In the pgsql_old installation you have symlinks pointing back to the current default location. As well pg_tablespace points back to /usr/local/pgsql/data/ The issue is that there is not actually anything there in the way of a

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Robert Haas
On Wed, Apr 16, 2014 at 1:42 PM, Peter Geoghegan p...@heroku.com wrote: On Wed, Apr 16, 2014 at 4:01 AM, Andres Freund and...@2ndquadrant.com wrote: Aren't you interested in the significance of the patch, and the test case? Not particularly in the specifics to be honest. The tradeoffs of the

Re: [HACKERS] Need Multixact Freezing Docs

2014-04-16 Thread Josh Berkus
You can see the current multixact value in pg_controldata output. Keep timestamped values of that somewhere (a table?) so that you can measure consumption rate. I don't think we provide SQL-level access to those values. Bleh. Do we provide SQL-level access in 9.4? If not, I think that's

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Peter Geoghegan
On Wed, Apr 16, 2014 at 10:56 AM, Robert Haas robertmh...@gmail.com wrote: I don't think he's being unreasonable, and I don't understand why you're getting bent out of shape about it. You proposed a patch, he articulated a problem, you don't want to fix it right now. All of which is fine.

Re: [HACKERS] The case against multixact GUCs

2014-04-16 Thread Josh Berkus
On 03/12/2014 09:45 AM, Heikki Linnakangas wrote: In hindsight, I think permanent multixids in their current form was a mistake. Before 9.3, the thing that made multixids special was that they could just be thrown away at a restart. They didn't need freezing. Now that they do, why not just use

Re: [HACKERS] The case against multixact GUCs

2014-04-16 Thread Andres Freund
On 2014-04-16 11:10:52 -0700, Josh Berkus wrote: On 03/12/2014 09:45 AM, Heikki Linnakangas wrote: In hindsight, I think permanent multixids in their current form was a mistake. Before 9.3, the thing that made multixids special was that they could just be thrown away at a restart. They

Re: [HACKERS] The case against multixact GUCs

2014-04-16 Thread Josh Berkus
On 04/16/2014 11:22 AM, Andres Freund wrote: I'm serious. The multixact stuff has been broken since 9.3 was released, and it's *still* broken. We can't give users any guidance or tools on how to set multixact stuff, and autovacuum doesn't handle it properly. Sorry, but I think you're

Re: [HACKERS] The case against multixact GUCs

2014-04-16 Thread Andres Freund
On 2014-04-16 11:25:49 -0700, Josh Berkus wrote: On 04/16/2014 11:22 AM, Andres Freund wrote: I'm serious. The multixact stuff has been broken since 9.3 was released, and it's *still* broken. We can't give users any guidance or tools on how to set multixact stuff, and autovacuum doesn't

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Merlin Moncure
On Wed, Apr 16, 2014 at 1:07 PM, Peter Geoghegan p...@heroku.com wrote: On Wed, Apr 16, 2014 at 10:56 AM, Robert Haas robertmh...@gmail.com wrote: I don't think he's being unreasonable, and I don't understand why you're getting bent out of shape about it. You proposed a patch, he articulated

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Tom Lane
Merlin Moncure mmonc...@gmail.com writes: Anyways, I'm still curious if you can post similar numbers basing the throttling on gross allocation counts instead of time. Meaning: some number of buffer allocations has to have occurred before you consider eviction. Besides being faster I think

Re: [HACKERS] The case against multixact GUCs

2014-04-16 Thread Josh Berkus
On 04/16/2014 11:30 AM, Andres Freund wrote: On 2014-04-16 11:25:49 -0700, Josh Berkus wrote: On 04/16/2014 11:22 AM, Andres Freund wrote: I'm serious. The multixact stuff has been broken since 9.3 was released, and it's *still* broken. We can't give users any guidance or tools on how to set

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Robert Haas
On Wed, Apr 16, 2014 at 2:07 PM, Peter Geoghegan p...@heroku.com wrote: On Wed, Apr 16, 2014 at 10:56 AM, Robert Haas robertmh...@gmail.com wrote: I don't think he's being unreasonable, and I don't understand why you're getting bent out of shape about it. You proposed a patch, he articulated

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Merlin Moncure
On Wed, Apr 16, 2014 at 1:44 PM, Tom Lane t...@sss.pgh.pa.us wrote: Merlin Moncure mmonc...@gmail.com writes: Anyways, I'm still curious if you can post similar numbers basing the throttling on gross allocation counts instead of time. Meaning: some number of buffer allocations has to have

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Peter Geoghegan
On Wed, Apr 16, 2014 at 12:17 PM, Robert Haas robertmh...@gmail.com wrote: I don't agree. I think it's perfectly appropriate to raise potential issues at the earliest possible time. If I didn't *strongly* emphasize my intent in writing the patch up front, I'd certainly agree. I just don't see

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Tom Lane
Merlin Moncure mmonc...@gmail.com writes: On Wed, Apr 16, 2014 at 1:44 PM, Tom Lane t...@sss.pgh.pa.us wrote: Yeah --- I think wall-clock-based throttling is fundamentally the wrong thing anyway. Are we going to start needing a CPU speed measurement to tune the algorithm with? Not the place

Re: [HACKERS] Need Multixact Freezing Docs

2014-04-16 Thread Alvaro Herrera
Josh Berkus wrote: You can see the current multixact value in pg_controldata output. Keep timestamped values of that somewhere (a table?) so that you can measure consumption rate. I don't think we provide SQL-level access to those values. Bleh. Do we provide SQL-level access in

Re: [HACKERS] Need Multixact Freezing Docs

2014-04-16 Thread Josh Berkus
On 04/16/2014 01:30 PM, Alvaro Herrera wrote: Josh Berkus wrote: You can see the current multixact value in pg_controldata output. Keep timestamped values of that somewhere (a table?) so that you can measure consumption rate. I don't think we provide SQL-level access to those values.

Re: [HACKERS] Need Multixact Freezing Docs

2014-04-16 Thread Alvaro Herrera
Josh Berkus wrote: Josh Berkus wrote: You can see the current multixact value in pg_controldata output. Keep timestamped values of that somewhere (a table?) so that you can measure consumption rate. I don't think we provide SQL-level access to those values. Bleh. Do we provide

Re: [HACKERS] bgworker crashed or not?

2014-04-16 Thread Petr Jelinek
On 16/04/14 18:34, Robert Haas wrote: On Wed, Apr 16, 2014 at 12:28 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-04-16 12:20:01 -0400, Robert Haas wrote: On Wed, Apr 16, 2014 at 12:11 PM, Andres Freund and...@2ndquadrant.com wrote: I'm still not seeing the problem. It's the

[HACKERS] New functions for sslinfo extension

2014-04-16 Thread Воронин Дмитрий
Hello all, postgresmen! I want to present some functions to sslinfo extension module:1) ssl_get_count_of_extensions() --- get count of X509v3 extensions from client certificate;2) ssl_get_extension_names() --- get short names of X509v3 extensions from client certificate;3)

Re: [HACKERS] bogus tsdict, tsparser, etc object identities

2014-04-16 Thread Alvaro Herrera
Alvaro Herrera wrote: This problem is new in 9.3, so backpatching to that. Prior to that we didn't have object identities. Done. -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Need Multixact Freezing Docs

2014-04-16 Thread Josh Berkus
So if age() doesn't mean anything, then how are users to know when the need to freeze? I don't understand. Autovacuum will freeze this automatically when the threshold is reached. Users don't need to do anything. What I'm asking is: - how do users know if Autovacuum is keeping up with

Re: [HACKERS] assertion failure 9.3.4

2014-04-16 Thread Alvaro Herrera
So, from top to bottom I see the following elements: * backend is executing a query * this query is getting captured by pg_stat_statements * the query is also getting captured by autoexplain, in chain from pg_stat_statements * autoexplain runs the query, which invokes a plpgsql function * this

Re: [HACKERS] improve the help message about psql -F

2014-04-16 Thread Bruce Momjian
On Tue, Feb 11, 2014 at 10:02:20PM -0500, Peter Eisentraut wrote: If you are going to change the help string for -F, you should also update the help string for -R, and possibly for -z and -0. Patch applied with all the suggestions merged in; commitfest item marked as committed: -F,

Re: [HACKERS] pg_get_viewdefs() indentation considered harmful

2014-04-16 Thread Bruce Momjian
On Sat, Jan 25, 2014 at 01:02:36PM -0500, Andrew Dunstan wrote: On 01/25/2014 11:06 AM, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Fri, Jan 24, 2014 at 8:53 PM, Greg Stark st...@mit.edu wrote: Indeed even aside from the performance questions, once you're indented 5-10

Re: [HACKERS] shouldn't we log permission errors when accessing the configured trigger file?

2014-04-16 Thread Bruce Momjian
On Mon, Jan 27, 2014 at 03:45:38PM +0100, Magnus Hagander wrote: On Mon, Jan 27, 2014 at 3:43 PM, Robert Haas robertmh...@gmail.com wrote: On Sun, Jan 26, 2014 at 1:03 PM, Andres Freund and...@2ndquadrant.com wrote: For some reason CheckForStandbyTrigger() doesn't report

[HACKERS] Re: test script, was Re: [COMMITTERS] pgsql: psql: conditionally display oids and replication identity

2014-04-16 Thread Bruce Momjian
On Tue, Apr 15, 2014 at 12:32:36PM -0700, David Fetter wrote: On Tue, Apr 15, 2014 at 02:46:34PM -0400, Bruce Momjian wrote: On Tue, Apr 15, 2014 at 02:32:53PM -0400, Tom Lane wrote: Bruce Momjian br...@momjian.us writes: psql: conditionally display oids and replication identity

Re: [HACKERS] Misaligned BufferDescriptors causing major performance problems on AMD

2014-04-16 Thread Bruce Momjian
On Thu, Feb 6, 2014 at 09:40:32AM +0100, Andres Freund wrote: On 2014-02-05 12:36:42 -0500, Robert Haas wrote: It may well be that your proposal is spot on. But I'd like to see some data-structure-by-data-structure measurements, rather than assuming that alignment must be a good thing.

  1   2   >