Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Peter Geoghegan
On Tue, Apr 15, 2014 at 9:44 PM, Robert Haas wrote: > On Mon, Apr 14, 2014 at 1:11 PM, Peter Geoghegan wrote: >> In the past, various hackers have noted problems they've observed with >> this scheme. A common pathology is to see frantic searching for a >> victim buffer only to find all buffer usa

Re: [HACKERS] Proposal: variant of regclass

2014-04-16 Thread Tatsuo Ishii
>> Well, I noticed that, too, but I didn't think it was my job to tell >> the patch author what functions he should have wanted. A follow-on >> patch to add to_regprocedure and to_regoperator wouldn't be much work, >> if you want that. > > And here is a patch for that. Looks good to me except du

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Andres Freund
Hi, It's good to see focus on this - some improvements around s_b are sorely needed. On 2014-04-14 10:11:53 -0700, Peter Geoghegan wrote: > 1) Throttles incrementation of usage_count temporally. It becomes > impossible to increment usage_count for any given buffer more > frequently than every 3 s

Re: [HACKERS] The question about the type numeric

2014-04-16 Thread amulsul
>But the sign is 0. >So is there anything wrong? have look in src/backend/utils/adt/numeric.c @ 154 & 155 for POS & NEG defination given as 154 #define NUMERIC_POS 0x 155 #define NUMERIC_NEG 0x4000 Regards, Amul Sul -- View this message in context

Re: [HACKERS] ECPG FETCH readahead

2014-04-16 Thread Boszormenyi Zoltan
2014-04-16 10:54 keltezéssel, Boszormenyi Zoltan írta: 2014-01-18 18:08 keltezéssel, Boszormenyi Zoltan írta: Hi, Alvaro Herrera wrote: Boszormenyi Zoltan escribió: Rebased patches after the regression test and other details were fixed in the infrastructure part. This thread started in 201

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Peter Geoghegan
On Wed, Apr 16, 2014 at 12:53 AM, Andres Freund wrote: > I think this is unfortunately completely out of question. For one a > gettimeofday() for every uffer pin will become a significant performance > problem. Even the computation of the xact/stm start/stop timestamps > shows up pretty heavily in

Re: [HACKERS] Patch: iff -> if

2014-04-16 Thread Andreas 'ads' Scherbaum
On 04/16/2014 12:19 AM, Andreas 'ads' Scherbaum wrote: stumbled over a number of "iff" in the source where "if" is meant - not sure what the real story behind this is, but attached is a patch to fix the about 80 occurrences. Looks like I missed something in my math lessons ... --

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Andres Freund
On 2014-04-16 01:58:23 -0700, Peter Geoghegan wrote: > On Wed, Apr 16, 2014 at 12:53 AM, Andres Freund > wrote: > > I think this is unfortunately completely out of question. For one a > > gettimeofday() for every uffer pin will become a significant performance > > problem. Even the computation of

[HACKERS] Dynamic Background Workers and clean exit

2014-04-16 Thread Petr Jelinek
Hello, I've been recently doing some work with dynamic bgworkers and noticed that I have no way of saying "I am done now and want to exit cleanly" because bgworkers get restarted automatically on exit code 0 no matter what is the restart interval set to. I understand the rationale for this b

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Peter Geoghegan
On Wed, Apr 16, 2014 at 2:18 AM, Andres Freund wrote: > *I* don't think any scheme that involves measuring the time around > buffer pins is going to be acceptable. It's better than I say that now > rather than when you've invested significant time into the approach, no? Well, I do think that it w

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Andres Freund
Hi, On 2014-04-16 02:57:54 -0700, Peter Geoghegan wrote: > Why should I be the one with all the answers? Who said you need to be? The only thing I am saying is that I don't agree with some of your suggestions? I only responded to the thread now because downthread (in CAM3SWZQa2OAVUrfPL-df=we1smo

Re: [HACKERS] BGWorkers, shared memory pointers, and postmaster restart

2014-04-16 Thread Craig Ringer
On 04/16/2014 02:37 PM, Craig Ringer wrote: > Hi all > > I've been using the dynamic BGWorker support for some recent work, and I > think I've found an issue with how postmaster restarts are handled. > > TL;DR: I don't think there's a safe way to use a BGWorker (static or > dynamic) with bgw_rest

Re: [HACKERS] BGWorkers, shared memory pointers, and postmaster restart

2014-04-16 Thread Andres Freund
On 2014-04-16 19:11:37 +0800, Craig Ringer wrote: > On 04/16/2014 02:37 PM, Craig Ringer wrote: > > Hi all > > > > I've been using the dynamic BGWorker support for some recent work, and I > > think I've found an issue with how postmaster restarts are handled. > > > > TL;DR: I don't think there's

Re: [HACKERS] Question about optimising (Postgres_)FDW

2014-04-16 Thread Etsuro Fujita
(2014/04/16 6:55), Hannu Krosing wrote: -- CREATE EXTENSION postgres_fdw; CREATE SERVER loop foreign data wrapper postgres_fdw OPTIONS (port '5432', dbname 'testdb'); CREATE USER MAPPING FOR PUBLIC SERVER loop; create table onemillion ( id serial primary

Re: [HACKERS] Request for Patch Feedback: Lag & Lead Window Functions Can Ignore Nulls

2014-04-16 Thread Nicholas White
Thanks for the detailed feedback, I'm sorry it took so long to incorporate it. I've attached the latest version of the patch, fixing in particular: > We have this block: I've re-written this so it only does a single pass through the window definitions (my patch originally added a second pass), and

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Robert Haas
On Wed, Apr 16, 2014 at 3:22 AM, Peter Geoghegan wrote: > It's possible that I've misunderstood what you mean here, but do you > really think it's likely that everything will be hot, in the event of > using something like what I've sketched here? I think it's an > important measure against this ge

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Ants Aasma
On Wed, Apr 16, 2014 at 7:44 AM, Robert Haas wrote: > I think that the basic problem here is that usage counts increase when > buffers are referenced, but they decrease when buffers are evicted, > and those two things are not in any necessary way connected to each > other. In particular, if no ev

Re: [HACKERS] BGWorkers, shared memory pointers, and postmaster restart

2014-04-16 Thread Craig Ringer
On 04/16/2014 07:21 PM, Andres Freund wrote: > On 2014-04-16 19:11:37 +0800, Craig Ringer wrote: >> On 04/16/2014 02:37 PM, Craig Ringer wrote: >>> Hi all >>> >>> I've been using the dynamic BGWorker support for some recent work, and I >>> think I've found an issue with how postmaster restarts are

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Merlin Moncure
On Tue, Apr 15, 2014 at 11:27 PM, Amit Kapila wrote: > On Wed, Apr 16, 2014 at 5:00 AM, Peter Geoghegan wrote: >> On Tue, Apr 15, 2014 at 3:59 PM, Ants Aasma wrote: >>> There's a paper on a non blocking GCLOCK algorithm, that does lock >>> free clock sweep and buffer pinning[7]. If we decide to

Re: [HACKERS] bgworker crashed or not?

2014-04-16 Thread Andres Freund
On 2014-02-03 11:29:22 -0500, Tom Lane wrote: > Robert Haas writes: > > On Mon, Feb 3, 2014 at 10:20 AM, Tom Lane wrote: > >> So > >> exit(0) - done, permanently > >> exit(1) - done until restart interval > >> exit(other) - crash > >> and there's no way to obtain the "restart immediately" behavio

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Andres Freund
On 2014-04-16 07:55:44 -0500, Merlin Moncure wrote: > > 1. Bgwriter needs to be improved so that it can help in reducing > > usage count and finding next victim buffer (run the clock sweep > > and add buffers to the free list). > > 2. SetLatch for bgwriter (wakeup bgwriter) when elements in

Re: [HACKERS] Question about optimising (Postgres_)FDW

2014-04-16 Thread Hannu Krosing
On 04/16/2014 01:35 PM, Etsuro Fujita wrote: > (2014/04/16 6:55), Hannu Krosing wrote: ... > > Maybe I'm missing something, but I think that you can do what I think > you'd like to do by the following procedure: No, what I'd like PostgreSQL to do is to 1. select the id+set from local table 2. sele

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Merlin Moncure
On Tue, Apr 15, 2014 at 11:44 PM, Robert Haas wrote: > I think that the basic problem here is that usage counts increase when > buffers are referenced, but they decrease when buffers are evicted, > and those two things are not in any necessary way connected to each > other. In particular, if no e

Re: [HACKERS] Need Multixact Freezing Docs

2014-04-16 Thread Alvaro Herrera
Josh Berkus wrote: > On 04/15/2014 02:25 PM, Josh Berkus wrote: > > Hackers, > > > > We need documentation on how users should intelligently set the > > multixact freeze settings. I'm happy to write the actual text, but I > > definitely don't have any idea how to set these myself. Under what > >

Re: [HACKERS] [BUG FIX] Compare returned value by socket() against PGINVALID_SOCKET instead of < 0

2014-04-16 Thread Alvaro Herrera
Bruce Momjian wrote: > > Yes, I saw that yesterday and fixed it. I also did a dry run of > backpatching and only 8.4 had conflicts, so I think we are good there. > (This is like the readdir() fix all over again.) > > Once this is applied I will work on changing the libpq socket type to > use por

Re: [HACKERS] bgworker crashed or not?

2014-04-16 Thread Petr Jelinek
On 16/04/14 15:10, Andres Freund wrote: I think we really should bite the bullet and change this before 9.4 comes out. The current static bgworker facility has only been out there for one release, and dynamic bgworkers aren't released yet at all. If we wait with this for 9.5, we'll annoy many mo

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Andres Freund
On 2014-04-16 08:25:23 -0500, Merlin Moncure wrote: > The downside of this approach was complexity and difficult to test for > edge case complexity. I would like to point out though that while i/o > efficiency gains are nice, I think contention issues are the bigger > fish to fry. That's my feeli

Re: [HACKERS] [BUG FIX] Compare returned value by socket() against PGINVALID_SOCKET instead of < 0

2014-04-16 Thread Bruce Momjian
On Wed, Apr 16, 2014 at 10:34:55AM -0300, Alvaro Herrera wrote: > Bruce Momjian wrote: > > > > Yes, I saw that yesterday and fixed it. I also did a dry run of > > backpatching and only 8.4 had conflicts, so I think we are good there. > > (This is like the readdir() fix all over again.) > > > > O

Re: [HACKERS] test failure on latest source

2014-04-16 Thread Marco Atzeri
On 13/04/2014 18:09, Tom Lane wrote: Andres Freund writes: On 2014-04-12 16:35:48 -0400, Tom Lane wrote: In principle, that commit shouldn't have affected behavior for pg_hba entries with numeric address fields ... Hm. getaddrinfo.c has this bit: /* Unsupported flags. */ if

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Robert Haas
On Wed, Apr 16, 2014 at 9:35 AM, Andres Freund wrote: > I think this is the wrong level to optimize things. Imo there's two > possible solutions (that don't exclude each other): > > * perform the clock sweep in one process so there's a very fast way to > get to a free buffer. Possibly in a parti

Re: [HACKERS] bgworker crashed or not?

2014-04-16 Thread Robert Haas
On Wed, Apr 16, 2014 at 9:10 AM, Andres Freund wrote: >> Agreed, but after further reflection it seems like if you've declared >> a restart interval, then "done until restart interval" is probably the >> common case. So how about >> >> exit(0) - done until restart interval, or permanently if ther

Re: [HACKERS] Archive recovery won't be completed on some situation.

2014-04-16 Thread Robert Haas
On Tue, Apr 15, 2014 at 2:52 AM, Kyotaro HORIGUCHI wrote: > Hello, thank you for the discussion. > > At Tue, 1 Apr 2014 11:41:20 -0400, Robert Haas wrote >>> I don't find that very radical at all. The backup_label file is >>> *supposed* to be removed on the master if it crashes during the >>> bac

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Andres Freund
On 2014-04-16 10:29:29 -0400, Robert Haas wrote: > On Wed, Apr 16, 2014 at 9:35 AM, Andres Freund wrote: > > I think this is the wrong level to optimize things. Imo there's two > > possible solutions (that don't exclude each other): > > > > * perform the clock sweep in one process so there's a ver

Re: [HACKERS] [BUG FIX] Compare returned value by socket() against PGINVALID_SOCKET instead of < 0

2014-04-16 Thread Bruce Momjian
On Fri, Apr 11, 2014 at 08:28:55AM -0400, Bruce Momjian wrote: > On Fri, Apr 11, 2014 at 10:03:08AM +0530, Amit Kapila wrote: > > On Fri, Apr 11, 2014 at 10:00 AM, Amit Kapila > > wrote: > > > On Thu, Apr 10, 2014 at 5:21 PM, Bruce Momjian wrote: > > >> On Thu, Apr 10, 2014 at 11:05:49AM +0530,

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Andres Freund
Hi, Stephen flagged a ENOPARSE: On 2014-04-16 16:49:55 +0200, Andres Freund wrote: > But I also agree with Merlin's that comment at the moment that the > scalability issues (concurrency and size of shared buffers). That should have been: But I also agree with Merlin's comment that at the momen

Re: [HACKERS] The question about the type numeric

2014-04-16 Thread Robert Haas
On Tue, Apr 15, 2014 at 5:37 AM, sure.postgres wrote: > Hi hackers, > > I am learning about numeric . > The comment of NumericShort format is: > * In the NumericShort format, the remaining 14 bits of the header word > * (n_short.n_header) are allocated as follows: 1 for sign (positive or > * ne

Re: [HACKERS] test failure on latest source

2014-04-16 Thread Alvaro Herrera
Marco Atzeri wrote: > On 13/04/2014 18:09, Tom Lane wrote: > >Andres Freund writes: > >>On 2014-04-12 16:35:48 -0400, Tom Lane wrote: > >>>In principle, that commit shouldn't have affected behavior for pg_hba > >>>entries with numeric address fields ... > > > >>Hm. getaddrinfo.c has this bit: > >>

Re: [HACKERS] bgworker crashed or not?

2014-04-16 Thread Andres Freund
On 2014-04-16 10:37:12 -0400, Robert Haas wrote: > On Wed, Apr 16, 2014 at 9:10 AM, Andres Freund wrote: > >> Agreed, but after further reflection it seems like if you've declared > >> a restart interval, then "done until restart interval" is probably the > >> common case. So how about > >> > >>

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Robert Haas
On Wed, Apr 16, 2014 at 10:49 AM, Andres Freund wrote: >> 1. Improving the rate at which we can evict buffers, which is what >> you're talking about here. >> >> 2. Improving the choice of which buffers we evict, which is what >> Peter's talking about, or at least what I think he's talking about. >

Re: [HACKERS] [BUG FIX] Compare returned value by socket() against PGINVALID_SOCKET instead of < 0

2014-04-16 Thread Bruce Momjian
On Fri, Apr 11, 2014 at 08:28:55AM -0400, Bruce Momjian wrote: > Once this is applied I will work on changing the libpq socket type to > use portable pgsocket, but I am not planning to backpatch that unless we > find a bug. Attached is a follow up patch which stores socket values in libpq as pgsoc

Re: [HACKERS] test failure on latest source

2014-04-16 Thread Marco Atzeri
On 16/04/2014 17:14, Alvaro Herrera wrote: Marco Atzeri wrote: On 13/04/2014 18:09, Tom Lane wrote: Andres Freund writes: On 2014-04-12 16:35:48 -0400, Tom Lane wrote: In principle, that commit shouldn't have affected behavior for pg_hba entries with numeric address fields ... Hm. getad

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Andres Freund
On 2014-04-16 11:28:04 -0400, Robert Haas wrote: > On Wed, Apr 16, 2014 at 10:49 AM, Andres Freund > wrote: > >> 1. Improving the rate at which we can evict buffers, which is what > >> you're talking about here. > >> > >> 2. Improving the choice of which buffers we evict, which is what > >> Peter

Re: [HACKERS] bgworker crashed or not?

2014-04-16 Thread Robert Haas
On Wed, Apr 16, 2014 at 11:28 AM, Andres Freund wrote: >> I actually think the right answer here might be to give background >> workers a way to change their configured restart interval. We've >> already got a shared memory area that the postmaster reads to know how >> what to do when starting a

Re: [HACKERS] test failure on latest source

2014-04-16 Thread Tom Lane
Alvaro Herrera writes: > I don't know if this is relevant, but perhaps we're defining the > constants in a way that conflicts with the values defined by cygwin. Hm, that's a thought, though I still don't see how it's relevant to the reported failure. Perhaps Cygwin is defining these constants so

Re: [HACKERS] bgworker crashed or not?

2014-04-16 Thread Andres Freund
On 2014-04-16 11:37:47 -0400, Robert Haas wrote: > > I think we probably also need a way to exit that's treated as an error, > > but doesn't lead to a PANIC restart. > > Why can't that be handled through ereport(ERROR/FATAL) rather than > through the choice of exit status? It seems to me that the

Re: [HACKERS] bgworker crashed or not?

2014-04-16 Thread Tom Lane
Andres Freund writes: > On 2014-04-16 11:37:47 -0400, Robert Haas wrote: >> Why can't that be handled through ereport(ERROR/FATAL) rather than >> through the choice of exit status? > I dislike that because it essentially requires the bgworker to have a > full error catching environment like Postg

Re: [HACKERS] bgworker crashed or not?

2014-04-16 Thread Andres Freund
On 2014-04-16 11:54:25 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2014-04-16 11:37:47 -0400, Robert Haas wrote: > >> Why can't that be handled through ereport(ERROR/FATAL) rather than > >> through the choice of exit status? > > > I dislike that because it essentially requires the bgwor

Re: [HACKERS] PostgreSQL in Windows console and Ctrl-C

2014-04-16 Thread Robert Haas
On Tue, Apr 15, 2014 at 2:23 PM, Christian Ullrich wrote: > * From: Robert Haas >> On Mon, Apr 14, 2014 at 2:16 AM, Christian Ullrich >> wrote: > >> > I meant creating a new one, yes. If, say, PGSQL_BACKGROUND_JOB was >> > set, the postmaster etc. would ignore the events. >> >> Why not just pass

Re: [HACKERS] bgworker crashed or not?

2014-04-16 Thread Robert Haas
On Wed, Apr 16, 2014 at 12:00 PM, Andres Freund wrote: > On 2014-04-16 11:54:25 -0400, Tom Lane wrote: >> Andres Freund writes: >> > On 2014-04-16 11:37:47 -0400, Robert Haas wrote: >> >> Why can't that be handled through ereport(ERROR/FATAL) rather than >> >> through the choice of exit status? >

Re: [HACKERS] Dynamic Shared Memory stuff

2014-04-16 Thread Robert Haas
On Tue, Apr 15, 2014 at 10:46 PM, Amit Kapila wrote: > On Wed, Apr 16, 2014 at 3:01 AM, Robert Haas wrote: >> On Tue, Apr 15, 2014 at 12:33 AM, Amit Kapila >> wrote: >>> On Mon, Apr 14, 2014 at 10:03 PM, Robert Haas wrote: For the create case, I'm wondering if we should put the block that

Re: [HACKERS] Question about optimising (Postgres_)FDW

2014-04-16 Thread Hannu Krosing
On 04/16/2014 03:16 PM, Hannu Krosing wrote: > On 04/16/2014 01:35 PM, Etsuro Fujita wrote: >> (2014/04/16 6:55), Hannu Krosing wrote: > ... >> Maybe I'm missing something, but I think that you can do what I think >> you'd like to do by the following procedure: > No, what I'd like PostgreSQL to do

Re: [HACKERS] bgworker crashed or not?

2014-04-16 Thread Andres Freund
On 2014-04-16 12:04:26 -0400, Robert Haas wrote: > And... so what's the problem? You seemed to be saying that the > background worker would need to a more developed error-handling > environment in order to do proper logging, but here you're saying > (rightly, I believe) that it doesn't. Even if i

Re: [HACKERS] BGWorkers, shared memory pointers, and postmaster restart

2014-04-16 Thread Robert Haas
On Wed, Apr 16, 2014 at 7:11 AM, Craig Ringer wrote: >> TL;DR: I don't think there's a safe way to use a BGWorker (static or >> dynamic) with bgw_restart_time != BGW_NEVER_RESTART and a bgw_main_arg >> Datum that points into shared memory, and think we might need a API >> change to fix that. > > A

Re: [HACKERS] bgworker crashed or not?

2014-04-16 Thread Robert Haas
On Wed, Apr 16, 2014 at 12:11 PM, Andres Freund wrote: > On 2014-04-16 12:04:26 -0400, Robert Haas wrote: >> And... so what's the problem? You seemed to be saying that the >> background worker would need to a more developed error-handling >> environment in order to do proper logging, but here you

Re: [HACKERS] Proposal: variant of regclass

2014-04-16 Thread Robert Haas
On Wed, Apr 16, 2014 at 3:27 AM, Tatsuo Ishii wrote: >>> Well, I noticed that, too, but I didn't think it was my job to tell >>> the patch author what functions he should have wanted. A follow-on >>> patch to add to_regprocedure and to_regoperator wouldn't be much work, >>> if you want that. >> >

Re: [HACKERS] Dynamic Background Workers and clean exit

2014-04-16 Thread Robert Haas
On Wed, Apr 16, 2014 at 5:27 AM, Petr Jelinek wrote: > Hello, > > I've been recently doing some work with dynamic bgworkers and noticed that I > have no way of saying "I am done now and want to exit cleanly" because > bgworkers get restarted automatically on exit code 0 no matter what is the > res

[HACKERS] AF_UNSPEC vs PF_UNSPEC

2014-04-16 Thread Tom Lane
While wondering what the heck is going on in http://www.postgresql.org/message-id/534e8fbb.9060...@gmail.com I chanced to notice that pgstat.c and a couple of other places set up arguments for getaddrinfo() like this: hints.ai_family = PF_UNSPEC; whereas the Single Unix Spec says clearly

Re: [HACKERS] bgworker crashed or not?

2014-04-16 Thread Andres Freund
On 2014-04-16 12:20:01 -0400, Robert Haas wrote: > On Wed, Apr 16, 2014 at 12:11 PM, Andres Freund > wrote: > > On 2014-04-16 12:04:26 -0400, Robert Haas wrote: > >> And... so what's the problem? You seemed to be saying that the > >> background worker would need to a more developed error-handlin

Re: [HACKERS] Ctrl+C from sh can shut down daemonized PostgreSQL cluster

2014-04-16 Thread Robert Haas
On Mon, Feb 17, 2014 at 8:26 PM, Tom Lane wrote: > Bruce Momjian writes: >> It certainly might be --- I have no idea. What surprised me is that we >> are relying solely on system() to block signals to pg_ctl-spawned >> servers. The question is whether that is sufficient and whether we >> should

Re: [HACKERS] bgworker crashed or not?

2014-04-16 Thread Robert Haas
On Wed, Apr 16, 2014 at 12:28 PM, Andres Freund wrote: > On 2014-04-16 12:20:01 -0400, Robert Haas wrote: >> On Wed, Apr 16, 2014 at 12:11 PM, Andres Freund >> wrote: >> > On 2014-04-16 12:04:26 -0400, Robert Haas wrote: >> >> And... so what's the problem? You seemed to be saying that the >> >>

Re: [HACKERS] Ctrl+C from sh can shut down daemonized PostgreSQL cluster

2014-04-16 Thread Tom Lane
Robert Haas writes: > On Mon, Feb 17, 2014 at 8:26 PM, Tom Lane wrote: >> Alternatively, we could do what the comments in pg_ctl have long thought >> desirable, namely get rid of use of system() in favor of fork()/exec(). >> With that, pg_ctl could do a setsid() inside the child process. > I lik

Re: [HACKERS] Ctrl+C from sh can shut down daemonized PostgreSQL cluster

2014-04-16 Thread Robert Haas
On Wed, Apr 16, 2014 at 12:35 PM, Tom Lane wrote: > Robert Haas writes: >> On Mon, Feb 17, 2014 at 8:26 PM, Tom Lane wrote: >>> Alternatively, we could do what the comments in pg_ctl have long thought >>> desirable, namely get rid of use of system() in favor of fork()/exec(). >>> With that, pg_c

[HACKERS] Tracking replication slot "blockings"

2014-04-16 Thread Magnus Hagander
I'm thinking it could be interesting to know how many times (or in some other useful unit than "times" - how often) a specific replication slot has "blocked" xlog rotation. Since this AFAIK only happens during checkpoints, it seems it should be "reasonably cheap" to track? It would serve as an indi

Re: [HACKERS] test failure on latest source

2014-04-16 Thread Marco Atzeri
On 16/04/2014 17:40, Tom Lane wrote: The bigger picture though is that this code isn't failing on the buildfarm. So what we need to ask is what's different about Marco's machine. good question. I checked again and I found that the fault is only on the cygwin 64 bit build but not on the cygwin

Re: [HACKERS] Tracking replication slot "blockings"

2014-04-16 Thread Andres Freund
Hi, On 2014-04-16 18:51:41 +0200, Magnus Hagander wrote: > I'm thinking it could be interesting to know how many times (or in some > other useful unit than "times" - how often) a specific replication slot has > "blocked" xlog rotation. Since this AFAIK only happens during checkpoints, > it seems i

Re: [HACKERS] Tracking replication slot "blockings"

2014-04-16 Thread Magnus Hagander
On Wed, Apr 16, 2014 at 6:56 PM, Andres Freund wrote: > Hi, > > On 2014-04-16 18:51:41 +0200, Magnus Hagander wrote: > > I'm thinking it could be interesting to know how many times (or in some > > other useful unit than "times" - how often) a specific replication slot > has > > "blocked" xlog rota

Re: [HACKERS] LDAP: bugfix and deprecated OpenLDAP API

2014-04-16 Thread Magnus Hagander
On Mon, Oct 21, 2013 at 3:31 PM, Albe Laurenz wrote: > Peter Eisentraut wrote: > >> --- 3511,3534 > >> } > >> > >> /* > >> ! * Perform an explicit anonymous bind. > >> ! * This is not necessary in principle, but we want to set a timeout > >> ! * of PGLDAP_TIMEOUT seconds

Re: [HACKERS] Tracking replication slot "blockings"

2014-04-16 Thread Andres Freund
On 2014-04-16 19:09:09 +0200, Magnus Hagander wrote: > On Wed, Apr 16, 2014 at 6:56 PM, Andres Freund wrote: > > The xlog removal code just check the "global minimum" required LSN - it > > doesn't check the individual slots. So you'd need to add a bit more code > > to that location. But it'd be eas

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Claudio Freire
On Wed, Apr 16, 2014 at 4:22 AM, Peter Geoghegan wrote: > > I don't want to dismiss what you're saying about heating and cooling > being unrelated, but I don't find the conclusion that not everything > can be hot obvious. Maybe "heat" should be relative rather than > absolute, and maybe that's act

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Peter Geoghegan
On Wed, Apr 16, 2014 at 4:01 AM, Andres Freund wrote: >> Aren't you interested in the significance of the patch, and the test case? > > Not particularly in the specifics to be honest. The tradeoffs of the > techniques you used in there seem prohibitive to me. It's easy to make > individual cases f

Re: [HACKERS] [GENERAL] pg_upgrade & tablespaces

2014-04-16 Thread Bruce Momjian
On Sun, Jan 12, 2014 at 11:04:41PM -0500, Bruce Momjian wrote: > > In the pgsql_old installation you have symlinks pointing back to the > > current default location. As well pg_tablespace points back to > > /usr/local/pgsql/data/ The issue is that there is not actually > > anything there in the way

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Robert Haas
On Wed, Apr 16, 2014 at 1:42 PM, Peter Geoghegan wrote: > On Wed, Apr 16, 2014 at 4:01 AM, Andres Freund wrote: >>> Aren't you interested in the significance of the patch, and the test case? >> >> Not particularly in the specifics to be honest. The tradeoffs of the >> techniques you used in there

Re: [HACKERS] Need Multixact Freezing Docs

2014-04-16 Thread Josh Berkus
> You can see the current multixact value in pg_controldata output. Keep > timestamped values of that somewhere (a table?) so that you can measure > consumption rate. I don't think we provide SQL-level access to those > values. Bleh. Do we provide SQL-level access in 9.4? If not, I think that

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Peter Geoghegan
On Wed, Apr 16, 2014 at 10:56 AM, Robert Haas wrote: > I don't think he's being unreasonable, and I don't understand why > you're getting bent out of shape about it. You proposed a patch, he > articulated a problem, you don't want to fix it right now. All of > which is fine. Why the ad hominem

Re: [HACKERS] The case against multixact GUCs

2014-04-16 Thread Josh Berkus
On 03/12/2014 09:45 AM, Heikki Linnakangas wrote: > In hindsight, I think permanent multixids in their current form was a > mistake. Before 9.3, the thing that made multixids special was that they > could just be thrown away at a restart. They didn't need freezing. Now > that they do, why not just

Re: [HACKERS] The case against multixact GUCs

2014-04-16 Thread Andres Freund
On 2014-04-16 11:10:52 -0700, Josh Berkus wrote: > On 03/12/2014 09:45 AM, Heikki Linnakangas wrote: > > In hindsight, I think permanent multixids in their current form was a > > mistake. Before 9.3, the thing that made multixids special was that they > > could just be thrown away at a restart. The

Re: [HACKERS] The case against multixact GUCs

2014-04-16 Thread Josh Berkus
On 04/16/2014 11:22 AM, Andres Freund wrote: >> I'm serious. The multixact stuff has been broken since 9.3 >> was released, and it's *still* broken. We can't give users any guidance >> or tools on how to set multixact stuff, and autovacuum doesn't handle it >> properly. > > Sorry, but I think you

Re: [HACKERS] The case against multixact GUCs

2014-04-16 Thread Andres Freund
On 2014-04-16 11:25:49 -0700, Josh Berkus wrote: > On 04/16/2014 11:22 AM, Andres Freund wrote: > >> I'm serious. The multixact stuff has been broken since 9.3 > >> was released, and it's *still* broken. We can't give users any guidance > >> or tools on how to set multixact stuff, and autovacuum d

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Merlin Moncure
On Wed, Apr 16, 2014 at 1:07 PM, Peter Geoghegan wrote: > On Wed, Apr 16, 2014 at 10:56 AM, Robert Haas wrote: >> I don't think he's being unreasonable, and I don't understand why >> you're getting bent out of shape about it. You proposed a patch, he >> articulated a problem, you don't want to f

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Tom Lane
Merlin Moncure writes: > Anyways, I'm still curious if you can post similar numbers basing the > throttling on gross allocation counts instead of time. Meaning: some > number of buffer allocations has to have occurred before you consider > eviction. Besides being faster I think it's a better imp

Re: [HACKERS] The case against multixact GUCs

2014-04-16 Thread Josh Berkus
On 04/16/2014 11:30 AM, Andres Freund wrote: > On 2014-04-16 11:25:49 -0700, Josh Berkus wrote: >> On 04/16/2014 11:22 AM, Andres Freund wrote: I'm serious. The multixact stuff has been broken since 9.3 was released, and it's *still* broken. We can't give users any guidance or tools

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Robert Haas
On Wed, Apr 16, 2014 at 2:07 PM, Peter Geoghegan wrote: > On Wed, Apr 16, 2014 at 10:56 AM, Robert Haas wrote: >> I don't think he's being unreasonable, and I don't understand why >> you're getting bent out of shape about it. You proposed a patch, he >> articulated a problem, you don't want to f

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Merlin Moncure
On Wed, Apr 16, 2014 at 1:44 PM, Tom Lane wrote: > Merlin Moncure writes: >> Anyways, I'm still curious if you can post similar numbers basing the >> throttling on gross allocation counts instead of time. Meaning: some >> number of buffer allocations has to have occurred before you consider >> e

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Peter Geoghegan
On Wed, Apr 16, 2014 at 12:17 PM, Robert Haas wrote: > I don't agree. I think it's perfectly appropriate to raise potential > issues at the earliest possible time. If I didn't *strongly* emphasize my intent in writing the patch up front, I'd certainly agree. I just don't see why what I've done c

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Tom Lane
Merlin Moncure writes: > On Wed, Apr 16, 2014 at 1:44 PM, Tom Lane wrote: >> Yeah --- I think wall-clock-based throttling is fundamentally the wrong >> thing anyway. Are we going to start needing a CPU speed measurement to >> tune the algorithm with? Not the place to be going. But driving it o

Re: [HACKERS] Need Multixact Freezing Docs

2014-04-16 Thread Alvaro Herrera
Josh Berkus wrote: > > > You can see the current multixact value in pg_controldata output. Keep > > timestamped values of that somewhere (a table?) so that you can measure > > consumption rate. I don't think we provide SQL-level access to those > > values. > > Bleh. Do we provide SQL-level acc

Re: [HACKERS] Need Multixact Freezing Docs

2014-04-16 Thread Josh Berkus
On 04/16/2014 01:30 PM, Alvaro Herrera wrote: > Josh Berkus wrote: >> >>> You can see the current multixact value in pg_controldata output. Keep >>> timestamped values of that somewhere (a table?) so that you can measure >>> consumption rate. I don't think we provide SQL-level access to those >>>

Re: [HACKERS] Need Multixact Freezing Docs

2014-04-16 Thread Alvaro Herrera
Josh Berkus wrote: > > Josh Berkus wrote: > >> > >>> You can see the current multixact value in pg_controldata output. Keep > >>> timestamped values of that somewhere (a table?) so that you can measure > >>> consumption rate. I don't think we provide SQL-level access to those > >>> values. > >>

Re: [HACKERS] bgworker crashed or not?

2014-04-16 Thread Petr Jelinek
On 16/04/14 18:34, Robert Haas wrote: On Wed, Apr 16, 2014 at 12:28 PM, Andres Freund wrote: On 2014-04-16 12:20:01 -0400, Robert Haas wrote: On Wed, Apr 16, 2014 at 12:11 PM, Andres Freund wrote: I'm still not seeing the problem. It's the background worker's job to make sure that the right

[HACKERS] New functions for sslinfo extension

2014-04-16 Thread Воронин Дмитрий
Hello all, postgresmen! I want to present some functions to sslinfo extension module:1) ssl_get_count_of_extensions() --- get count of X509v3 extensions from client certificate;2) ssl_get_extension_names() --- get short names of X509v3 extensions from client certificate;3) ssl_get_extension_value(t

Re: [HACKERS] bogus tsdict, tsparser, etc object identities

2014-04-16 Thread Alvaro Herrera
Alvaro Herrera wrote: > This problem is new in 9.3, so backpatching to that. Prior to that we > didn't have object identities. Done. -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Need Multixact Freezing Docs

2014-04-16 Thread Josh Berkus
>> So if age() doesn't mean anything, then how are users to know when the >> need to freeze? > > I don't understand. Autovacuum will freeze this automatically when the > threshold is reached. Users don't need to do anything. What I'm asking is: - how do users know if Autovacuum is keeping up

Re: [HACKERS] assertion failure 9.3.4

2014-04-16 Thread Alvaro Herrera
So, from top to bottom I see the following elements: * backend is executing a query * this query is getting captured by pg_stat_statements * the query is also getting captured by autoexplain, in chain from pg_stat_statements * autoexplain runs the query, which invokes a plpgsql function * this

Re: [HACKERS] improve the help message about psql -F

2014-04-16 Thread Bruce Momjian
On Tue, Feb 11, 2014 at 10:02:20PM -0500, Peter Eisentraut wrote: > If you are going to change the help string for -F, you should also > update the help string for -R, and possibly for -z and -0. Patch applied with all the suggestions merged in; commitfest item marked as committed: -F, --field

Re: [HACKERS] pg_get_viewdefs() indentation considered harmful

2014-04-16 Thread Bruce Momjian
On Sat, Jan 25, 2014 at 01:02:36PM -0500, Andrew Dunstan wrote: > > On 01/25/2014 11:06 AM, Tom Lane wrote: > >Robert Haas writes: > >>On Fri, Jan 24, 2014 at 8:53 PM, Greg Stark wrote: > >>>Indeed even aside from the performance questions, once you're indented > >>>5-10 times the indention stop

Re: [HACKERS] shouldn't we log permission errors when accessing the configured trigger file?

2014-04-16 Thread Bruce Momjian
On Mon, Jan 27, 2014 at 03:45:38PM +0100, Magnus Hagander wrote: > On Mon, Jan 27, 2014 at 3:43 PM, Robert Haas wrote: > > On Sun, Jan 26, 2014 at 1:03 PM, Andres Freund > wrote: > > For some reason CheckForStandbyTrigger() doesn't report permission > > errors when stat()int the

[HACKERS] Re: test script, was Re: [COMMITTERS] pgsql: psql: conditionally display oids and replication identity

2014-04-16 Thread Bruce Momjian
On Tue, Apr 15, 2014 at 12:32:36PM -0700, David Fetter wrote: > On Tue, Apr 15, 2014 at 02:46:34PM -0400, Bruce Momjian wrote: > > On Tue, Apr 15, 2014 at 02:32:53PM -0400, Tom Lane wrote: > > > Bruce Momjian writes: > > > > psql: conditionally display oids and replication identity > > > > > > Bu

Re: [HACKERS] Misaligned BufferDescriptors causing major performance problems on AMD

2014-04-16 Thread Bruce Momjian
On Thu, Feb 6, 2014 at 09:40:32AM +0100, Andres Freund wrote: > On 2014-02-05 12:36:42 -0500, Robert Haas wrote: > > >> It may well be that your proposal is spot on. But I'd like to see some > > >> data-structure-by-data-structure measurements, rather than assuming that > > >> alignment must be a

Re: [HACKERS] assertion failure 9.3.4

2014-04-16 Thread Tom Lane
Alvaro Herrera writes: > I'm not quite clear on why the third query, the one in ri_PerformCheck, > is invoking a sequence. It's not --- SeqNext is the next-tuple function for a sequential scan. Nothing to do with sequences. Now, it *is* worth wondering why the heck a query on the table's primary

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-16 Thread Peter Geoghegan
On Wed, Apr 16, 2014 at 7:29 AM, Robert Haas wrote: > 2. Improving the choice of which buffers we evict, which is what > Peter's talking about, or at least what I think he's talking about. > > Those things are both important, but they're different, and I'm not > sure that working on one precludes

  1   2   >