Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

Here is a benchmark that is similar to my earlier one, but with a rate limit of 125 tps, to help us better characterize how the prototype patch helps performance: http://postgres-benchmarks.s3-website-us-east-1.amazonaws.com/3-sec-delay-limit/ Again, these are 15 minute runs with unlogged tables

Re: [HACKERS] assertion failure 9.3.4

On 2014-04-21 19:43:15 -0400, Andrew Dunstan wrote: > > On 04/21/2014 02:54 PM, Andres Freund wrote: > >Hi, > > > >I spent the last two hours poking arounds in the environment Andrew > >provided and I was able to reproduce the issue, find a assert to > >reproduce it much faster and find a possible

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

On 22/04/14 09:25, Andres Freund wrote: On 2014-04-21 17:21:20 -0400, Bruce Momjian wrote: On Mon, Apr 21, 2014 at 02:08:51PM -0700, Joshua Drake wrote: If the community had more *BSD presence I think it would be great but it isn't all that viable at this point. I do know however that no-one in

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

On Tue, Apr 22, 2014 at 9:58 AM, Tatsuo Ishii wrote: > > * Tatsuo Ishii (is...@postgresql.org) wrote: > >> I observe performance degradation with PostgreSQL 9.3 vs 9.2 on Linux > >> as well. The hardware is HP DL980G7, 80 cores, 2TB mem, RHEL 6, > >> pgbench is used (read only query), scale fact

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

>> Can you isolate the sysv-vs-mmap patch and see what happens with just >> that change..? > > Unfortunately I don't have an access to the machine at this moment. Where is the patch? I would like to test it on a smaller machine for now. Best regards, -- Tatsuo Ishii SRA OSS, Inc. Japan English:

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

On 04/21/2014 06:19 PM, Andrew Dunstan wrote: If we never start we'll never get there. I can think of several organizations that might be approached to donate hardware. Like .Org? We have a hardware farm, a rack full of hardware and spindles. It isn't the most current but it is there. S

Re: [HACKERS] Composite Datums containing toasted fields are a bad idea(?)

On Mon, Apr 21, 2014 at 10:57:34AM -0400, Tom Lane wrote: > Noah Misch writes: > > I wonder how it would work out to instead delay this new detoast effort > > until > > toast_insert_or_update(). > > That would require toast_insert_or_update() to know about every container > datatype. I doubt it

Re: [HACKERS] DISCARD ALL (Again)

On Mon, Apr 21, 2014 at 12:00 AM, Stephen Frost wrote: > > * Robert Haas (robertmh...@gmail.com) wrote: > > On Fri, Apr 18, 2014 at 2:24 AM, Tom Lane wrote: > > > 2. While I'm no Python expert, I believe GD is just a specific instance > > > of a general capability for global state in Python. Are

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

On Mon, Apr 21, 2014 at 5:59 PM, Peter Geoghegan wrote: > LRU-K, and 2Q have roughly the same advantages. I'm > reasonably confident you can have the best of both worlds, or > something closer to it. Having said that, a big part of what I'd like to accomplish here is to address the more general p

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

On Mon, Apr 21, 2014 at 6:12 PM, Tom Lane wrote: > Did you read the commit message I pointed to? Yes. > (See also 4e8af8d27.) Oh, I wasn't actually aware of the fact that 2Q made it into the tree. I thought that the first commit message you referred to just referenced on-list discussion of 2Q.

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

On 04/21/2014 09:16 PM, Peter Geoghegan wrote: On Mon, Apr 21, 2014 at 6:08 PM, Tatsuo Ishii wrote: What we would need is a way to graph the results - that's something beyond my very rudimentary expertise in web programming. If anyone feels like collaborating I'd be glad to hear from them (The

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

On Mon, Apr 21, 2014 at 6:08 PM, Tatsuo Ishii wrote: >> What we would need is a way to graph the results - that's something >> beyond my very rudimentary expertise in web programming. If anyone >> feels like collaborating I'd be glad to hear from them (The web site >> is programmed in perl + Templ

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

Peter Geoghegan writes: > On Mon, Apr 21, 2014 at 5:50 PM, Tom Lane wrote: >> ARC *was* the predecessor algorithm. See commit 5d5087363. > I believe that the main impetus for replacing ARC with clock sweep > came from patent issues, though. That was one issue, but performance gains were a larg

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

> This is exactly why we need a benchfarm. > > I actually have a client working based on Greg Smith's pgbench tools. > > What we would need is a way to graph the results - that's something > beyond my very rudimentary expertise in web programming. If anyone > feels like collaborating I'd be glad

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

On 04/21/2014 08:49 PM, Stephen Frost wrote: * Tatsuo Ishii (is...@postgresql.org) wrote: I observe performance degradation with PostgreSQL 9.3 vs 9.2 on Linux as well. The hardware is HP DL980G7, 80 cores, 2TB mem, RHEL 6, pgbench is used (read only query), scale factor is 1,000 (DB size 15GB

Re: [HACKERS] assertion failure 9.3.4

All, I've taken a stab at creating a reproduceable test case based on the characterisitics of the production issues I'm seeing. But clearly there's an element I'm missing, because I'm not able to produce the bug with a pgbench-based test case. My current test has FKs, updating both FK'd tables,

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

On Mon, Apr 21, 2014 at 5:50 PM, Tom Lane wrote: > ARC *was* the predecessor algorithm. See commit 5d5087363. I believe that the main impetus for replacing ARC with clock sweep came from patent issues, though. It was a happy coincidence that clock sweep happened to be better than ARC, but that d

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

> * Tatsuo Ishii (is...@postgresql.org) wrote: >> I observe performance degradation with PostgreSQL 9.3 vs 9.2 on Linux >> as well. The hardware is HP DL980G7, 80 cores, 2TB mem, RHEL 6, >> pgbench is used (read only query), scale factor is 1,000 (DB size >> 15GB). > > Can you isolate the sysv-vs

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

Peter Geoghegan writes: > On Mon, Apr 21, 2014 at 5:28 PM, Tom Lane wrote: >> We used to have one. It was a big bottleneck --- and this was years >> ago, when the buffer manager was much less scalable than it is today. >> (IIRC, getting rid of a central lock was one of the main advantages >> of

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

* Tatsuo Ishii (is...@postgresql.org) wrote: > I observe performance degradation with PostgreSQL 9.3 vs 9.2 on Linux > as well. The hardware is HP DL980G7, 80 cores, 2TB mem, RHEL 6, > pgbench is used (read only query), scale factor is 1,000 (DB size > 15GB). Can you isolate the sysv-vs-mmap patc

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

On 04/21/2014 03:08 PM, Jim Nasby wrote: On 4/21/14, 4:08 PM, Joshua D. Drake wrote: If the community had more *BSD presence I think it would be great but it isn't all that viable at this point. I do know however that no-one in this community would turn down a team of FreeBSD advocates helping

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

On Mon, Apr 21, 2014 at 5:28 PM, Tom Lane wrote: > We used to have one. It was a big bottleneck --- and this was years > ago, when the buffer manager was much less scalable than it is today. > (IIRC, getting rid of a central lock was one of the main advantages > of the current clock sweep code ov

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

On Mon, Apr 21, 2014 at 4:10 AM, Andres Freund wrote: > If there are indeed such large regressions on FreeBSD we need to treat > them as postgres regressions. It's nicer not to add config options for > things that don't need it, but apparently that's not the case here. +1, but I think this is som

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

> I see performance degradation with PostgreSQL 9.3 vs 9.2 on FreeBSD, and I'm > wondering who to poke to mitigate the problem. In reference to this thread > [1], who where the FreeBSD people that Francois mentioned? If mmap needs to > perform well in the kernel, I'd like to know of someone with

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

Jim Nasby writes: > How *certain* are we that a single freelist lock (that actually ONLY > protects the freelist) would be that big a deal? We used to have one. It was a big bottleneck --- and this was years ago, when the buffer manager was much less scalable than it is today. (IIRC, getting rid

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

Stephen Frost wrote > * Alfred Perlstein ( > alfred@ > ) wrote: >> On 4/21/14, 12:47 PM, Stephen Frost wrote: >> > Asking for help to address the FreeBSD performance would have >> >been much better received. Thanks, Stephen >> >> That is exactly what I did, I asked for a version of postgresql t

Re: [HACKERS] assertion failure 9.3.4

On 04/21/2014 02:54 PM, Andres Freund wrote: Hi, I spent the last two hours poking arounds in the environment Andrew provided and I was able to reproduce the issue, find a assert to reproduce it much faster and find a possible root cause. What's the assert that makes it happen faster? That m

Re: [HACKERS] AXLE Plans for 9.5 and 9.6

On 04/21/2014 03:41 PM, Simon Riggs wrote: > Storage Efficiency > * Compression > * Column Orientation You might look at turning this: http://citusdata.github.io/cstore_fdw/ ... into a more integrated part of Postgres. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via

Re: [HACKERS] Removing dependency to wsock32.lib when compiling code on WIndows

On Mon, Apr 21, 2014 at 12:54 PM, Michael Paquier wrote: > When doing some work on Windows, I noticed that the mkvc specs in > src/tools/msvc use wsock32.lib, which is as far as I understand an > old, obsolete version of the Windows socket library. Wouldn't it make > sense to update the specs to

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

On Mon, Apr 21, 2014 at 8:07 PM, David G Johnston wrote: > Jim Nasby-2 wrote >>> I feel that if there is no memory pressure, frankly it doesnt matter much >>> about what gets out and what not. The case I am specifically targeting is >>> when the clocksweep gets to move about a lot i.e. high memory

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

Jim Nasby-2 wrote >> I feel that if there is no memory pressure, frankly it doesnt matter much >> about what gets out and what not. The case I am specifically targeting is >> when the clocksweep gets to move about a lot i.e. high memory pressure >> workloads. Of course, I may be totally wrong here

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

On 4/16/14, 10:28 AM, Robert Haas wrote: Also, I think the scalability problems around buffer eviction are eminently solvable, and in particular I'm hopeful that Amit is going to succeed in solving them. Suppose we have a background process (whether the background writer or some other) that runs

[HACKERS] AXLE Plans for 9.5 and 9.6

I've discussed 2ndQuadrant's involvement in the AXLE project a few times publicly, but never on this mailing list. The project relates to innovation and improvement in Business Intelligence for systems based upon PostgreSQL in the range of 10-100TB. Our work will span the 9.5 and 9.6 cycles. We're

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

On 4/18/14, 2:51 PM, Atri Sharma wrote: I feel that if there is no memory pressure, frankly it doesnt matter much about what gets out and what not. The case I am specifically targeting is when the clocksweep gets to move about a lot i.e. high memory pressure workloads. Of course, I may be to

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

On 4/15/14, 1:15 PM, Peter Geoghegan wrote: On Tue, Apr 15, 2014 at 9:30 AM, Merlin Moncure wrote: >There are many reports of improvement from lowering shared_buffers. >The problem is that it tends to show up on complex production >workloads and that there is no clear evidence pointing to probl

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

On 4/21/14, 4:08 PM, Joshua D. Drake wrote: If the community had more *BSD presence I think it would be great but it isn't all that viable at this point. I do know however that no-one in this community would turn down a team of FreeBSD advocates helping us make PostgreSQL awesome for PostgreSQ

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

On 4/21/14, 2:23 PM, Stephen Frost wrote: Alfred, * Alfred Perlstein (alf...@freebsd.org) wrote: On 4/21/14, 12:47 PM, Stephen Frost wrote: Asking for help to address the FreeBSD performance would have been much better received. Thanks, Stephen That is exactly what I did, I asked for a ver

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

On 2014-04-21 17:21:20 -0400, Bruce Momjian wrote: > On Mon, Apr 21, 2014 at 02:08:51PM -0700, Joshua Drake wrote: > > If the community had more *BSD presence I think it would be great > > but it isn't all that viable at this point. I do know however that > > no-one in this community would turn dow

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

On Mon, Apr 21, 2014 at 02:08:51PM -0700, Joshua Drake wrote: > If the community had more *BSD presence I think it would be great > but it isn't all that viable at this point. I do know however that > no-one in this community would turn down a team of FreeBSD advocates > helping us make PostgreSQL

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

Alfred, * Alfred Perlstein (alf...@freebsd.org) wrote: > On 4/21/14, 12:47 PM, Stephen Frost wrote: > > Asking for help to address the FreeBSD performance would have > >been much better received. Thanks, Stephen > > That is exactly what I did, I asked for a version of postgresql that > was easy

Re: [HACKERS] assertion failure 9.3.4

> Josh, how long does it take you to reproduce the issue? A couple hours. > And can you > reproduce it outside of a production environment? Not yet, still working on that. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@pos

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

On 04/21/2014 10:39 AM, Alfred Perlstein wrote: What I am seeing here is unfortunately a very strong departure from FreeBSD support by the community from several of the developers. In fact over drinks at pgcon last year there were a TON of jokes making fun of FreeBSD users and developers which

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

On Mon, Apr 21, 2014 at 01:52:48PM -0700, Alfred Perlstein wrote: > > On 4/21/14, 12:47 PM, Stephen Frost wrote: > > Asking for help to address the FreeBSD performance would have > >been much better received. Thanks, Stephen > > That is exactly what I did, I asked for a version of postgresql tha

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

On 4/21/14, 12:47 PM, Stephen Frost wrote: Asking for help to address the FreeBSD performance would have been much better received. Thanks, Stephen That is exactly what I did, I asked for a version of postgresql that was easy to switch at runtime between two behaviors. That would make it

Re: [HACKERS] assertion failure 9.3.4

On 2014-04-21 12:31:09 -0700, Josh Berkus wrote: > On 04/21/2014 12:26 PM, Tom Lane wrote: > > Andres Freund writes: > >> I spent the last two hours poking arounds in the environment Andrew > >> provided and I was able to reproduce the issue, find a assert to > >> reproduce it much faster and find

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

* Andres Freund (and...@2ndquadrant.com) wrote: > On 2014-04-21 15:47:31 -0400, Stephen Frost wrote: > > That's certainly unfortunate. For my 2c, I'd recommend that you write a > > minimal implementation that allows you to test just the sysv-vs-mmap > > case (which could certainly take an option,

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

On 2014-04-21 15:47:31 -0400, Stephen Frost wrote: > That's certainly unfortunate. For my 2c, I'd recommend that you write a > minimal implementation that allows you to test just the sysv-vs-mmap > case (which could certainly take an option, to avoid having to > recompile during testing), or even

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

Alfred, * Alfred Perlstein (alf...@freebsd.org) wrote: > Stephen, please calm down on the hyperbole, seriously, picking > another db is not an attack. Perhaps it wasn't intended by you, but to those of us reading it, your comments came across clearly as "if you don't fix this, then we won't use P

Re: [HACKERS] assertion failure 9.3.4

On 2014-04-21 15:26:03 -0400, Tom Lane wrote: > Andres Freund writes: > > I spent the last two hours poking arounds in the environment Andrew > > provided and I was able to reproduce the issue, find a assert to > > reproduce it much faster and find a possible root cause. > > Hmm ... is this the s

Re: [HACKERS] assertion failure 9.3.4

On 04/21/2014 03:26 PM, Tom Lane wrote: Andres Freund writes: I spent the last two hours poking arounds in the environment Andrew provided and I was able to reproduce the issue, find a assert to reproduce it much faster and find a possible root cause. Hmm ... is this the same thing Josh is rep

Re: [HACKERS] assertion failure 9.3.4

On 04/21/2014 12:26 PM, Tom Lane wrote: > Andres Freund writes: >> I spent the last two hours poking arounds in the environment Andrew >> provided and I was able to reproduce the issue, find a assert to >> reproduce it much faster and find a possible root cause. > > Hmm ... is this the same thing

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

On 4/21/14, 11:14 AM, Stephen Frost wrote: Alfred, * Alfred Perlstein (alf...@freebsd.org) wrote: On 4/21/14, 9:51 AM, Andres Freund wrote: On 2014-04-21 09:42:06 -0700, Alfred Perlstein wrote: Sure, to be fair, we are under the gun here for a product, it may just mean that the end result of

Re: [HACKERS] assertion failure 9.3.4

Andres Freund writes: > I spent the last two hours poking arounds in the environment Andrew > provided and I was able to reproduce the issue, find a assert to > reproduce it much faster and find a possible root cause. Hmm ... is this the same thing Josh is reporting? If so, why the apparent conn

Re: [HACKERS] assertion failure 9.3.4

Hi, I spent the last two hours poking arounds in the environment Andrew provided and I was able to reproduce the issue, find a assert to reproduce it much faster and find a possible root cause. Since the symptom of the problem seem to be multixacts with more than one updating xid, I added a check

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

Alfred, * Alfred Perlstein (alf...@freebsd.org) wrote: > On 4/21/14, 9:51 AM, Andres Freund wrote: > >On 2014-04-21 09:42:06 -0700, Alfred Perlstein wrote: > >>Sure, to be fair, we are under the gun here for a product, it may just mean > >>that the end result of that conversation is "mysql". > >Pe

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

On 4/21/14, 9:52 AM, Alvaro Herrera wrote: Alfred Perlstein wrote: I am unsure of the true overhead of making this a runtime tunable so pardon if I'm asking for "a lot". From the perspective of both an OS developer and postgresql user (I am both) it really makes more sense to have it a runtim

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

* Alfred Perlstein (alf...@freebsd.org) wrote: > How high on the hierarchy of PostgreSQL's "needs" is making a single > option a tunable versus compile time thing? I mean seriously you > mean to stick on this one point when one of your users are asking > you about this? That is pretty concerning

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

On 4/21/14, 9:51 AM, Andres Freund wrote: On 2014-04-21 09:42:06 -0700, Alfred Perlstein wrote: Sure, to be fair, we are under the gun here for a product, it may just mean that the end result of that conversation is "mysql". Personally arguments in that vain are removing just about any incenti

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

Alfred Perlstein wrote: > How high on the hierarchy of PostgreSQL's "needs" is making a single > option a tunable versus compile time thing? I mean seriously you > mean to stick on this one point when one of your users are asking > you about this? That is pretty concerning to me. I think the s

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

On 4/21/14, 9:51 AM, Andrew Dunstan wrote: On 04/21/2014 12:44 PM, Alfred Perlstein wrote: On 4/21/14 9:38 AM, Andrew Dunstan wrote: On 04/21/2014 12:25 PM, Alfred Perlstein wrote: 1. OS developers are not the target audience for GUCs. If the OS developers want to test and can't be bo

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

On 4/21/14 4:10 AM, Andres Freund wrote: Hi, On 2014-04-20 11:24:38 +0200, Palle Girgensohn wrote: I see performance degradation with PostgreSQL 9.3 vs 9.2 on FreeBSD, and I'm wondering who to poke to mitigate the problem. In reference to this thread [1], who where the FreeBSD people that Fra

Re: [HACKERS] assertion failure 9.3.4

Josh Berkus writes: > 1) I've confirmed at the 2nd site that the issue doesn't happen if > pg_stat_statements.so is not loaded. So this seems to be confirmation > that either auto_explain, pg_stat_statements, or both need to be loaded > (but not necessarily created as extensions) in order to have

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

Alfred Perlstein wrote: > I am unsure of the true overhead of making this a runtime tunable so > pardon if I'm asking for "a lot". From the perspective of both an > OS developer and postgresql user (I am both) it really makes more > sense to have it a runtime tunable for the following reasons: >

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

On 2014-04-21 09:42:06 -0700, Alfred Perlstein wrote: > Sure, to be fair, we are under the gun here for a product, it may just mean > that the end result of that conversation is "mysql". Personally arguments in that vain are removing just about any incentive I have to work on the problem. Greetin

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

On 04/21/2014 12:44 PM, Alfred Perlstein wrote: On 4/21/14 9:38 AM, Andrew Dunstan wrote: On 04/21/2014 12:25 PM, Alfred Perlstein wrote: 1. OS developers are not the target audience for GUCs. If the OS developers want to test and can't be botherrd with building with a couple of differ

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

On 4/21/14 9:38 AM, Andrew Dunstan wrote: On 04/21/2014 12:25 PM, Alfred Perlstein wrote: 1. OS developers are not the target audience for GUCs. If the OS developers want to test and can't be botherrd with building with a couple of different parameters then I'm not very impressed. 2. W

Re: [HACKERS] Composite Datums containing toasted fields are a bad idea(?)

Merlin Moncure wrote: > On Mon, Apr 21, 2014 at 10:40 AM, Andres Freund > wrote: > > And too bad that infomask bits are so scarce :(. We really need to > > reclaim HEAP_MOVED_OFF and HEAP_MOVED_IN. > > The only consequence of that is losing support for in-place update for > pre-9.0 (of which th

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

On 4/21/14 9:34 AM, Stephen Frost wrote: * Alfred Perlstein (alf...@freebsd.org) wrote: There is definitely hope, however changes to the FreeBSD vm are taken as seriously as changes to core changes to Postresql's store. In addition changes to vm is somewhat in the realm of complexity of Postgres

Re: [HACKERS] assertion failure 9.3.4

> Can you get the infomask bits..? What's does pg_controldata report wrt > the MultiXid's? Can't get the infomask bits. pg_controldata attached, with some redactions. Unfortunately, it appears that they've continued to do tests on this system, so the XID counter has advanced somewhat. pg_cont

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

On 04/21/2014 12:25 PM, Alfred Perlstein wrote: 1. OS developers are not the target audience for GUCs. If the OS developers want to test and can't be botherrd with building with a couple of different parameters then I'm not very impressed. 2. We should be trying to get rid of GUCs where

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

* Alfred Perlstein (alf...@freebsd.org) wrote: > There is definitely hope, however changes to the FreeBSD vm are > taken as seriously as changes to core changes to Postresql's store. > In addition changes to vm is somewhat in the realm of complexity of > Postgresql store as well so it may not be co

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

On 4/21/14 9:24 AM, Andrew Dunstan wrote: On 04/21/2014 11:59 AM, Alfred Perlstein wrote: On 4/21/14 8:45 AM, Andrew Dunstan wrote: On 04/21/2014 11:39 AM, Magnus Hagander wrote: On Mon, Apr 21, 2014 at 4:51 PM, Andres Freund mailto:and...@2ndquadrant.com>> wrote: On 2014-04-21 10:45:2

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

On Mon, Apr 21, 2014 at 05:43:46PM +0200, Andres Freund wrote: > > On 2014-04-21 17:39:39 +0200, Magnus Hagander wrote: > > But do we really want a *guc* for it though? Isn't it enough (and in fact > > better) with a configure switch to pick the implementation when multiple > > are available, that

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

On 04/21/2014 11:59 AM, Alfred Perlstein wrote: On 4/21/14 8:45 AM, Andrew Dunstan wrote: On 04/21/2014 11:39 AM, Magnus Hagander wrote: On Mon, Apr 21, 2014 at 4:51 PM, Andres Freund mailto:and...@2ndquadrant.com>> wrote: On 2014-04-21 10:45:24 -0400, Tom Lane wrote: > Andres Freun

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

On 4/21/14 9:13 AM, Stephen Frost wrote: * Alfred Perlstein (alf...@freebsd.org) wrote: Can the package builder not set the default for the runtime tunable? Yeah, I was thinking about that also, but at least in this case it seems pretty clear that the 'right' answer is known at build time. Ho

Re: [HACKERS] assertion failure 9.3.4

* Josh Berkus (j...@agliodbs.com) wrote: > 1) I've confirmed at the 2nd site that the issue doesn't happen if > pg_stat_statements.so is not loaded. So this seems to be confirmation > that either auto_explain, pg_stat_statements, or both need to be loaded > (but not necessarily created as extensio

Re: [HACKERS] assertion failure 9.3.4

All, More on this: 1) I've confirmed at the 2nd site that the issue doesn't happen if pg_stat_statements.so is not loaded. So this seems to be confirmation that either auto_explain, pg_stat_statements, or both need to be loaded (but not necessarily created as extensions) in order to have the iss

Re: [HACKERS] Composite Datums containing toasted fields are a bad idea(?)

On Mon, Apr 21, 2014 at 10:40 AM, Andres Freund wrote: > On 2014-04-21 11:30:57 -0400, Tom Lane wrote: >> Andres Freund writes: >> > I unfortunately haven't followed this in detail, but shouldn't it be >> > relatively easily to make this even cheaper by checking for >> > HEAP_HASEXTERNAL? If it'

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

On 2014-04-21 11:58:10 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2014-04-21 11:45:49 -0400, Andrew Dunstan wrote: > >> That seems to make more sense. I can't imagine why this would be a runtime > >> parameter as opposed to build time. > > > Because that implies that packagers and port

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

* Alfred Perlstein (alf...@freebsd.org) wrote: > Can the package builder not set the default for the runtime tunable? Yeah, I was thinking about that also, but at least in this case it seems pretty clear that the 'right' answer is known at build time. > Honestly we're about to select a db platfor

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

Andres Freund writes: > On 2014-04-21 11:45:49 -0400, Andrew Dunstan wrote: >> That seems to make more sense. I can't imagine why this would be a runtime >> parameter as opposed to build time. > Because that implies that packagers and porters need to make that > decision. If it's a GUC people can

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

On 4/21/14 8:58 AM, Tom Lane wrote: Andres Freund writes: On 2014-04-21 11:45:49 -0400, Andrew Dunstan wrote: That seems to make more sense. I can't imagine why this would be a runtime parameter as opposed to build time. Because that implies that packagers and porters need to make that decisi

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

On 4/21/14 8:45 AM, Andrew Dunstan wrote: On 04/21/2014 11:39 AM, Magnus Hagander wrote: On Mon, Apr 21, 2014 at 4:51 PM, Andres Freund mailto:and...@2ndquadrant.com>> wrote: On 2014-04-21 10:45:24 -0400, Tom Lane wrote: > Andres Freund mailto:and...@2ndquadrant.com>> writes: > >

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

Hi, On 2014-04-21 11:45:49 -0400, Andrew Dunstan wrote: > That seems to make more sense. I can't imagine why this would be a runtime > parameter as opposed to build time. Because that implies that packagers and porters need to make that decision. If it's a GUC people can benchmark it and decide.

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

On 04/21/2014 11:39 AM, Magnus Hagander wrote: On Mon, Apr 21, 2014 at 4:51 PM, Andres Freund > wrote: On 2014-04-21 10:45:24 -0400, Tom Lane wrote: > Andres Freund mailto:and...@2ndquadrant.com>> writes: > > If there are indeed such large regressions

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

Hi, On 2014-04-21 17:39:39 +0200, Magnus Hagander wrote: > But do we really want a *guc* for it though? Isn't it enough (and in fact > better) with a configure switch to pick the implementation when multiple > are available, that could then be set by default for example by the freebsd > ports buil

Re: [HACKERS] Composite Datums containing toasted fields are a bad idea(?)

On 2014-04-21 11:30:57 -0400, Tom Lane wrote: > Andres Freund writes: > > I unfortunately haven't followed this in detail, but shouldn't it be > > relatively easily to make this even cheaper by checking for > > HEAP_HASEXTERNAL? If it's not set we don't need to iterate over the > > composite's co

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

On Mon, Apr 21, 2014 at 4:51 PM, Andres Freund wrote: > On 2014-04-21 10:45:24 -0400, Tom Lane wrote: > > Andres Freund writes: > > > If there are indeed such large regressions on FreeBSD we need to treat > > > them as postgres regressions. It's nicer not to add config options for > > > things th

Re: [HACKERS] Composite Datums containing toasted fields are a bad idea(?)

Andres Freund writes: > I unfortunately haven't followed this in detail, but shouldn't it be > relatively easily to make this even cheaper by checking for > HEAP_HASEXTERNAL? If it's not set we don't need to iterate over the > composite's columns, right? That's the point I made further down: we

Re: [HACKERS] Composite Datums containing toasted fields are a bad idea(?)

Hi, On 2014-04-20 15:38:23 -0400, Tom Lane wrote: > Some poking around with oprofile says that much of the added time is > coming from added syscache and typcache lookups, as I suspected. > > I did some further work on the patch to make it possible to cache > the element tuple descriptor across c

Re: [HACKERS] Composite Datums containing toasted fields are a bad idea(?)

Noah Misch writes: > I wonder how it would work out to instead delay this new detoast effort until > toast_insert_or_update(). That would require toast_insert_or_update() to know about every container datatype. I doubt it could lead to an extensible or maintainable solution. I'm actually planni

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

On 2014-04-21 10:45:24 -0400, Tom Lane wrote: > Andres Freund writes: > > If there are indeed such large regressions on FreeBSD we need to treat > > them as postgres regressions. It's nicer not to add config options for > > things that don't need it, but apparently that's not the case here. > > >

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

Andres Freund writes: > If there are indeed such large regressions on FreeBSD we need to treat > them as postgres regressions. It's nicer not to add config options for > things that don't need it, but apparently that's not the case here. > Imo this means we need to add GUC to control wether anon

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

On 4/21/14 4:10 AM, Andres Freund wrote: Hi, On 2014-04-20 11:24:38 +0200, Palle Girgensohn wrote: I see performance degradation with PostgreSQL 9.3 vs 9.2 on FreeBSD, and I'm wondering who to poke to mitigate the problem. In reference to this thread [1], who where the FreeBSD people that Fra

[HACKERS] New functions for sslinfo extension

Hello, I make an a patch, which adds 4 functions to sslinfo extension module:1) ssl_get_count_of_extensions() --- get count of X509v3 extensions from client certificate;2) ssl_get_extension_names() --- get short names of X509v3 extensions from client certificate;3) ssl_get_extension_value(text) ---

Re: [HACKERS] Re: [DOCS] Docs incorrectly claiming equivalence between show and pg_settings

On Saturday 19 April 2014 13:38:16 Tom Lane wrote: > > 3 document this restriction > > As for (3), I might be wrong, but I don't think the documentation mentions > the possibility of abusing SET this way at all. Restrictions in > undocumented quasi-features are likewise undocumented. h

Re: [HACKERS] New functions for sslinfo extension

Hello, I make an a patch, which adds 4 functions to sslinfo extension module.  You can view some information of certificate's extensions with those functions. The descriptions of functions I posted in my first letter and in source code. What do you think about it? -- Best regards, Dmitry Voronin --

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

Hi, On 2014-04-20 11:24:38 +0200, Palle Girgensohn wrote: > I see performance degradation with PostgreSQL 9.3 vs 9.2 on FreeBSD, and I'm > wondering who to poke to mitigate the problem. In reference to this thread > [1], who where the FreeBSD people that Francois mentioned? If mmap needs to > p

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

> 21 apr 2014 kl. 11:26 skrev Francois Tigeot : > >> On Sun, Apr 20, 2014 at 04:07:25PM +0200, Palle Girgensohn wrote: >> If mmap needs to perform well in the kernel, I'd like to know of someone with FreeBSD kernel knowledge who is interested in working with mmap perfocmance. I

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

On Sun, Apr 20, 2014 at 04:07:25PM +0200, Palle Girgensohn wrote: > > >> If mmap needs to perform well in the kernel, I'd like to know of someone > >> with FreeBSD kernel knowledge who is interested in working with mmap > >> perfocmance. If mmap is indeed the cuplrit, I've just tested 9.2.8 vs

  1   2   >