Re: [HACKERS] [PERFORM] DELETE vs TRUNCATE explanation

2012-07-17 Thread Greg Smith
not always clear that this is a positive thing. Here's Greg Smith commenting on the more-is-worse phenonmenon: http://archives.postgresql.org/pgsql-hackers/2012-02/msg00564.php You can add crash recovery to the list of things where the interaction with the OS write cache matters a lot too

Re: [HACKERS] Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)

2012-07-17 Thread Greg Smith
heartbeat the BGW is running at. I could even see people changing the frequencies for each independently depending on expected system load. Tune for lower power when you don't expect many users, that sort of thing. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL

[HACKERS] Oracle porting sample instr function

2012-07-02 Thread Greg Smith
A web site doc comment from user skong today points out a small issue around the sample INSTR function given in plpgsql-porting.html that I can't confirm (none of those dirty Oracle instances here today), but it sounds legit. A look at Oracle's documentation on the INSTR function at

Re: [HACKERS] pgbench--new transaction type

2012-06-20 Thread Greg Smith
that logic cleanly while being backward compatible is a bit complicated; I'm going to think about that for a bit. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Backport of fsync queue compaction

2012-06-20 Thread Greg Smith
this and related OS tuning is a better topic for pgsql-performance, will raise this there when I've sorted that out a bit more clearly. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com From

Re: [HACKERS] Backport of fsync queue compaction

2012-06-19 Thread Greg Smith
to move some people off of 8.1 and 8.2 even with the carrot of 8.3 will be much faster at everything. I fear 8.3 is going to end up like 7.4, where a 7 year long support window passes and people are still using it years later. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore

Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-23 Thread Greg Smith
adding buffers the background writer finds reusable to the free list have been on the TODO list since 2007; neither ideas nor code are the problem here. Proving that a) a new policy helps on some workloads and b) it doesn't harm any important workload, those are the hard parts here. -- Greg

Re: [HACKERS] unite recovery.conf and postgresql.conf

2012-05-09 Thread Greg Smith
to fix it in 9.3, or not? Greg Smith was going to allow for files in configuration directories, and that was somehow going to make it easier to manage the configuration files and remove recovery.conf. I don't think Greg did it; I didn't see it in the release notes I just wrote. That was actually

Re: [HACKERS] Future In-Core Replication

2012-05-04 Thread Greg Smith
it to the community once it's moved onto being a proven concept. Since the community has a clear set of guidelines for how and when to submit new features, we make sure the development plans line up with them. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL

Re: [HACKERS] Future In-Core Replication

2012-05-03 Thread Greg Smith
On 05/01/2012 09:09 AM, Robert Haas wrote: I think we ought to be sharing and debugging designs in public, not internally within 2ndQuadrant - or any other company, or any other mailing list other than this one. OK. You go first. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com

Re: [HACKERS] xReader, double-effort

2012-04-27 Thread Greg Smith
(or here) at the beginning of the month: http://archives.postgresql.org/pgsql-students/2012-04/msg2.php but the next time I heard about the subject was the acceptance notice. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support

Re: [HACKERS] xReader, double-effort

2012-04-27 Thread Greg Smith
On 04/27/2012 06:48 PM, Kevin Grittner wrote: That's largely my fault. It may not have come out that way, but I was trying more to point out the complexity of the story rather than assign blame. I find it hard to point at any obvious don't do that again target here. -- Greg Smith

Re: [HACKERS] Bug tracker tool we need

2012-04-17 Thread Greg Smith
to that; I just know that Trac, Redmine, and GitHub haven't felt like a good fit, having used every one of that trio for multiple years now at some point. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com

Re: [HACKERS] Bug tracker tool we need

2012-04-17 Thread Greg Smith
into a fixed in table. -Generate a web page out of the database. I think I've outlined that in a way that would make useful steps toward adopting one of the full packages, too. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7

Re: [HACKERS] Bug tracker tool we need

2012-04-17 Thread Greg Smith
be a useful exercise, one that's likely to benefit any tracker adoption. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Last gasp

2012-04-16 Thread Greg Smith
those, compared with having to be the person saying this isn't ready to commit only at the end. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] Last gasp

2012-04-15 Thread Greg Smith
, the easy path is to just do it yourself and be done. Non committing reviewers can't get that efficiency boost in the cases it's appropriate. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via

Re: [HACKERS] Last gasp

2012-04-15 Thread Greg Smith
more like marketing now though. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] Last gasp

2012-04-15 Thread Greg Smith
for this one. Trying to quantify how much time investment the CF manager role really involves is one of my important projects to chew on. Whoever ends up doing that should at least have an idea what scale of problem they're getting into. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-14 Thread Greg Smith
. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Last gasp

2012-04-14 Thread Greg Smith
On 04/14/2012 03:02 AM, Alex wrote: I didn't follow this whole thread, but have we considered Redmine[1]? It comes up every couple of years in contexts near this one, such as http://wiki.postgresql.org/wiki/TrackerDiscussion -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore

Re: [HACKERS] Last gasp

2012-04-12 Thread Greg Smith
I aimed for in my book wouldn't have been possible without Kevin Grittner, Scott Marlowe, and Jim Mlodgenski as reviewers; it didn't require anyone with commit bits though. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www

Re: [HACKERS] Last gasp

2012-04-12 Thread Greg Smith
just dumped a home-made FUD bomb on them. It's not a high percentage path toward credibility. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] Last gasp

2012-04-11 Thread Greg Smith
usually not this bad, but in every case it's pulling resources off of more valuable jobs. I'd like to dump around 50 pages of new material into the docs as a start, but I don't want to take so much time away from the code oriented committers to chew on that much. -- Greg Smith 2ndQuadrant US

Re: [HACKERS] Last gasp

2012-04-10 Thread Greg Smith
enough data to establish meaningful statistics on feedback. I had Robert send me a dump of the data that's in the CF app the other day. I'm hoping to do some useful data mining on it before PGCon. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Greg Smith
right now will start looking fat in a year or two, and more things might need to be shifted to usec instead. Checkpoint timing can survive having less resolution because its primary drumbeat is very unlikely to drop below the minutes range. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com

Re: [HACKERS] Last gasp

2012-04-10 Thread Greg Smith
On 04/10/2012 01:33 PM, Robert Haas wrote: I also think that people were more receptive to my reviews before I got a commit bit. That's not true; many people were just as annoyed at you back then. (Robert knows I'm kidding. I hope.) -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com

Re: [HACKERS] Last gasp

2012-04-10 Thread Greg Smith
in a positive way. All I see coming for next year is a dramatic increase in this class of problem. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] Initial 9.2 pgbench write results

2012-04-10 Thread Greg Smith
of usage count. Your can find both opportunity and problem in that overlap. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] Last gasp

2012-04-08 Thread Greg Smith
late. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql

Re: [HACKERS] Last gasp

2012-04-06 Thread Greg Smith
in April isn't is this an important feature? anymore. It should be is this such an important feature that it's worth delaying the release and taking a risk on recently churning code for? Let's not kill the messenger who delivers that reminder just because the news is painful. -- Greg Smith

Re: [HACKERS] Last gasp

2012-04-06 Thread Greg Smith
. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] checkpoint patches

2012-04-06 Thread Greg Smith
, it will trash VACUUM write performance in the process. That's one of the reasons I submitted the MB/s logging to VACUUM for 9.2, to make it easier to measure what happens to that as write cache changes are made. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL

Re: [HACKERS] Switching to Homebrew as recommended Mac install?

2012-04-03 Thread Greg Smith
a critical mass convinced. --Documentation Homebrew will have to become more complicated if it's going to try and wander down this path. With complexity and backward compatibility come increased needs for documentation. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD

Re: [HACKERS] performance-test farm

2012-04-03 Thread Greg Smith
where things like work_mem matter, the first step there is to pick a completely different benchmark workload. You're not going to do it with simple pgbench calls. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www

Re: [HACKERS] checkpoint patches

2012-04-03 Thread Greg Smith
submitted patch and seeing what happened. I threw mine out in hopes that some larger checkpoint patch shoot-out might find it useful. Didn't happen, sorry I didn't get to looking more at the other horses. I do have some more neat benchmarks to share though -- Greg Smith 2ndQuadrant USg

Re: [HACKERS] Hot Standby Failover Scenario

2012-02-28 Thread Greg Smith
this should be asked on either the pgsql-admin or pgsql-general mailing list. I'm not going to answer additional questions like this from you here on this list, and I doubt anyone else will either. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training

Re: [HACKERS] pgstat documentation tables

2012-02-27 Thread Greg Smith
always assumed that the reason this didn't work, but the Fedoras did, was because of a difference between 1.3 and 1.4. Maybe a forward port of that 1.3 patch would help. I'm not sure what state RHEL6 is really in though, I don't fully trust my SL6 system yet for this task. -- Greg Smith

Re: [HACKERS] Initial 9.2 pgbench write results

2012-02-27 Thread Greg Smith
buffers out at the latest possible moment works significantly better in some cases here. But that fact isn't new to 9.2; it's just has a slightly higher potential to get in the way, now that the writing happens during the sync phase. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com

Re: [HACKERS] pgstat documentation tables

2012-02-27 Thread Greg Smith
. And this one also works fine on the openjade 1.3 based SL6 server here, just need a different path for the library files: $ openjade -t tex -d /usr/share/sgml/docbook/utils-0.6.14/docbook-utils.dsl#print /usr/share/sgml/xml.dcl test2.docbook -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com

Re: [HACKERS] swapcache-style cache?

2012-02-27 Thread Greg Smith
it could be done perfectly fine by the OS, without any database knowledge of what is going on. Your results seem to validate that idea. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers

Re: [HACKERS] Initial 9.2 pgbench write results

2012-02-23 Thread Greg Smith
the competing I/O from backends mixed in. Note that the clients-sets graph still shows a strong jump from 9.0 to 9.1 at high client counts; I'm pretty sure that's the fsync compaction at work. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services

Re: [HACKERS] foreign key locks, 2nd attempt

2012-02-23 Thread Greg Smith
catalog conversion for in-place upgrade. That luck won't hold forever. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] foreign key locks, 2nd attempt

2012-02-23 Thread Greg Smith
On 02/23/2012 01:04 PM, Alvaro Herrera wrote: manual vacuum is teh sux0r I think you've just named my next conference talk submission. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] Initial 9.2 pgbench write results

2012-02-23 Thread Greg Smith
grabbing, I'd think that its individual impact would be a small slice of the total activity. I will of course reserve arguing that point until I've benchmarked to support it though. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7

Re: [HACKERS] swapcache-style cache?

2012-02-23 Thread Greg Smith
tier of read cache, not as fast as RAM but larger and faster than real disk seeks. In all of those potential win cases, though, I don't see why the OS couldn't just manage the whole thing for us. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training

[HACKERS] pg_test_timing tool for EXPLAIN ANALYZE overhead

2012-02-22 Thread Greg Smith
://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf VMware timer suggestions for various Linux versions: http://kb.vmware.com/selfservice/microsites/search.do?language=en_UScmd=displayKCexternalId=1006427 -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL

Re: [HACKERS] pg_test_timing tool for EXPLAIN ANALYZE overhead

2012-02-22 Thread Greg Smith
stating explicitly though. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [HACKERS] WIP: URI connection string support for libpq

2012-02-22 Thread Greg Smith
what I'd like to do with this next, regardless of that. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] pg_test_timing tool for EXPLAIN ANALYZE overhead

2012-02-22 Thread Greg Smith
finding this on their own. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [HACKERS] Displaying accumulated autovacuum cost

2012-02-21 Thread Greg Smith
I think. This one is pretty deep and already far off its titled topic. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] Displaying accumulated autovacuum cost

2012-02-21 Thread Greg Smith
into shared_buffers in particular are likely to see a deceptively small write count. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] Initial 9.2 pgbench write results

2012-02-19 Thread Greg Smith
something documenting this possibility before 9.2 is officially released. I'm a lot less stressed that there's really a problem here now. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql

Re: [HACKERS] Initial 9.2 pgbench write results

2012-02-19 Thread Greg Smith
just on that test. I think that's more than pgbench testing deserves though. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] wal_buffers

2012-02-19 Thread Greg Smith
as other bottlenecks are smashed too. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] Future of our regular expression code

2012-02-19 Thread Greg Smith
.html -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql

[HACKERS] Initial 9.2 pgbench write results

2012-02-14 Thread Greg Smith
is another 26%. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com ,,9.0,9.0,9.0,9.0,9.1,9.1,9.1,9.1,9.2,9.2,9.2,9.2,9.0 to 9.1,9.1 to 9.2,9.0 to 9.2 scale,clients,tps,avg_latency,90%,max_latency,tps,avg_latency,90

Re: [HACKERS] Initial 9.2 pgbench write results

2012-02-14 Thread Greg Smith
On 02/14/2012 01:45 PM, Greg Smith wrote: scale=1000, db is 94% of RAM; clients=4 Version TPS 9.0 535 9.1 491 (-8.4% relative to 9.0) 9.2 338 (-31.2% relative to 9.1) A second pass through this data noted that the maximum number of buffers cleaned by the background writer is =2785 in 9.0

Re: [HACKERS] random_page_cost vs seq_page_cost

2012-02-12 Thread Greg Smith
. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Review of pg_archivecleanup -x option patch

2012-02-12 Thread Greg Smith
. I've certainly made the mistake you've shown when using the patched version of the program myself, just didn't think about catching and correcting it myself before. We can rev this to add that feature and resubmit easily enough, will turn that around soon. -- Greg Smith 2ndQuadrant USg

Re: [HACKERS] CUDA Sorting

2012-02-12 Thread Greg Smith
of race, which happened (IMHO) only because Intel's Itanium failed to prioritize backwards compatibility highly enough. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing

Re: [HACKERS] Vacuum rate limit in KBps

2012-02-08 Thread Greg Smith
, to be more likely to gain community buy-in as an automated improvement. That's a longer term project though, which I'll hopefully be able to revisit for 9.3. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com

Re: [HACKERS] double writes using double-write buffer approach [WIP]

2012-02-07 Thread Greg Smith
that would want the double write buffer to move as well. And if it's inside pg_xlog, existing base backup scripts won't need to be changed--the correct ones already exclude pg_xlog files. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7

Re: [HACKERS] some longer, larger pgbench tests with various performance-related patches

2012-02-07 Thread Greg Smith
seconds of any write-heavy test always have an inflated speed. You're writing into the OS cache at maximum speed, and none of those writes are making it to physical disk--except perhaps for the WAL, which is all fast and sequential. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com

Re: [HACKERS] Checkpoint sync pause

2012-02-07 Thread Greg Smith
of concurrent processes as another input, and the complexity of the resulting model is high. -- Greg Smith 2ndQuadrant usg...@2ndquadrant.comBaltimore, MD PostgreSQL Training, Services, and 24x7 Supportwww.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

Re: [HACKERS] random_page_cost vs seq_page_cost

2012-02-07 Thread Greg Smith
to a first-time reader of this section. I stopped just short of recommending a value for the completely cached case. I normally use 1.01 there; I know others prefer going fully to 1.0 instead. That argument seems like it could rage on for some time. -- Greg Smith 2ndQuadrant USg

Re: [HACKERS] basic pgbench runs with various performance-related patches

2012-02-04 Thread Greg Smith
, interleaved, or b) rebuild the filesystem in between each initdb. I'm not sure that's the problem you're running into, but it's the only one I've been hit by that matches the suspicious part of your results. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training

Re: [HACKERS] Group commit, revised

2012-01-29 Thread Greg Smith
in the process though. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref

Re: [HACKERS] Vacuum rate limit in KBps

2012-01-25 Thread Greg Smith
will ultimately fit together. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] Publish checkpoint timing and sync files summary data to pg_stat_bgwriter

2012-01-23 Thread Greg Smith
overall review work pile) then. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] Vacuum rate limit in KBps

2012-01-23 Thread Greg Smith
that happened 10X as often as it does now. You'll waste a moderate amount of CPU and disk resources, but a tuning error that leans toward analyzing too frequently isn't that expensive. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7

Re: [HACKERS] Publish checkpoint timing and sync files summary data to pg_stat_bgwriter

2012-01-22 Thread Greg Smith
Jeff Janes wrote: I'm finding the backend_writes column pretty unfortunate. The only use I know of for it is to determine if the bgwriter is lagging behind. Yet it doesn't serve even this purpose because it lumps together the backend writes due to lagging background writes, and the backend

Re: [HACKERS] Vacuum rate limit in KBps

2012-01-22 Thread Greg Smith
Jim Nasby wrote: Your two comments together made me realize something... at the end of the day people don't care about MB/s. They care about impact to other read and write activity in the database. What would be interesting is if we could monitor how long all *foreground* IO requests took.

Re: [HACKERS] Vacuum rate limit in KBps

2012-01-20 Thread Greg Smith
fundamentally broken, by being too difficult to use, to most people in its current form. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] Vacuum rate limit in KBps

2012-01-19 Thread Greg Smith
see any way to wander down this path that doesn't end up introducing multiple new GUCs, which is the opposite of what I'd hoped to do--which was at worst to keep the same number, but reduce how many were likely to be touched. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore

Re: [HACKERS] Vacuum rate limit in KBps

2012-01-19 Thread Greg Smith
vacuum easily is the top limiting factor on overall performance. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Vacuum rate limit in KBps

2012-01-19 Thread Greg Smith
that vacuum page misses introduce will not be exactly the same as vacuum_cost_read_limit--but it will be below that limit, which is all it claims to be. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent

Re: [HACKERS] Publish checkpoint timing and sync files summary data to pg_stat_bgwriter

2012-01-19 Thread Greg Smith
the pg_stat_bgwriter numbers. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] Patch review for logging hooks (CF 2012-01)

2012-01-19 Thread Greg Smith
that the hook exists and what the ground rules are for grabbing it. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] Vacuum rate limit in KBps

2012-01-18 Thread Greg Smith
with a way to make that work as it does now in the new code, that's a problem. I don't think it really is, it's just that people in that situation will need to all three upwards. It's still a simpler thing to work out than the current situation, and this is an unusual edge case. -- Greg

Re: [HACKERS] WIP patch for parameterized inner paths

2012-01-17 Thread Greg Smith
on a fast track unrelated to the normal schedule. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-01-17 Thread Greg Smith
. But this existing one is already bad enough to justify shipping something to help measure/manage it in my mind. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com diff --git a/contrib/Makefile b/contrib/Makefile index

Re: [HACKERS] automating CF submissions (was xlog location arithmetic)

2012-01-17 Thread Greg Smith
On 01/17/2012 11:50 AM, Peter Eisentraut wrote: On mån, 2012-01-16 at 17:25 -0500, Greg Smith wrote: The most reasonable answer to this is for people to publish a git repo URL in addition to the official submission of changes to the list in patch form. Note that the original complaint

Re: [HACKERS] [WIP] Double-write with Fast Checksums

2012-01-17 Thread Greg Smith
possible they have that data but just didn't post just to keep the summary size managable, since dbt-2 collects a lot of information. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql

Re: [HACKERS] logging in high performance systems.

2012-01-16 Thread Greg Smith
, hopefully later this week. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org

[HACKERS] CommitFest 2012-01 kick-off

2012-01-16 Thread Greg Smith
features have certainly been stacking on each other usefully so far. So many performance improvements that we may not be able to absorb them all at once...there are worse problems to have. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services

Re: [HACKERS] automating CF submissions (was xlog location arithmetic)

2012-01-16 Thread Greg Smith
pretty happy with how well that's going. We don't need to add infrastructure to enable people to push code to github and link to their branch comparison repo viewer as a way to view the patch; that's already available to anyone who wants is. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com

Re: [HACKERS] Review of: pg_stat_statements with query tree normalization

2012-01-16 Thread Greg Smith
of code changes, but not a lot of code complexity. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription

Re: [HACKERS] Why is CF 2011-11 still listed as In Progress?

2012-01-16 Thread Greg Smith
tinkered with shouldn't delay a date-driven alpha drop. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Checkpoint sync pause

2012-01-16 Thread Greg Smith
relations to sync and you have to wait 10 seconds between syncs to get latency down, the server is going to inform you an hour between checkpoints is all you can do here. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www

Re: [HACKERS] Why is CF 2011-11 still listed as In Progress?

2012-01-16 Thread Greg Smith
seen enough feedback on to acknowledge and accept as impractical. If the flood of last minute submissions right before the freeze submission deadline takes 6 weeks to clear now, that still seems a whole lot better than what I remember of 8.3 and 8.4 development. -- Greg Smith 2ndQuadrant US

[HACKERS] Vacuum rate limit in KBps

2012-01-15 Thread Greg Smith
upgrading all old pages before moving forward. While this could be an interesting small test case for that sort of thing, I'd rather not be patient #1 for that part of the long-term in-place upgrade path right now. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL

Re: [HACKERS] automating CF submissions (was xlog location arithmetic)

2012-01-15 Thread Greg Smith
. Works fine out of the box, you just won't have things like all the PLs installed. Yes, I am aware I'm going at this bottom-up. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers

Re: [HACKERS] IDLE in transaction introspection

2012-01-15 Thread Greg Smith
that removes a bunch of docs the previous patch added. I have to be careful to always do something like git diff origin/master to avoid this class of problem, until I got into that habit I did this sort of thing regularly. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD

Re: [HACKERS] pgstat documentation tables

2012-01-15 Thread Greg Smith
, then adopting the same style of reorg to the rest until they're all converted. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

Re: [HACKERS] pgstat documentation tables

2012-01-15 Thread Greg Smith
be willing to work on doing that in a way that improves what is documented, too. The difficulty of working with the existing tables has been the deterrent for improving that section to me. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services

Re: [HACKERS] Vacuum rate limit in KBps

2012-01-15 Thread Greg Smith
details about issues like this is something I think PostgreSQL needs to accept as part of the release cycle. Expecting that pg_upgrade can transparently handle every possible change would be setting a high bar to clear, higher than I think is expected by the database industry at large. -- Greg

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-01-15 Thread Greg Smith
. pg_stat_statements is easy enough to continue work on outside of core. I could see a path where that happens in parallel with adding a better core wait event infrastructure, just to get the initial buffer wait info into people's hands earlier. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com

[HACKERS] Publish checkpoint timing and sync files summary data to pg_stat_bgwriter

2012-01-15 Thread Greg Smith
. That all works against this being stable. The main consumer of this data I use regularly is the pg_stat_bgwriter graphs that Munin produces, and my expectation is that myself and/or Magnus will make sure it's compatible with any changes made here for 9.2. -- Greg Smith 2ndQuadrant USg

Re: [HACKERS] Publish checkpoint timing and sync files summary data to pg_stat_bgwriter

2012-01-15 Thread Greg Smith
On 01/16/2012 01:28 AM, Greg Smith wrote: -I can't tell for sure if this is working properly when log_checkpoints is off. This now collects checkpoint end time data in all cases, whereas before it ignored that work if log_checkpoints was off. ...and there's at least one I missed located

<    1   2   3   4   5   6   7   8   9   10   >