Re: [PERFORM] Millions of tables

2016-10-05 Thread Greg Spiegelberg
here; autovac treats toast tables as just another table, with their > own stats and their own freeze needs. If you're generating a lot of toast > records that might make a difference. > I do not anticipate TOAST entering the picture. No single column or record > 8KB or even approaching it. We have a few databases that (ab)use pg_toast and I want to avoid those complications. -Greg

Re: [PERFORM] Millions of tables

2016-09-28 Thread Greg Spiegelberg
On Wed, Sep 28, 2016 at 11:27 AM, Stephen Frost <sfr...@snowman.net> wrote: > Greg, > > * Greg Spiegelberg (gspiegelb...@gmail.com) wrote: > > Bigger buckets mean a wider possibility of response times. Some buckets > > may contain 140k records and some 100X more. >

Re: [PERFORM] Millions of tables

2016-09-28 Thread Greg Spiegelberg
dy allows for many database servers. 40 is okay, 100 isn't terrible but if it's thousands then operations might lynch me. -Greg

Re: [PERFORM] Millions of tables

2016-09-27 Thread Greg Spiegelberg
On Tue, Sep 27, 2016 at 10:15 AM, Terry Schmitt <tschm...@schmittworks.com> wrote: > > > On Sun, Sep 25, 2016 at 7:50 PM, Greg Spiegelberg <gspiegelb...@gmail.com> > wrote: > >> Hey all, >> >> Obviously everyone who's been in PostgreSQL or almost

Re: [PERFORM] Millions of tables

2016-09-27 Thread Greg Spiegelberg
On Tue, Sep 27, 2016 at 9:42 AM, Mike Sofen <mso...@runbox.com> wrote: > *From:* Mike Sofen *Sent:* Tuesday, September 27, 2016 8:10 AM > > *From:* Greg Spiegelberg *Sent:* Monday, September 26, 2016 7:25 AM > I've gotten more responses than anticipated and have answere

Re: [PERFORM] Millions of tables

2016-09-27 Thread Greg Spiegelberg
On Tue, Sep 27, 2016 at 8:30 AM, Craig James <cja...@emolecules.com> wrote: > On Sun, Sep 25, 2016 at 7:50 PM, Greg Spiegelberg <gspiegelb...@gmail.com> > wrote: > >> Hey all, >> >> Obviously everyone who's been in PostgreSQL or almost any RDBMS for

Re: [PERFORM] Millions of tables

2016-09-26 Thread Greg Spiegelberg
On Sun, Sep 25, 2016 at 8:50 PM, Greg Spiegelberg <gspiegelb...@gmail.com> wrote: > Hey all, > > Obviously everyone who's been in PostgreSQL or almost any RDBMS for a time > has said not to have millions of tables. I too have long believed it until > recently. > &

Re: [PERFORM] Millions of tables

2016-09-26 Thread Greg Spiegelberg
10M rows in a table is not a problem for the query > times you are referring to. So instead of millions of tables, unless I'm > doing my math wrong, you probably only need thousands of tables. > > > > On Mon, Sep 26, 2016 at 5:43 AM, Stuart Bishop <stu...@stuartbishop.net> &

Re: [PERFORM] Millions of tables

2016-09-26 Thread Greg Spiegelberg
larger group and the option to reduce the 8M tables to ~4000 is an option however the problem then becomes rather than having an anticipated 140k records/table to 140M to 500M records/table. I'm concerned read access times will go out the window. It is on the docket to test. -Greg

Re: [PERFORM] Millions of tables

2016-09-26 Thread Greg Spiegelberg
options. :) -Greg On Mon, Sep 26, 2016 at 6:54 AM, Yves Dorfsman <y...@zioup.com> wrote: > Something that is not talked about at all in this thread is caching. A > bunch > of memcache servers in front of the DB should be able to help with the 30ms > constraint (doesn't have to

Re: [PERFORM] Millions of tables

2016-09-26 Thread Greg Spiegelberg
On Mon, Sep 26, 2016 at 3:43 AM, Stuart Bishop <stu...@stuartbishop.net> wrote: > On 26 September 2016 at 11:19, Greg Spiegelberg <gspiegelb...@gmail.com> > wrote: > >> I did look at PostgresXL and CitusDB. Both are admirable however neither >> could support

Re: [PERFORM] Millions of tables

2016-09-26 Thread Greg Spiegelberg
Following list etiquette response inline ;) On Mon, Sep 26, 2016 at 2:28 AM, Álvaro Hernández Tortosa <a...@8kdata.com> wrote: > > > On 26/09/16 05:50, Greg Spiegelberg wrote: > >> Hey all, >> >> Obviously everyone who's been in PostgreSQL or almost any RDBM

Re: [PERFORM] Millions of tables

2016-09-25 Thread Greg Spiegelberg
lake you must accept the possibility of scanning the entire lake. However, if all fish were in barrels where each barrel had a particular kind of fish of specific length, size, color then the problem is far simpler. -Greg On Sun, Sep 25, 2016 at 9:04 PM, julyanto SUTANDANG <julya...@equni

Re: [PERFORM] Millions of tables

2016-09-25 Thread Greg Spiegelberg
chema and it's intended use is complete. You'll have to trust me on that one. -Greg On Sun, Sep 25, 2016 at 9:23 PM, Mike Sofen <mso...@runbox.com> wrote: > *From:* Greg Spiegelberg *Sent:* Sunday, September 25, 2016 7:50 PM > … Over the weekend, I created 8M tables with 1

[PERFORM] Millions of tables

2016-09-25 Thread Greg Spiegelberg
inheritance (I am abusing it in my test case) are no-no's. A system or database crash could take potentially hours to days to recover. There are likely other issues ahead. You may wonder, "why is Greg attempting such a thing?" I looked at DynamoDB, BigTable, and Cassandra. I like Greenplum

Re: [PERFORM] pgbouncer issue

2015-07-05 Thread Greg Sabino Mullane
and nobody else has complained, it's probably not too important as far as day to day pgbouncer use. :) - -- Greg Sabino Mullane g...@turnstep.com End Point Corporation http://www.endpoint.com/ PGP Key: 0x14964AC8 201507051040 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8

Re: [PERFORM] Yet another abort-early plan disaster on 9.3

2014-10-19 Thread Greg Stark
that algorithm is needed at all. -- greg -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] Yet another abort-early plan disaster on 9.3

2014-10-17 Thread Greg Stark
filter then do a second pass (possibly for the same sample?) keeping counts only for values that the counting bloom filter said hashed to the most common hash values. That might not be exactly the most common values but should be at least a representative sample of the most common values. -- greg

Re: [PERFORM] Yet another abort-early plan disaster on 9.3

2014-10-10 Thread Greg Stark
the entire table -- and it still had pretty poor results. All the research I could find went into how to analyze the whole table while using a reasonable amount of scratch space and how to do it incrementally. -- greg -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make

Re: [PERFORM] Yet another abort-early plan disaster on 9.3

2014-09-29 Thread Greg Stark
surprisingly far before they run into problems. That may not be the best thing for users in the long run but that's a problem that should be solved by better development tools to help users identify scalability problems early. -- greg -- Sent via pgsql-performance mailing list (pgsql-performance

Re: [PERFORM] Yet another abort-early plan disaster on 9.3

2014-09-20 Thread Greg Stark
On 19 Sep 2014 19:40, Josh Berkus j...@agliodbs.com wrote: On 09/19/2014 10:15 AM, Merlin Moncure wrote: On Wed, Sep 17, 2014 at 7:11 PM, Josh Berkus j...@agliodbs.com wrote: This is the core issue with abort-early plans; they depend on our statistics being extremely accurate, which we

Re: [PERFORM] Poor OFFSET performance in PostgreSQL 9.1.6

2013-08-28 Thread Greg Spiegelberg
Two solutions come to mind. First possibility is table partitioning on the column you're sorting. Second, depending on your application, is to use a cursor. Cursor won't help with web applications however a stateful application could benefit. HTH -Greg On Wed, Aug 28, 2013 at 2:39 PM, fburg

Re: [PERFORM] Fw: [osdldbt-general] Running DBT5 on remote database server

2013-07-31 Thread Greg Smith
. You will also need to setup the postgresql.conf and pg_hba.conf on the system to allow remote connections, the same way as this is normally done with Postgres. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www

[PERFORM] Re: bgwriter autotuning might be unnecessarily penalizing bursty workloads

2013-07-17 Thread Greg Smith
minutes: -[ RECORD 1 ]---+--- alloc_mbps | 246.019686474412 checkpoint_mbps | 0.0621780475463596 clean_mbps | 2.38631188442859 backend_mbps| 0.777490109599045 write_mbps | 3.22598004157399 -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD

Re: [PERFORM] PostgreSQL settings for running on an SSD drive

2013-07-17 Thread Greg Smith
On 6/20/13 4:32 PM, Josh Berkus wrote: First, cc'ing Greg Smith to see if he can address this with the Fusion folks so that they stop giving out a bad guide. I'm working on a completely replacement of that guide, one that actually gives out a full set of advice. Right now I'm between

[PERFORM] Dynamic queries in stored procedure

2013-07-05 Thread Greg Jaskiewicz
Hey, We have a search method that depending on search params will join 3-5 tables, craft the joins and where section. Only problem is, this is done in rather horrible java code. So using pgtap for tests is not feasible. I want to move the database complexity back to database, almost writing the

Re: [PERFORM] Check Pointer

2013-05-30 Thread Greg Smith
you're seeing there. It doesn't actually use any significant amount of memory on its own. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-performance mailing list (pgsql-performance

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

2013-05-24 Thread Greg Smith
, I'll be kicking off a brand new round of SSD tests on a 24 core server here soon. All those will appear on my blog. The 320 drive is returning as the bang for buck champ, along with a DC S3700 and a Seagate 1TB Hybrid drive with NAND durable write cache. -- Greg Smith 2ndQuadrant USg

Re: [PERFORM] pgbench: spike in pgbench results(graphs) while testing pg_hint_plan performance

2013-05-24 Thread Greg Smith
. -Does this happen on every test run? Is it at the same time? -You can run top -bc to dump snapshots of what the system is doing every second. With some work you can then figure out what was actually happening during the two seconds around when the throughput dropped. -- Greg Smith

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

2013-05-22 Thread Greg Smith
will look like that is optimistic, and it sets unreasonable expectations. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

2013-05-22 Thread Greg Smith
data before I feel comfortable saying exactly what the worst case looks like. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

2013-05-22 Thread Greg Smith
is lucky to hit 10K TPS though, so it can't compete against what a PCI-E card like the FusionIO drives are capable of. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-performance mailing

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

2013-05-22 Thread Greg Smith
On 5/22/13 3:06 PM, Joshua D. Drake wrote: Greg, can you elaborate on the SSD + Xlog issue? What type of burn through are we talking about? You're burning through flash cells at a multiple of the total WAL write volume. The system I gave iostat snapshots from upthread (with the Intel 710

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

2013-05-22 Thread Greg Smith
a 710 since the announcement. However, hit the street is still an issue. No one has been able to keep DC S3700 drives in stock very well yet. It took me three tries through Newegg before my S3700 drive actually shipped. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

2013-05-22 Thread Greg Smith
. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

2013-05-22 Thread Greg Smith
. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

2013-05-22 Thread Greg Smith
written, either. Several of the SMART attributes are labeled Vendor-specific, but you'll need to guess what they track and read the associated values using third-party software. That's a serious problem for most business use of this sort of drive. -- Greg Smith 2ndQuadrant USg

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

2013-05-21 Thread Greg Smith
S3700 was $250. That's still not two orders of magnitude faster though. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org

Re: [PERFORM] Deleting Rows From Large Tables

2013-05-19 Thread Greg Spiegelberg
; COMMIT; HTH. -Greg On Fri, May 17, 2013 at 5:26 AM, Rob Emery re-pg...@codeweavers.net wrote: Hi All, We've got 3 quite large tables that due to an unexpected surge in usage (!) have grown to about 10GB each, with 72, 32 and 31 million rows in. I've been tasked with cleaning out about

Re: [PERFORM] Hardware suggestions for maximum read performance

2013-05-19 Thread Greg Smith
. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql

Re: [PERFORM] Deleting Rows From Large Tables

2013-05-19 Thread Greg Smith
on in the background. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

2013-05-19 Thread Greg Smith
to disk. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

2013-05-19 Thread Greg Smith
the last base backup happened. That can easily result in a week of downtime if you're only shipping backups once per month, for example. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql

Re: [PERFORM] slow joins?

2013-04-05 Thread Greg Williamson
and on line_items ? What are the stats settings for these tables ? HTH, Greg WIlliamson -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] slow joins?

2013-04-05 Thread Greg Williamson
Joe -- From: Joe Van Dyk j...@tanga.com To: Greg Williamson gwilliamso...@yahoo.com Cc: pgsql-performance@postgresql.org pgsql-performance@postgresql.org Sent: Friday, April 5, 2013 7:56 PM Subject: Re: [PERFORM] slow joins? On Fri, Apr 5, 2013 at 6:54 PM

Re: [PERFORM] New server setup

2013-03-13 Thread Greg Jaskiewicz
On 13 Mar 2013, at 15:33, John Lister john.lis...@kickstone.com wrote: On 12/03/2013 21:41, Gregg Jaskiewicz wrote: Whilst on the hardware subject, someone mentioned throwing ssd into the mix. I.e. combining spinning HDs with SSD, apparently some raid cards can use small-ish (80GB+) SSDs

Re: [PERFORM] sniff test on some PG 8.4 numbers

2013-03-10 Thread Greg Smith
8.4 only has a little over a year before it won't get bug fixes anymore. Also, your server would really appreciate the performance gains added to 9.2. If that's a bit too leading edge for you, I don't recommend deploying at version below 9.1 anymore. -- Greg Smith 2ndQuadrant USg

Re: [PERFORM] New server setup

2013-03-10 Thread Greg Smith
be, 2ndQuadrant does offer a hardware benchmarking service to do that sort of thing: http://www.2ndquadrant.com/en/hardware-benchmarking/ I think we're even generating those reports in German now. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training

Re: [PERFORM] sniff test on some PG 8.4 numbers

2013-03-10 Thread Greg Smith
what they thought they were buying. The potential downside of HT isn't so big that its worth opening that can of worms, unless you've run real application level tests to prove it hurts. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7

Re: [PERFORM] pgbench intriguing results: better tps figures for larger scale factor

2013-03-04 Thread Greg Smith
there really is no reason to consider running pgbench on a system with a smaller scale than that. I normally get a rough idea of things by running with scales 100, 250, 500, 1000, 2000. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7

Re: [PERFORM] How to keep queries low latency as concurrency increases

2012-10-30 Thread Greg Williamson
queries a second to the readonly boxes, about the same to a beefier read / write master. This is a slightly old pgbouncer at that ... used is a fairly basic mode. Greg Williamson -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription

Re: [PERFORM] Inserts in 'big' table slowing down the database

2012-10-01 Thread Greg Williamson
that are bigger than 1 hour -- too many partitions doesn't help. Greg Williamson -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] [repost] Help me develop new commit_delay advice

2012-09-05 Thread Greg Smith
of the siblings was one of the challenges I kept butting into then. Making the GUC settings even more complicated for this doesn't seem a productive step forward for the average user. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7

Re: [PERFORM] exponential performance decrease in ISD transaction

2012-09-05 Thread Greg Smith
runs will give you an idea if write cache filling is actually an issue here. If that number just keeps going up and speeds keep on dropping, that's at least one cause here. This could easily be both that and an autovacuum related too though. -- Greg Smith 2ndQuadrant USg

Re: [PERFORM] Index Bloat Problem

2012-08-18 Thread Greg Williamson
Thanks for this description--we have index bloat problems on a massively active (but small) database.This may help shed light on our problems. Sorry for top-posting--challenged email reader. Greg W. From: Jeff Janes jeff.ja...@gmail.com To: Strahinja

Re: [PERFORM] slow query, different plans

2012-08-03 Thread Greg Williamson
Midge -- Sorry for top-quoting -- challenged mail. Perhaps a difference in the stats estimates -- default_statistics_target ? Can you show us a diff between the postgres config files for each instance ? Maybe something there ... Greg Williamson From: Midge

Re: [PERFORM] ZFS vs. UFS

2012-07-26 Thread Greg Smith
? to protect databases, and the standby server for that doesn't need to be an expensive system. That said, there is no reason to set things up so that they only work with that Intel RAID controller, given that it's not a very good piece of hardware anyway. -- Greg Smith 2ndQuadrant USg

[PERFORM] Linux memory zone reclaim

2012-07-17 Thread Greg Smith
, where the memory it needs is at, and whether the server wants to reclaim memory (and just what that means its own complicated topic) as part of that. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent

Re: [PERFORM] Paged Query

2012-07-09 Thread Greg Spiegelberg
recommended. -Greg

Re: [PERFORM] Paged Query

2012-07-06 Thread Greg Spiegelberg
page count. Luck. -Greg

Re: [PERFORM] Introducing the TPC-V benchmark, and its relationship to PostgreSQL

2012-07-05 Thread Greg Smith
in. Anyway, guessing at causes here is premature speculation. When there's some code for the test kit published, at that point discussing the particulars of why it's not running well will get interesting. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services

Re: [PERFORM] The need for clustered indexes to boost TPC-V performance

2012-07-05 Thread Greg Smith
? and how can we get them sponsored to focus on it? I can tell from your comments yet what role(s) in that process VMWare wants to take on internally, and which it's looking for help with. The job of convincing people it's a useful feature isn't necessary--we know that's true. -- Greg Smith

Re: [PERFORM] MemSQL the world's fastest database?

2012-07-05 Thread Greg Smith
, their TPS numbers are useless without a contest of how big each transaction is, and we don't know. I can take MemSQL seriously when there's a press release describing how to replicate their benchmark independently. Then it's useful to look at the absolute number. -- Greg Smith 2ndQuadrant USg

Re: [PERFORM] MemSQL the world's fastest database?

2012-06-30 Thread Greg Smith
. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql

Re: [PERFORM] SSD, Postgres and safe write cache

2012-06-30 Thread Greg Smith
even matter. Or that the bottleneck is somewhere else entirely. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes

Re: [PERFORM] SSD selection

2012-05-28 Thread Greg Smith
that are done more efficiently in terms of flash longevity (710). You can't get both at the same time. The 710 may ultimately throttle its speed back to meet lifetime specifications as the drive fills, it's really hard to benchmark the differences between the two series. -- Greg Smith

[PERFORM] Millions of relations (from Maximum number of sequences that can be created)

2012-05-25 Thread Greg Spiegelberg
On Fri, May 25, 2012 at 9:04 AM, Craig James cja...@emolecules.com wrote: On Fri, May 25, 2012 at 4:58 AM, Greg Spiegelberg gspiegelb...@gmail.com wrote: On Sun, May 13, 2012 at 10:01 AM, Craig James cja...@emolecules.com wrote: On Sun, May 13, 2012 at 1:12 AM, Віталій Тимчишин tiv

Re: [PERFORM] Configuration Recommendations

2012-05-15 Thread Greg Sabino Mullane
fsync = off ;) - -- Greg Sabino Mullane g...@turnstep.com End Point Corporation http://www.endpoint.com/ PGP Key: 0x14964AC8 201205151351 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -BEGIN PGP SIGNATURE- iEYEAREDAAYFAk+yl8YACgkQvJuQZxSWSshB+QCghfweMspFIqmP4rLv6

Re: [PERFORM] Parallel Scaling of a pgplsql problem

2012-04-26 Thread Greg Spiegelberg
can prepare the data ahead of time as it changes via a trigger or client-side code then your problem will go away pretty quickly. -Greg

Re: [PERFORM] Configuration Recommendations

2012-04-25 Thread Greg Sabino Mullane
, separating pg_xlog, etc. - -- Greg Sabino Mullane g...@turnstep.com End Point Corporation http://www.endpoint.com/ PGP Key: 0x14964AC8 201204251304 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -BEGIN PGP SIGNATURE- iEYEAREDAAYFAk

Re: [PERFORM] Tablespaces on a raid configuration

2012-03-30 Thread Greg Spiegelberg
for the indexes for the OLTP but we ran out of drives to create a RAID Group D and the above configuration works well enough. Before going with RAID 5, please review http://www.baarf.com/. -Greg

Re: [PERFORM] Very long deletion time on a 200 GB database

2012-02-23 Thread Greg Spiegelberg
but with the LOCK it should be safe. This transaction took perhaps 30 minutes and removed 100k rows and once the table was VACUUM'd afterward it freed up close to 20 GB on the file system. HTH -Greg

Re: [PERFORM] Very long deletion time on a 200 GB database

2012-02-23 Thread Greg Spiegelberg
On Thu, Feb 23, 2012 at 11:11 AM, Andy Colson a...@squeakycode.net wrote: On 2/23/2012 12:05 PM, Shaun Thomas wrote: On 02/23/2012 11:56 AM, Greg Spiegelberg wrote: I know there are perils in using ctid but with the LOCK it should be safe. This transaction took perhaps 30 minutes

Re: [PERFORM] random_page_cost = 2.0 on Heroku Postgres

2012-02-08 Thread Greg Sabino Mullane
decrement rpc slowly and find out at one points it changes, which would be more interesting than testing arbitrary numbers. Would lead to some really sweet graphs as well. :) - -- Greg Sabino Mullane g...@turnstep.com End Point Corporation http://www.endpoint.com/ PGP Key: 0x14964AC8 201202082338 http

Re: [PERFORM] how to demonstrate the effect of direct I/O ?

2012-02-07 Thread Greg Smith
acceleration here and b) on Solaris. You won't find a compelling performance improvement listed at https://ext4.wiki.kernel.org/articles/c/l/a/Clarifying_Direct_IO%27s_Semantics_fd79.html and Linux has generally ignored direct I/O as something important to optimize for. -- Greg Smith 2ndQuadrant US

Re: [PERFORM] wal_level=archive gives better performance than minimal - why?

2012-01-16 Thread Greg Smith
archive To make sure the difference wasn't some variation on gets slower after each run. pgbench suffers a lot from problems in that class. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql

[PERFORM] Slow nested loop execution on larger server

2011-12-16 Thread Greg Smith
pg_locks for this 12634 shows all granted ones, nothing exciting there. I asked how well this executes with enable_nestloop turned off, hoping to see that next. This all seems odd, and I get interested and concerned when that start showing up specifically on newer hardware. -- Greg Smith

Re: [PERFORM] SSD endurance calculations

2011-11-23 Thread Greg Smith
several of them. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [PERFORM] probably cause (and fix) for floating-point assist faults on itanium

2011-11-19 Thread Greg Matthews
Looks good to me. I built PG with this change, no kernel warnings after ~10 minutes of running. I'll continue to monitor but I think this fixes the syndrome. Thanks Tom. -Greg On Fri, 18 Nov 2011, Tom Lane wrote: Claudio Freire klaussfre...@gmail.com writes: On Thu, Nov 17, 2011 at 10:07

Re: [PERFORM] Benchmarking tools, methods

2011-11-19 Thread Greg Smith
as you'd like though. http://www.2ndquadrant.com/en/talks/ has some updated material about things discovered since the book was published. The Bottom-Up Database Benchmarking there shows the tests I'm running nowadays, which have evolved a bit in the last year. -- Greg Smith 2ndQuadrant US

Re: [PERFORM] SSD options, small database, ZFS

2011-11-18 Thread Greg Smith
on that program's page is to help people navigate this whole maze, and have some data points to set expectations against. See https://github.com/gregs1104/stream-scaling for the code and the samples. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services

[PERFORM] probably cause (and fix) for floating-point assist faults on itanium

2011-11-17 Thread Greg Matthews
, and the extra floating point handling may affect some other process(es). -Greg -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] What's the state of postgresql on ext4 now?

2011-11-15 Thread Greg Smith
changes. In just about every other way but commit performance, ext4 is faster than most other filesystems. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us

Re: [PERFORM] Slow queries / commits, mis-configuration or hardware issues?

2011-11-15 Thread Greg Smith
of their servers if I try to make that problem bad, you're only seeing the middle range of latency issues so far. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-performance mailing list (pgsql

Re: [PERFORM] What's the state of postgresql on ext4 now?

2011-11-15 Thread Greg Smith
to see how it's doing soon, just haven't had time/requests for it. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make

Re: [PERFORM] WAL partition filling up after high WAL activity

2011-11-11 Thread Greg Smith
in the section that talks about parameters to configure; there really should be. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make

Re: [PERFORM] WAL partition filling up after high WAL activity

2011-11-09 Thread Greg Smith
as well set it to a large number, say 128, and let checkpoints get driven by time instead. The existing limit isn't working effectively anyway, and having more segments lets the checkpoint spreading code work more evenly. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD

Re: [PERFORM] does update of column with no relation imply a relation check of other column?

2011-10-19 Thread Greg Jaskiewicz
On 19 Oct 2011, at 17:51, Anibal David Acosta wrote: For example: Table A -id (PK) -name Table B -table_a_id (PK, FK) -address When I do an insert on table B, the database check if value for column “table_a_id” exists in table A But, if I do an update of column “address” of

Re: [PERFORM] Composite keys

2011-10-12 Thread Greg Smith
. There are plenty of times that the reason behind why isn't it using my index? is the index is too fat to navigate efficiently, because the actual number of blocks involved is factored into the cost computations. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training

Re: [PERFORM] Adding more memory = hugh cpu load

2011-10-11 Thread Greg Smith
with this unusual vmstat output. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http

Re: [PERFORM] Adding more memory = hugh cpu load

2011-10-11 Thread Greg Smith
enough that it's hard to innovate in this area within Postgres. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make

Re: [PERFORM] Adding more memory = hugh cpu load

2011-10-11 Thread Greg Smith
this earlier this year, and that was as much documentation as I could justify at the time. If there's a user-visible behavior changes here, that's the point where an update to the manual would be in order. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL

Re: [PERFORM] Adding more memory = hugh cpu load

2011-10-10 Thread Greg Smith
that in the position you're in now though. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http

Re: [PERFORM] : PG9.0 - Checkpoint tuning and pg_stat_bgwriter

2011-10-04 Thread Greg Smith
be useful. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref

Re: [PERFORM] : PG9.0 - Checkpoint tuning and pg_stat_bgwriter

2011-10-04 Thread Greg Smith
. The advantage to making them happen less often is that you get less total writes. People need to be careful about going a long *time* between checkpoints. But there's very few cases where you need to worry about the segment count going too high before another one is triggered. -- Greg

Re: [PERFORM] Suggestions for Intel 710 SSD test

2011-10-02 Thread Greg Smith
rough to use. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us attachment: array-vs-ssd.png -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription

Re: [PERFORM] PostgreSQL-9.0 Monitoring System to improve performance

2011-09-28 Thread Greg Smith
across someone yet who wants to fund that size of project for this purpose yet. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org

Re: Show_database_bloat reliability? [was: Re: [PERFORM] REINDEX not working for wastedspace]

2011-09-21 Thread Greg Smith
can be useful for determining if bloat is likely increasing or decreasing--which is the purpose of that query. The value returned is a rough estimate, and should not be considered useful as any sort of absolute measurement. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD

Re: [PERFORM] : Performance Improvement Strategy

2011-09-21 Thread Greg Smith
that. A multi-gigabyte table can easily be unavailable for several hours if you execute VACUUM FULL against it. CLUSTER is almost always faster. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent

Re: [PERFORM] Query optimization using order by and limit

2011-09-21 Thread Greg Smith
can't figure out if you're running into a basic error here, where constraint exclusion just isn't working at all, or if you are only having this problem because the query is too complicated. Figuring that out will narrow the potential solutions. -- Greg Smith 2ndQuadrant USg

  1   2   3   4   5   6   7   8   9   10   >