Re: [PERFORM] archive wal's failure and load increase.

2006-10-02 Thread Simon Riggs
On Fri, 2006-09-29 at 11:55 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > PreallocXlogFiles() adds only a *single* xlog file, sometimes. > > Hm, you are right. I wonder why it's so unaggressive ... perhaps > because under normal circumstances

Re: [PERFORM] commit so slow program looks frozen

2006-10-28 Thread Simon Riggs
d all of those things have potential secondary effects, all of which can also conflict with other user tasks and especially with a CHECKPOINT. Then there's various forms of contention caused by misconfiguration. I do think we need some better instrumentation for this kind of thing. -- Si

Re: [PERFORM] partitioned table performance

2006-10-30 Thread Simon Riggs
On Sun, 2006-10-29 at 00:28 +0200, Andreas Kostyrka wrote: > Any ideas how to make the optimizer handle partitioned tables more > sensible? Yes, those are known inefficiencies in the current implementation which we expect to address for 8.3. -- Simon Riggs EnterpriseDB

Re: [PERFORM] partitioned table performance

2006-10-30 Thread Simon Riggs
On Mon, 2006-10-30 at 22:58 +0100, Andreas Kostyrka wrote: > Am Montag, den 30.10.2006, 08:18 + schrieb Simon Riggs: > > On Sun, 2006-10-29 at 00:28 +0200, Andreas Kostyrka wrote: > > > > > Any ideas how to make the optimizer handle partitioned tables more > > &g

Re: [PERFORM] Database-wide vacuum can take a long time, duringwhich tables are not being analyzed

2006-11-02 Thread Simon Riggs
nterested to hear more about your system. -- Simon Riggs EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq

Re: [PERFORM] Help w/speeding up range queries?

2006-11-02 Thread Simon Riggs
GeneX is on > some chromosome from position 10->40) Do you know about www.biopostgres.org ? I believe they provide some additional indexing mechanisms for just this type of data. -- Simon Riggs EnterpriseDB http://www.enterprisedb.com --

Re: [PERFORM] [BUGS] BUG #2737: hash indexing large table fails,while btree of same index works

2006-11-11 Thread Simon Riggs
) > addressing the last block it intends to use whenever it allocates a new > batch of blocks, whereupon md.c could adopt a saner API: allow > smgrextend but not other calls to address blocks beyond the current EOF. > Thoughts? Yes, do it. -- Simon Riggs EnterpriseDB http

Re: [PERFORM] Context switch storm

2006-11-15 Thread Simon Riggs
tes memory with many connected users since the PrivRefCount array uses memory that could have been used as filesystem cache. -- Simon Riggs EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings

Re: [PERFORM] [BUGS] BUG #2737: hash indexing large tablefails,while btree of same index works

2006-11-17 Thread Simon Riggs
d be good to brain dump some starting places for an investigation. Does anybody have a perf test that shows hash indexes beating btrees by any significant margin? (Not saying there isn't one...) I can see the argument that fixed hash indexes would be faster, but there are obviously major down

Re: [PERFORM] [BUGS] BUG #2737: hash indexing largetablefails,while btree of same index works

2006-11-17 Thread Simon Riggs
On Fri, 2006-11-17 at 08:26 -0600, Kenneth Marshall wrote: > I certainly hold out some hope that they can improved. I would like to see > them still included. Once they are gone, it will be much harder to ever > add them back. OK, you got it - keep hash indexes then. -- Si

Re: [PERFORM] PostgreSQL underestimates sorting

2006-11-23 Thread Simon Riggs
ted data) and the cost model doesn't get this right, as you observe. Try enabling trace_sort (available in both 8.1 and 8.2) and post the results here please, which would be very useful to have results on such a large real-world sort. -- Simon Riggs

Re: [PERFORM] PostgreSQL and Linux 2.6 kernel.

2004-04-14 Thread Simon Riggs
x27;ve got working on the optimizer, its how many accurate field reports of less-than perfect optimization reach them. In that case, PostgreSQL is likely in a better position than Microsoft, since the accessibility of the pg discussion lists makes such cases much more likely to get aired. Any thoug

Re: [PERFORM] PostgreSQL and Linux 2.6 kernel.

2004-04-15 Thread Simon Riggs
> Bruno Wolff > Simon Riggs <[EMAIL PROTECTED]> wrote: > > > > I guess what I'm saying is it's not how many people you've > got working > > on the optimizer, its how many accurate field reports of less-than > > perfect optimization reach them

Re: [PERFORM] PostgreSQL and Linux 2.6 kernel.

2004-04-16 Thread Simon Riggs
or DBT-3. ...sounds like there's more to discuss here, so I'll duck out now and get back to my current project... Best Regards, Simon Riggs ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate

Re: [PERFORM] What is the best way to do attribute/values?

2004-08-25 Thread Simon Riggs
in total logical volume, so 4Gb of RAM should be easily sufficient. Best Regards, Simon Riggs > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] Behalf Of Daniel > Ceregatti > Sent: 19 August 2004 19:03 > To: [EMAIL PROTECTED] > Subject: [PERFORM]

Re: [PERFORM] Data Warehouse Reevaluation - MySQL vs Postgres -- merge tables

2004-09-13 Thread Simon Riggs
, > including Oracle and DB2. Agreed. You can reduce the time for the ANALYZE by ignoring some of the (measures) columns not used in WHERE clauses. Also, if you're sure that each load is very similar to the last, you might even consider directly updating pg_statistic rows with the stat

Re: [PERFORM] Data Warehouse Reevaluation - MySQL vs Postgres --

2004-09-14 Thread Simon Riggs
e added soon. > Yes, I think it is an important feature for both Data Warehousing (used in set-operation mode for bulk processing) and OLTP (saves a round-trip to the database, so faster on single rows also). It's in my top 10 for 2005. Best Regards, Simon Riggs ---

Re: [PERFORM] Data Warehouse Reevaluation - MySQL vs Postgres -- merge tables

2004-09-14 Thread Simon Riggs
> Jim C. Nasby > On Mon, Sep 13, 2004 at 11:07:35PM +0100, Simon Riggs wrote: > > PostgreSQL's functionality is in many ways similar to Oracle > Partitioning. > > > > Loading up your data in many similar tables, then creating a view like: > > > > C

Re: [PERFORM] Data Warehouse Reevaluation - MySQL vs Postgres -- merge tables?

2004-09-15 Thread Simon Riggs
Chris Browne <[EMAIL PROTECTED]> wrote on 15.09.2004, 04:34:53: > [EMAIL PROTECTED] ("Simon Riggs") writes: > > Well, its fairly straightforward to auto-generate the UNION ALL view, and > > important as well, since it needs to be re-specified each time a new > &

Re: [PERFORM] Data Warehouse Reevaluation - MySQL vs Postgres --

2004-09-15 Thread Simon Riggs
Joe, Your application is very interesting. I've just read your OSCON paper. I'd like to talk more about that. Very similar to Kalido. ...but back to partitioning momentarily: Does the performance gain come from partition elimination of the inherited tables under the root? Best Rega

Re: [PERFORM] Data Warehouse Reevaluation - MySQL vs Postgres --

2004-09-17 Thread Simon Riggs
ostgreSQL* that allows partitions to be excluded from queries as a way of speeding up queries against very large tables: UNION ALL with appended constants. Best Regards, Simon Riggs ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org

Re: [PERFORM] First set of OSDL Shared Mem scalability results, some wierdness ...

2004-10-14 Thread Simon Riggs
also in far fewer words, but I'll leave that to Jan to agree/disagree... I say this: ARC in 8.0 PostgreSQL allows us to sensibly allocate as large a shared_buffers cache as is required by the database workload, and this should not be constrained to a small percentage of server RAM. Best Regards,

Re: [Testperf-general] Re: [PERFORM] First set of OSDL Shared Memscalability results, some wierdness ...

2004-10-15 Thread Simon Riggs
that helps you decide how big the shared_buffers SHOULD BE, by making the LRU list bigger than the cache itself, so you'd be able to see whether there is beneficial effect in increasing shared_buffers. ...remember that this applies to other databases too, and with those we find that they ha

Re: [Testperf-general] Re: [PERFORM] First set of OSDL Shared Memscalability results, some wierdness ...

2004-10-15 Thread Simon Riggs
allocated. I've got a few ideas, but I need to check out the code first. I'll investigate both simple/complex options as an 8.1 feature. Best Regards, Simon Riggs ---(end of broadcast)--- TIP 8: explain analyze is your friend

Re: [PERFORM] how much mem to give postgres?

2004-10-19 Thread Simon Riggs
ssages, then its high enough. > Are there any other settings I should be concerned with? I've heard > about the effective_cache_size setting, but I haven't seen anything on > what the size should be. wal_buffers if the databases are heavily updated. > Any help woul

Re: [PERFORM] [PATCHES] [HACKERS] ARC Memory Usage analysis

2004-11-01 Thread Simon Riggs
He points out that it's not > really related to the *real* probability of any particular table being > cached. At least, if I'm reading him right. Yes, that was how Jan originally meant to discuss it, but not what I meant. Best regards, Simon Riggs ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]

Re: [PERFORM] Performance difference when using views

2004-11-01 Thread Simon Riggs
_code_id = zp.zip_code_id LEFT OUTER JOIN city ci ON ci.city_id = zp.city_id LEFT OUTER JOIN state u ON ci.state_id = s.state_id ) USING (person_id) WHERE a.adress_type = 2 AND p.person_id = 19257; Which should return the same answer, and also hopefully the same plan. -- Best Regards

Re: [PERFORM] Restricting Postgres

2004-11-03 Thread Simon Riggs
led to the machine dropping outright. >Any help on this matter would be appreciated. You can limit the number of connections overall? -- Best Regards, Simon Riggs ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster

Re: [PERFORM] Restricting Postgres

2004-11-04 Thread Simon Riggs
On Wed, 2004-11-03 at 21:25, Martin Foster wrote: > Simon Riggs wrote: > > On Tue, 2004-11-02 at 23:52, Martin Foster wrote: > > > >>Is there a way to restrict how much load a PostgreSQL server can take > >>before dropping queries in order to safeguard the ser

Re: [PERFORM] Anything to be gained from a 'Postgres Filesystem'?

2004-11-04 Thread Simon Riggs
)" call... > > That would mean that a seq scan or a vacuum wouldn't force useful data > out of cache. ARC does almost exactly those two things in 8.0. Seq scans do get put in cache, but in a way that means they don't spoil the main bulk of the cache. -- Best Regar

Re: [PERFORM] Anything to be gained from a 'Postgres Filesystem'?

2004-11-04 Thread Simon Riggs
On Thu, 2004-11-04 at 19:34, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > On Thu, 2004-11-04 at 15:47, Chris Browne wrote: > >> Something like a "read_uncached()" call... > >> > >> That would mean that a seq scan or a

Re: [PERFORM] poor performance in migrated database

2004-11-06 Thread Simon Riggs
XPLAIN ANALYZE will show you whether this is the case. It seems likely that the estimated cardinality of certain joins is incorrect. -- Best Regards, Simon Riggs ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ?

Re: [PERFORM] mis-estimation on data-warehouse aggregate creation

2004-11-16 Thread Simon Riggs
planner will be able to correctly estimate the join selectivity. i.e. put product.id onto shd_productline ahead of time, so you can avoid the complex join. Setting join_collapse_limit lower doesn't look like it would help, since the plan already shows joining the sub-queries together first.

Re: [PERFORM] Analyzer is clueless

2004-11-17 Thread Simon Riggs
have zero effect on this *plan* though might still benefit execution time. Please post EXPLAIN ANALYZE output for any further questions. -- Best Regards, Simon Riggs ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match

Re: [PERFORM] Speeding up pg_dump

2004-12-14 Thread Simon Riggs
k resources. The lower compression levels are fast and nearly as good (in my testing) as full compression. Using compression tends to use up the CPU that would otherwise be wasted since the pg_dump is disk intensive, and then saves further I/O by reducing the output file size. -- Best Reg

Re: [PERFORM] query rewrite using materialized views

2005-01-04 Thread Simon Riggs
d a little > insight into what your doing? A basic table structure, and indexes > involved would be handy. You may change column and table names if you > like. There's a known issue using UNION ALL views in 8.0 that makes them slightly more inefficient than using a s

Re: [PERFORM] Swapping on Solaris

2005-01-19 Thread Simon Riggs
ou can look at the data that really matters. -- Best Regards, Simon Riggs ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]

Re: [PERFORM] PostgreSQL clustering VS MySQL clustering

2005-01-23 Thread Simon Riggs
o doing some work to try to avoid that could frequently pay off. This would be isolated to updating. This wouldn't remove the need for vacuuming, but it would act to prevent severe performance degradation caused by frequent re-updating. What do you think? -- Best Regards, Simon Riggs

[PERFORM] Faster and more frequent VACUUM (was PostgreSQL clustering VS MySQL clustering)

2005-01-24 Thread Simon Riggs
On Sun, 2005-01-23 at 15:40 -0500, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > Changing the idea slightly might be better: if a row update would cause > > a block split, then if there is more than one row version then we vacuum > > the whole block

Re: [PERFORM] Possible interesting extra information for explain

2005-02-28 Thread Simon Riggs
t Regards, Simon Riggs ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster

Re: [PERFORM] What needs to be done for real Partitioning?

2005-03-22 Thread Simon Riggs
ll do not understand. It does work, well. And has many of the wrinkles ironed out, even if not all of them are easy to understand why they exist at first glance. I think it most likely that Phase I should be a simplified blend of both ideas, with a clear view towards minimum impact and

Re: [PERFORM] CPU 0.1% IOWAIT 99% for decisonnal queries

2005-03-22 Thread Simon Riggs
On the other hand...just go for more RAM, as you suggest...but you should create a RAMdisk, rather than use too large shared_buffers....that way your data is always in RAM, rather than maybe in RAM. Best Regards, Simon Riggs ---(end of broadcast)---

Re: [PERFORM] CPU 0.1% IOWAIT 99% for decisonnal queries

2005-03-24 Thread Simon Riggs
s the bgwriter's default settings are a problem. You don't need it, so I suggest turning it off. Best Regards, Simon Riggs ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]

Re: [PERFORM] [BUGS] BUG #1552: massive performance hit between 7.4 and 8.0.1

2005-03-25 Thread Simon Riggs
On Wed, 2005-03-23 at 14:22 -0500, Keith Browne wrote: > Simon Riggs wrote: > > > The EXPLAINs you've enclosed are for SELECTs, yet your bug report > > describes INSERTs as being the things that are slow. > > [You may find better performance from using COPY] &

Re: [PERFORM] 12 hour table vacuums

2007-10-23 Thread Simon Riggs
uite a lot. The default maintenance_work_mem is sufficient for this table. -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-

Re: [PERFORM] Speed difference between select ... union select ... and select from partitioned_table

2007-10-27 Thread Simon Riggs
oning improvements in 8.4 or beyond. -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq

Re: [PERFORM] partitioned table and ORDER BY indexed_field DESC LIMIT 1

2007-10-27 Thread Simon Riggs
ork? Do you have timings/different plan? -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org

Re: [PERFORM] partitioned table and ORDER BY indexed_field DESCLIMIT 1

2007-10-28 Thread Simon Riggs
ng this technique. I agree its annoying and I have a way of doing this, but that's an 8.4 thing now. Anybody think different? -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings

Re: [PERFORM] partitioned table and ORDER BY indexed_field DESCLIMIT 1

2007-10-28 Thread Simon Riggs
On Sun, 2007-10-28 at 12:53 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > I agree its annoying and I have a way of doing this, but that's an 8.4 > > thing now. > > It was an 8.4 thing quite some time ago, since no working patch was ever >

Re: [PERFORM] work_mem and shared_buffers

2007-11-18 Thread Simon Riggs
still slower than 16M. Remember that what you have shown is that for *this* dataset 16Mb is the optimum value. It is not a recommended value for all cases. -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com ---(end of broadcast)--- TIP 1: i

Re: [PERFORM] TB-sized databases

2007-11-27 Thread Simon Riggs
allow us to get bigger and faster databases. If anybody has specific concerns then I'd like to hear them so I can consider those things in the planning stages. -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com ---(end of broadcast)--

Re: [PERFORM] TB-sized databases

2007-11-28 Thread Simon Riggs
ple like me to re-write and resubmit this patch for 8.4? Tom's previous concerns were along the lines of "How would know what to set it to?", given that the planner costs are mostly arbitrary numbers. Any bright ideas, or is it we want it and we don't care about the possible d

Re: [PERFORM] TB-sized databases

2007-11-28 Thread Simon Riggs
On Tue, 2007-11-27 at 18:06 -0500, Pablo Alcaraz wrote: > Simon Riggs wrote: > > All of those responses have cooked up quite a few topics into one. Large > > databases might mean text warehouses, XML message stores, relational > > archives and fact-based business data warehou

Re: [PERFORM] 7.4 Checkpoint Question

2007-11-29 Thread Simon Riggs
n background at that time, updates loads of blocks which need to be written out at checkpoint time. That slows queries down at that time but not others. -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster

Re: [PERFORM] TB-sized databases

2007-11-29 Thread Simon Riggs
On Thu, 2007-11-29 at 10:45 -0500, Tom Lane wrote: > Gregory Stark <[EMAIL PROTECTED]> writes: > > "Simon Riggs" <[EMAIL PROTECTED]> writes: > >> Tom's previous concerns were along the lines of "How would know what to > >> set it to?&

Re: [PERFORM] TB-sized databases

2007-11-30 Thread Simon Riggs
On Fri, 2007-11-30 at 17:41 +1100, Russell Smith wrote: > Simon Riggs wrote: > > On Tue, 2007-11-27 at 18:06 -0500, Pablo Alcaraz wrote: > > > >> Simon Riggs wrote: > >> > >>> All of those responses have cooked up quite a few topics in

Re: [PERFORM] Training Recommendations

2007-12-04 Thread Simon Riggs
; 2ndQuandrant (sp?) for example recently announced a PITR training. Just posting to get the spelling right; thanks Josh for mentioning. -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com ---(end of broadcast)--- TIP 4: Hav

Re: [PERFORM] TB-sized databases

2007-12-06 Thread Simon Riggs
facility, or would the difficulty in setting it exactly prevent you from using it for real? -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org

Re: [PERFORM] database tuning

2007-12-07 Thread Simon Riggs
ds you mention, so best to concentrate your efforts on index planning for your most frequently executed queries. -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings

Re: [PERFORM] TB-sized databases

2007-12-11 Thread Simon Riggs
On Fri, 2007-12-07 at 12:45 -0500, Robert Treat wrote: > On Thursday 06 December 2007 04:38, Simon Riggs wrote: > > > I think you're completly overlooking the effect of disk latency has on > > > query times. We run queries all the time that can vary from 4 hours to &g

Re: [PERFORM] Limited performance on multi core server

2007-12-13 Thread Simon Riggs
ng you've said suggests you've hit the scalability limit of 7.4, which had a buffer manager that got worse with larger settings, fixed in 8.0. Most of the scalability stuff has been added since then and 8.3 looks to be really fast, but we would still like some more performance numbe

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Simon Riggs
e vacuumed, and I noticed something odd. Even > though there have been no rollbacks, updates, or deletes on this > table, the vacuum is writing as much as it is reading while dealing > with the TOAST data. Writing hint bits. Annoying isn't it? :-( -- Simon Riggs 2ndQuadr

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Simon Riggs
what its supposed to be doing. That's a lot of work to fiddle with the knobs to improve things and there's higher things on the list AFAICS. -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Simon Riggs
On Thu, 2007-12-13 at 11:46 -0600, Kevin Grittner wrote: > >>> On Thu, Dec 13, 2007 at 10:11 AM, in message > <[EMAIL PROTECTED]>, Simon Riggs <[EMAIL PROTECTED]> > wrote: > > On Thu, 2007-12-13 at 09:46 -0600, Kevin Grittner wrote: > > > >> T

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Simon Riggs
On Thu, 2007-12-13 at 13:52 -0500, Tom Lane wrote: > Alvaro Herrera <[EMAIL PROTECTED]> writes: > > Simon Riggs wrote: > >> We might be able to have bgwriter set hint bits on dirty blocks, > > > I don't think that works, because the bgwriter has no access to

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Simon Riggs
On Thu, 2007-12-13 at 15:19 -0600, Kevin Grittner wrote: > >>> On Thu, Dec 13, 2007 at 12:12 PM, in message > <[EMAIL PROTECTED]>, Simon Riggs <[EMAIL PROTECTED]> > wrote: > > On Thu, 2007-12-13 at 11:46 -0600, Kevin Grittner wrote: > >> I

Re: [PERFORM] More shared buffers causes lower performances

2007-12-26 Thread Simon Riggs
red_buffers=1024MB: 1559 tps Can you try with bgwriter_lru_maxpages = 0 So we can see if the bgwriter has any hand in this? -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com ---(end of broadcast)--- TIP 5: don't forget to incre

Re: [PERFORM] Commit takes a long time.

2008-01-04 Thread Simon Riggs
commit record > takes a long time. I've seen 3 other reasons for this in the field while tuning people's systems. In 8.3 we've fixed one, reduced the other and the third is amenable to tuning via wal_buffers even in 8.1 -- Simon Riggs 2ndQuadrant http://www.2ndQuadran

Re: [PERFORM] big database performance

2008-01-09 Thread Simon Riggs
On Wed, 2008-01-09 at 10:18 +0200, Adrian Moisey wrote: > We recently converted to postgres (from mssql) and we're having > performance issues. I think you need to say more about what the performance issues actually are, otherwise everybody will just speculate you to death. --

Re: [PERFORM] Linux/PostgreSQL scalability issue - problem with 8 cores

2008-01-25 Thread Simon Riggs
discussion to -patches now that will do this. It seems straightforward enough to include in 8.3, but that may rise a few eyebrows, but read the patch first. -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster

[PERFORM] Benchmark Data requested

2008-02-04 Thread Simon Riggs
/ Multiple runs from different people/different hardware is useful since they help to iron-out differences in hardware and test methodology. So don't worry if you see somebody else doing this also. Thanks, -- Simon Riggs 2ndQuadrant http://www.2ndQuadran

Re: [PERFORM] Benchmark Data requested

2008-02-04 Thread Simon Riggs
u just referring to the Cray paper? (Which used fixed length tuples and specific vector hardware). (With regard to benchmarks, I'd rather not download Monet at all. Helps avoid legal issues around did-you-look-at-the-code questions.) -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com

Re: [PERFORM] Benchmark Data requested

2008-02-04 Thread Simon Riggs
On Mon, 2008-02-04 at 15:09 -0500, Greg Smith wrote: > On Mon, 4 Feb 2008, Simon Riggs wrote: > > > Would anybody like to repeat these tests with the latest production > > versions of these databases (i.e. with PGSQL 8.3) > > Do you have any suggestions on how people shou

Re: [PERFORM] Benchmark Data requested

2008-02-05 Thread Simon Riggs
what we do for discussion, so it's hardly breaking the spirit of the TPC rules to publish results here, in the hope of focusing development effort. -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com ---(end of broadcast)--- TIP 1: if

Re: [PERFORM] Benchmark Data requested

2008-02-05 Thread Simon Riggs
ests in response to my request, then *any* scale factor is interesting, on any hardware. If that means Scale Factor 1, 3, 10 or 30 then that's fine by me. -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com ---(end of broadcast)--- TIP 1

Re: [PERFORM] Benchmark Data requested

2008-02-05 Thread Simon Riggs
On Tue, 2008-02-05 at 14:43 +, Richard Huxton wrote: > Simon Riggs wrote: > > On Tue, 2008-02-05 at 15:06 +0100, Dimitri Fontaine wrote: > >> > >> Le lundi 04 février 2008, Jignesh K. Shah a écrit : > > >>> Multiple table loads ( 1 per table) spawne

Re: [PERFORM] Benchmark Data requested

2008-02-05 Thread Simon Riggs
n PostgreSQL trigger or rules > as used for partitioning currently, but ISTM Jignesh quoted § is just about > that. Much better than triggers and rules, but it will be hard to get it to work. -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings

Re: [PERFORM] Benchmark Data requested

2008-02-05 Thread Simon Riggs
est not. > > Hmm - the table-locking requirement is true enough, but why would > pg_loader cause problems after the first few rows? It runs a stream of COPY statements, so only first would be optimized with the "empty table optimization". -- Simon Riggs

Re: [PERFORM] Benchmark Data requested

2008-02-05 Thread Simon Riggs
On Tue, 2008-02-05 at 18:15 +0100, Dimitri Fontaine wrote: > Le mardi 05 février 2008, Simon Riggs a écrit : > > I'll look at COPY FROM internals to make this faster. I'm looking at > > this now to refresh my memory; I already had some plans on the shelf. > >

Re: [PERFORM] Benchmark Data requested

2008-02-05 Thread Simon Riggs
many people would choose that then blame us for delivering unreliable software. e.g. direct path loader on Oracle used to corrupt a PK index if you loaded duplicate rows with it (whether it still does I couldn't care). That kind of behaviour is simply incompatible with production usage, even if it do

Re: [PERFORM] Benchmark Data requested

2008-02-05 Thread Simon Riggs
can timeslice it well for the second process to use > those wait times for the index population work. If Dimitri is working on parallel load, why bother? -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com ---(end of broadcast)--- TI

Re: [PERFORM] Optimizer : query rewrite and execution plan ?

2008-02-06 Thread Simon Riggs
at would perform additional checks for redundant clauses of various kinds. The default for that would be "off" since most people don't suffer from this problem. BO isn't the only SQL generating-client out there, so I think this is a fairly wide problem. -- Simon Riggs 2ndQua

Re: [PERFORM] Benchmark Data requested --- pgloader CE design ideas

2008-02-06 Thread Simon Riggs
may just need multiple sessions from pg_loader. Experience from doing the non-routing parallel version may help in deciding whether to go for the routing version. -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq

Re: [PERFORM] Optimizer : query rewrite and execution plan ?

2008-02-11 Thread Simon Riggs
On Wed, 2008-02-06 at 11:00 -0500, Tom Lane wrote: > Theo Kramer <[EMAIL PROTECTED]> writes: > > On Wed, 2008-02-06 at 11:53 +0000, Simon Riggs wrote: > >> Since the SQL is not your fault and difficult to control, it is an > >> argument in favour of an optiona

Re: [PERFORM] multi-threaded pgloader needs your tests

2008-03-01 Thread Simon Riggs
l if you don't get any replies here. New feature is very important for us. -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com PostgreSQL UK 2008 Conference: http://www.postgresql.org.uk ---(end of broadcast)--- TIP 9: In vers

Re: [PERFORM] Very slow (2 tuples/second) sequential scan after bulk insert; speed returns to ~500 tuples/second after commit

2008-03-10 Thread Simon Riggs
On Mon, 2008-03-10 at 11:01 +, Heikki Linnakangas wrote: > According > to oprofile, all the time is spent in TransactionIdIsInProgress. I recently submitted a patch to optimise this. Your comments would be welcome on the patch. -- Simon Riggs 2ndQuadrant http://www.2ndQuadra

Re: [PERFORM] SQL Function Slowness, 8.3.0

2008-04-17 Thread Simon Riggs
k we need to realise that many people will never post their SQL at all. -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] SQL Function Slowness, 8.3.0

2008-04-17 Thread Simon Riggs
On Thu, 2008-04-17 at 12:12 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > I think it would help if there was some way to prepare functions to > > allow them to be posted and understood more easily. These would help: > > > * a name obfuscator

Re: [PERFORM] SQL Function Slowness, 8.3.0

2008-04-17 Thread Simon Riggs
On Thu, 2008-04-17 at 12:41 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > On Thu, 2008-04-17 at 12:12 -0400, Tom Lane wrote: > >> Aren't these suggestions mutually contradictory? > > > No, they're orthogonal. The pretty printer

Re: [PERFORM] Backup causing poor performance - suggestions

2008-05-05 Thread Simon Riggs
reports about Skype Tools, which has both a Slony-like replicator > (not as configurable as Slony, but easier to set up and use), plus an > entirely separate set of scripts that simplifies "warm standby" using > WAL logging. I think we should mention Warm Standby via pg_standby, w

Re: [PERFORM] Join runs for > 10 hours and then fills up >1.3TB of disk space

2008-05-16 Thread Simon Riggs
e an estimated 3 billion rows? Can you cope if that estimate is wrong and the true figure is much higher? Do you think the estimate is realistic? -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-performance mailing list (pgsql-perfor

Re: [PERFORM] I/O on select count(*)

2008-05-16 Thread Simon Riggs
the transactionid is assigned the page of the clog that contains that transactionid is checked to see if it already exists and if not, it is initialised. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-performance mailing list (pgsql-pe

[PERFORM] Outer joins and equivalence

2008-05-27 Thread Simon Riggs
ON (b.id = c.id) though that restriction on outer join equivalence is not relevant here. (SQL, EXPLAINs etc available off-list only, by request). I'm looking into this more now. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-per

Re: [PERFORM] I/O on select count(*)

2008-05-27 Thread Simon Riggs
> > > Or even better: set the hint-bits, but don't dirty the page. > > Which in fact is what Simon suggested, not the other thing. Just raised this on -hackers, BTW. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via

Re: [PERFORM] Outer joins and equivalence

2008-05-28 Thread Simon Riggs
On Wed, 2008-05-28 at 11:45 +0100, Matthew Wakeling wrote: > On Tue, 27 May 2008, Simon Riggs wrote: > > I do recognise that we would *not* be able to deduce this form of SQL > > > > A JOIN B ON (a.id = c.id) LEFT JOIN C ON (b.id = c.id) > > Surely that would not b

Re: [PERFORM] GEQO Benchmark

2008-05-28 Thread Simon Riggs
put; sometimes it gets the right plan, other times it doesn't get close. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.p

Re: [PERFORM] 2GB or not 2GB

2008-05-31 Thread Simon Riggs
his improved in 8.3? There is an optimum for each specific sort. Your results cannot be used to make a global recommendation about the setting of work_mem. So not finding any benefit in your tests *and* Magnus seeing an improvement are not inconsistent events. -- Simon Riggs www.2nd

Re: [PERFORM] 2GB or not 2GB

2008-06-01 Thread Simon Riggs
your view of serious I suppose. I would say it is an acceptable situation, but needs further optimization. I threw some ideas around on Hackers around Dec/New Year, but I don't have time to work on this further myself in this dev cycle. Further contributions welcome. -- Simon Riggs

Re: [PERFORM] Outer joins and equivalence

2008-06-02 Thread Simon Riggs
On Tue, 2008-05-27 at 17:43 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > I have a complex query where making a small change to the SQL increases > > run-time by > 1000 times. > > > The first SQL statement is of the form > > >

Re: [PERFORM] Outer joins and equivalence

2008-06-02 Thread Simon Riggs
On Mon, 2008-06-02 at 18:10 +0100, Simon Riggs wrote: > On Tue, 2008-05-27 at 17:43 -0400, Tom Lane wrote: > > Simon Riggs <[EMAIL PROTECTED]> writes: > > > I have a complex query where making a small change to the SQL increases > > > run-time by > 1000 times

<    1   2   3   4   >