Re: [PERFORM] How much expensive are row level statistics?

2005-12-16 Thread Simon Riggs
On Thu, 2005-12-15 at 19:06 -0500, Tom Lane wrote: Michael Fuhr [EMAIL PROTECTED] writes: Does the backend support, or could it be easily modified to support, a mechanism that would post the command string after a configurable amount of time had expired, and then continue processing the

Re: [PERFORM] Crashing DB or Server?

2005-12-16 Thread Harry Jackson
On 12/16/05, Moritz Bayer [EMAIL PROTECTED] wrote: This is really weird, just a few hours ago the machine run very smooth serving the data for a big portal. Can you log the statements that are taking a long time and post them to the list with the table structures and indexes for the tables

Re: [PERFORM] Overriding the optimizer

2005-12-16 Thread Kyle Cordes
Kevin Brown wrote: Craig A. James wrote: Hints are dangerous, and I consider them a last resort. If you consider them a last resort, then why do you consider them to be a better alternative than a workaround such as turning off enable_seqscan, when all the other tradeoffs are

[PERFORM] ALTER TABLE SET TABLESPACE and pg_toast

2005-12-16 Thread PostgreSQL
We're storing tif images in a table as bytea. We were running low on our primary space and moved several tables, including the one with the images, to a second tablespace using ALTER TABLE SET TABLESPACE. This moved quite cleaned out quite a bit of space on the original tablespace, but not as

Re: [PERFORM] How much expensive are row level statistics?

2005-12-16 Thread Merlin Moncure
Now there goes Tom with his skeptical eye again, and here comes me saying oops again. Further tests show that for this application I made the same mistake, fwiw. The big hit comes with command_string. However, row level stats bring a big enough penalty (~10% on my usage) that I keep them

Re: [PERFORM] Overriding the optimizer

2005-12-16 Thread Jaime Casanova
On 12/16/05, Kyle Cordes [EMAIL PROTECTED] wrote: Kevin Brown wrote: Craig A. James wrote: Hints are dangerous, and I consider them a last resort. If you consider them a last resort, then why do you consider them to be a better alternative than a workaround such as turning off

Re: [PERFORM] Overriding the optimizer

2005-12-16 Thread Tomasz Rybak
Dnia 16-12-2005, pią o godzinie 16:16 +1300, Mark Kirkwood napisał(a): Craig A. James wrote: What would be cool would be some way the developer could alter the plan, but they way of doing so would strongly encourage the developer to send the information to this mailing list.

Re: [PERFORM] ALTER TABLE SET TABLESPACE and pg_toast

2005-12-16 Thread Tom Lane
PostgreSQL [EMAIL PROTECTED] writes: We're storing tif images in a table as bytea. We were running low on our primary space and moved several tables, including the one with the images, to a second tablespace using ALTER TABLE SET TABLESPACE. This moved quite cleaned out quite a bit of space

Re: [PERFORM] Overriding the optimizer

2005-12-16 Thread Craig A. James
Jaime Casanova wrote: The context is this - in a busy OLTP system, sometimes a query comes through that, for whatever reason (foolishness on my part as a developer, unexpected use by a user, imperfection of the optimizer, etc.), takes a really long time to run, usually because it table-scans one

Re: [PERFORM] Overriding the optimizer

2005-12-16 Thread Mitch Skinner
On Thu, 2005-12-15 at 18:23 -0800, Craig A. James wrote: So, you still have no problem is exactly wrong, because Postgres picked the wrong plan. Postgres decided that applying myfunc() to 10,000,000 rows was a better plan than an index scan of 50,000 row_nums. So I'm screwed. FWIW, The

[PERFORM] 8.1 - pg_autovacuum question

2005-12-16 Thread Chris Hoover
In PostgreSQL 8.1, is the pg_autovacuum daemon affected by the vacuum_cost_* variables? I need to make sure that if we turn autovacuuming on when we upgrade to 8.1, we don't cause any i/o issues. Thanks, Chris ---(end of broadcast)--- TIP 1: if

Re: [PERFORM] 8.1 - pg_autovacuum question

2005-12-16 Thread Alvaro Herrera
Chris Hoover wrote: In PostgreSQL 8.1, is the pg_autovacuum daemon affected by the vacuum_cost_* variables? I need to make sure that if we turn autovacuuming on when we upgrade to 8.1, we don't cause any i/o issues. What pg_autovacuum daemon? The contrib one? I don't know. The integrated

Re: [PERFORM] Simple Join

2005-12-16 Thread Mark Kirkwood
David Lang wrote: On Fri, 16 Dec 2005, Mark Kirkwood wrote: Right on. Some of these coerced plans may perform much better. If so, we can look at tweaking your runtime config: e.g. effective_cache_size random_page_cost default_statistics_target to see if said plans can be chosen naturally.

Re: [PERFORM] Lots of postmaster processes (fwd)

2005-12-16 Thread Jim C. Nasby
Dunno if this has gotten a reply elsewhere, but during a checkpoint the database can become quite busy. If that happens and performance slows down, other queries will slow down as well. If you have an app where a a high rate of incomming requests (like a busy website), existing backends won't be

Re: [PERFORM] SAN/NAS options

2005-12-16 Thread Jim C. Nasby
On Wed, Dec 14, 2005 at 08:28:56PM +1300, Mark Kirkwood wrote: Another interesting thing to try is rebuilding the database ufs filesystem(s) with 32K blocks and 4K frags (as opposed to 8K/1K or 16K/2K - can't recall the default on 4.x). I found this to give a factor of 2 speedup on random

Re: [PERFORM] SAN/NAS options

2005-12-16 Thread Jim C. Nasby
On Wed, Dec 14, 2005 at 01:56:10AM -0500, Charles Sprickman wrote: You'll note that I'm being somewhat driven by my OS of choice, FreeBSD. Unlike Solaris or other commercial offerings, there is no nice volume management available. While I'd love to keep managing a dozen or so FreeBSD

Re: [PERFORM] Overriding the optimizer

2005-12-16 Thread Jim C. Nasby
On Thu, Dec 15, 2005 at 09:48:55PM -0800, Kevin Brown wrote: Craig A. James wrote: Kevin Brown wrote: Hints are dangerous, and I consider them a last resort. If you consider them a last resort, then why do you consider them to be a better alternative than a workaround such as turning

Re: [PERFORM] Overriding the optimizer

2005-12-16 Thread Jim C. Nasby
On Fri, Dec 16, 2005 at 03:31:03PM +1300, Mark Kirkwood wrote: After years of using several other database products (some supporting hint type constructs and some not), I have come to believe that hinting (or similar) actually *hinders* the development of a great optimizer. I don't think you

Re: [PERFORM] Overriding the optimizer

2005-12-16 Thread Jim C. Nasby
On Fri, Dec 16, 2005 at 04:16:58PM +1300, Mark Kirkwood wrote: Craig A. James wrote: What would be cool would be some way the developer could alter the plan, but they way of doing so would strongly encourage the developer to send the information to this mailing list. Postgres would

Re: [PERFORM] SAN/NAS options

2005-12-16 Thread Mark Kirkwood
Jim C. Nasby wrote: On Wed, Dec 14, 2005 at 08:28:56PM +1300, Mark Kirkwood wrote: Another interesting thing to try is rebuilding the database ufs filesystem(s) with 32K blocks and 4K frags (as opposed to 8K/1K or 16K/2K - can't recall the default on 4.x). I found this to give a factor of 2

Re: [PERFORM] SAN/NAS options

2005-12-16 Thread Michael Stone
On Fri, Dec 16, 2005 at 04:18:01PM -0600, Jim C. Nasby wrote: Even if you're doing a lot of random IO? I would think that random IO would perform better if you use smaller (8K) blocks, since there's less data being read in and then just thrown away that way. The overhead of reading an 8k block

Re: [PERFORM] SAN/NAS options

2005-12-16 Thread Jim C. Nasby
On Fri, Dec 16, 2005 at 05:51:03PM -0500, Michael Stone wrote: On Fri, Dec 16, 2005 at 04:18:01PM -0600, Jim C. Nasby wrote: Even if you're doing a lot of random IO? I would think that random IO would perform better if you use smaller (8K) blocks, since there's less data being read in and then

Re: [PERFORM] SAN/NAS options

2005-12-16 Thread Michael Stone
On Fri, Dec 16, 2005 at 06:25:25PM -0600, Jim C. Nasby wrote: True, but now you've got 4x the amount of data in your cache that you probably don't need. Or you might be 4x more likely to have data cached that's needed later. If you're hitting disk either way, that's probably more likely than

Re: [PERFORM] Overriding the optimizer

2005-12-16 Thread Kyle Cordes
Jaime Casanova wrote: What I would really like is for my DBMS to give me a little more pushback - I'd like to ask it to run a query, and have it either find a good way to run the query, or politely refuse to run it at all. set statement_timeout in postgresql.conf That is what I am

Re: [PERFORM] How much expensive are row level statistics?

2005-12-16 Thread Bruce Momjian
Tom Lane wrote: Michael Fuhr [EMAIL PROTECTED] writes: Further tests show that for this application the killer is stats_command_string, not stats_block_level or stats_row_level. I tried it with pgbench -c 10, and got these results: 41% reduction in TPS rate for

Re: [PERFORM] Should Oracle outperform PostgreSQL on a complex

2005-12-16 Thread Mark Kirkwood
Bruce Momjian wrote: How are star joins different from what we do now? --- Recall that a star query with n tables means a query where there are (n - 1) supposedly small tables (dimensions) and 1 large table (fact) -

Re: [PERFORM] Overriding the optimizer

2005-12-16 Thread Tom Lane
Craig A. James [EMAIL PROTECTED] writes: How about this: Instead of arguing in the abstract, tell me in concrete terms how you would address the very specific example I gave, where myfunc() is a user-written function. To make it a little more challenging, try this: myfunc() can behave very