Re: [PERFORM] NOT LIKE much faster than LIKE?

2006-01-10 Thread Matteo Beccati
Hi, I did just think of something we could improve though. The pattern selectivity code doesn't make any use of the statistics about most common values. For a constant pattern, we could actually apply the pattern test with each common value and derive answers that are exact for the portion of

Re: [PERFORM] help tuning queries on large database

2006-01-10 Thread Ron
At 12:23 PM 1/9/2006, peter royal wrote: On Jan 8, 2006, at 4:35 PM, Ron wrote: Areca ARC-1220 8-port PCI-E controller Make sure you have 1GB or 2GB of cache. Get the battery backup and set the cache for write back rather than write through. The card we've got doesn't have a SODIMM

[PERFORM] How to handle a large DB and simultaneous accesses?

2006-01-10 Thread Charles A. Landemaine
Hello, I have to develop a companies search engine (looks like the Yellow pages). We're using PostgreSQL at the company, and the initial DB is 2GB large, as it has companies from the entire world, with a fair amount of information. What reading do you suggest so that we can develop the search

Re: [PERFORM] NOT LIKE much faster than LIKE?

2006-01-10 Thread Andrea Arcangeli
On Tue, Jan 10, 2006 at 10:11:18AM -0500, Greg Stark wrote: Andrea Arcangeli [EMAIL PROTECTED] writes: Fixing this with proper stats would be great indeed. What would be the most common value for the kernel_version? You can see samples of the kernel_version here

Re: [PERFORM] 500x speed-down: Wrong statistics!

2006-01-10 Thread Tom Lane
Alessandro Baretta [EMAIL PROTECTED] writes: I have no clue as to how or why the statistics were wrong yesterday--as I vacuum-analyzed continuously out of lack of any better idea--and I was stupid enough to re-timestamp everything before selecting from pg_stats. Too bad. I would be

Re: [PERFORM] Index isn't used during a join.

2006-01-10 Thread Robert Creager
When grilled further on (Mon, 9 Jan 2006 22:58:18 -0700), Michael Fuhr [EMAIL PROTECTED] confessed: On Mon, Jan 09, 2006 at 09:23:38PM -0700, Robert Creager wrote: I'm working with a query to get more info out with a join. The base query works great speed wise because of index usage. When

Re: [PERFORM] NOT LIKE much faster than LIKE?

2006-01-10 Thread Greg Stark
Andrea Arcangeli [EMAIL PROTECTED] writes: Fixing this with proper stats would be great indeed. What would be the most common value for the kernel_version? You can see samples of the kernel_version here http://klive.cpushare.com/2.6.15/ . That's the string that is being searched against

Re: [PERFORM] NOT LIKE much faster than LIKE?

2006-01-10 Thread Tom Lane
Andrea Arcangeli [EMAIL PROTECTED] writes: There's only one preempt near the end, not sure if it would work? Not with that data, but maybe if you increased the statistics target for the column to 100 or so, you'd catch enough values to get reasonable results. regards,

Re: [PERFORM] NOT LIKE much faster than LIKE?

2006-01-10 Thread Tom Lane
Matteo Beccati [EMAIL PROTECTED] writes: I did just think of something we could improve though. The pattern selectivity code doesn't make any use of the statistics about most common values. For a constant pattern, we could actually apply the pattern test with each common value and derive

Re: [PERFORM] NOT LIKE much faster than LIKE?

2006-01-10 Thread Simon Riggs
On Tue, 2006-01-10 at 12:49 -0500, Tom Lane wrote: Matteo Beccati [EMAIL PROTECTED] writes: I did just think of something we could improve though. The pattern selectivity code doesn't make any use of the statistics about most common values. For a constant pattern, we could actually apply

Re: [PERFORM] NOT LIKE much faster than LIKE?

2006-01-10 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: I think its OK to use the MCV, but I have a problem with the current heuristics: they only work for randomly generated strings, since the selectivity goes down geometrically with length. We could certainly use a less aggressive curve for that. You got a

Re: [PERFORM] NOT LIKE much faster than LIKE?

2006-01-10 Thread Simon Riggs
On Tue, 2006-01-10 at 17:21 -0500, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: I think its OK to use the MCV, but I have a problem with the current heuristics: they only work for randomly generated strings, since the selectivity goes down geometrically with length. We could

Re: [PERFORM] help tuning queries on large database

2006-01-10 Thread Mark Lewis
Ron, A few days back you mentioned: Upgrade your kernel to at least 2.6.12 There's a known issue with earlier versions of the 2.6.x kernel and 64b CPUs like the Opteron. See kernel.org for details. I did some searching and couldn't find any obvious mention of this issue (I gave up after

[PERFORM] Left Join Performance vs Inner Join Performance

2006-01-10 Thread Dave Dutcher
Hello, I have an inner join query that runs fast, but I when I change to a left join the query runs 96 times slower. I wish I could always do an inner join, but there are rare times when there isnt data in the right hand table. I could expect a small performance hit, but the difference

Re: [PERFORM] NOT LIKE much faster than LIKE?

2006-01-10 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: I meant use the same sampling approach as I was proposing for ANALYZE, but do this at plan time for the query. That way we can apply the function directly to the sampled rows and estimate selectivity. I think this is so unlikely to be a win as to not even

Re: [PERFORM] How to handle a large DB and simultaneous accesses?

2006-01-10 Thread David Lang
On Tue, 10 Jan 2006, Charles A. Landemaine wrote: Hello, I have to develop a companies search engine (looks like the Yellow pages). We're using PostgreSQL at the company, and the initial DB is 2GB large, as it has companies from the entire world, with a fair amount of information. What

Re: [PERFORM] Left Join Performance vs Inner Join Performance

2006-01-10 Thread Tom Lane
Dave Dutcher [EMAIL PROTECTED] writes: I have an inner join query that runs fast, but I when I change to a left join the query runs 96 times slower. This looks like an issue that is fixed in the latest set of releases, namely that OUTER JOIN ON conditions that reference only the inner side of

Re: [PERFORM] Index isn't used during a join.

2006-01-10 Thread Robert Creager
Ok, I'm back, and in a little better shape. The query is now correct, but still is slow because of lack of index usage. I don't know how to structure the query correctly to use the index. Taken individually: weather=# explain analyze select * from doy_agg where doy = extract( doy from now()

Re: [PERFORM] help tuning queries on large database

2006-01-10 Thread Ron
At 07:28 PM 1/10/2006, Mark Lewis wrote: Ron, A few days back you mentioned: Upgrade your kernel to at least 2.6.12 There's a known issue with earlier versions of the 2.6.x kernel and 64b CPUs like the Opteron. See kernel.org for details. I did some searching and couldn't find any

Re: [PERFORM] Index isn't used during a join.

2006-01-10 Thread Michael Fuhr
On Tue, Jan 10, 2006 at 10:10:55PM -0700, Robert Creager wrote: The query is now correct, but still is slow because of lack of index usage. I don't know how to structure the query correctly to use the index. Have you tried adding restrictions on doy in the WHERE clause? Something like this, I