Re: [PERFORM] ERROR: "invalid memory alloc request size" or "unexpected end of data" on large table

2007-11-12 Thread Tom Lane
Stephane Bailliez <[EMAIL PROTECTED]> writes: > ERROR: invalid memory alloc request size 1664639562 This sounds like corrupt data --- specifically, 1664639562 showing up where a variable-width field's length word ought to be. It may or may not be relevant that the ASCII equivalent of that bit pat

[PERFORM] ERROR: "invalid memory alloc request size" or "unexpected end of data" on large table

2007-11-12 Thread Stephane Bailliez
(posting on pgsql-perf as I'm questioning the pertinence of the settings, might not be the best place for the overall pb: apologies) Postgresql 8.1.10 Linux Ubuntu: 2.6.17-12-server 4GB RAM, machine is only used for this I do have less than 30 tables, 4 of them having between 10-40 million rows

Re: [PERFORM] difference between a unique constraint and a unique index ???

2007-11-12 Thread Dave Cramer
On 12-Nov-07, at 11:37 AM, Tom Lane wrote: Alvaro Herrera <[EMAIL PROTECTED]> writes: Well, AFAIK the index with varchar_pattern_ops is used for LIKE queries, whereas the other one is going to be used for = queries. So you need to keep both indexes. Given the current definition of text

Re: [PERFORM] Need to run CLUSTER to keep performance

2007-11-12 Thread Heikki Linnakangas
Scott Marlowe wrote: On Nov 12, 2007 11:01 AM, Heikki Linnakangas <[EMAIL PROTECTED]> wrote: Scott Marlowe wrote: So, between the first and second vacuum you had a long running transaction that finally ended and let you clean up the dead rows. No, before 8.3, CLUSTER throws away non-removable

Re: [PERFORM] Need to run CLUSTER to keep performance

2007-11-12 Thread Scott Marlowe
On Nov 12, 2007 11:01 AM, Heikki Linnakangas <[EMAIL PROTECTED]> wrote: > > Scott Marlowe wrote: > > So, between the first and second vacuum you had a long running > > transaction that finally ended and let you clean up the dead rows. > > No, before 8.3, CLUSTER throws away non-removable dead tuple

Re: [PERFORM] Need to run CLUSTER to keep performance

2007-11-12 Thread Heikki Linnakangas
Scott Marlowe wrote: On Nov 12, 2007 10:11 AM, Rafael Martinez <[EMAIL PROTECTED]> wrote: Sending this just in case it can help Checking all the log files from these vacuum jobs we have been running, we found one that looks difference from the rest, specially on the amount of removed page

Re: [PERFORM] difference between a unique constraint and a unique index ???

2007-11-12 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > Well, AFAIK the index with varchar_pattern_ops is used for LIKE queries, > whereas the other one is going to be used for = queries. So you need to > keep both indexes. Given the current definition of text equality, it'd be possible to drop ~=~ and have

Re: [PERFORM] Need to run CLUSTER to keep performance

2007-11-12 Thread Bill Moran
In response to Heikki Linnakangas <[EMAIL PROTECTED]>: > Rafael Martinez wrote: > > DETAIL: 83623 dead row versions cannot be removed yet. > > Looks like you have a long-running transaction in the background, so > VACUUM can't remove all dead tuples. I didn't see that in the vacuum > verbose o

Re: [PERFORM] Need to run CLUSTER to keep performance

2007-11-12 Thread Scott Marlowe
On Nov 12, 2007 10:11 AM, Rafael Martinez <[EMAIL PROTECTED]> wrote: > Sending this just in case it can help > > Checking all the log files from these vacuum jobs we have been running, > we found one that looks difference from the rest, specially on the > amount of removed pages. > > We are s

Re: [PERFORM] Curious about dead rows.

2007-11-12 Thread Andrew Sullivan
On Sat, Nov 10, 2007 at 09:22:58PM -0500, Jean-David Beyer wrote: > > > > So, there are NO failed inserts, and no updates? Cause that's what > > I'd expect to create the dead rows. > > > So would I. Hence the original question. Foreign keys with cascading deletes or updates? A -- Andrew Sull

Re: [PERFORM] Need to run CLUSTER to keep performance

2007-11-12 Thread Heikki Linnakangas
Rafael Martinez wrote: DETAIL: 83623 dead row versions cannot be removed yet. Looks like you have a long-running transaction in the background, so VACUUM can't remove all dead tuples. I didn't see that in the vacuum verbose outputs you sent earlier. Is there any backends in "Idle in transac

Re: [PERFORM] Need to run CLUSTER to keep performance

2007-11-12 Thread Rafael Martinez
Rafael Martinez wrote: > > We have more information about this 'problem'. > Sending this just in case it can help Checking all the log files from these vacuum jobs we have been running, we found one that looks difference from the rest, specially on the amount of removed pages. We are send

Re: [PERFORM] Need to run CLUSTER to keep performance

2007-11-12 Thread Rafael Martinez
Tom Lane wrote: > Rafael Martinez <[EMAIL PROTECTED]> writes: >> Heikki Linnakangas wrote: >>> On a small table like that you could run VACUUM every few minutes >>> without much impact on performance. That should keep the table size in >>> check. > >> Ok, we run VACUUM ANALYZE only one time a day,

Re: [PERFORM] PostgreSQL vs MySQL, and FreeBSD

2007-11-12 Thread Erik Jones
On Nov 11, 2007, at 2:17 PM, Joshua D. Drake wrote: Dimitri wrote: Seems to me there is more thread model implementation problem on FreeBSD, and databases just reflecting it... Most of the test I done on Solaris show the same performance level on the same short READ- only queries for MySQL a

Re: [PERFORM] difference between a unique constraint and a unique index ???

2007-11-12 Thread Dave Cramer
On 12-Nov-07, at 9:56 AM, Alvaro Herrera wrote: Dave Cramer wrote: In order to get like queries to use an index with database initialized with a UTF-8 character set I added a unique index to a table with a varchar_pattern_ops This table already had a unique constraint on the column so I d

Re: [PERFORM] difference between a unique constraint and a unique index ???

2007-11-12 Thread Alvaro Herrera
Dave Cramer wrote: > In order to get like queries to use an index with database initialized with > a UTF-8 character set I added a unique index to a table with a > varchar_pattern_ops > > This table already had a unique constraint on the column so I dropped the > unique constraint. > > I can't g

[PERFORM] difference between a unique constraint and a unique index ???

2007-11-12 Thread Dave Cramer
In order to get like queries to use an index with database initialized with a UTF-8 character set I added a unique index to a table with a varchar_pattern_ops This table already had a unique constraint on the column so I dropped the unique constraint. I can't give exact measurements howev

Re: [PERFORM] PostgreSQL vs MySQL, and FreeBSD

2007-11-12 Thread Shane Ambler
Steinar H. Gunderson wrote: On Sun, Nov 11, 2007 at 08:27:02PM +0100, Dimitri wrote: As well to demonstrate OS parallelism it's not so impressive to see 4CPU server results rather 8CPU or 32threaded Niagara... Don't know why they did not present similar performance graphs for these platform, str

Re: [PERFORM] work_mem and shared_buffers

2007-11-12 Thread Cédric Villemain
Bill Moran a écrit : On Fri, 9 Nov 2007 12:08:57 -0600 "Campbell, Lance" <[EMAIL PROTECTED]> wrote: How do you know when you should up the value of work_mem? Just play with the number. Is there a query I could do that would tell me if PostgreSql is performing SQL that could use more memory