Re: [PERFORM] pg_autovacuum not having enough suction ?

2005-03-25 Thread Bruce Momjian
M FULL if table is nearly empty? I don't think autovacuum is every going to be smart enough to recycle during the delete, especially since the rows can't be reused until the transaction completes. One problem with VACUUM FULL would be autovacuum waiting for an exclusiv

Re: [PERFORM] pg_autovacuum not having enough suction ?

2005-03-25 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian writes: > > Tom Lane wrote: > >> I'm not sure if autovacuum could be taught to do that --- it could > >> perhaps launch a vacuum as soon as it notices a large fraction of the > >> table got deleted, but do we really want t

Re: [PERFORM] which dual-CPU hardware/OS is fastest for PostgreSQL?

2005-03-28 Thread Bruce Momjian
ives from caching writes? I asked 3ware this at the Linuxworld Boston show and they said their controller keeps the information in cache until they are sure it is on the platters and not just in the disk cache, but that is far from a 100% reliable report. -- Bruce Momjian| h

Re: [PERFORM] Compressing WAL

2005-04-10 Thread Bruce Momjian
never heard anyone talk about it, but it seems useful. I think compressing the page images written on first page modification since checkpoint would be a big win. Is this a TODO? -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (

Re: [PERFORM] 8.0.1 much slower than 7.4.2?

2005-04-14 Thread Bruce Momjian
#x27; > 2005-04-04 18:05:00 CST PARSELOG: statement: SELECT attnotnull FROM > pg_catalog.pg_attribute WHERE attrelid = $1 AND attnum = $2 > 2005-04-04 18:05:00 CST PARSELOG: statement: SELECT def.adsrc FROM > pg_catalog.pg_class c JOIN pg_catalog.pg_attribute a O

Re: [PERFORM] Compressing WAL

2005-04-18 Thread Bruce Momjian
Added to TODO: * Compress WAL entries [wal] I have also added this email to TODO.detail. --- Simon Riggs wrote: > On Sun, 2005-04-10 at 21:12 -0400, Bruce Momjian wrote: > > Jim C. Nasby wrote: > >

Re: [PERFORM] How to improve db performance with $7K?

2005-04-18 Thread Bruce Momjian
if another shows up, you send that too, and the drive optimizes the grouping at a later time, knowing what the drive is doing, rather queueing in the kernel. -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your l

Re: [PERFORM] How to improve db performance with $7K?

2005-04-18 Thread Bruce Momjian
ommand queueing obviously helps, and I am not sure what else does. -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| N

Re: [PERFORM] How to improve db performance with $7K?

2005-04-19 Thread Bruce Momjian
AFAIK only SCSI >drives/controllers do that for now. What is 'multiple initiators' used for in the real world? -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive,

Re: [PERFORM] How to improve db performance with $7K?

2005-04-19 Thread Bruce Momjian
t part? --- > -Original Message- > From: Bruce Momjian [mailto:[EMAIL PROTECTED] > Sent: Tuesday, April 19, 2005 12:10 PM > To: Mohan, Ross > Cc: pgsql-performance@postgresql.org > Subject: Re: [PERFORM] How to improve db performance with $7K? > > > Mohan

Re: [PERFORM] Opteron vs Xeon (Was: What to do with 6 disks?)

2005-04-20 Thread Bruce Momjian
as PCIexpress, faster > FSB 667MHz, which doesn't match up with AMD's total IO bandwidth, but > much better than previous 6650s). Dell cuts too many corners to be a good server. -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us

Re: [PERFORM] Joel's Performance Issues WAS : Opteron vs Xeon

2005-04-22 Thread Bruce Momjian
uery tool, but it certainly seems encouraging. > > I am going to visit Josh's tests he wanted me to run on the LINUX server. > > Joel Fradkin > > > > > ---(end of broadcast)--- > TIP 7: don't forget to increas

Re: ODBC driver overpopulation (was Re: [PERFORM] Final decision)

2005-04-27 Thread Bruce Momjian
sive because they have a team working on improving the existing driver. I am sure they would want to work together too. -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Robert

Re: [PERFORM] Final decision

2005-04-27 Thread Bruce Momjian
ng us the current > > driver is being replaced. > > Well I don't think anyone knew that the current driver is still being > maintained? We have been looking for someone to take over ODBC and Pervasive agreed to do it, but there wasn't a big announcement about it. I have dis

Re: [PERFORM] [GENERAL] "Hash index" vs. "b-tree index" (PostgreSQL

2005-05-09 Thread Bruce Momjian
ting a warning/notice on hash-index creation is something I've > > suggested in the past -- that would be fine with me. > > Probably not a bad idea. Agreed. -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610)

Re: [PERFORM] Intel SRCS16 SATA raid?

2005-05-11 Thread Bruce Momjian
icator of this. There are a lot more variabilities between these two types of drives than I knew. I recommend it for anyone who is choosing drives for a system. -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your li

Re: [PERFORM] Profiler for PostgreSQL

2005-07-13 Thread Bruce Momjian
of time. Kindly let me know if any of you knows of such a > tool for PostgreSQL. > Agha Asif Raza Sure see log_statement in postgresql.conf. There are a lot of settings in there to control what is logged. -- Bruce Momjian| http://candle.pha.pa

Re: [PERFORM] [PATCHES] COPY FROM performance improvements

2005-07-29 Thread Bruce Momjian
osen. We've had some > apparent driver issues with a client running Emulex on Linux, even using > many different versions of the kernel. Where is the most recent version of the COPY patch? -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us

Re: [PERFORM] Read/Write block sizes

2005-08-24 Thread Bruce Momjian
; > >>>* With 64bit 1GB file chunks is also moot.. Maybe it should be > >>>tuneable too like 100GB without recompiling the code. > >>> > >>>Why recompiling is bad? Most companies that will support Postgres > >>>will support their own binaries a

Re: [PERFORM] PostgreSQL overall design

2005-09-27 Thread Bruce Momjian
ir == NULL) > ereport(ERROR, > (errcode_for_file_access(), > errmsg("could not open directory \"%s\": %m", fromdir))); > > > > -- > with thanks & regards, > S.Gnanavel > Satyam Computer Services Ltd. -- Bruce Momjian|

Re: [PERFORM] Effects of cascading references in foreign keys

2005-10-29 Thread Bruce Momjian
) and RI_FKey_keyequal_upd_fk(). The first is for primary keys (pk), the second for foreign keys (fk). These are called by src/backend/command/triggers.c::AfterTriggerSaveEvent(). The checks prevent the trigger from being registered at all if there is no change in the primary/foreign ke

Re: [PERFORM] Effects of cascading references in foreign keys

2005-10-29 Thread Bruce Momjian
ged, we needn't enqueue the trigger, since the update cannot violate the foreign key. This optimization was previously applied in the RI trigger function, but it is more efficient to avoid firing the trigger altogether. Per recent discussion on pgsql-hackers. -- Bruce

Re: [PERFORM] Effects of cascading references in foreign keys

2005-10-30 Thread Bruce Momjian
big win. Hard to say, perhaps: Prevent referential integrity triggers from firing if referenced columns are not changed by an UPDATE Previously, triggers would fire but do nothing. However, the description seems more complex than it is worth. -- Bruce Momjian

Re: [PERFORM] insert performance for win32

2005-11-04 Thread Bruce Momjian
--- can anyone see > a reason not to? Nope. > I'm inclined to treat this as an outright bug, not just a minor > performance issue, because it implies that a sufficiently long psql > script would probably crash a Windows machine. Agreed. -- Bruce Momjian

Re: [HACKERS] [PERFORM] insert performance for win32

2005-11-04 Thread Bruce Momjian
use it implies that a sufficiently long psql > > script would probably crash a Windows machine. > > Ouch. In light of this, are we *sure* what we've got a is a candidate > for release? Good point. It is something we would fix in a minor release, so it doesn't seem wo

Re: [PERFORM] Hardware/OS recommendations for large databases (

2005-11-22 Thread Bruce Momjian
k "create a thread" approach that could easily bog us down as outlined above. Right now the file system will do read-ahead for a heap scan (but not an index scan), but even then, there is time required to get that kernel block into the PostgreSQL shared buffers, backing up Luke'

Re: [PERFORM] Hardware/OS recommendations for large databases (

2005-11-22 Thread Bruce Momjian
Alan Stange wrote: > Bruce Momjian wrote: > > Right now the file system will do read-ahead for a heap scan (but not an > > index scan), but even then, there is time required to get that kernel > > block into the PostgreSQL shared buffers, backing up Luke's observation &

Re: [PERFORM] TSearch2 vs. Apache Lucene

2005-12-06 Thread Bruce Momjian
her database, plug-ins can't fully function as integrated, but in PostgreSQL, everything is really a plug-in because it is all abstracted. -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is

Re: [PERFORM] TSearch2 vs. Apache Lucene

2005-12-06 Thread Bruce Momjian
Michael Riess wrote: > Bruce Momjian schrieb: > > Oleg Bartunov wrote: > >> Folks, > >> > >> tsearch2 and Lucene are very different search engines, so it'd be unfair > >> comparison. If you need full access to metadata and instant indexing >

Re: [PERFORM] High context switches occurring

2005-12-06 Thread Bruce Momjian
> about meaningless, for two reasons: > > 1. You don't want number of clients (-c) much higher than scaling factor > (-s in the initialization step). The number of rows in the "branches" > table will equal -s, and since every transaction updates one Shoul

Re: [PERFORM] Effects of cascading references in foreign keys

2005-12-06 Thread Bruce Momjian
7;m still > wondering though if the KeysEqual tests in the other RI triggers aren't > now a waste of cycles. > > regards, tom lane > > ---(end of broadcast)--- > TIP 3: Have you che

Re: [PERFORM] How much expensive are row level statistics?

2005-12-16 Thread Bruce Momjian
41% reduction in TPS rate for stats_command_string Woh, 41%. That's just off the charts! What are we doing internally that would cause that? -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your lif

Re: [PERFORM] Should Oracle outperform PostgreSQL on a complex

2005-12-16 Thread Bruce Momjian
-(end of broadcast)--- > TIP 9: In versions below 8.0, the planner will ignore your desire to >choose an index scan if your joining column's datatypes do not >match > -- Bruce Momjian

Re: [PERFORM] Simple Join

2005-12-17 Thread Bruce Momjian
ome other > choices by fooling with the planner enable switches (eg set > enable_seqscan = off) but I doubt you'll find much improvement. There > are too many rows being pulled from ordered_products to make an index > nestloop a good idea. We do have an FAQ item: 4.

Re: [PERFORM] Should Oracle outperform PostgreSQL on a complex

2005-12-17 Thread Bruce Momjian
subselects to create bitmaps of each dimmension table match and merge the bitmaps to perform the join on the fact table. --- Simon Riggs wrote: > On Fri, 2005-12-16 at 23:28 -0500, Bruce Momjian wrote: > > How are s

Re: [PERFORM] Should Oracle outperform PostgreSQL on a complex

2005-12-17 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian writes: > > Added to TODO: > > * Allow star join optimizations > > See my response to Simon for reasons why this doesn't seem like a > particularly good TODO item. Yes, TODO removed. I thought we were waiting for bitmap joins before

Re: [PERFORM] What's the best hardver for PostgreSQL 8.1?

2005-12-27 Thread Bruce Momjian
>>same number of drives with 8 drives or less in a single array beat > >>a RAID 10 with the same number of drives. > > > >Well, those are frankly little arrays, probably on lousy controllers... > Nah. Regardless of controller I can take any RAID 5 an

Re: [PERFORM] Direct I/O issues

2006-11-23 Thread Bruce Momjian
n eye toward the "Determine optimal > fdatasync/fsync, O_SYNC/O_DSYNC options" to-do item, which is what I'd > really like to have. Please send an updated patch for test_fsync.c so we can get it working for 8.2. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp:/

Re: RES: [PERFORM] Priority to a mission critical transaction

2006-11-28 Thread Bruce Momjian
> TIP 7: You can help support the PostgreSQL project by donating at > > http://www.postgresql.org/about/donate -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backu

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-13 Thread Bruce Momjian
hinking just reexecuting a single SELECT over and over again would be a better test of the CPU optimizations. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broad

Re: [PERFORM] URGENT: Out of disk space pg_xlog

2006-12-22 Thread Bruce Momjian
ansactions, since > all the operations are in pg_xlog. Might take a little while to replay > all that log though :-( Amazing that all works. What I did not see is confirmation from the user that the data directory filled up _before_ pg_xlog filled up. -- Bruce Momjian [EMAIL PROTECTED

Re: [PERFORM] URGENT: Out of disk space pg_xlog

2006-12-23 Thread Bruce Momjian
spicious here. Thanks for all of your comments and > suggestions. Even though I haven't figured out the root of the problem > yet, they've been very informative. The bottom line is that we know of now cases where a long-running transaction would delay recycling of the WAL fi

Re: [PERFORM] High update activity, PostgreSQL vs BigDBMS

2007-01-08 Thread Bruce Momjian
I think this is vastly improved from 8.0x > to 8.1 and forward; others might be able to comment whether count() is > now as fast in Postgres as Oracle. The "idiom" to replace count() was ^^ BigDBMS == Oracle. ;-) -- Bruce Momjian [E

Re: [PERFORM] stats collector process high CPU utilization

2007-03-02 Thread Bruce Momjian
> TIP 1: if posting/reading through Usenet, please send an appropriate >subscribe-nomail command to [EMAIL PROTECTED] so that your >message can get through to the mailing list cleanly -- Bruce Momjian <[EMAIL PROTECTED]> http://momjian.us

Re: [PERFORM] Estimate the size of the SQL file generated by pg_dump utility

2007-03-05 Thread Bruce Momjian
error please delete it and notify the sender > immediately. Before opening any mail and > attachments please check them for viruses and defect. > > ----------- -- Bruc

Re: [PERFORM] Estimate the size of the SQL file generated by pg_dump utility

2007-03-05 Thread Bruce Momjian
Bricklen Anderson wrote: > Bruce Momjian wrote: > > Ravindran G-TLS,Chennai. wrote: > >> Note: Please bear with us for the disclaimer because it is automated in > >> the exchange server. > >> Regards, > >> Ravi > > > > FYI, we are getting

Re: [PERFORM] SCSI vs SATA

2007-04-04 Thread Bruce Momjian
I is technically superior or inferior to SCSI as > an interface, in any place that counts. You should probably read this to learn the difference between desktop and enterprise-level drives: http://www.seagate.com/content/docs/pdf/whitepaper/D2c_More_than_Interface_ATA_vs_SCSI_042003.pdf

Re: [PERFORM] SCSI vs SATA

2007-04-04 Thread Bruce Momjian
Joshua D. Drake wrote: > Bruce Momjian wrote: > > [EMAIL PROTECTED] wrote: > >> On Wed, Apr 04, 2007 at 08:50:44AM -0700, Joshua D. Drake wrote: > >>>> difference. OTOH, the SCSI discs were way less reliable than the SATA > >>>> discs, that might hav

Re: [PERFORM] SCSI vs SATA

2007-04-06 Thread Bruce Momjian
n environment of 55/60C, I'm pretty sure one of them was > lying. We are silently replacing the failed drives with drives from > manufacturer S. > > Based on experience I think that on average server drives are more > reliable than consumer grade drives, and can take more punishm

Re: [PERFORM] general PG network slowness (possible cure) (repost)

2007-05-28 Thread Bruce Momjian
ar TCP_CORK, or 200ms have passed with no send. > > Peter > > ---(end of broadcast)--- > TIP 7: You can help support the PostgreSQL project by donating at > > http://www.postgresql.org/about/donate -- Bruce Momjian <

Re: [PERFORM] Semantics of PostgreSQL Server Log Stats

2007-07-31 Thread Bruce Momjian
> hit rate = 0.00% > ! Direct blocks: 0 read, 0 written > > What each element of this report mean? This outlines the shared/local buffer I/O and file system I/O performed. -- Bruce Momjian <[EMAIL PROTECTED]> http://momjian.us E

Re: [PERFORM] When/if to Reindex

2007-09-13 Thread Bruce Momjian
egards, tom lane > > ---(end of broadcast)------- > TIP 1: if posting/reading through Usenet, please send an appropriate >subscribe-nomail command to [EMAIL PROTECTED] so that your >message can

Re: [PERFORM] Effects of cascading references in foreign keys

2007-09-26 Thread Bruce Momjian
;t they need to account for the special case mentioned in the > comment in triggers.c, that the RI check must still be done if we are > looking at a row updated by the same transaction that created it? > > regards, tom lane > >

Re: [PERFORM] mysql to postgresql, performance questions

2010-03-31 Thread Bruce Momjian
avour of actually admiting that some well known filesystems and > saftware raid systems have had trouble with their write barriers? I thought the issue was that many file systems do not issue the drive ATAPI flush command, and I suppose drives are allowed not to flush on write if they hon

Re: [PERFORM] temp table "on commit delete rows": transaction overhead

2010-03-31 Thread Bruce Momjian
operations at commit (basically an "ftruncate"). > I don't recall any operations at transaction start for such tables, > but there may be some. I think one of the problems is that we do the truncate even if the table has not be touched by the query, which is poor behavior. --

Re: [PERFORM] temp table "on commit delete rows": transaction overhead

2010-04-05 Thread Bruce Momjian
Artiom Makarov wrote: > 2010/4/1 Bruce Momjian : > > > I think one of the problems is that we do the truncate even if the table > > has not be touched by the query, which is poor behavior. > > Thank you for the support. > Will be this problem registered? I have it o

Re: *** PROBABLY SPAM *** [PERFORM] Does the psql executable support a "fetch many" approach when dumping large queries to stdout?

2010-04-12 Thread Bruce Momjian
t. > > I think it grabs the whole result set to calculate the display column > widths. I think there is an option to tweak this but don't remember which, > have a look at the psql commands (\?), formatting section. See the FETCH_COUNT variable mentioned in the psql manual pa

Re: [PERFORM] Occasional giant spikes in CPU load

2010-04-14 Thread Bruce Momjian
either get a battery-backed RAID controller or risk losing a few transaction with synchronous_commit = off. If you already have a battery-backed RAID controller, there is little benefit to turning synchronous_commit off, and some major downsides (possible data loss). -- Bruce Momjian http://m

Re: [PERFORM] Autovacuum in postgres.

2010-05-27 Thread Bruce Momjian
on one database every hour. As a result, on my database > autovacuum is run once in 5 hours. Is there any way to make it run it every > hour. What settings did you change to make it run every hour? Also, it will only vacuum tables that need vacuuming. What version of Postgres are you using?

Re: [PERFORM] Autovacuum in postgres.

2010-05-27 Thread Bruce Momjian
wasn't very capable. --- > On Thu, May 27, 2010 at 8:03 PM, Bruce Momjian wrote: > > > venu madhav wrote: > > > Hi All, > > >In my application we are using postgres which runs on an embedded > > > box. I have con

Re: [PERFORM] Weird XFS WAL problem

2010-06-04 Thread Bruce Momjian
D controller card writes to the drive, but the data isn't on the platers, how does it know when it can discard that information from the BBU RAID cache? -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + None of us is going to

Re: [PERFORM] Weird XFS WAL problem

2010-06-04 Thread Bruce Momjian
Kevin Grittner wrote: > Bruce Momjian wrote: > > > On another topic, I am a little unclear on how things behave when > > the drive is write-back. If the RAID controller card writes to the > > drive, but the data isn't on the platers, how does it know when it >

Re: [PERFORM] Weird XFS WAL problem

2010-06-04 Thread Bruce Momjian
Kevin Grittner wrote: > Bruce Momjian wrote: > > Kevin Grittner wrote: > > >> The controller waits for the drive to tell it that it has made it > >> to the platter before it discards it. What made you think > >> otherwise? > > > > Becau

Re: [PERFORM] Weird XFS WAL problem

2010-06-04 Thread Bruce Momjian
Kevin Grittner wrote: > Bruce Momjian wrote: > > Kevin Grittner wrote: > > >> Any decent RAID controller will ensure that the drives themselves > >> aren't using write-back caching. When we've mentioned write-back > >> versus write-th

Re: [PERFORM] PostgreSQL as a local in-memory cache

2010-06-23 Thread Bruce Momjian
stgresql.org/pgsql-performance/2010-06/msg00300.php -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + None of us is going to be here forever. + -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To ma

Re: [PERFORM] PostgreSQL as a local in-memory cache

2010-06-23 Thread Bruce Momjian
hancements possible if durability is not required. The patch also documents that synchronous_commit = false has potential committed transaction loss from a database crash (as well as an OS crash). -- Bruce Momjian http://momjian

Re: [PERFORM] PostgreSQL as a local in-memory cache

2010-06-23 Thread Bruce Momjian
Pavel Stehule wrote: > 2010/6/23 Bruce Momjian : > > Tom Lane wrote: > >> Dimitri Fontaine writes: > >> > Josh Berkus writes: > >> >> a) Eliminate WAL logging entirely > > > > If we elimiate WAL logging, that means a reinstall is req

Re: [PERFORM] PostgreSQL as a local in-memory cache

2010-06-23 Thread Bruce Momjian
Robert Haas wrote: > On Wed, Jun 23, 2010 at 3:37 PM, Bruce Momjian wrote: > > Tom Lane wrote: > >> Dimitri Fontaine writes: > >> > Josh Berkus writes: > >> >> a) Eliminate WAL logging entirely > > > > If we elimiate WAL logging, tha

Re: [PERFORM] PostgreSQL as a local in-memory cache

2010-06-23 Thread Bruce Momjian
h Robert that a way to not log content > updates for individual user tables is likely to be much more useful in > practice. OK, TODO removed. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + None of us is going to be h

Re: [PERFORM] ALTER Table and CLUSTER does adding a new column rewrite clustered? (8.4.3)

2010-06-24 Thread Bruce Momjian
that > cause a rewrite but does not say what the state is after the rewrite > with respect to CLUSTER, FILLFACTOR, and index bloat. I have added a documentation patch to mention the indexes are rebuilt; applied patch attached. The gory details can be found in src/backend/commands/ta

Re: [PERFORM] cpu bound postgresql setup.

2010-06-28 Thread Bruce Momjian
hing went back to its normal state. Did you get a db server log message suggesting in increasing that setting? I hope so. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + None of us is going to be here forever. + -- Sent via pgsq

Re: [PERFORM] PostgreSQL as a local in-memory cache

2010-06-28 Thread Bruce Momjian
Bruce Momjian wrote: > Tom Lane wrote: > > Dimitri Fontaine writes: > > > Josh Berkus writes: > > >> a) Eliminate WAL logging entirely > > >> b) Eliminate checkpointing > > >> c) Turn off the background writer > > >> d) H

Re: [PERFORM] PostgreSQL as a local in-memory cache

2010-06-29 Thread Bruce Momjian
Robert Haas wrote: > On Mon, Jun 28, 2010 at 5:57 PM, Bruce Momjian wrote: > >> The patch also documents that synchronous_commit = false has > >> potential committed transaction loss from a database crash (as well as > >> an OS crash). > > Is this actually t

Re: [PERFORM] cpu bound postgresql setup.

2010-06-29 Thread Bruce Momjian
Benjamin Krajmalnik wrote: > > -Original Message- > > From: Bruce Momjian [mailto:br...@momjian.us] > > Sent: Monday, June 28, 2010 3:45 PM > > To: Benjamin Krajmalnik > > Cc: Rajesh Kumar Mallah; Kevin Grittner; pgsql- > > performa...@postgresql.org

Re: [PERFORM] PostgreSQL as a local in-memory cache

2010-06-29 Thread Bruce Momjian
Robert Haas wrote: > On Tue, Jun 29, 2010 at 9:32 AM, Bruce Momjian wrote: > > Robert Haas wrote: > >> On Mon, Jun 28, 2010 at 5:57 PM, Bruce Momjian wrote: > >> >> The patch also documents that synchronous_commit = false has > >> >> potential comm

Re: [PERFORM] PostgreSQL as a local in-memory cache

2010-06-29 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian writes: > >>> I asked on IRC and was told it is true, and looking at the C code it > >>> looks true. ?What synchronous_commit = false does is to delay writing > >>> the wal buffers to disk and fsyncing them, not just fsync, w

Re: [PERFORM] PostgreSQL as a local in-memory cache

2010-06-29 Thread Bruce Momjian
Jignesh Shah wrote: > On Tue, Jun 29, 2010 at 2:45 PM, Bruce Momjian wrote: > > Tom Lane wrote: > >> Bruce Momjian writes: > >> >>> I asked on IRC and was told it is true, and looking at the C code it > >> >>> looks true. ?What synchronous_co

Re: [PERFORM] PostgreSQL as a local in-memory cache

2010-06-30 Thread Bruce Momjian
Brad Nicholson wrote: > > > > Ah, very good point. ?I have added a C comment to clarify why this is > > > > the current behavior; ?attached and applied. > > > > > > > > -- > > > > ?Bruce Momjian ? ? ? ? ?http://momjian.us > > >

Re: [PERFORM] Analysis Function

2010-06-30 Thread Bruce Momjian
e creation of timestamps using parameters * http://archives.postgresql.org/pgsql-performance/2010-06/msg00232.php -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + None of us is going to be here forever. + -- Sent via pgsql-pe

Re: [PERFORM] No hash join across partitioned tables?

2010-07-01 Thread Bruce Momjian
er periodic manual analyze commands > > in this situation, or > > (c) do nothing. > > > Thoughts? > > The objections to (a) are that it might result in excessive ANALYZE work > if not done intelligently, and that we haven't got a patch ready anyway. > I would

Re: [PERFORM] No hash join across partitioned tables?

2010-07-02 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian writes: > > Tom Lane wrote: > >> I would have liked to get to this for 9.0 but I feel it's a bit late > >> now. > > > What do we want to do about the above issue? > > TODO item. Added to TODO: Have autoanaly

Re: [PERFORM] No hash join across partitioned tables?

2010-07-02 Thread Bruce Momjian
Robert Haas wrote: > On Fri, Jul 2, 2010 at 4:58 PM, Tom Lane wrote: > > Bruce Momjian writes: > >> I am surprised there is no documentation update requirement for this. > > > > Somebody put something about it in the docs a few days ago, IIR

Re: [PERFORM] Weird XFS WAL problem

2010-07-07 Thread Bruce Momjian
Then the "blow out the RAID cache when barriers are on" > behavior will only show up during checkpoint fsyncs, which will make > things a lot better (albeit still not ideal). Great information! I have added the attached documentation patch to explain the write-barrier/BBU interact

Re: [PERFORM] Need help in performance tuning.

2010-07-12 Thread Bruce Momjian
g in > performance on workloads where queries are big and expensive but there > are relatively few of them running at a time. Agreed. We certainly are going to have to go in that direction someday. We have TODO items for these. -- Bruce Momjian http://momjian.us EnterpriseDB

Re: [PERFORM] Testing Sandforce SSD

2010-08-11 Thread Bruce Momjian
entation: http://www.postgresql.org/docs/9.0/static/wal-reliability.html -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-performance mailing list (pgsql-perform

Re: [PERFORM] Questions on query planner, join types, and work_mem

2010-08-11 Thread Bruce Momjian
ybe refine > the CPU cost model more, as all-in-RAM cases get more common. This confused me. If we are assuing the data is in effective_cache_size, why are we adding sequential/random page cost to the query cost routines? -- Bruce Momjian http://momjian.us EnterpriseDB

Re: [PERFORM] Advice configuring ServeRAID 8k for performance

2010-08-13 Thread Bruce Momjian
Linux ext3 file system. Journaled file systems do improve boot speed after a crash. Should this be changed? -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + --

Re: [PERFORM] Advice configuring ServeRAID 8k for performance

2010-08-16 Thread Bruce Momjian
requirement no matter how your array is set up underneath. Do we need to document this? -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-performance mailing list (p

Re: [PERFORM] BBU Cache vs. spindles

2010-10-20 Thread Bruce Momjian
something else? > > A BBU is, what, $100 or so? Adding one seems a no-brainer to me. > Dedicated WAL spindles are nice and all, but they're still spinning > media. Raid card cache is wy faster, and while it's best at bursty > writes, it sounds like bursty writes are precisely

Re: [PERFORM] Slow count(*) again...

2010-10-20 Thread Bruce Momjian
nd costs. If the row number is large, just round it to the nearest thousand and return it to the application as a count --- this is what Google does for searches (just try it). If the row count/cost are low, run the query and return an exact count. -- Bruce Momjian http://momjian.us Enter

Re: [PERFORM] New wiki page on write reliability

2010-10-21 Thread Bruce Momjian
dded into there once > he's finished tweaking it. My presentation is done and is now on the wiki too: http://momjian.us/main/writings/pgsql/hw_selection.pdf -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's

Re: [PERFORM] BBU Cache vs. spindles

2010-10-21 Thread Bruce Momjian
Scott Marlowe wrote: > On Wed, Oct 20, 2010 at 8:25 PM, Joshua D. Drake > wrote: > > On Wed, 2010-10-20 at 22:13 -0400, Bruce Momjian wrote: > >> Ben Chobot wrote: > >> > On Oct 7, 2010, at 4:38 PM, Steve Crawford wrote: > >> > > >> >

Re: [PERFORM] BBU Cache vs. spindles

2010-10-21 Thread Bruce Momjian
Kevin Grittner wrote: > Bruce Momjian wrote: > > > With a BBU you can turn off full_page_writes > > My understanding is that that is not without risk. What happens if > the WAL is written, there is a commit, but the data page has not yet > been written to the controll

Re: [PERFORM] How does PG know if data is in memory?

2010-10-21 Thread Bruce Momjian
; we add more knobs which are this hard to tune correctly, we would > risk inundation with complaints from people to tried to use it and > made things worse. Agreed. Here is a blog entry that explains some of the tradeoffs of adding knobs: http://momjian.us/main/blogs/pgblog/2009.h

Re: [PERFORM] BBU Cache vs. spindles

2010-10-21 Thread Bruce Momjian
OS cache > so that the window might occasionally be rather large? If the write fails to the controller, the page is not flushed and PG does not continue. If the write fails, the fsync never happens, and hence PG stops. -- Bruce Momjian http://momjian.us EnterpriseDB

Re: [PERFORM] BBU Cache vs. spindles

2010-10-21 Thread Bruce Momjian
Kevin Grittner wrote: > Bruce Momjian wrote: > > > If the write fails to the controller, the page is not flushed and > > PG does not continue. If the write fails, the fsync never > > happens, and hence PG stops. > > PG stops? This case at issue is when the OS c

Re: [PERFORM] BBU Cache vs. spindles

2010-10-21 Thread Bruce Momjian
Kevin Grittner wrote: > Bruce Momjian wrote: > > > I assume we send a full 8k to the controller, and a failure during > > that write is not registered as a write. > > On what do you base that assumption? I assume that we send a full > 8K to the OS cache, and

Re: [PERFORM] BBU Cache vs. spindles

2010-11-30 Thread Bruce Momjian
where I was assuming that an 8k-block based file system would write to the disk atomically in 8k segments, which of course it cannot. My bet is that even if you write to the kernel in 8k pages, and have an 8k file system, the disk is still accessed via 512-byte blocks, even with a BBU. -- Bruce

Re: [PERFORM] BBU Cache vs. spindles

2010-11-30 Thread Bruce Momjian
patch, if you are using ZFS then you can turn off full-page writes, so full-page writes are still useful. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent vi

Re: [PERFORM] BBU Cache vs. spindles

2010-11-30 Thread Bruce Momjian
> clarification. Given that, it seems to me there are only two situations > where full_page_writes is safe to turn off: > > 1) The operating system block size is exactly the same database block > size, and all writes are guaranteed to be atomic to that block size. Is that true

  1   2   3   4   5   >