Re: [ADMIN] proper tuning for restoring from pg_dump in 8.3.7

2010-07-14 Thread David Kerr
On Wed, Jul 14, 2010 at 05:30:17PM -0400, Burgholzer, Robert (DEQ) wrote: - >. The options I suggested be turned off just for the load process - > ... - > not safe if the other databases are in the same PostgreSQL Cluster - - They are, so I won't --- thanks for the heads up on that too. - - Jus

Re: [ADMIN] proper tuning for restoring from pg_dump in 8.3.7

2010-07-14 Thread Burgholzer, Robert (DEQ)
Kevin, Thanks a ton for your responses (and forwarding to the list). As you gathered, I had two issues - long load time, performance hit on other databases on the same machine. I simply didn't conceptualize my problem properly, so I wasn't able to ask for the right help... so: > and *those* are

Re: [ADMIN] proper tuning for restoring from pg_dump in 8.3.7

2010-07-14 Thread Kevin Grittner
[Please be careful to keep the list copied.] "Burgholzer, Robert (DEQ)" wrote: >> Exactly what are you seeing which causes you to say that? > > An httpd request that is filled in fractions of a second now takes > 10-30 seconds to complete - this being a page that calls a php > script, and ne

Re: [ADMIN] proper tuning for restoring from pg_dump in 8.3.7

2010-07-14 Thread Kevin Grittner
"Burgholzer, Robert (DEQ)" wrote: > my performance in most definitely suffering Exactly what are you seeing which causes you to say that? (Not in terms of what you think might be the *cause* of the performance problem, but what the *symptoms* are.) -Kevin -- Sent via pgsql-admin mailing l

Re: [ADMIN] proper tuning for restoring from pg_dump in 8.3.7

2010-07-14 Thread Kris Deugau
Burgholzer, Robert (DEQ) wrote: OK, thanks to multiple folks for letting me know that I was looking at the wrong "top" metric. That said, my performance in most definitely suffering -- does this "swap" number seem excessive (looks like ~100 G to me): Swap: 10232k total Total swap isn't an

Re: [ADMIN] proper tuning for restoring from pg_dump in 8.3.7

2010-07-14 Thread Kevin Grittner
"Burgholzer, Robert (DEQ)" wrote: > does this "swap" number seem excessive (looks like ~100 G to me): > Swap: 10232k total That's probably how much disk space you have set aside for swapping. What matters is how much of that is *used*. For example: kgri...@plato:/home/ccsa> free -m

Re: [ADMIN] proper tuning for restoring from pg_dump in 8.3.7

2010-07-14 Thread Burgholzer, Robert (DEQ)
OK, thanks to multiple folks for letting me know that I was looking at the wrong "top" metric. That said, my performance in most definitely suffering -- does this "swap" number seem excessive (looks like ~100 G to me): Swap: 10232k total > Cached data is not a problem. Don't worry about that

Re: [ADMIN] proper tuning for restoring from pg_dump in 8.3.7

2010-07-14 Thread Tom Lane
"Burgholzer, Robert (DEQ)" writes: > That said, the time to restore is explainable, but is there something in > my tuning that is causing all of my memory to be eaten? That's normal behavior. A working Linux/Unix system *should* have near zero free memory. If it doesn't, either the filesystem c

Re: [ADMIN] proper tuning for restoring from pg_dump in 8.3.7

2010-07-14 Thread Kevin Grittner
"Burgholzer, Robert (DEQ)" wrote: > We seem to have some undiagnosed issue whereby opening and closing > large files on the system leaves a lot in the cache Cached data is not a problem. Don't worry about that. -Kevin -- Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org) To ma

Re: [ADMIN] proper tuning for restoring from pg_dump in 8.3.7

2010-07-14 Thread Joshua D. Drake
On Wed, 2010-07-14 at 13:07 -0400, Burgholzer, Robert (DEQ) wrote: > That said, the time to restore is explainable, but is there something in > my tuning that is causing all of my memory to be eaten? Please don't top post. > > We seem to have some undiagnosed issue whereby opening and closing la

Re: [ADMIN] proper tuning for restoring from pg_dump in 8.3.7

2010-07-14 Thread Kevin Grittner
"Burgholzer, Robert (DEQ)" wrote: > cat dumpfile | psql db_name Try: psql -1 -f dumpfile db_name > The trouble is that my system free memory (according to top) goes > to about 60M, What do you get from free or vmstat? (Oon't trust top too far on memory usage reporting.) > temp_buffers =

Re: [ADMIN] proper tuning for restoring from pg_dump in 8.3.7

2010-07-14 Thread David Kerr
- -Original Message- - From: Joshua D. Drake [mailto:j...@commandprompt.com] - Sent: Wednesday, July 14, 2010 12:58 PM - To: Burgholzer, Robert (DEQ) - Cc: pgsql-admin@postgresql.org - Subject: Re: [ADMIN] proper tuning for restoring from pg_dump in 8.3.7 - - On Wed, 2010-07-14 at 12:33 -0

Re: [ADMIN] proper tuning for restoring from pg_dump in 8.3.7

2010-07-14 Thread Burgholzer, Robert (DEQ)
- From: Joshua D. Drake [mailto:j...@commandprompt.com] Sent: Wednesday, July 14, 2010 12:58 PM To: Burgholzer, Robert (DEQ) Cc: pgsql-admin@postgresql.org Subject: Re: [ADMIN] proper tuning for restoring from pg_dump in 8.3.7 On Wed, 2010-07-14 at 12:33 -0400, Burgholzer, Robert (DEQ) wrote: > I

Re: [ADMIN] proper tuning for restoring from pg_dump in 8.3.7

2010-07-14 Thread Joshua D. Drake
On Wed, 2010-07-14 at 12:33 -0400, Burgholzer, Robert (DEQ) wrote: > I am restoring a fairly sizable database from a pg_dump file (COPY FROM > STDIN style of data) -- the pg_dump file is ~40G. > > My system has 4 cores, and 12G of RAM. I drop, then recreate the > database, and I do this restore v

[ADMIN] proper tuning for restoring from pg_dump in 8.3.7

2010-07-14 Thread Burgholzer, Robert (DEQ)
I am restoring a fairly sizable database from a pg_dump file (COPY FROM STDIN style of data) -- the pg_dump file is ~40G. My system has 4 cores, and 12G of RAM. I drop, then recreate the database, and I do this restore via a: cat dumpfile | psql db_name. The trouble is that my system free memory