[ADMIN] Is my database now too big?

2007-10-06 Thread Darren Reed
I'm starting to wonder if my database has finally grown too big for my computer. But, rather than peform badly, I see a number of "bad things" happen: * pg_dump fails to run, causing an "out of memory" error (I don't understand why this happens, but it does); * restarting postgres across a reb

Re: [ADMIN] Is my database now too big?

2007-10-07 Thread Darren Reed
On Sat, 06 Oct 2007 23:05:23 -0700, "Joshua D. Drake" <[EMAIL PROTECTED]> said: > Darren Reed wrote: > > I'm starting to wonder if my database has finally grown too big for my > > computer. > > > > But, rather than peform badly, I see a number of

Re: [ADMIN] Is my database now too big?

2007-10-07 Thread Darren Reed
Scott Marlowe wrote: On 10/7/07, Darren Reed <[EMAIL PROTECTED]> wrote: > Scott Marlowe wrote: > > ... > > > > Any reasonably modern version of pgsql should simply stop accepting > > requests rather than suffering loss due to txid wraparound.So,I can > &g

[ADMIN] postgres: will not recognise existing tables

2007-10-08 Thread Darren Reed
I've run into this problem twice in the last few days. postgresql is running, shutdown, reboot, restart, and tables are missing. The shutdown is clean: LOG: unexpected EOF on client connection LOG: received smart shutdown request LOG: shutting down LOG: database system is shut down LOG: dat

Re: [ADMIN] Is my database now too big?

2007-10-08 Thread Darren Reed
Scott Marlowe wrote: ... Any reasonably modern version of pgsql should simply stop accepting requests rather than suffering loss due to txid wraparound.So,I can think of two possibilities here. Bad hardware or operator error. Assuming you've checked out your machine thoroughly for bad hardware,

Re: [ADMIN] Is my database now too big?

2007-10-14 Thread Darren Reed
Scott Marlowe wrote: On 10/7/07, Darren Reed <[EMAIL PROTECTED]> wrote: > Scott Marlowe wrote: > > On 10/7/07, Darren Reed <[EMAIL PROTECTED]> wrote: > > > Scott Marlowe wrote: > A few days ago I did: > pg_dumpall > foo > What I was doing yesterday was

Re: [ADMIN] Is my database now too big?

2007-10-14 Thread Darren Reed
Darren Reed wrote: Scott Marlowe wrote: On 10/7/07, Darren Reed <[EMAIL PROTECTED]> wrote: > Scott Marlowe wrote: > > On 10/7/07, Darren Reed <[EMAIL PROTECTED]> wrote: > > > Scott Marlowe wrote: > A few days ago I did: > pg_dumpall > foo > What I

Re: [ADMIN] Is my database now too big?

2007-10-15 Thread Darren Reed
So having upgraded to 8.1.10, I thought I'd try out a few things... # /usr/pkg/bin/pg_dumpall -- -- PostgreSQL database cluster dump -- \connect postgres -- -- Roles -- CREATE ROLE postgres; ALTER ROLE postgres WITH SUPERUSER INHERIT CREATEROLE CREATEDB LOGIN; CREATE ROLE root; ALTER ROLE roo

Re: [ADMIN] Is my database now too big?

2007-10-15 Thread Darren Reed
Darren Reed wrote: Scott Marlowe wrote: On 10/7/07, Darren Reed <[EMAIL PROTECTED]> wrote: > Scott Marlowe wrote: > > On 10/7/07, Darren Reed <[EMAIL PROTECTED]> wrote: > > > Scott Marlowe wrote: > A few days ago I did: > pg_dumpall > foo > What I

Re: [ADMIN] Is my database now too big?

2007-10-15 Thread Darren Reed
Tom Lane wrote: Darren Reed <[EMAIL PROTECTED]> writes: > # /usr/pkg/bin/psql -U postgres template1 > psql: FATAL: out of memory > DETAIL: Failed on request of size 20. I'm starting to think there is something very broken about your machine :-(. Have you run any hardwa

Re: [ADMIN] Is my database now too big?

2007-10-15 Thread Darren Reed
Tom Lane wrote: Darren Reed <[EMAIL PROTECTED]> writes: > # /usr/pkg/bin/psql -U postgres template1 > psql: FATAL: out of memory > DETAIL: Failed on request of size 20. I'm starting to think there is something very broken about your machine :-(. Have you run any hardwa

Re: [ADMIN] Is my database now too big?

2007-10-15 Thread Darren Reed
Tom Lane wrote: Darren Reed <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> Anyway, the above error should have also produced a map of per-context >> memory usage in the postmaster log (ie, postmaster stderr). If you >> could show us that, it might be revealing.

Re: [ADMIN] Is my database now too big?

2007-10-15 Thread Darren Reed
Scott Marlowe wrote: ... Again, I'm kinda shooting in the dark here as you reveal more and more what you are doing a little at a time. A test case that can invoke this failure would be most useful. After seeing this today: ERROR: duplicate key violates unique constraint "ers_pkey" ERROR:

Re: [ADMIN] Is my database now too big?

2007-10-15 Thread Darren Reed
Scott Marlowe wrote: On 10/15/07, Darren Reed <[EMAIL PROTECTED]> wrote: > Scott Marlowe wrote: > > ... > > > > Again, I'm kinda shooting in the dark here as you reveal more and more > > what you are doing a little at a time. A test case that can invoke

Re: [ADMIN] Is my database now too big?

2007-10-17 Thread Darren Reed
For better or worse, it seems to be behaving itself again for a while... There was, however, one change to my procedure and that was to drop the triggers/functions when restoring the data (using copy into). Cheers, Darren ---(end of broadcast)---

Re: [ADMIN] Is my database now too big?

2007-10-17 Thread Darren Reed
Darren Reed wrote: For better or worse, it seems to be behaving itself again for a while... There was, however, one change to my procedure and that was to drop the triggers/functions when restoring the data (using copy into). Perhaps I spoke too soon... # /usr/pkg/bin/pg_dumpall -O -U

[ADMIN] Why dos select work when pg_dump fails?

2007-11-29 Thread Darren Reed
For whatever reason, I'm back dealing with corrupted tables but curiously while this fails: pg_dump -U postgres -a -t table -f /data/table.dump this works: psql -U postgres -A -o /data/table.dump -c "SELECT * FROM table;" Graned the format of the data is different, but regardless, the data is

[ADMIN] Power outage borked things (8.1.10)...

2008-02-20 Thread Darren Reed
Starting up postgres, I get the log contents below. Is it really as bad as it suggests, namely that I need to recover from backup? The pg_clog directory has the following: # ls -al pg_clog/ total 1216 drwx-- 2 postgres wheel 512 Jan 17 09:09 ./ drwx-- 10 postgres wheel 512 F

Re: [ADMIN] Power outage borked things (8.1.10)...

2008-02-20 Thread Darren Reed
Tom Lane wrote: Darren Reed <[EMAIL PROTECTED]> writes: > Starting up postgres, I get the log contents below. > Is it really as bad as it suggests, namely that I > need to recover from backup? Probably :-( I've started a new db in parallel with the old data and I'm

Re: [ADMIN] Power outage borked things (8.1.10)...

2008-02-20 Thread Darren Reed
Tom Lane wrote: Darren Reed <[EMAIL PROTECTED]> writes: > # su postgres -c "/usr/pkg/bin/pg_resetxlog -f /data/db" > Transaction log reset > PANIC: could not access status of transaction 4570963 > DETAIL: could not read from file "pg_clog/0004" at offset

[ADMIN] How to reindex when unable to open relation?

2008-02-25 Thread Darren Reed
Inserts into my table started generating an error message (see below), and there was a log message that suggested I reindex. postgres seems to start and stop ok, but no operations on the tables seem to work. l# /usr/pkg/bin/pg_dump -s -U postgres pg_dump: failed sanity check, parent table OID

Re: [ADMIN] How to reindex when unable to open relation?

2008-02-25 Thread Darren Reed
Tom Lane wrote: ... There's no info here to suggest exactly what went wrong; Darren, do you want to fess up to any hardware problems, tripped-over power cords, or such? What PG version is this anyway? This is 8.2.6... and I'm starting to believe the hardware is just borked, with all of the

[ADMIN] Retrieving data from the data files.

2008-02-26 Thread Darren Reed
For whatever reason, I seem to be plagued with issues... Whether they're hardware or operating system or a bad combinationI'm not sure yetand I find myself needing to recover from postgres not wanting to play all too often. In looking at the database data files, I notice that in (for exam

Re: [ADMIN] How to reindex when unable to open relation?

2008-02-26 Thread Darren Reed
Shoaib Mir wrote: On Tue, Feb 26, 2008 at 5:13 PM, Darren Reed <[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>> wrote: Inserts into my table started generating an error message (see below), and there was a log message that suggested I reindex. postgres seems to start

[ADMIN] Retrieving data from the data files.

2008-02-26 Thread Darren Reed
For whatever reason, I seem to be plagued with issues... Whether they're hardware or operating system or a bad combinationI'm not sure yetand I find myself needing to recover from postgres not wanting to play all too often. In looking at the database data files, I notice that in (for exam

Re: [ADMIN] How to reindex when unable to open relation?

2008-02-26 Thread Darren Reed
Tom Lane wrote: Darren Reed <[EMAIL PROTECTED]> writes: > Hmmm, from the logfile, signal 11, so it core dumped... > Of course the default built binary is -O2... > (gdb) where > #0 0x082114b8 in RelationCacheInitializePhase2 () > #1 0x0821fa67 in InitPostgres () > #2 0x

Re: [ADMIN] How to reindex when unable to open relation?

2008-02-26 Thread Darren Reed
Tom Lane wrote: Darren Reed <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> Hm, if you are really lucky, this is because of a corrupt >> pg_internal.init file. > How do I know which files those are? find $PGDATA -name pg_internal.init

Re: [ADMIN] How to reindex when unable to open relation?

2008-02-26 Thread Darren Reed
Tom Lane wrote: Darren Reed <[EMAIL PROTECTED]> writes: >>> How do I know which files those are? >> >> find $PGDATA -name pg_internal.init > Doesn't exist. Sucks to be you, then :-( I'm curious though exactly where the failure is, because there

[ADMIN] postgres 8.2.6 core dump when initialising.

2008-04-13 Thread Darren Reed
Some minutes after doing a live reindex of all my tables, I started to get these messages: ERROR: could not open relation with OID 16461 ...for every table(!) So I shutdown... LOG: received smart shutdown request LOG: shutting down LOG: database system is shut down And reboot LOG: dat

Re: [ADMIN] postgres 8.2.6 core dump when initialising.

2008-04-14 Thread Darren Reed
Tom Lane wrote: Darren Reed <[EMAIL PROTECTED]> writes: > Program terminated with signal 11, Segmentation fault. > #0 0x0829845c in RelationCacheInitializePhase2 () at relcache.c:2400 > 2400LOAD_CRIT_INDEX(TriggerRelidNameIndexId); > (gdb) where &

[ADMIN] Some additions to pg_filedump...

2008-04-21 Thread Darren Reed
I did a bit of hacking on pg_filedump to make it slightly more useful...but it wasn't clear where or to whom the patches should be sent, so I'm sending an email to pgsql-admins 'cause they're the kind of folks that probably care about them... The changes I added are two command line options: -F t

Re: [ADMIN] Some additions to pg_filedump...

2008-04-23 Thread Darren Reed
On Mon, 21 Apr 2008 23:19:48 +0300, "Devrim GÜNDÜZ" <[EMAIL PROTECTED]> said: > Hi, > > On Mon, 2008-04-21 at 13:51 +0200, Darren Reed wrote: > > I did a bit of hacking on pg_filedump to make it slightly more > > useful...but it wasn't clear where

Re: [ADMIN] Some additions to pg_filedump...

2008-04-23 Thread Darren Reed
On Wed, 23 Apr 2008 09:20:49 -0400, "Alvaro Herrera" <[EMAIL PROTECTED]> said: > Darren Reed wrote: > > > I saw this, but when I went in search of activity, there was nothing. > > > > Look in: > > http://sources.redhat.com/ml/rhdb/ > >

Re: [ADMIN] Some additions to pg_filedump...

2008-05-01 Thread Darren Reed
Alvaro Herrera wrote: Darren Reed wrote: I saw this, but when I went in search of activity, there was nothing. Look in: http://sources.redhat.com/ml/rhdb/ http://sources.redhat.com/ml/rhdb-announce/ http://sources.redhat.com/ml/rhdb-cvs/ All of the archives for 2008 are empty... It

Re: [ADMIN] Bus error in postgres 8.3

2008-05-02 Thread Darren Reed
Filip Svendsen wrote: On Tue, Apr 29, 2008 at 8:01 AM, Tom Lane <[EMAIL PROTECTED]> wrote: "Peter Koczan" <[EMAIL PROTECTED]> writes: > I'm going to try to force a crash on my test server to see if I can > find out anything more. Does anyone know how to get useful debugging > info at the t

[ADMIN] postgres doesn't detect other end of the unix socket is gone...

2008-05-07 Thread Darren Reed
It would appear that sometimes, when a disconnect a unix stream (ie perl DBI) from postgres in a forceful manner (^C), postgres doesn't appear to notice and keeps processing the transaction rather than aborting it. As an example, I currently see the following open files: postgres postgres 28780