Interesting space issue
Hi, I have an interesting problem. Over the weekend our syslog forwarder went beserk generating over 300,000 messages to about 6 people. This morning our three new Cyrus systems went belly up, (yes that is a technical term), actually the master daemon seemed to eventually freeze up. The only real error msgs I can find are these: Mar 10 00:18:16 postoffice8 lmtpd[27393]: [ID 729713 local6.error] DBERROR: opening /opt/cyrus/mailboxes.db: Not enough space Mar 10 08:04:46 postoffice8 pop3d[2183]: [ID 729713 local6.error] DBERROR: opening /opt/cyrus/mailboxes.db: Not enough space Mar 10 08:12:58 postoffice8 imapd[2489]: [ID 729713 local6.error] DBERROR: opening /opt/cyrus/mailboxes.db: Not enough space Mar 10 08:14:05 postoffice8 imapd[2731]: [ID 729713 local6.error] DBERROR: opening /opt/cyrus/mailboxes.db: Not enough space Mar 10 08:27:59 postoffice8 imapd[3951]: [ID 729713 local6.error] DBERROR: opening /opt/cyrus/mailboxes.db: Not enough space Now I'm been running older versions of Cyrus (1.5.19) for years at 300,000 messages a day with no trouble. I don't believe space is really an issue, here is a df -k from one of the systems. Filesystemkbytesused avail capacity Mounted on /dev/md/dsk/d0 1984564 904568 102046047%/ /proc 0 0 0 0%/proc fd 0 0 0 0%/dev/fd mnttab 0 0 0 0%/etc/mnttab /dev/md/dsk/d1962573 255248 64957129%/var swap 28642528 32 28642496 1%/var/run swap 28655440 12944 28642496 1%/tmp /dev/md/dsk/d4 50408148134 4982272 1%/users /dev/md/dsk/d3 5040814 452439 453796710%/opt /dev/vx/dsk/po8_dg01/logvol01 5160542 115891 4993046 3%/logs /dev/vx/dsk/po8_dg01/mqueuevol01 103218844986 10213680 1%/mqueue /dev/vx/dsk/po8_dg01/cyrus_data_vol01 41287586 126222 40748489 1%/opt/cyrus /dev/vx/dsk/po8_dg01/sendmailvol01 41287586 603402 40271309 2%/opt/sendmail_vol /dev/vx/dsk/po8_dg01/cyrus_app_vol01 41287586 147526 40727185 1%/opt/cyrus_vol /dev/vx/dsk/po8_dg01/spoolvol01 103218991 679107 101507695 1%/var/spool/mail swap 28642640 144 28642496 1%/opt/cyrus/proc /dev/vx/dsk/po8_dg01/appvol01 20643785 54397 20382951 1%/applications This is all with Cyrus 2.1.11 on a V880 with 32GB of memory with Solaris 8 and, Sendmail 8.12.8. Anyone seen this before? Thanks. Jim
Re: Interesting space issue
On Mon, Mar 10, 2003 at 03:13:30PM -0500, Jim Howell wrote: Now I'm been running older versions of Cyrus (1.5.19) for years at 300,000 messages a day with no trouble. I don't believe space is really an issue, here is a df -k from one of the systems. Did you by any chance run out of inodes on the filesystem? df -i (YMMV) should help you discover that. Gr, Nils.
Re: Interesting space issue
Date: Mon, 10 Mar 2003 15:13:30 -0500 From: Jim Howell [EMAIL PROTECTED] [...] This is all with Cyrus 2.1.11 on a V880 with 32GB of memory with Solaris 8 and, Sendmail 8.12.8. Anyone seen this before? Thanks. What sort of database backend are you using? (cyradm version should help) Are you sure there weren't many many different copies of mailboxes.db sticking around that were taking up space? Enough inodes? Collecting truss output from a process giving this error message could be helpful. Larry
Re: Interesting space issue
Hi, Good idea, but no I didn't run out of inodes. Jim At 09:30 PM 3/10/2003 +0100, Nils Vogels wrote: On Mon, Mar 10, 2003 at 03:13:30PM -0500, Jim Howell wrote: Now I'm been running older versions of Cyrus (1.5.19) for years at 300,000 messages a day with no trouble. I don't believe space is really an issue, here is a df -k from one of the systems. Did you by any chance run out of inodes on the filesystem? df -i (YMMV) should help you discover that. Gr, Nils.
Re: Interesting space issue
At 03:39 PM 3/10/2003 -0500, Lawrence Greenfield wrote: Date: Mon, 10 Mar 2003 15:13:30 -0500 From: Jim Howell [EMAIL PROTECTED] [...] This is all with Cyrus 2.1.11 on a V880 with 32GB of memory with Solaris 8 and, Sendmail 8.12.8. Anyone seen this before? Thanks. What sort of database backend are you using? (cyradm version should help) BerkleyDB 4.1 Are you sure there weren't many many different copies of mailboxes.db sticking around that were taking up space? I don't know, why would there be more than one? Enough inodes? Yes Collecting truss output from a process giving this error message could be helpful. I wasn't able to unfortunately. Those processes were gone and things were messed up by time I knew it. Larry
Re: Interesting space issue
On Mon, Mar 10, 2003 at 03:13:30PM -0500, Jim Howell wrote: Hi, I have an interesting problem. Over the weekend our syslog forwarder went beserk generating over 300,000 messages to about 6 people. This morning our three new Cyrus systems went belly up, (yes that is a technical term), actually the master daemon seemed to eventually freeze up. The only real error msgs I can find are these: Mar 10 00:18:16 postoffice8 lmtpd[27393]: [ID 729713 local6.error] DBERROR: opening /opt/cyrus/mailboxes.db: Not enough space Mar 10 08:04:46 postoffice8 pop3d[2183]: [ID 729713 local6.error] DBERROR: opening /opt/cyrus/mailboxes.db: Not enough space Mar 10 08:12:58 postoffice8 imapd[2489]: [ID 729713 local6.error] DBERROR: opening /opt/cyrus/mailboxes.db: Not enough space Mar 10 08:14:05 postoffice8 imapd[2731]: [ID 729713 local6.error] DBERROR: opening /opt/cyrus/mailboxes.db: Not enough space Mar 10 08:27:59 postoffice8 imapd[3951]: [ID 729713 local6.error] DBERROR: opening /opt/cyrus/mailboxes.db: Not enough space ... * Exceeed some sort of operating system user/group quota limit ? * Exceeded 2Gb on some file ? * Tried fscking /opt ? -- Alain Williams #include std_disclaimer.h
