Hello,
Perhaps this is a problem with SIGPIPE from the fflush(ci->tx)
on line 158 of pop3.c (in pop3_handle_connection()). I'll have
to dig up Ryan's patch and look at it. The current signal handler
should call _exit(1) on SIGPIPE, which is pretty hard to screw up :),
but if yours doesn't do that, perhaps you're experiencing the same
behavior noted in ChildSigHandler:
serverchild.c: line 122-ish
/*
* For some reason i have not yet determined the process starts eating up
* all CPU time when trying to disconnect.
* For now: just bail out :-)
*/
_exit(1);
Jesse
---- Original Message ----
From: Jason Burfield <[email protected]>
To: [email protected]
Subject: Re: [Dbmail] High Load Average and dbmail-pop3d ?
Sent: 19 May 2003 19:41:49 -0400
> Jesse...
>
> Some are using APOP, some are not.
>
> And, it finally crapped out with a trace_level of 5 as well...here is
> the relevant log info from that session:
>
> ---
> May 19 18:47:27 vault dbmail/pop3d[25871]: PerformChildTask(): waiting
> for connection
> May 19 18:57:25 vault dbmail/pop3d[25871]: PerformChildTask(): incoming
> connection from [141.155.157.55]
> May 19 18:57:25 vault dbmail/pop3d[25871]: PerformChildTask(): client
> info init complete, calling client handler
> May 19 18:57:27 vault dbmail/pop3d[25871]: pop3(): incoming buffer:
> [APOP [EMAIL PROTECTED] e9624c82649cacdf5b9905391b07d4d3]
> May 19 18:57:27 vault dbmail/pop3d[25871]: pop3(): command issued :cmd
> [APOP], value [EMAIL PROTECTED] e9624c82649cacdf5b9905391b07d4d3]
> May 19 18:57:27 vault dbmail/pop3d[25871]: pop3(): command looked up as
> commandtype 11
> May 19 18:57:27 vault dbmail/pop3d[25871]: __auth_query(): executing
> query [SELECT user_idnr FROM users WHERE userid='[EMAIL PROTECTED]']
> May 19 18:57:27 vault dbmail/pop3d[25871]: __auth_query(): executing
> query [SELECT encryption_type FROM users WHERE user_idnr = 78]
> May 19 18:57:27 vault dbmail/pop3d[25871]: pop3(): APOP auth, username
> [EMAIL PROTECTED], md5_hash [EMAIL PROTECTED]
> May 19 18:57:27 vault dbmail/pop3d[25871]: __auth_query(): executing
> query [SELECT passwd,user_idnr FROM users WHERE
> userid="[EMAIL PROTECTED]"]
> May 19 18:57:27 vault dbmail/pop3d[25871]: auth_md5_validate():
> apop_stamp=[<[EMAIL PROTECTED]>], userpw=[pass]
> May 19 18:57:27 vault dbmail/pop3d[25871]: auth_md5_validate():
> checkstring for md5 [<[EMAIL PROTECTED]>pass] -> result
> [e9624c82649cacdf5b9905391b07d4d3]
> May 19 18:57:27 vault dbmail/pop3d[25871]: auth_md5_validate():
> validating md5_apop_we=[e9624c82649cacdf5b9905391b07d4d3]
> md5_apop_he=[<[EMAIL PROTECTED]>pass]
> May 19 18:57:27 vault dbmail/pop3d[25871]: auth_md5_validate(): user
> [EMAIL PROTECTED] could not be validated
> ---
>
>
> That is everything for that PID for that specific user, which is what
> caused it to take off again.
>
> Up to that point that same PID was handling connections with no
> problems.
>
> That same process actually handled this same user logging in several
> times before, with no problems.
>
> I have had it happen with a few different users, but this specific user
> seems to have it happen a lot. He has his password set correctly and all
> other info is set correctly as well. He can log in and get mail several
> times, then all of a sudden this happens.
>
> We are using usernames in the form of '[EMAIL PROTECTED]' but I wouldn't
> think this would make a difference to dbmail would it?
>
> Is this helping at all? Any additional info that would help I can get...
>
> Again, many thanks!
>
> -- Jason
>
>
> On Mon, 2003-05-19 at 17:36, Jesse Norell wrote:
> > Hello Jason,
> >
> > An earlier post showed auth_md5_validate() in the logs, which is
> > what does APOP validation - this time, no mention. Can you forward
> > the logs at trace level 5 (when the problem "should" happen). Are
> > you using APOP or not? (Maybe after running through APOP command
> > a variable is reused for USER/PASS without initialization or
> > something.) Also, I assume the user(s) this happens for does have
> > the password saved correctly, as this doesn't happen 100% of the time?
> > Or do they manually enter it (ie. sometime it'll be right, sometimes
> > not)?
> >
> > Jesse
> >
> > ---- Original Message ----
> > From: Jason Burfield <[email protected]>
> > To: [email protected]
> > Subject: Re: [Dbmail] High Load Average and dbmail-pop3d ?
> > Sent: 19 May 2003 17:24:21 -0400
> >
> > > Well...I have found one thing that is interesting...
> > >
> > > With the trace_level set to 5, I do NOT have any problems at all.
> > > However, at 0,1,2,3 or 4 I have problems.
> > >
> > > Does this make any sense to anyone?
> > >
> > > The most I can get in a log file is at trace_level=4 and it is just
> > > this:
> > >
> > > ---
> > > May 19 17:18:51 vault dbmail/pop3d[24695]: PerformChildTask(): incoming
> > > connection from [xx.xxx.xxx.xx]
> > > May 29 17:18:51 vault dbmail/pop3d[24695]: user [EMAIL PROTECTED] could
> > > not be validated
> > > ---
> > >
> > > At this point it seems to go into a loop or something, using up all
> > > processor ticks and NOT returning an error to the user.
> > >
> > > When I go in and issue 'killall dbmail-pop3d' is when the user gets a
> > > 'password incorrect' message on his/her screen.
> > >
> > > So far it will not freak out when it's at trace_level 5, but then the
> > > logs are huge.
> > >
> > > I'm still baffled.
> > >
> > > Thanks...
> > >
> > > -- Jason
> > >
> > >
> > > On Sun, 2003-05-18 at 13:34, Micah Stevens wrote:
> > > > go through the logs and find where he's getting an invalid PW error, if
> > > > you
> > > > have the trace level set to 5 it should provide some usefull
> > > > information
> > > > there. Post that bit if you can maybe others can help.
> > > >
> > > > I would restart the pop server and purposely log in with an incorrect
> > > > password
> > > > too to see what happens.
> > > >
> > > > I know I have at least 1 OSX user and this isn't a problem, I'm using a
> > > > CVS
> > > > from February though.
> > > >
> > > > -Micah
> > > >
> > > >
> > > > On Saturday 17 May 2003 3:30 pm, Jason Burfield wrote:
> > > > > Another quick follow-up...
> > > > >
> > > > > My server just freaked and ran up to a load average over 32.00. That
> > > > > is out
> > > > > of control! Wow!
> > > > >
> > > > > There were 30 dbmail-pop3d processes all carrying a load. I killed the
> > > > > server and then with a copy of the log file, traced each of the pids
> > > > > back.
> > > > >
> > > > > Every single one of them ended with:
> > > > >
> > > > > --
> > > > > May 17 17:31:21 vault dbmail/pop3d[24001]: auth_md5_validate(): user
> > > > > [EMAIL PROTECTED] could not be validated
> > > > > --
> > > > >
> > > > > Obviously, the address has been changed.
> > > > >
> > > > > I found that this user is using the Apple Mail program on OS X to
> > > > > check
> > > > > mail. And, he is checking mail every one minute. (yeah, I know...but
> > > > > what
> > > > > can I do?)
> > > > >
> > > > > It seems that most of his connections work fine, but then, for
> > > > > whatever
> > > > > reason, he gets an error about his password being incorrect.
> > > > >
> > > > > This seems to be when the server takes off and creates a huge load.
> > > > >
> > > > > I'm really getting concerned now because when the server is left
> > > > > alone for
> > > > > a few hours, it gets to be out of control.
> > > > >
> > > > > If there is any additional info anyone would like to see, please let
> > > > > me
> > > > > know...I'd like to help figure this one out!
> > > > >
> > > > > Thanks again...
> > > > >
> > > > > -- Jason
> > > > >
> > > > > on 5/17/03 2:23 AM, Eelco van Beek - IC&S at [EMAIL PROTECTED] wrote:
> > > > > > The most important thing pop & imap share is the connection handling
> > > > > > code. Roel is checking that out for bugs.
> > > > > >
> > > > > > Best regards,
> > > > > >
> > > > > > eelco
> > > > > >
> > > > > > On zaterdag, mei 17, 2003, at 04:22 Europe/Amsterdam, Chris Nolan
> > > > > > wrote:
> > > > > >> Hi all,
> > > > > >>
> > > > > >> I am having an extremely similar problem with dbmail-imapd (running
> > > > > >> dbmail 1.1). And, to simplify things somewhat, I can tell you a bit
> > > > > >> about the problem.
> > > > > >>
> > > > > >> 1. MySQL is *not* the cause of this problem. I am using MySQL
> > > > > >> 4.0.12
> > > > > >> and killing dbmail-imapd very quickly brings the load average down.
> > > > > >> 2. The load average grows linearly, so one day after having
> > > > > >> dbmail-imapd running, the load average will read "1.00 1.00 1.00",
> > > > > >> the
> > > > > >> second day "2.00 2.00 2.00" the third day "3.00 3.00 3.00" etc.
> > > > > >> 3. I am running Mandrake 9.0 (one of the last glibc 2.2-based
> > > > > >> distros).
> > > > > >> 4. DBMail very happily maxes out MySQL's connection pool. I'm
> > > > > >> betting
> > > > > >> this has something to do with it (trying to connect from any other
> > > > > >> machine with any MySQL client or another DBMail instance will
> > > > > >> always
> > > > > >> fail after dbmail-imapd has been running for a few days).
> > > > > >> 5. I have many indexes and foreign key constraints in place as have
> > > > > >> been suggested on this list.
> > > > > >> 6. I only have 8 people using this server - 6 with IMAP clients
> > > > > >> and 2
> > > > > >> with POP3 clients.
> > > > > >> 7. SquirrelMail's performance degrades as the load average goes up,
> > > > > >> and massively. It takes 10 seconds to render each frame due to the
> > > > > >> IMAP side of things being so heavily choked.
> > > > > >>
> > > > > >> If anyone has any other questions or any suggestion as to where the
> > > > > >> problem is in the source (I'm a 4th year Software Engineering
> > > > > >> student
> > > > > >> at a uni where C is the primary language and Unix is the primary
> > > > > >> platform) I'll have a go at fixing it.
> > > > > >>
> > > > > >> Interestingly, Roel said that they only get this problem every few
> > > > > >> months with some of their customer sites....
> > > > > >>
> > > > > >> Chris
> > > > > >>
> > > > > >> lou wrote:
> > > > > >>> In some email I received from Jason Burfield <[EMAIL PROTECTED]>
> > > > > >>> on
> > > > > >>> 16 May 2003 14:54:16
> > > > > >>> -0400, wrote:
> > > > > >>>
> > > > > >>> Hi jason,
> > > > > >>> did you try to attach a debugger to the process?
> > > > > >>> in any way if you can get us dump image off the process that
> > > > > >>> would be
> > > > > >>> great.
> > > > > >>>
> > > > > >>> kill -ABRT <pid>
> > > > > >>> should make the process exit with a core dump.
> > > > > >>>
> > > > > >>> anyway, might be an infinite loop or something?
> > > > > >>>
> > > > > >>> cheers.
> > > > > >>>
> > > > > >>>> It's dbmail-pop3d.
> > > > > >>>>
> > > > > >>>> Currently TOP is showing this:
> > > > > >>>>
> > > > > >>>> PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME
> > > > > >>>> COMMAND
> > > > > >>>> 21088 root 15 0 564 564 500 S 99.9 0.0 181:42
> > > > > >>>> dbmail-pop3d
> > > > > >>>>
> > > > > >>>> The database currently has all proper indexes...and it does not
> > > > > >>>> seem
> > > > > >>>> to
> > > > > >>>> be a MySQL issue at all.
> > > > > >>>>
> > > > > >>>> And, the load average continues to grow the longer that
> > > > > >>>> dbmail-pop3d
> > > > > >>>> continues to run.
> > > > > >>>>
> > > > > >>>> It is currently at 11.57
> > > > > >>>>
> > > > > >>>> This is the highest I personally have EVER seen one of my
> > > > > >>>> machines
> > > > > >>>> go.
> > > > > >>>>
> > > > > >>>> Also, when I check the maillog it does not seem to be doing
> > > > > >>>> anything
> > > > > >>>> out
> > > > > >>>> of the ordinary. It almost looks like one child process has just
> > > > > >>>> run
> > > > > >>>> off
> > > > > >>>> and refuses to stop...but the log is not showing tons of stuff
> > > > > >>>> going
> > > > > >>>> on.
> > > > > >>>>
> > > > > >>>> I'm totally freaked and confused now!
> > > > > >>>>
> > > > > >>>> Thanks!
> > > > > >>>>
> > > > > >>>> -- Jason
> > > > > >>>>
> > > > > >>>> On Fri, 2003-05-16 at 14:39, Aaron Stone wrote:
> > > > > >>>>> When you run top, do you find that most of the CPU time is being
> > > > > >>>>> taken by
> > > > > >>>>> dbmail-pop3d itself, or by mysqld? In the latter case, you may
> > > > > >>>>> do
> > > > > >>>>> well to
> > > > > >>>>> tune the database a bit in my.cnf, and to add the indices
> > > > > >>>>> posted to
> > > > > >>>>> the
> > > > > >>>>> list a couple of times. If it's pop3d.. umm... sounds bad, post
> > > > > >>>>> more info!
> > > > > >>>>>
> > > > > >>>>> Aaron
> > > > > >>>>>
> > > > > >>>>> On 16 May 2003, Jason Burfield wrote:
> > > > > >>>>>> I have searched through back messages and found one thread with
> > > > > >>>>>> someone
> > > > > >>>>>> having huge load averages caused by dbmail-smtpd.
> > > > > >>>>>>
> > > > > >>>>>> Well...I'm getting HUGE load averages with dbmail-pop3d.
> > > > > >>>>>>
> > > > > >>>>>> For whatever reason the load average on my box is jumping up to
> > > > > >>>>>> well
> > > > > >>>>>> over 7.00 after dbmail-pop3d has been running for a few
> > > > > >>>>>> minutes.
> > > > > >>>>>>
> > > > > >>>>>> This is on a dual 2.4ghz Xeon with 4 gigs of RAM...so I can't
> > > > > >>>>>> for
> > > > > >>>>>> the
> > > > > >>>>>> life of me, imagine what would cause that type of load.
> > > > > >>>>>>
> > > > > >>>>>> This mail server went up last night...so in the past 24 hours
> > > > > >>>>>> it
> > > > > >>>>>> has
> > > > > >>>>>> accepted something around 2000 messages. We do have lots of
> > > > > >>>>>> clients
> > > > > >>>>>> checking mail frequently, around every 5 minutes, however, I
> > > > > >>>>>> wouldn't
> > > > > >>>>>> think that would cause this type of load.
> > > > > >>>>>>
> > > > > >>>>>> I'm running dmail-1.1, MySQL and Postfix on Linux. (RH 8)
> > > > > >>>>>>
> > > > > >>>>>> As an example, at this very moment the box has a load average
> > > > > >>>>>> of
> > > > > >>>>>> 5.62.
> > > > > >>>>>> There are currently 6 clients connected to the pop server.
> > > > > >>>>>>
> > > > > >>>>>> Does anyone have any idea what may be causing this? We don't
> > > > > >>>>>> have
> > > > > >>>>>> that
> > > > > >>>>>> many clients, only about 300 email clients...what type of load
> > > > > >>>>>> should be
> > > > > >>>>>> expected from that type of base?
> > > > > >>>>>>
> > > > > >>>>>> Any help is greatly appreciated.
> > > > > >>>>>>
> > > > > >>>>>> Thanks!
> > > > > >>>>>>
> > > > > >>>>>> -- Jason
> > > > > >>>>>>
> > > > > >>>>>> -- Jason Burfield
> > > > > >>>>>> -- [EMAIL PROTECTED]
> > >
> > > _______________________________________________
> > > Dbmail mailing list
> > > [email protected]
> > > https://mailman.fastxs.nl/mailman/listinfo/dbmail
> > >
> > -- End Original Message --
> >
> >
> > --
> > Jesse Norell
> > jesse (at) kci.net
> >
> > _______________________________________________
> > Dbmail mailing list
> > [email protected]
> > https://mailman.fastxs.nl/mailman/listinfo/dbmail
> >
>
> _______________________________________________
> Dbmail mailing list
> [email protected]
> https://mailman.fastxs.nl/mailman/listinfo/dbmail
>
-- End Original Message --
--
Jesse Norell
jesse (at) kci.net
