Ok, taking a quick look at it again, this is what I gather:
The time is spent in db_send_message_lines when sending a RETR reply
from pop3.c calling:
fprintf ((FILE *)stream,"+OK %llu octets\r\n",((struct message
*)tmpelement->data)->msize);
return db_send_message_lines ((void *)stream, ((struct message
*)tmpelement->data)->realmessageid,-2, 0);
Looking in dbpgsql.c, in db_send_message_lines, we allocate a
1MB buffer (ie. READ_BLOCK_SIZE from db.h * 2) and read the db
return data into it one char at a time, handling \n -> \r\n
conversion and escaping periods, and then flush that buffer at every
3000 characters, then memset the whole 1MB buffer back to null,
in this block:
if (rowlength%3000==0) /* purge buffer at every 3000 bytes */
{
/* fprintf ((FILE *)fstream,"%s",buffer); */
/* fflush ((FILE *)fstream); */
fwrite (buffer, sizeof(char), strlen(buffer), (FILE
*)fstream);
/* cleanup the buffer */
memset (buffer, '\0', (READ_BLOCK_SIZE*2));
}
I don't know how long it takes to write to 1MB of memory, but
that seems a bit inefficient. I might see if I can use a smaller
working buffer and see if that fixes it. Any avid C coders see
that as an immediate alarm condition?
jn
---- Original Message ----
From: Jesse Norell <[email protected]>
To: [email protected]
Subject: Re: [Dbmail] 100% cpu usage with dbmail-pop3d
Sent: Mon, 6 Oct 2003 12:11:37 -0600 (MDT)
>
> Hello,
>
> Yes, that's exactly what it acts like. I looked over the code a
> bit once and nothing stood out as definitely wrong, but I really
> don't do a lot of C network coding, so it could easily be.
>
>
> ---- Original Message ----
> From: Bret Baptist <[email protected]>
> To: [email protected]
> Subject: Re: [Dbmail] 100% cpu usage with dbmail-pop3d
> Sent: Mon, 6 Oct 2003 12:48:16 -0500
>
> > Sarge is testing.
> >
> > Well I think we can rule out the postgres libs, since I am running MySQL.
> > I
> > am thinking it must be the SQL code they are using, instead of idling while
> > waiting for the client to download, it is just taking all the cpu.
> >
> > Just silly.
> >
> >
> > Bret.
> >
> >
> >
> > On Monday 06 October 2003 12:36 pm, Jesse Norell wrote:
> > > Hello,
> > >
> > > Take comfort in the fact that you're not alone ... but I don't
> > > have a solution for you. This is exactly what we're seeing after
> > > having upgraded to debian unstable (is unstable now sarge?). I
> > > alluded to this briefly in a reply the other day - I think the
> > > problem has to either be in the linux kernel or the postgres libs,
> > > not dbmail itsself, but maybe would be possible to work around
> > > it if it could be nailed down. We've not had much time to look
> > > into it, just have enough pop3 front ends that we can put it off
> > > for a while. It has persisted through a few kernel updates (and
> > > tried smp kernel, same effects), and at least one postgres lib
> > > update also...
> > >
> > >
> > >
> > > ---- Original Message ----
> > > From: Bret Baptist <[email protected]>
> > > To: [email protected]
> > > Subject: [Dbmail] 100% cpu usage with dbmail-pop3d
> > > Sent: Mon, 6 Oct 2003 12:24:21 -0500
> > >
> > > > I recently switched over to a separate dbmail front end server and
> > > > database server. When I set the new front end server up I used debian
> > > > sarge. Since that time I have been having huge problems with load being
> > > > caused by dbmail-pop3d. It appears to attempt to use all the available
> > > > cpu while transferring messages to the client. I had never seen this
> > > > before when running the front end and database on the same server.
> > > >
> > > > Does anyone have any clues? This is where the logs are at when it is
> > > > taking 100% cpu:
> > > > Oct 6 12:02:52 destiny dbmail/pop3d[20773]: pop3():RETR command,
> > > > retrieving message
> > > > Oct 6 12:02:52 destiny dbmail/pop3d[20773]: pop3(): RETR command,
> > > > selecting message
> > > > Oct 6 12:02:52 destiny dbmail/pop3d[20773]: db_send_message_lines():
> > > > request for [-2] lines
> > > > Oct 6 12:02:52 destiny dbmail/pop3d[20773]: db_send_message_lines():
> > > > executing query [SELECT * FROM messageblks WHERE message_idnr=2458388
> > > > ORDER BY messageblk_idnr ASC]
> > > > Oct 6 12:02:52 destiny dbmail/pop3d[20773]: db_query(): executing
> > > > [SELECT * FROM messageblks WHERE message_idnr=2458388 ORDER BY
> > > > messageblk_idnr ASC] Oct 6 12:02:52 destiny dbmail/pop3d[20773]:
> > > > db_send_message_lines(): sending [-2] lines from message [2458388]
> > > > Oct 6 12:02:52 destiny dbmail/pop3d[20773]: db_send_message_lines():
> > > > getting nextblock [1]
> > > >
> > > > I am running the latest CVS version available, and still seeing this
> > > > issue.
> > > >
> > > >
> > > > --
> > > > Bret Baptist
> > > > Systems and Technical Support Specialist
> > > > [EMAIL PROTECTED]
> > > > Internet Exposure, Inc.
> > > > http://www.iexposure.com
> > > >
> > > > (612)676-1946 x17
> > > > Web Development-Web Marketing-ISP Services
> > > > ------------------------------------------
> > > >
> > > >
> > > > Today is the tomorrow you worried about yesterday.
> > > >
> > > > _______________________________________________
> > > > Dbmail mailing list
> > > > [email protected]
> > > > https://mailman.fastxs.nl/mailman/listinfo/dbmail
> > >
> > > -- End Original Message --
> > >
> > >
> > > --
> > > Jesse Norell
> > > jesse (at) kci.net
> > >
> > >
> > > _______________________________________________
> > > Dbmail mailing list
> > > [email protected]
> > > https://mailman.fastxs.nl/mailman/listinfo/dbmail
> >
> > --
> > Bret Baptist
> > Systems and Technical Support Specialist
> > [EMAIL PROTECTED]
> > Internet Exposure, Inc.
> > http://www.iexposure.com
> >
> > (612)676-1946 x17
> > Web Development-Web Marketing-ISP Services
> > ------------------------------------------
> >
> >
> > Today is the tomorrow you worried about yesterday.
> >
> > _______________________________________________
> > Dbmail mailing list
> > [email protected]
> > https://mailman.fastxs.nl/mailman/listinfo/dbmail
> >
> -- End Original Message --
>
>
> --
> Jesse Norell
> jesse (at) kci.net
>
>
> _______________________________________________
> Dbmail mailing list
> [email protected]
> https://mailman.fastxs.nl/mailman/listinfo/dbmail
>
-- End Original Message --
--
Jesse Norell
jesse (at) kci.net
