Re: patch submission?
On Wed, 23 Mar 2005, Clifton Royston wrote: > Isn't this almost precisely what the "-F" flag or "fast-update" > config file option does?? I believe that it is, though I had been left with the impression that this feature requires (or perhaps implies?) that "server mode" also be used, and that this was not suitable in our environment. On Wed, 23 Mar 2005, Greg Earle wrote: > Jeez. Where were you with this patch a year ago, when I needed it? Using Qpopper-3.something, but acheiving performance milestones with other parts of the system and different software packages. :-) The POP server started showing up as a performance "problem" only about a month or so ago. First I upgraded our installation to 4.0.6b3, to get a feel for how much it would improve things by itself, then I started looking for ways to reduce its I/O impact without moving into a POP-only situation (because we're not a POP-only service). > Thanks anyway for sending it in though ... I'm glad to do it. Hopefully, since my performance patch isn't helpful to you, the lastlog patch might still be ... -- -- Sylvain Robitaille [EMAIL PROTECTED] Systems analyst Concordia University Instructional & Information TechnologyMontreal, Quebec, Canada --
Re: patch submission?
On Mar 23, 2005, at 2:27 PM, Sylvain Robitaille wrote: Appended below my signature is what actually amounts to two separate patches against Qpopper-4.0.6b3: The first to reduce the amount of mail spool copying that takes place in "normal" (non-server) mode, and the second to have the POP server update the system "lastlog" file when the user is authenticated. Jeez. Where were you with this patch a year ago, when I needed it? :-P (I've been migrating people off of an old Sun SPARCserver 20/71 mail server running Qpopper 4.0.5 to a SunFire V210. Most of the people with big spool files - that were killing the SS20 - have been moved to the new V210 already, so my need for this isn't anywhere near what it was a year ago. Thanks anyway for sending it in though ... ) - Greg
Re: patch submission?
On Wed, Mar 23, 2005 at 05:27:13PM -0500, Sylvain Robitaille wrote: > Earlier, I wrote: > > > What I'm wondering is, is this an appropriate place to submit that > > patch? Are the Qpopper developpers known to use this mailing list? > > Are other people interested in my patch? > > > Joe Maimon replied: > > > Yes > ... > For the first patch, pop_dropcopy.c and pop_updt.c are changed as > follows: > > In pop_dropcopy.c, link the mail spool to the temp spool, acquire a > lock on the mail spool, unlink and recreate the mail spool, and release > the lock. If the link fails, (either because the temp spool is on a > different file system than the mail spool, or because the temp spool > already exists), fall back to the original code that copies the mail > spool into the temp spool. > > In pop_updt.c, if no updates are pending (no messages were read, > requiring Status: or X-UIDL: header updates, and no no messages were > marked to be deleted), after the lock has been acquired on the mail > spool, and any new messages have been appended to the temp spool, > rename the temp spool to mail spool (overwriting the existing mail > spool whose contents have already been copied), rather than copying > the contents back. Isn't this almost precisely what the "-F" flag or "fast-update" config file option does?? See the Qpopper 4.0 Manual PDF, p.33 > The second patch permits us to use the system lastlog file to monitor > which accounts are used regularly, or more to the point, which ones > haven't been used in any way (interactively, POP, IMAP, FTP, etc.; I've > made the same mod to IMAP and FTP servers) in some extended period of > time. I added a function to pop_pass.c to update the lastlog file, then > added a call to that after the user authentication succeeds. This sounds like a nice idea. -- Clifton -- Clifton Royston -- [EMAIL PROTECTED] Tiki Technologies Lead Programmer/Software Architect "I'm gonna tell my son to grow up pretty as the grass is green And whip-smart as the English Channel's wide..." -- 'Whip-Smart', Liz Phair
Re: patch submission?
Earlier, I wrote:
> What I'm wondering is, is this an appropriate place to submit that
> patch? Are the Qpopper developpers known to use this mailing list?
> Are other people interested in my patch?
Joe Maimon replied:
> Yes
Ok, so here goes ...
Appended below my signature is what actually amounts to two separate
patches against Qpopper-4.0.6b3: The first to reduce the amount of mail
spool copying that takes place in "normal" (non-server) mode, and the
second to have the POP server update the system "lastlog" file when the
user is authenticated.
For the first patch, pop_dropcopy.c and pop_updt.c are changed as
follows:
In pop_dropcopy.c, link the mail spool to the temp spool, acquire a
lock on the mail spool, unlink and recreate the mail spool, and release
the lock. If the link fails, (either because the temp spool is on a
different file system than the mail spool, or because the temp spool
already exists), fall back to the original code that copies the mail
spool into the temp spool.
In pop_updt.c, if no updates are pending (no messages were read,
requiring Status: or X-UIDL: header updates, and no no messages were
marked to be deleted), after the lock has been acquired on the mail
spool, and any new messages have been appended to the temp spool,
rename the temp spool to mail spool (overwriting the existing mail
spool whose contents have already been copied), rather than copying
the contents back.
This significantly reduces I/O on the system, particular in the case
where users who leave their mail on the server check frequently for
new mail. In our case, it has made the difference between a system
that had been otherwise rendered unusable (user's do use this system
for other purposes than POP, including an IMAP based webmail interface
and extensive filtering with Procmail in at least some cases), and a
system whose response has returned to what we consider normal for the
capabilities of this hardware.
The second patch permits us to use the system lastlog file to monitor
which accounts are used regularly, or more to the point, which ones
haven't been used in any way (interactively, POP, IMAP, FTP, etc.; I've
made the same mod to IMAP and FTP servers) in some extended period of
time. I added a function to pop_pass.c to update the lastlog file, then
added a call to that after the user authentication succeeds.
Note that some of the line numbers affected in pop_pass.c will be
different than a stock source tree, as I've removed from the diff some
site-specific changes we also make to that file.
We've been using the lastlog patch with various versions of the Qpopper
server for years without any trouble. The reduced I/O patch has been in
place for nearly two weeks, (though it did undergo some development
during the first week or so), on a system with approximately 20,000
users, (though only 1656 of them have used the POP service in the past
two weeks). Of those users that do use the POP service, enough of them
have their client software configured to check mail frequently enough
that it was rendering a 1 GHz dual Alpha system (with 10GB RAM, but
memory was not a problem at all) completely unusable. I upgraded from
Qpopper-3.0.2 to Qpopper-4.0.6b3, then added this reduced I/O
functionality to the non-server mode in Qpopper-4.0.6b3. The difference
in performance has been quite remarkable.
I hope others find this useful, and certainly if anyone finds errors or
other reasons to be concerned they'll bring it up ...
--
--
Sylvain Robitaille [EMAIL PROTECTED]
Systems analyst / Postmaster Concordia University
Instructional & Information TechnologyMontreal, Quebec, Canada
--
--- popper/pop_dropcopy.c.original 2004-12-14 21:36:18.0 -0500
+++ popper/pop_dropcopy.c 2005-03-18 15:39:59.0 -0500
@@ -1485,7 +1485,95 @@
(long unsigned) geteuid(),
(long unsigned) getegid() );
-dfd = open ( p->temp_drop, O_RDWR | O_CREAT, 0660 );
+/*
+ * 2005/03/10 Sylvain Robitaille: We want the temporary drop to be
+ * opened with the same permission as the real mail spool. Start
+ * by gathering that information.
+ */
+if ( stat(p->drop_name, &mybuf) == -1 ) {
+pop_log ( p, POP_PRIORITY, HERE,
+ "[SYS/TEMP] Unable to stat maildrop "
+ "'%s': %s (%d)",
+ p->drop_name, STRERROR(errno), errno );
+return pop_msg ( p, POP_FAILURE, HERE,
+ "[SYS/TEMP] Failed to stat %s with uid %lu, "
+ "gid %lu. Change permissions.",
+ p->drop_name,
+ (long unsigned) pwp->pw_uid,
+ (long unsigned) pwp->pw_gid );
+}
+/*
+ * 2005/03/11
Re: patch submission?
> Hello ... > > I have patched our installation of Qpopper-4.0b3 in an attempt to get > some of the benefit that's available in the "server" mode of that > version of the POP server, without needing to use "server" mode. > > What I'm wondering is, is this an appropriate place to submit that > patch? Are the Qpopper developpers known to use this mailing list? > Are other people interested in my patch? > Yes Thanks.
