Re: [vchkpw] cdb -V- Sql
On Mar 24, 2007, at 4:36 AM, Peter Normann wrote: Quey wrote: I've read that use of SQL for vpopmail is really only advantageous when you get many domains. Not exactly. SQL is advantageous if you somewhere down the road want to implement a web based management system for administering accounts. SQL is advantageous if you want to scale the cluster beyond more than one box. If the data is stored in CDB, then you must replicate or share the CDB file(s) to each machine in the cluster. This is not difficult, as you can NFS share the CDB, or you can rsync/rdist/scp the files to each system. But if the CDB file(s) are frequently updated, and can be updated by more than one system, you'll run into problems with the CDB getting munged. This is most pronounced with the etc/tcp.smtp when using POP before SMTP on a large cluster, but I've also seen the vpasswd files get munged. But you shouldn't be using POP before SMTP any longer. And it's been a long time since I've used CDB for vpasswd but I can recall writing a script to rebuild the vpasswd file after it got munged. You have to be very careful to make sure you limit the writers of a CDB to one at a time else you run into problems. With SQL, it's easy for multiple systems to all access the same SQL tables and concurrency issues are taken care of for you. However, what about 1 domain, at how many users would it be faster to use SQL over the default cdb file? Never. SQL (any flavor) is at least an order of magnitude slower than CDB, on a single box. In 2000, the best I could get out of one top of the line dual PIII system was about 400 queries per second. Of course, qps will vary based on your ratio of read and writes. Writes are very slow with SQL because they must be committed to disk in order to complete. With todays hardware and the latest MySQL, I'd guess you'd be looking at somewhere in the neighborhood of 1,000-1,500 qps under normal usage. That assumes you've got a large enough data set to invest 4-8 hours tuning MySQL and your queries to get the best performance. The last time I benchmarked CDB performance (in 2000), I was able to get well over 6,000 qps on servers half the CPU of my SQL boxes. That kind of performance is expected from a CDB because it's a file. Any good Unix-like OS will mmap it and access it from RAM. Is it beneficial at 10K users or 50K users in the same domain, or no real gain at all until 100K users, or never? I am uncertain whether SQL provides performance gains under any circumstances. Maybe someone could expand on this... Where SQL beats the pants off CDB is scalability. CDB has file size limits and you can't have multiple writers. Before any write is completed you must rebuild the CDB from the plain text file. With tiny CDB files, this is never an issue. But when your CDB gets large and takes seconds, or minutes to compile, soaking up gobs of CPU and RAM in the process, this becomes a big problem. Under those conditions, SQL kicks CDB's tail all over town. One SQL write/update and you're done. You can throw a bunch of hardware at MySQL and achieve many thousands of queries per second such as sites like Wikipedia and Friendster do, pushing upwards of 15,000 queries per second. And unlike CDB, they have a lot of redundancy built in because the entire data set exists in multiple databases. If your data access is almost entirely reads, CDB is fantastic. If your data set is tiny or small, CDB is excellent. If you need frequent writes of a large or huge data set, CDB is probably inappropriate. Matt
Re: [vchkpw] authdaemond and MySQL server has gone away
You might want to make sure that MySQL is started before authdaemond. Edit /usr/local/etc/rc.d/courier-authdaemond.sh and add this line: REQUIRE: LOGIN mysql-server To understand REQUIRE a little better, cd /usr/local/etc/rc.d and grep for REQUIRE in the startup files. It's purpose should become obvious. http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/ configtuning-rcng.html http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/ configtuning-starting-services.html Matt On Jun 22, 2005, at 8:34 PM, Billy Newsom wrote: I have been having a strange issue with authdaemond ever since it split into a seperate auth port. I am running FreeBSD 5.4, net- qmail, vpopmail, Courier-IMAP, and using a mysql backend to vpopmail. The only authentication package I use or need is the vchkpw. Most or all of these are pretty late versions of these programs. Well, what seems to be the problem is during a server *reboot* 1. authdaemond boots up, using an rc.d script (FreeBSD's autoexec files) 2. I think this is before mysql is loaded. 3. auth requests come in to the IMAP server almost immediately. 4. ALL AUTHs FAIL until I do the following. When I get to the root shell a few hours later, I can get AUTHs working by restarting the daemons (I frantically restart imap, authdaemond, and mysql). But, by the way, the authdaemond script is broken, and I have to stop and start it (typing /usr/local/etc/ rc.d/courier-authdaemond.sh restart only stops the daemon). At this stage, all AUTHs now work! Yeah! But what is going on? During the AUTH failures, nobody can login, and everyone has to retype their mail passwords (Mozilla, for example, resets the IMAP password) Here is what mysql logs said. Notice, it appears that mysql started *AFTER* the first AUTH attempt. 050601 02:28:45 mysqld started 050601 2:28:49 InnoDB: Started; log sequence number 0 43740 /usr/local/libexec/mysqld: ready for connections. Version: '4.1.11' socket: '/tmp/mysql.sock' port: 3306 FreeBSD port: mysql-server-4.1.11_1 Here is my mail and debug log. I tried to put in spaces just to show different user login attempts. Jun 1 02:28:40 ibm authdaemond: modules=authvchkpw, daemons=5 Jun 1 02:28:40 ibm authdaemond: Installing libauthvchkpw Jun 1 02:28:40 ibm authdaemond: Installation complete: authvchkpw Jun 1 02:28:41 ibm imapd: Connection, ip=[192.168.0.11] Jun 1 02:28:41 ibm authdaemond: received auth request, service=imap, authtype=login Jun 1 02:28:41 ibm authdaemond: authvchkpw: trying this module Jun 1 02:28:41 ibm authdaemond: vchkpw: user does not exist Jun 1 02:28:41 ibm authdaemond: authvchkpw: REJECT - try next module Jun 1 02:28:41 ibm authdaemond: FAIL, all modules rejected Jun 1 02:28:41 ibm imapd: LOGIN FAILED, user=test, ip=[192.168.0.11] Jun 1 02:28:42 ibm pop3d: Connection, ip=[192.168.0.18] Jun 1 02:28:42 ibm authdaemond: received auth request, service=pop3, authtype=login Jun 1 02:28:42 ibm authdaemond: authvchkpw: trying this module Jun 1 02:28:42 ibm authdaemond: vchkpw: user does not exist Jun 1 02:28:42 ibm authdaemond: authvchkpw: REJECT - try next module Jun 1 02:28:42 ibm authdaemond: FAIL, all modules rejected Jun 1 02:28:42 ibm pop3d: LOGIN FAILED, user=tester, ip= [192.168.0.18] Jun 1 02:28:44 ibm pop3d: Connection, ip=[192.168.0.17] Jun 1 02:28:44 ibm authdaemond: received auth request, service=pop3, authtype=login Jun 1 02:28:44 ibm authdaemond: authvchkpw: trying this module Jun 1 02:28:44 ibm authdaemond: vchkpw: user does not exist Jun 1 02:28:44 ibm authdaemond: authvchkpw: REJECT - try next module Jun 1 02:28:44 ibm authdaemond: FAIL, all modules rejected Jun 1 02:28:44 ibm pop3d: LOGIN FAILED, user=ppp, ip=[192.168.0.17] Jun 1 02:28:44 ibm imapd: Connection, ip=[127.0.0.1] Jun 1 02:28:44 ibm imapd: LOGOUT, ip=[127.0.0.1] Jun 1 02:28:45 ibm pop3d: Connection, ip=[192.168.0.9] Jun 1 02:28:45 ibm authdaemond: received auth request, service=pop3, authtype=login Jun 1 02:28:45 ibm authdaemond: authvchkpw: trying this module Jun 1 02:28:45 ibm authdaemond: vchkpw: user does not exist Jun 1 02:28:45 ibm authdaemond: authvchkpw: REJECT - try next module Jun 1 02:28:45 ibm authdaemond: FAIL, all modules rejected Jun 1 02:28:45 ibm pop3d: LOGIN FAILED, user=, ip=[192.168.0.9] Jun 1 02:28:46 ibm imapd: Disconnected, ip=[192.168.0.11], time=5 Jun 1 02:28:46 ibm pop3d: Connection, ip=[192.168.0.6] Jun 1 02:28:46 ibm authdaemond: received auth request, service=pop3, authtype=login Jun 1 02:28:46 ibm authdaemond: authvchkpw: trying this module Jun 1 02:28:46 ibm authdaemond: vchkpw: user does not exist Jun 1 02:28:46 ibm authdaemond: authvchkpw: REJECT - try next module Jun 1 02:28:46 ibm authdaemond: FAIL, all modules rejected Jun 1 02:28:46 ibm authdaemond: vmysql: sql error[3]: MySQL server has gone away Jun 1 02:28:46 ibm pop3d: LOGIN FAILED, user=jjj, ip=[192.168.0.6] Jun 1
Re: [vchkpw] vpopmail authentications failing randomly
SOLVED! Read on On May 25, 2005, at 1:37 PM, Tom Collins wrote: On May 24, 2005, at 7:25 PM, Matt Simerson wrote: So, I turned to vmysql.c and noticed a timeout setting there that affects the mysql connection timeout. I bumped it up from 2 to 5, and it has reduced the failure frequency but it's still happening fairly regularly. There are some slow queries in MySQL, but only 15 from the last day, so that doesn't even closely correspond with 300 no found errors. So, anyone got ideas on how to debug this issue further? Thanks for taking such a detailed look at this problem. I think you've nailed it right there -- the server (or MySQL) gets bogged down to the point where it can't answer the query in under 5 seconds. Vpopmail times out and its only option is to reply that the login failed. Increasing that timeout to 10 or even 15 seconds shouldn't have a negative impact -- it will make vpopmail more tolerant of slow responses. -- Tom Collins - [EMAIL PROTECTED] I would love to believe that this issue was that simple. Last night while running the buildworld, the system was very heavily loaded. During that time, I had the timeout raised to 5 seconds and was running my test script which runs one query every second. The queries that failed were failing much sooner than the 5 second timeout. Further, the other program that I was accessing MySQL with never once failed, and it never took more than a second or two to return the results either. Further still is that the timeout value that I altered in vmysql.c ONLY affects the MySQL connection timeout (MYSQL_OPT_CONNECT_TIMEOUT) which is only used in mysql_real_connect. Since it's always connecting to MySQL in well under 5 seconds, that certainly is not the problem, and raising it to any arbitrarily higher number isn't going to help. As Matt B pointed out in his post where he saw this issue previously, vpopmail is successfully connecting to the MySQL server, and in 99.9% of the cases (even where authentication failure occurs), successfully switching to the vpopmail database correctly. Today I added a new data point to the mix. I upgraded another mail server for a different client, also FreeBSD. Previously using MySQL 3.x. Fairly moderate load, but a much more busy server that's been happily busy since I build it for him, two years ago. I also upgraded this box to MySQL 4.0 w/linuxthreads. I began getting the failed authentications in the logs again. He complained that qmailadmin wasn't working correctly (while I was still compiling) and sure enough, when the system was under moderate load, I was getting Apache errors in the log file: vmysql: sql error[5]: Commands out of sync; You can't run this command now vmysql: sql error[5]: Commands out of sync; You can't run this command now vmysql: sql error[5]: Commands out of sync; You can't run this command now vmysql: sql error[5]: Commands out of sync; You can't run this command now vmysql: sql error[5]: Commands out of sync; You can't run this command now vmysql: sql error[5]: Commands out of sync; You can't run this command now vmysql: sql error[5]: Commands out of sync; You can't run this command now vmysql: sql error[3]: MySQL server has gone away I didn't bother to read the qmailadmin source and understand it's error messages, I just compiled a new mysql-server w/o linuxthreads, installed it and all the MySQL problems went away. It may simply be coincidence that this fixes the problem on two different systems, but it's a happy coincidence for me. Matt
[vchkpw] vpopmail authentications failing randomly
Problem: vpopmail authentications failing randomly
grep vchkpw /var/log/maillog | grep -v success | grep dlb
May 24 11:45:03 mail01 vpopmail[40833]: vchkpw-pop3: vpopmail user
not found [EMAIL PROTECTED]:67.92.111.22
May 24 11:50:03 mail01 vpopmail[41401]: vchkpw-pop3: vpopmail user
not found [EMAIL PROTECTED]:67.92.111.22
May 24 11:55:04 mail01 vpopmail[42117]: vchkpw-pop3: vpopmail user
not found [EMAIL PROTECTED]:67.92.111.22
May 24 12:00:04 mail01 vpopmail[42735]: vchkpw-pop3: vpopmail user
not found [EMAIL PROTECTED]:67.92.111.22
May 24 12:50:06 mail01 vpopmail[51623]: vchkpw-pop3: vpopmail user
not found [EMAIL PROTECTED]:67.92.111.22
May 24 12:55:07 mail01 vpopmail[52208]: vchkpw-pop3: vpopmail user
not found [EMAIL PROTECTED]:67.92.111.22
May 24 13:00:06 mail01 vpopmail[52799]: vchkpw-pop3: vpopmail user
not found [EMAIL PROTECTED]:67.92.111.22
May 24 13:20:16 mail01 vpopmail[55953]: vchkpw-pop3: vpopmail user
not found [EMAIL PROTECTED]:67.92.111.22
May 24 13:48:23 mail01 vpopmail[13650]: vchkpw-pop3: vpopmail user
not found [EMAIL PROTECTED]:67.92.111.22
These are happening for various accounts, on a seemingly random
basis. Out about 4,000 authentications today, it failed 100 times.
qmail-pop3d is run as follows:
exec softlimit -m 4096000 tcpserver -H -R -c50 0 pop3 qmail-popup
mail01.example.net vchkpw qmail-pop3d Maildir 21
System Specs:
Pentium III 1.0GHz - 1.0GB RAM
36GB ATA disk
FreeBSD 4.10-stable
MySQL 4.0.24 with linuxthreads, running locally
Vpopmail 5.4.2-5.4.10
~400 users.
Previous versions of MySQL were not compiled with linuxthreads but
this problem existed then as well. The my.cnf file is based on my-
medium.cnf, with two tweaks appropriate for this system (skip-innodb,
bin-log disabled).
I have used vpopmail versions 5.4.2, 5.4.8 and 5.4.10 and all exhibit
this problem. I also see this error in the maillogs on occasion:
delivery 14578: failure: vmysql:_sql_error
[1]:_Can't_create_database_'vpopmail'._Database_exists/
vmysql:_sql_error[3]:_No_Database_Selected/
Sorry,_no_mailbox_here_by_that_name._vpopmail_(#5.1.1)/
delivery 14641: failure: vmysql:_sql_error
[1]:_Can't_create_database_'vpopmail'._Database_exists/
vmysql:_sql_error[3]:_No_Database_Selected/
Sorry,_no_mailbox_here_by_that_name._vpopmail_(#5.1.1)/
There are no errors reported in MySQL's .err log. There are no other
related errors reported in the system logs. The MySQL load is quite
light, with less than one query executed every few seconds. The
system load is also fairly light, mostly hovering between 0.4 and 0.5.
All the symptoms are indicative of vpopmail having an issue with
MySQL, so I set up a little perl script to test for me while I tried
to replicate the problem.
#!/usr/bin/perl
my $email = $1 || '[EMAIL PROTECTED]';
my $limit = $2 || 1000;
print checking for $email $limit times (each . = success) ;
for ( my $i = 0; $i $limit; $i++){
my $dir =`~vpopmail/bin/vuserinfo -d $email`;
chomp $dir;
-d $dir ? print . : print $i fail\n;
sleep 1;
};
So, I run this script in one terminal window while trying to trigger
the problem in another. As near as I can tell, the problem is always
during times when the system is busy. I cannot replicate this problem
on any of my own servers. To help narrow down the problem, I put the
system under sustained heavy load (make buildworld). As expected, I
get frequent authentication and test (~45%) failures.
So, there is certainly a problem with vpopmail and it's MySQL
interaction. Again, this is with versions including 5.4.2, 5.4.8, and
5.4.10. I've seen random cases of this but I've never had an
instance where it was repeatable.
In such a case, shouldn't the vpopmail programs be returning a error
indicative of the problem instead of a user not found error? A
user not found error is not even close to accurately describing
what the problem is.
During this heavy load testing, I wanted to see if the issue was
MySQL or vpopmail. To test this I ran another MySQL client and see
if it too has problems interacting with the MySQL server. This is
another C program, also compiled to access the vpopmail database and
do a query. I also ran this program simultaneously with the test
script and the make buildworld. While the vpopmail test script was
failing every other authentication, the other program succeeded,
every single time, never failing once.
So, I turned to vmysql.c and noticed a timeout setting there that
affects the mysql connection timeout. I bumped it up from 2 to 5, and
it has reduced the failure frequency but it's still happening fairly
regularly. There are some slow queries in MySQL, but only 15 from the
last day, so that doesn't even closely correspond with 300 no found
errors.
So, anyone got ideas on how to debug this issue further?
Matt
``
Show me a piano falling down a mineshaft and I'll show you A-flat minor.
Re: [vchkpw] many roaming users
On May 18, 2005, at 3:39 PM, Jeremy Kitchen wrote: On Wednesday 18 May 2005 01:26 pm, Payal Rathod wrote: On Wed, May 18, 2005 at 01:15:22PM -0500, Jeremy Kitchen wrote: don't use vpopmail's roaming-users functionality if you want pop-before-smtp authentication, use Bruce Guenter's relay-ctrl package. Any particular reason why? vpopmail's roaming-users support is poorly designed, slow, and prone to failure. I'd like to see some evidence to back up this assertion. One could argue all day about whether it's poorly designed, but as I recall, you were not there when it was designed. While I completely agree that smtp-auth is a better method of allowing users to relay, anyone with even a little bit of experience working in large scale ISP or other hosting environments knows exactly how much pain and real cost is involved in getting even a small number (hundreds) of clients to update their email settings. Most places assign a dollar value to every support call and when you start doing the math, if you have 10% of this fellows 3,000 users calling in because they can't figure out how to update their ancient version of (Eudora|Outlook|Netscape] from 1995 to use SMTP-AUTH, it's going to cost his company thousands of dollars. Telling him to depreciate the use of POP before SMTP simply isn't terribly practical advise. How exactly is vpopmail roaming-users slow? You authenticate and the IP is immediately stuffed into open-smtp, which is compiled into tcp.smtp that very same second. How is that slow? The user can relay immediately. I have 600,000 users who have never once complained about it being slow. I'm sure this fellows 3,000 users have never had a problem with it being slow and never will. It's only prone to failure if you're using clusters of servers, in which case you'll have lock contention when re-writing the tcp.smtp file on a NFS mount. This does not affect many users of vpopmail and would certainly not show up on a system with only 3,000 users. I was the first to use vpopmail in such an environment and had over 10,000 users on the system when I ran into it. The tcpserver-MySQL patch was written and it's worked great for me and many others since. That hack has worked extraordinarily well for thousands of mail servers since. relay-ctrl is not, and is even, in fact, safe to use over NFS (I've done it) Using relay-ctrl on NFS is no less of a hack. On any well designed system that uses NFS, a primary limitation of scale will be NFS r/w operations between the NFS clients and server. Most often the point of using NFS is scalability. Having a cluster of boxes delivering mail for hundreds of thousands of users can quickly saturate even a the beefiest of NFS servers. Thus, a wise systems engineer will do everything he can to avoid adding to that load. So, the question becomes whether you prefer to litter /var with thousands of IP address files or use a MySQL table to store IPs. Databases were invented just for such purposes and do the job quite admirably. Matt However, I wouldn't even use pop-before-smtp.. I would set up SMTP authentication and require that. His usersuMe too. But his users have grown used to it. I suggested starting SMTP-Auth on another port and slowly switching pop-before- smtp completely off. that's what email is so handy for. You send your users an email telling them they have to change in their mail clients, and give them a URL with some pictures and instructions, and notify your support staff about the change, and train them how to handle the situation.. then send that email every week for 3 months. After the 3 months is up, shut pop-before-smtp off. -Jeremy -- Jeremy Kitchen ++ Systems Administrator ++ Inter7 Internet Technologies, Inc. [EMAIL PROTECTED] ++ inter7.com ++ 866.528.3530 ++ 815.776.9465 int'l kitchen @ #qmail #gentoo on EFnet IRC ++ scriptkitchen.com/qmail GnuPG Key ID: 481BF7E2 ++ jabber:[EMAIL PROTECTED] `` Matt Simersonhttp://matt.simerson.net The Network People Inc. http://www.tnpi.biz Show me a piano falling down a mineshaft and I'll show you A-flat minor.
Re: [vchkpw] courier-imap roaming-users problem
On Feb 4, 2005, at 10:25 AM, Tom Collins wrote: On Feb 4, 2005, at 6:11 AM, Jesús Arnáiz wrote: The problem is that, although I compiled courier with authvchkpw-auth it seems not to call this program. I think so because I see (with ktrace and truss) that courier authenticate my users reading the mysql database and it adds also the info data to the relay table on MySQL directly (I mean, not using vchkpw as it should do). So, the user is authenticated and can read his e-mail but he isn't added to the tcp.smtp.cdb file (tcpserver). Actually authvchkpw is a version of vchkpw that's integrated with courier-imap. It never calls vchkpw directly, so it's behaving as expected. Well, at least as documented. Unfortunately, I don't know about using roaming users (pop before smtp) with courier. I've always used SMTP AUTH as the sole relaying mechanism on our servers. Newer version of courier-imap will no longer be able to write the IPs into the relay table. This is because all authentication has now been moved out of courier (and sqwebmail) and into the courier-authlib package. As has always been the case, authdaemon doesn't support POP before SMTP. That's why I chose never to use authdaemon. Now that courier no longer supports POP before SMTP via the authvchkpw module, it's just one more good reason not to use courier-imap for POP3 services. Use qmails pop3d instead. That way POP users still get roaming access. IMAP users are users newer email clients that support SMTP AUTH and should be directed to use that instead. Matt `` Matt Simersonhttp://matt.simerson.net The Network People Inc. http://www.tnpi.biz The chief danger in life is that you may take too many precautions. - Alfred Adler ``
Re: [vchkpw] temp-problems with vpopmail and finding users
My first guess is that you need to figure out why SQL queries to your server fail and you'll have the answer to your problem. Write a script that asks your server lots and lots of queries just like a busy mail server would. Check and verify the results and I'm sure you'll find some clues as to exactly what tree you should be barking up. Matt On Dec 4, 2003, at 3:26 AM, Mandy wrote: Hi Group. I installed vpopmail 5.3.30 and what happens ?? The same :-( Where can i search for the ground of this fault ? I'm using mysql 4.0.16. regards Mandy Mandy wrote: hi i'm new in vpopmail I installed a little system with 4 domains using the great howto by matt. I'm using vpopmail 5.3.29. Sometime the system can't find a user. But the user exists ?!? SQL-Server was up and runninng the hole time. This also happens sometimes while getting mails from the server with eg. fetchmail. Mostly everthing works fine. But sometimes there is and auth error while fetching? Can anyone help me ? regards mandy /var/qmail/log/send/current mail one to [EMAIL PROTECTED] @40003fbc6f9a223ff3b4 starting delivery 1759: msg 1586646 to local [EMAIL PROTECTED] @40003fbc6f9a22400354 status: local 1/10 remote 0/20 @40003fbc6f9a228fa60c delivery 1759: success: vmysql:_sql_error[1]: _Can't_create_database_'vpopmail'._Database_exists/vmys ql:_sql_error[3]:_No_Database_Selected/ user_does_not_exist,_but_will_deliver_to_/home/vpopmail/domains/ bar.de/catchall//did _0+0+1/ mail two to [EMAIL PROTECTED] @40003fbc86f00c683f24 starting delivery 1772: msg 1586646 to local [EMAIL PROTECTED] @40003fbc86f00c684adc status: local 1/10 remote 0/20 @40003fbc86f00d3e675c delivery 1772: success: did_0+0+1/
Re: [vchkpw] Inter7 mail server doesn't have reverse DNS!
On Thursday, March 27, 2003, at 06:28 PM, Raboo wrote: lol ya all :-) btw matt simerson, if you code hacks to make your smtpd do more loging like you said before that you have made it do logging of what kinds of blocks it does, please feel free to publish them if you like, if not allready, There's no code hacks involved to get more logging. You simply configure your smtp invocation to log the denied connections. For example, my smtp invocation looks like this: exec softlimit -m 800 tcpserver -S -R -c20 -x /usr/local/vpopmail/etc/tcp.smtp.cdb -u 89 -g 89 0 smtp rblsmtpd -r sbl.spamhaus.org -r rbl-plus.mail-abuse.org -r list.dsbl.org -r bl.spamcop.net -r relays.ordb.org qmail-smtpd mail.cadillac.net /usr/local/vpopmail/bin/vchkpw /usr/bin/true 21 If you read the rblsmtpd man page, you'll learn that it writes it's output to file descriptor 2. In order for that to get logged, we have to put the little redirect (21) in there at the end of the invocation. Now we actually get a little output from our smtp daemon. I wrote a little script that gets run as a multilog processor and it gets triggered every 5 minutes to roll the log, parse out the counts. Those get collected via SNMP and stuffed into a database which another script reads to create the graph you'll see here: http://matt.simerson.net/cgi-bin/rrdutil.pl Eventually I'll hack up SpamAssassin some more and get it to log all the delivered messages, and their spam scores and I'll graph that too. That'll should prove interesting. :) Greetings /Raboo P.S. this is probibly one of the top ten biggest posts in the vchkpw mailinglist Religion zealots are everywhere. Who'd have thought I could make such a fuss simply by doing DNS checks on my very own personal mail server? Matt
[vchkpw] vpopmail 5.3.1[6,9] logging
I'm not exactly sure when it quit but I've noticed that authentications via vchkpw are no longer logged. Vpopmail is configured as follows and my pop3 daemon is qmail, run as shown below. I've also attempted logging via syslog (rather than multilog) and I still get no authentication logging. service/pop/run #!/bin/sh exec softlimit -m 200 tcpserver -R 0 pop3 qmail-popup mail.cadillac.net \ /usr/local/vpopmail/bin/vchkpw qmail-pop3d Maildir 21 vpopmail 5.3.19 Current settings --- vpopmail directory = /usr/local/vpopmail uid = 89 gid = 89 ip alias = OFF --enable-ip-alias-domains=n (default) address extentions = OFF --enable-qmail-ext=n (default) roaming users = ON --enable-roaming-users=y tcpserver file = /usr/local/vpopmail/etc/tcp.smtp open_smtp file = /usr/local/vpopmail/etc/open-smtp user quota = 1S,1C -enable-hardquota=1S,1C domain quotas = OFF --enable-domainquotas=n (default) auth module = mysql --enable-mysql=y mysql replication = ON --enable-mysql-replication=y table optimization = many domains --enable-many-domains=y (default) system passwords = OFF --enable-passwd=n (default) file locking = ON --enable-file-locking=y (default) file sync = OFF --enable-file-sync=n disable vdelivermail fsync (default) make seekable = ON --enable-make-seekable=y (default) auth logging = ON --enable-auth-logging=y (default) mysql logging = OFF --enable-mysql-logging=n (default) clear passwd = ON --enable-clear-passwd=y (default) users big dir = ON --enable-users-big-dir=y (default) valias processing = OFF --enable-valias=n mysql limits = OFF --enable-mysql-limits=n (default) pop syslog = show failure attempts with clear text password --enable-logging=p default domain = --enable-default-domain= auth inc = -I/usr/local/include/mysql auth lib = -L/usr/local/lib/mysql -lmysqlclient -lz
[vchkpw] Inter7 mail server doesn't have reverse DNS!
Apparently a server named ns1.inter7.com is doing the delivery for the vchkpw mailing list. This wouldn't be a problem except that it doesn't have reverse DNS. I started blocking connections to my mail server from servers who don't have DNS and my vpopmail and qmailadmin list traffic stopped. After a day of no messages, I determined that by then I really *should* have had some messages arrive. I removed my DNS checks and voila, mailing list messages started flowing in. Someone at Inter7 ought to have a look at that. Ken? Vol? Matt [EMAIL PROTECTED]:/var/log/mail/smtp # dig ns1.inter7.com. ;; ANSWER SECTION: ns1.inter7.com. 2d23h56m3s IN A 209.218.8.2 [EMAIL PROTECTED]:/var/log/mail/smtp # dig -x 209.218.8.2 ; DiG 8.3 -x ;; res options: init recurs defnam dnsrch ;; got answer: ;; -HEADER- opcode: QUERY, status: NXDOMAIN, id: 2 ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0 ;; QUERY SECTION: ;; 2.8.218.209.in-addr.arpa, type = ANY, class = IN Delivered-To: mailing list [EMAIL PROTECTED] Received: (qmail 60493 invoked by uid 85); 27 Mar 2003 14:15:27 - Received: from [EMAIL PROTECTED] by cadillac.mi.us by uid 89 with qmail-scanner-1.15 (spamassassin: 2.44. Clear:SA:0(-0.5/10.0):. Processed in 1.900033 secs); 27 Mar 2003 14:15:27 - Received: from unknown (HELO ns1.inter7.com) (209.218.8.2) by matt-serv2.cdlc.mi.voyager.net with SMTP; 27 Mar 2003 14:15:25 -
Re: [vchkpw] Inter7 mail server doesn't have reverse DNS!
On Thursday, March 27, 2003, at 10:35 AM, Ron Guerin wrote: On Thu, 2003-03-27 at 10:21, Matt Simerson wrote: Apparently a server named ns1.inter7.com is doing the delivery for the vchkpw mailing list. This wouldn't be a problem except that it doesn't have reverse DNS. I started blocking connections to my mail server from servers who don't have DNS and my vpopmail and qmailadmin list traffic stopped. You'll be losing a lot more legit mail than just this list if you do that. - Ron I have managed several dozen mail systems supporting over 100,000 users. I am not naive to that simple fact. However, this is my personal mail server and I've decided that if you can't set up DNS for your mail server properly, I don't want to receive mail from you. It's that simple. In this case, I'm sure it's an oversight on the part of the Inter7 guys. As soon as they get it straightened out, I'll be once again blocking connections from servers without reverse DNS. I'm writing a logging program that parses out log smtp logs and lets me know how many connections I've blocked, which rule blocked the connection (spamhaus, spamcop, rbl, reverse dns, etc), and the IP I blocked. Matt
Re: [vchkpw] Inter7 mail server doesn't have reverse DNS!
Oh my, Nick top-posted. Quickly, someone call out the firing squad. Where you choose to reply to in a message body is a matter of personal preference. It is NOT a breach of email etiquette to prefer a way other than your personal preference. In a list thread, many tend to prefer a top-post as it's assumed that they've already read the thread and thus no longer have a need for what follows, except possibly as a reference (hence the choice to include rather than snip it). The rules of email etiquette exist for the benefit of email users, not for anal retentive users to LART others with. On to the matter of DNS blocks; we aren't going to agree on this matter. You consider my mail server broken. I consider it optimized. I, and many others, will continue to block connections from mail servers without reverse DNS. Live with it. Per Arie's question on exactly what I block based upon, it's quite simple. Your reverse DNS must be set. Period. It doesn't have to match. Asking it to match would be, IMHO, a bad idea. The how is quite simple: Contents of ~vpopmail/etc/tcp.smtp 1.127.:allow,RELAYCLIENT= 2.209.218.8.2:allow 3.=:allow,QMAILQUEUE=/var/qmail/bin/qmail-scanner-queue.pl 4.:allow,RBLSMTPD=Blocked - Reverse DNS queries for your IP fail. You cannot send me mail. 5.#:allow,RBLSMTPD=-Blocked - Reverse DNS queries for your IP fail. You cannot send me mail. 1. Obvious, allowing localhost to relay. 2. Allows traffic from the inter7 mailing list (with no reverse DNS) 3. Matches any mail message with reverse dns. 4. Matches what's left (no reverse DNS). By setting the RBLSMTPD environment variable, we actually get to pass a message back to the mail server we're blocking telling them why. That gives them a chance to fix it before the messages bounces. They'll get a message like this in their mail logs: Mar 27 08:40:43 seattle qmail: 1048783243.397888 info msg 6469: bytes 258 from [EMAIL PROTECTED] qp 13226 uid 0 Mar 27 08:40:43 seattle qmail: 1048783243.438981 starting delivery 533: msg 6469 to remote [EMAIL PROTECTED] Mar 27 08:40:43 seattle qmail: 1048783243.979048 delivery 533: deferral: 207.89.154.94_does_not_like_recipient./Remote_host_said:_451_Blocked_- _Reverse_DNS_queries_for_your_IP_failed._You_cannot_send_me_mail./ Giving_up_on_207.89.154.94./ Mar 27 08:40:43 seattle qmail: 1048783243.979779 status: local 0/10 remote 0/2 5. If you want to be a little more aggressive about it, use the 5th line instead of the fourth. Notice the '-' character in there. That tells rblsmtpd to return a permanent error (ie, don't try again!). Mar 27 08:42:40 seattle qmail: 1048783360.776812 info msg 6475: bytes 250 from [EMAIL PROTECTED] qp 13464 uid 0 Mar 27 08:42:40 seattle qmail: 1048783360.805534 starting delivery 534: msg 6475 to remote [EMAIL PROTECTED] Mar 27 08:42:41 seattle qmail: 1048783361.259737 delivery 534: failure: 207.89.154.94_does_not_like_recipient./Remote_host_said:_553_Blocked_- _Reverse_DNS_queries_for_your_IP_fail._You_cannot_send_me_mail./ Giving_up_on_207.89.154.94./ Mar 27 08:42:41 seattle qmail: 1048783361.269637 bounce msg 6475 qp 13467 Mar 27 08:42:41 seattle qmail: 1048783361.270564 end msg 6475 Notice that in the second case, the message bounces immediately. It's your mail server, you have to decide what policy you think is best. Bouncing messages seems to get more attention, and gets it faster than deferring connections. That's all there is to it. Of course, that assumes you are running rblsmtpd as part of your smtp invocation. Matt On Thursday, March 27, 2003, at 11:12 AM, Ron Guerin wrote: On Thu, 2003-03-27 at 11:05, Nick Harring wrote: Rather than questioning why we would refuse to accept from non-reversible hosts, why don't we ask why anyone would set a host up without reverse DNS? Rather than question why you've deliberately broken your mail server, I should explain to you why some people running legit servers don't comply with your arbitrary requirements? A better question is why I'm wasting my time trying to explain things to someone who top-posts and sends HTML to mailing lists. - Ron
Re: [vchkpw] About to release new devel version 5.3.20
On Tuesday, March 25, 2003, at 04:25 PM, Robin Bowes wrote: I'm about to release a new 5.3.20 devel version. Does anyone have any patches they would like to submit? No, but would it be difficult to move mysql options from vmysql.h to configure options, e.g. ./configure \ --with-mysql-update-server=servername \ --with-mysql-update-user=username \ --with-mysql-update-password=password \ --with-mysql-read-server=servername \ --with-mysql-read-user=username \ --with-mysql-read-password=password The --with-mysql-read-* options could be optional, i.e. use the update server values if the read server is not present. I'd really like to see this. Cheers, R. I would like to see that as well. :) Matt
Re: [vchkpw] Inter7 mail server doesn't have reverse DNS!
On Thursday, March 27, 2003, at 01:18 PM, Rick Updegrove wrote: - Original Message - From: Matt Simerson [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Thursday, March 27, 2003 7:21 AM Subject: [vchkpw] Inter7 mail server doesn't have reverse DNS! I started blocking connections to my mail server from servers who don't have DNS and my vpopmail and qmailadmin list traffic stopped. Matt, How much actual SPAM did your now-misconfigured mailserver actually block with this tactic during that time? Before anyone flames me, consider that this is a serious question and I am very interested in the answer. Thanks. Rick Up Todays Total Blocks: 875 spamhaus=637 spamcop = 127 reverse = 66 dsbl = 44 ordb = 1 Due to reverse DNS failure: 66 Unique mail server IP block: 23 Matt
Re: [vchkpw] Inter7 mail server doesn't have reverse DNS!
On Thursday, March 27, 2003, at 01:35 PM, Tom Collins wrote: On Thursday, March 27, 2003, at 10:22 AM, Paul Theodoropoulos wrote: rather than trumping up your argument with etiquette fascism, how about pointing out a relevant RFC that backs up your [baseless] opinion that a mailserver must accept messages from a site without reverse DNS? ever heard of RFC 2505? apparently not. I hadn't read RFC2505 http://zvon.org/tmRFC/RFC2505/Output/index.html until now, but I took the time to do so. It has some good advice, but I didn't see any mention of refusing mail from hosts without reverse DNS. It does talk about refusing mail based on the FQDN that reverse DNS resolves to (section 2.5), but I think it's a stretch to extend that to IP addresses that don't have reverse DNS. I don't think that's stretching at all. Having an IP not resolve at all *is* a form of resolution. It resolves to an empty value. If you look at how I'm doing the block, I am doing almost exactly what the RFC describes and refusing based on the FQDN of the mail server. If the FQDN is empty, I refuse the connection. I agree with others that the first post should have gone to Inter7 (perhaps [EMAIL PROTECTED]) and not this list. I made the original post, and I agree, partially. If my intent was solely to get Inter7 to fix the reverse DNS, then I would agree completely. I've already fixed the problem on my end by adding a special rule for their mail servers IP. However, I also wanted to hear what a few other postmasters had to say about it. Posting privately would not have afforded that luxury. I have enjoyed hearing what a few others think about blocking based on DNS. I haven't done it in quite a few years. If someone wants to add spam filters to their personal mail server that deny mail from hosts without reverse DNS, that's fine with me. If they think it's a good idea and tell others about it, I think it's a good idea for others to provide constructive feedback on why they disagree. I couldn't agree more. However, calling a mail server broken because it's not set up the way you'd prefer isn't constructive. :) If it's true that spammers don't have reverse DNS on their IP addresses, I wouldn't mind seeing the MTA adding a header like X-Possible-Spam: Host 209.218.8.2 does not have reverse DNS. and even X-Possible-Spam: Host 209.218.8.2 resolves to spam.com which does not resolve to 209.218.8.2. Then an email client could filter on that header or SpamAssassin could add a few points to the message's spam score. That's easy enough to do in qmail-scanner, and on my production servers, that's just another check in SpamAssassin to determine a messages spam score. What I'm attempting to do is block the spam *before* it has to be processed by the more heavy weight utilities like AV and SA. Matt
Re: [vchkpw] NFS shared
Sharing the tcp.smtp.cdb file over TCP is not a good idea. Once your traffic starts picking up, you'll have all sorts of problems with that. Check out the writeup I have on exactly this scenario: http://matt.simerson.net/computing/mail/qmail/qmail.toaster.open- smtp_writeup.txt. There are other considerations but to minimize traffic on your backend (NFS) network, you really should be planning to only share what's necessary (~vpopmail/domains) via NFS. Matt On Saturday, March 15, 2003, at 02:27 AM, Jason wrote: Hello All, If I'm going to install a couple of mailservers(vpopmail+qmail) and use an NFS server for storage, is the Maildir the only directory that will be shared by the mail servers. I think the tcp.smtp.cdb also needs to be shared. Any others files? TIA
Re: [vchkpw] Configure options for MySQL support?
I now do this with my mail toaster setup
(http://matt.simerson.net/computing/mail/toaster/).
When you run the perl script (toaster_setup.pl -s vpopmail) it asks you
a bunch of questions and then rewrites vmysql.h with your answers
instead of the default values. Then it runs ./configure with the
appropriate --enable-* stuff based on the interview.
Here's the useful logic:
sub set_up_vmysql {
my ($mysql_repl, $my_write, $my_read, $my_user, $my_pass) = @_;
copy(vmysql.h, vmysql.h.orig);
my @lines = read_file(vmysql.h);
foreach my $line (@lines) {
chomp $line;
if ( $line =~ /^#define MYSQL_UPDATE_SERVER/ ) {
if ($mysql_repl) {
$line = #define MYSQL_UPDATE_SERVER \$my_write\;
} else {
$line = #define MYSQL_UPDATE_SERVER \$my_read\;
};
} elsif ( $line =~ /^#define MYSQL_UPDATE_USER/ ) {
$line = #define MYSQL_UPDATE_USER \$my_user\;
} elsif ( $line =~ /^#define MYSQL_UPDATE_PASSWD/ ) {
$line = #define MYSQL_UPDATE_PASSWD \$my_pass\;
} elsif ( $line =~ /^#define MYSQL_READ_SERVER/ ) {
$line = #define MYSQL_READ_SERVER \$my_read\;
} elsif ( $line =~ /^#define MYSQL_READ_USER/ ) {
$line = #define MYSQL_READ_USER \$my_user\;
} elsif ( $line =~ /^#define MYSQL_READ_PASSWD/ ) {
$line = #define MYSQL_READ_PASSWD \$my_pass\;
};
};
write_file(vmysql.h, @lines);
};
On Tuesday, February 25, 2003, at 10:14 AM, Andre Fortin wrote:
This sounds like a good idea to me.. I'd like to see this in the
future.
Andre
-Original Message-
From: Robin Bowes [mailto:[EMAIL PROTECTED]
Sent: Tuesday, February 25, 2003 10:13 AM
To: [EMAIL PROTECTED]
Subject: Re: [vchkpw] Configure options for MySQL support?
David,
On Tue, February 25, 2003 2:13 pm, David Richardson said:
I don't plan on changing my MySQL params very often, but I'm _certain_
that I'll forget to update vmysql.h during a recompile and I'll
foobar my
Vpopmail.
Can you guess what bit me resulting in my raising this as an issue?!!
Perhaps we could just cobble together a helper perl script to query
for
and rewrite the vmysql.h params as part of the compile script(s) you
use.
That's another possibility...
Without a lot of other me too votes, is there interest from the
vpopmail
team in having this idea go a particular direction (into config/make
or
external perl)? Is this even the appropriate venue to ask???
Let's wait and see...
R.
--
Robin Bowes | http://robinbowes.com
[vchkpw] Defines on the command line.
I'm sure I'm about to display my ignorance of C but: Is there a way to override the values find in a .h file from the command line? Specifically, I want to pass along a couple defines to set the MYSQL_MASTER_PASSWD and related fields from vmysql.h without having to edit the vmysql.h file itself. It seems I recall a way to do that. Failing that, I'll just parse the the file in Perl. That just seems a kludgey way to do it. Matt
Re: [vchkpw] Re: Defines on the command line.
OK, I attempted to pass them via make: make -DMYSQL_UPDATE_SERVER=\db.simerson.net\ -DMYSQL_UPDATE_USER=\vpopmail\ -DMYSQL_UPDATE_PASSWD=\secret\ -DMYSQL_READ_SERVER=\localhost\ -DMYSQL_READ_USER=\vpopmail\ -DMYSQL_READ_PASSWD=\secret\ but that didn't work. So, I set the CFLAGS environment variable as follows: setenv CFLAGS '-DMYSQL_UPDATE_SERVER=\db.simerson.net\ -DMYSQL_UPDATE_USER=\vpopmail\ -DMYSQL_UPDATE_PASSWD=\secret\ -DMYSQL_READ_SERVER=\localhost\ -DMYSQL_READ_USER=\vpopmail\ -DMYSQL_READ_PASSWD=\secret\' Interestingly enough, the CFLAGS env variable is set properly but configure fails when CFLAGS is set and it tests gcc. If I run configure without CFLAGS set and then set it, and compile via make, then the CFLAGS don't get passed to gcc. So the only way I've been able to set the compiler flags and have them effect is by editing the Makefile. In that case they actually get passed to gcc and it warns about redefining them but the values I pass don't end up overriding the ones in the vmysql.h file as the resulting binaries don't work. The closest thing to a hack I've found is by echo'ing the new values to the bottom of the vmysql.h file. Then I get the expected redefine errors and the resulting binaries work as expected. However, that leaves me no better off than when I started. In order to do it right I still have to parse through the file, dropping the MYSQL_UPDATE_* and MYSQL_READ_* lines. What I really, really want is to override the defined values in vmysql.h without actually altering any files in any way. Maybe it's not possible, but it just seems like it should be. Matt On Saturday, February 8, 2003, at 08:14 PM, Peter Palmreuther wrote: I don't know if there's a way to override the '#define's. A quick test gave me warning: `... redefined So I guess your best bet is to comment out the defines (well knowing it _will not_ compile anymore w/o defining the values on command line) and passing '-D' arguments to the compiler. E.g.: -DMYSQL_UPDATE_SERVER=\localhost\ \ -DMYSQL_UPDATE_USER=\vpopmail\ \ -DMYSQL_UPDATE_PASSWD=\bla\ and so on. The important part is escaping the quotes with '\'. Else they'll be interpreted _before_ defining the value as delimiter to avoid argument splitting by e.g. space characters. The '\' tells the compiler they're not surrounding the value, but they're part of the value. HTH -- Best regards Peter Palmreuther Features should be discovered, not documented.
Re: [vchkpw] Re: SMTP-AUTH, yet again...
Kit, you're reinventing the wheel! Yes, there are significant problems with simply merging together a bunch of the qmail patches. It took me quite some time to get all the patches I wanted to play nicely together. Now that it's done, feel free to use it: http://matt.simerson.net/computing/mail/toaster/. Pay particular attention to the Install Qmail (with a few hacks) section. Bill Shupp also has similar patches that I've also heard work well but are linux oriented where mine focuses on the FreeBSD platform. It wouldn't take too much effort to use my setup on Open/NetBSD but those are down the list for me, after making it work on Darwin. (Mac OS X) :) Matt On Thursday, December 5, 2002, at 12:23 AM, Kit Halsted wrote: Thanks for the pointers, everybody... At 11:22 PM -0500 12/3/02, Kit Halsted wrote: ... Tried 0.31 tonight, no luck. Maybe my other patches are interfering? I'll try it at home as the only patch see how that goes. 2.) Try to run qmail-smtpd as root. Just for testing, but this avoids access denied to vpasswd.cdb and therefore excludes one possible culprit. Also no luck. Urgh. Just tried again on my home box, which starts qmail/vpopmail from rc.local instead daemontools. (OpenBSD 3.1, virgin qmail 1.03 + elysium.pl 0.31 auth patch only, vpopmail 5.2.1... D'oh!, okay, vpopmail 5.3.9 now.) I'm back to square one now with relay by IP only, but it sure did fail interestingly for a while. Below is what I was trying to do, I've since reverted back to a working setup so I can send receive. - From rc.local: /usr/local/bin/tcpserver -u 1001 -g 1000 -x /home/vpopmail/etc/tcp.smtp.cdb 0 25 \ /var/qmail/bin/qmail-smtpd yabox.kithalsted.com /home/vpopmail/bin/vchkpw /usr/b in/true \ 21 | /var/qmail/bin/splogger smtpd 3 ... /usr/local/bin/tcpserver -u 1001 -g 1000 -H -R 0 110 \ /var/qmail/bin/qmail-popup yabox.kithalsted.com \ /home/vpopmail/bin/vchkpw /var/qmail/bin/qmail-pop3d Maildir (Linewraps courtesy of less, the files are wrapped correctly.) - yabox# ls -al /var/qmail/bin/qmail-smtpd -rwxr-xr-x 1 vpopmail vchkpw 40960 Dec 4 20:54 /var/qmail/bin/qmail-smtpd yabox# ls -al /home/vpopmail/etc/tcp.smtp.cdb -rwxr-xr-x 1 vpopmail vchkpw 4359 Dec 4 22:09 /home/vpopmail/etc/tcp.smtp.cdb - Log entries corresponding to failed send/check from Eudora on my TiBook: yabox# tail /var/log/maillog Dec 4 23:36:51 yabox qmail: 1039063011.757314 end msg 889600 Dec 4 23:41:18 yabox qmail: 1039063278.163871 status: local 0/10 remote 0/20 Dec 4 23:41:18 yabox qmail: 1039063278.270637 new msg 889600 Dec 4 23:41:18 yabox qmail: 1039063278.270823 info msg 889600: bytes 230 from [EMAIL PROTECTED] qp 24677 uid 0 Dec 4 23:41:18 yabox qmail: 1039063278.328081 end msg 889600 Dec 4 23:41:36 yabox vpopmail[5686]: vchkpw-smtp: password fail [EMAIL PROTECTED]:208.36.84.242 Dec 4 23:41:36 yabox vpopmail[32015]: vchkpw-pop3: setgid 1001 failed errno 1 [EMAIL PROTECTED]:208.36.84.242 Dec 4 23:41:37 yabox vpopmail[26305]: vchkpw-pop3: setgid 1001 failed errno 1 [EMAIL PROTECTED]:208.36.84.242 Dec 4 23:41:56 yabox vpopmail[30076]: vchkpw-pop3: setgid 1001 failed errno 1 [EMAIL PROTECTED]:208.36.84.242 Dec 4 23:41:56 yabox vpopmail[18542]: vchkpw-pop3: setgid 1001 failed errno 1 [EMAIL PROTECTED]:208.36.84.242 - (Yes, uid 1001 is vpopmail gid 1000 is vchkpw.) -Kit -- They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety. -Benjamin Franklin ...qui desiderat pacem, praeparet bellum (...if you would have peace, be prepared for war) -Flavius Vegetius Renatus
Re: [vchkpw] Qmail AntiSpaming
I've been working on integrating SpamAssassin into my FreeBSD mail toaster setup (and it'll work similarly with Bill's or any other vpopmail setup). One of the biggest problems I've found is the lack of good vpopmail support in SpamAssassin. I've hacked it up and added some of my own code to it so that you can use SpamAssassin via qmail-queue at the server level (IE, for every incoming message) or strictly at the user level (via maildrop/procmail/etc). However, my goal is NOT to publish my version of SpamAssassin. I've sent several emails to Justin (SA author) regarding this but have yet to get a response from him. Maybe he added the vpopmail support to SA and got offended that I didn't think it was good enough. I don't know. SpamAssassin works just fine with vpopmail as it's published, but only if every domain is owned by vpopmail, and you run spamd with the -v -u vpopmail flags. I think there was something else I had to do but I don't recall what. The only problem with this type of setup is there's no way to provide vpopmail users with individual spamassassin settings (like whitelists, etc). That's the functionality I added to my version of SpamAssassin. It also necessitated a couple hacks to qmail-queue. Matt On Wednesday, October 16, 2002, at 05:20 PM, Rick Romero wrote: I compiled qmail with the qmail-queue patch, and added Qmail-Scanner and SpamAssassin. There are docs floating around to on how to add SpamAssassin via .qmail files if you don't want to recompile, or didn't compile with the qmail-queue patch. Rick On Wed, 2002-10-16 at 16:01, Werner Schalk wrote: Hello, what about spam filtering methods or programs that work with vpopmail and qmail without compiling qmail completly new? Has anyone got spamassassin or anything like that to work? Bye and thanks, Werner.
Re: [vchkpw] Check Space Usage
On Wednesday, October 9, 2002, at 08:43 PM, Justin R. Miller wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Said Matt Simerson on Wed, Oct 09, 2002 at 03:30:55PM -0400: Otherwise, Doug's heading down the right path. Write yourself a script that loops for each domain, and then each user within the domain, and go through and start counting up the bits. You'll likely have issues with depending on the maildirquota file so make sure that if that file doesn't exist, you fall back to a more expensive but accurate method like du. Is there an efficient way to do this if you _don't_ use quotas, ideally something other than 'du'? Efficient is a relative term. If you're asking if there's a way to do this that's nearly as efficient as using file system quotas, then the answer is an easy no. The reason quotas are so efficient is that the kernel is involved, keeping track of disk operations on a per user basis. It always has a real time idea of how much disk space is in use by a customer. There is no other highly efficient way of doing it. The next best utility (based on my personal experience, the extensive reading I've done, and advise of peers) for such a task is du or ls, both if which suffer from varying degrees of inefficiency. I haven't ever compared the speed of using du/ls to using perls File::* utilities. If using file system quota's isn't an option, experimenting with all three could be a fruitful endeavor, the results of which I'd find interesting. Matt
Re: [vchkpw] vpopmail vpasswd file difference(s) between 4.9.10 and 5.3.8 (convert from cdb to mysql)
Just in case anyone else runs into this, I wrote a little Perl script
that parses out /var/qmail/users/assign to fetch a list of domains that
are active on the mail server. Then, for each domain it goes out and
fetches the vpasswd file and feeds its contents into a SQL database. It
works quite nicely.
Matt
#!/usr/bin/perl
use strict;
use DBI;
my $host = 'localhost';
my $user = 'vpopmail';
my $password = 'secret';
my $db = 'vpopmail';
my $domainfile=/var/qmail/users/assign;
my @connect = (dbi:mysql:database=$db:host=$host, $user, $password);
my $dbh = DBI-connect(@connect);
warn @connect unless $dbh;
my @domainlist = get_domainlist( $domainfile );
foreach my $file ( @domainlist ) {
#print $file-{'dom'} \t $file-{'dir'} \n;
add_users( $file-{'dir'}, $file-{'dom'} );
};
$dbh-disconnect;
print All done.\n\n;
##
## Subdomains
##
sub get_domainlist {
my @domainlist;
print Reading in the file $_[0]...;
open ( DOMAINLIST, $_[0] ) || die couldn't open $_[0]: $!\n;
my @lines = DOMAINLIST;
close (DOMAINLIST) || die couldn't close $_[0]: $!\n;
print done.\n\n;
print Extracting the list of domains...;
my $row = 0;
foreach my $line ( @lines ) {
my @fields = split(:, $line);
my %domain = (
dir = $fields[4]/vpasswd,
dom = $fields[1]
);
$domainlist[$row] = \%domain;
$row++;
};
print done.\n\n.;
return @domainlist;
}
sub add_users {
if ( -e $_[0] ) {
open ( USERLIST, $_[0] ) || warn couldn't open $_[0]: $!\n;
my @users = USERLIST;
close (USERLIST);
foreach my $user (@users) {
chomp $user;
my @f = split(:, $user);
print $user\n;
my $query = INSERT INTO vpopmail SET pw_name=\$f[0]\,
pw_domain=\$_[1]\, pw_passwd=\$f[1]\, pw_uid=\$f[2]\,
pw_gid=\$f[3]\, pw_gecos=\$f[4]\, pw_dir=\$f[5]\,
pw_shell=\$f[6]\;
my $sth = $dbh-prepare($query);
$sth-execute;
my $x = $DBI::errstr;
if ($x) { print Error running $query\n; };
$sth-finish;
};
};
}
On Tuesday, September 24, 2002, at 04:03 PM, Matt Simerson wrote:
I'm rebuilding a dinosaur mail server that I built a couple years ago.
It's running vpopmail 4.9.10 using cdb authentication. I'm upgrading
to 5.3.8 with MySQL authentication.
I've build out the new 5.3.8 system and cluster of machines. It's all
up, running and happy. I've NFS mounted all the home directories from
the old cluster onto the new system so that I have ready access to the
vpasswd and vpasswd.cdb files. They are mounted onto the local
filesystem so that the paths in /var/qmail/users/assign are correct.
I run vconvert -c -m test.domain.com and it exits quietly indicating
success:
mail8# ~vpopmail/bin/vconvert -c -m -v -d test.simerson.net
version: 5.3.8
converting test.simerson.net done
However, when I check the MySQL server, there's no entries in the
vpopmail table that correspond to that domain. :-(
I've tested this one step further. I've manually added a valid user
([EMAIL PROTECTED]) into the new mail system by manually adding
it into the MySQL table. It works fine, authenticates, etc. That
basically makes it work with my new mail system and the old one at the
same time. Cool. Anyway, if I run vconvert again, it'll actually
delete that entry from the vpopmail table. Is that supposed to happen?
Everything on the new system works just fine except the vconvert
function. Can anyone verify that vconvert actually does work? Can you
sanitize a copy of a newer vpasswd file and send it my way so I can
compare it to my ancient system to see if the format is still the same?
Matt
Re: [vchkpw] Check Space Usage
This is a valid argument for having each domain created/owned by an unprivileged system user. When you do that, you have handy tools like repquota to help you manage your disk space usage. When used in conjunction with a perl script, I can format and report the disk space for 10,000 domains in about 6 seconds. I've further taken that report and for all domains that are within 90% of their quota, check the disk usage for each user within the domain and report the disk space hogs. That report takes a couple minutes to run but is invaluable. Otherwise, Doug's heading down the right path. Write yourself a script that loops for each domain, and then each user within the domain, and go through and start counting up the bits. You'll likely have issues with depending on the maildirquota file so make sure that if that file doesn't exist, you fall back to a more expensive but accurate method like du. Matt On Wednesday, October 9, 2002, at 08:27 AM, Doug Clements wrote: Michael Funk wrote: Is there a way to use vuserinfo to produce a report of user space utilization, per user, for all users in the domain? Most of the scripting stuff I have tried takes HOURS with 50,000 users. vuserinfo doesn't store that data.. you pretty much just have to either count up the mails, or look in the maildirquota file if you use those kinds of quotas. I think parsing the quota file would be a bit easier, but the perl script I use to look for abusive customers also takes quite a while with a large amount of uses. I've spent a bit of time optimizing my script, so if you want to post it, I'll take a look and see if it can sped up. --Doug
Re: [vchkpw] vpopmail vpasswd file difference(s) between 4.9.10 and 5.3.8
That was it, thanks HD. Upon comparing his vpasswd file and mine, there's an extra field in the vpasswd file for the clear text password. If I merely add another colon to the end of each line in the vpasswd file, vconvert works just fine. I'd suggest updating the vconvert program so that it does not require the clear text password field to be present in order to function properly. That will make legacy installs much easier to upgrade. Lastly, I'm not entirely certain *why* running vconvert against my older vpasswd file deleted all mysql entries in the vpopmail table but it's not a safe behavior. Maybe it just removes all entries that match the domain before importing and then does the import. It would be a really good idea to verify that there's some new data capable of being inserted before deleting the old entries. Matt On Wednesday, September 25, 2002, at 11:18 AM, H. D. Lee wrote: Hi Matt, On 2002.09.24_16:03:52_+, Matt Simerson wrote: Everything on the new system works just fine except the vconvert function. Can anyone verify that vconvert actually does work? Yes, it actually works. Below are the steps I have taken to come up with that statement: 1. Compile and install 5.3.8 with cdb. 2. Test create a domain (devel.dutnux.com) and some users. 3. Recompile with mysql support, install, using 5.3.8. 4. Invoke ./vconvert -c -m -v -d devel.dutnux.com version: 5.3.8 could not connect to mysql update server Unknown database 'vpopmail' with database converting devel.dutnux.com done 5. I checked mysql, it created vpopmail database and table for me. The vpopmail table contain exactly the user I created. Can you sanitize a copy of a newer vpasswd file and send it my way so I can compare it to my ancient system to see if the format is still the same? This is the first line of the vpasswd file I have. It was created using 5.3.8 with password learning support: postmaster:$1$R.O7J$cRKPnHXAkWuZ6epI94R.k/:1:0:Postmaster: /home/vpopmail/domains/devel.dutnux.com/postmaster:10M:dutnux Sorry for the wrapped line. Matt HTH -- H. D. Lee
[vchkpw] vpopmail vpasswd file difference(s) between 4.9.10 and 5.3.8
I'm rebuilding a dinosaur mail server that I built a couple years ago. It's running vpopmail 4.9.10 using cdb authentication. I'm upgrading to 5.3.8 with MySQL authentication. I've build out the new 5.3.8 system and cluster of machines. It's all up, running and happy. I've NFS mounted all the home directories from the old cluster onto the new system so that I have ready access to the vpasswd and vpasswd.cdb files. They are mounted onto the local filesystem so that the paths in /var/qmail/users/assign are correct. I run vconvert -c -m test.domain.com and it exits quietly indicating success: mail8# ~vpopmail/bin/vconvert -c -m -v -d test.simerson.net version: 5.3.8 converting test.simerson.net done However, when I check the MySQL server, there's no entries in the vpopmail table that correspond to that domain. :-( I've tested this one step further. I've manually added a valid user ([EMAIL PROTECTED]) into the new mail system by manually adding it into the MySQL table. It works fine, authenticates, etc. That basically makes it work with my new mail system and the old one at the same time. Cool. Anyway, if I run vconvert again, it'll actually delete that entry from the vpopmail table. Is that supposed to happen? Everything on the new system works just fine except the vconvert function. Can anyone verify that vconvert actually does work? Can you sanitize a copy of a newer vpasswd file and send it my way so I can compare it to my ancient system to see if the format is still the same? Matt
Re: [vchkpw] Using tcpserver/mysql
On Friday, September 6, 2002, at 07:10 AM, David Phillips wrote: Paul Fries writes: Yeah I have been really pulling my hair out over that one. =) With as many Pop-3 connections/sec that our servers handle I thought that I would make things a little smoother if Vpopmail didn't write the cdb. I am afraid that the hash would become corrupt and cause relay problems for my users. Assuming that vpopmail has proper locking for the tcprules(1) source file and uses unique temporary files everytime it runs tcprules, then this isn't a problem. tcprules atomically updates the cdb file. In theory, you are correct David. In practice, that file gets truncated regularly on a high volume mail server. Discovering that, and subsequent problems because of it caused me to write the mysql patch in the first place. Matt
Re: [vchkpw] Using tcpserver/mysql
Paul, You are exactly correct. Even when using my patch, vpopmail continues to update the tcp.smtp.cdb file. There's one reason for that. The first is that my suggestion for adding a --dont-update-tcprulesfile=true was never implemented. Granted, it would only benefit a few users (really big ones like me) and it really doesn't make that much difference. So, in short you are right. You can do as I do and manually disable the routines in the vpopmail code that writes out the file, or you can just ignore it. If you're really clever, you can code in a --don't-update-tcprulesfile function but to save some time, you might want to search through the archives of this list as I've already been down that road. Matt On Tuesday, September 3, 2002, at 12:08 PM, Paul Fries wrote: Greetings, I am trying to accomplish using vpopmail with the tcpserver/mysql patch. Based on the documentation found on step 13 of http://www.inter7.com/vpopmail/INSTALL: -- For sites using the mysql module and --enable-roaming-users=y it is highly suggested to use Matt Simersons tcpserver-mysql patch. This removes the need for vpopmail to compile a tcp.smtp.cdb file for each pop authentication. -- I have implemented the patch, and verified that tcpserver is reading my /var/qmail/control/sql file properly (if I chmod 000 this file, tcpserver complains about it on startup). Also with --enable-roaming-users=y in vpopmail, vchkpw properly writes the authenticated IP addresses/timestamp to the vpopmail.relay table in MySQL. My problem is this: Even though it is properly writing the IPs to the vpopmail.relay table, it is also writing them into the tcp.smtp.cdb hash file. I would like it to ONLY write to MySQL and not to the .cdb. I have tried --enable-roaming-users=n, but this causes vchkpw to stop writing the IP addresses to either location. It is almost like there should be a --dont-update-tcprulesfile=true variable upon configuration. Am I missing something here? Has someone else configured vpopmail in this manner and gotten around this problem? Thanks! Regards, Paul Fries [EMAIL PROTECTED]
RE: vpopmail w/sqwebmail
Title: Message Gee, I think so. One of my mail systems has 6,000 domains and way more users than that. shameless plug Someone even went through the bother of documenting how to do it at http://matt.simerson.net/computing/qmail/qmail.toaster.shtml /plug Matt PS: Version 2.0 of the recipe will be finished Really Soon Now (within a couple weeks) and includes vpopmail 5.0, sqwebmail 3.2, courier-imap 1.3.12, newest qmail smtp-auth patches, qmail-tls, and other goodies. -Original Message-From: Chris Zecca [mailto:[EMAIL PROTECTED]]Sent: Wednesday, December 05, 2001 9:37 PMTo: Vchkpw@Inter7. ComSubject: vpopmail w/sqwebmail I am looking at setting up a web based freeBSD mail server that can handle 40,000 users and more. Can vpopmail with sqwebmail handle that amount of users? Has anyone had any experience using vpopmail and sqwebmail with a large amount of users? Best regards,Chris
vpopmail auth question
Just wondering really quickly, what types of encryption does vpopmail use? I'm guessing that on FreeBSD, if I have the default set to MD5, then vpopmail will only be able to verify MD5 passwords. Is that correct? If so, how do I extend it to support DES as well (like FreeBSD's crypt routines)? Is it just a matter of updating FreeBSD's libcrypt and then recompiling vpopmail? Matt
RE: vpop and pine..
Hardcore unix veterans don't use PINE. :-P Hardcore folks won't use anything that doesn't use vi as it's editor. I'm not hardcore, I've only been using unix for 12 years so I use pine because gack a) I've already hacked it to support Maildir b) It supports POP/IMAP accessing my other mailboxes c) it's installed on every mail system I have an account on (including those I don't administer). Of course you have to be running PINE as the user that the maildir is owned as but I do it this way on my personal mailbox: ~matt/domains/simerson.net/matt/Maildir is my maildir. In my $HOME I have a symlink that points Maildir to that directory. It works just fine. Matt -Original Message- From: Alex Hathaway [mailto:[EMAIL PROTECTED]] Sent: Wednesday, July 18, 2001 11:16 AM To: [EMAIL PROTECTED] Subject: vpop and pine.. I'm using qmail+vpop+qmailadmin and I'm curious if there is a way to use pine w/ this setup? I have some hard core unix veterans who are whining. -Lexx
RE: Hardware platform
Title: OT: Hardware platform It all depends on a lot of factors, most important ones being the speed of your disk(s), the drive layout, drive optimizations, MTA performance optimizations, activity of users, and a whole bunch more factors. I wouldn't put many more than 10,000 on any one system although it mightbe able to handle it. Matt -Original Message-From: Leonard Cooper [mailto:[EMAIL PROTECTED]]Sent: Tuesday, June 12, 2001 11:11 AMTo: '[EMAIL PROTECTED]'Subject: OT: Hardware platform Hi All, I know this is WAY off topic, and even out of context for the list, But I need an Answer on this ASAP. How many users can we support on the following platform? Compaq Alpha DS10 1Gb Ram 4x100baseTX - peered to 400mb/s All SMTP / POP3? A guestimate as to how many pop3 users I could support, 80% on Dial-up? Thanks Leonard
RE: Port of Matt SImerson's Mysql Relay + cdb auth patchtovpopm ail-4.9.10
Oh, _that's_ what you did. Yes, you were correct. You should probably refer to the version of my toaster recipe when you specify what you did. Then I can figure it out better. I have systems that use each type of recipe (full mysql relay only mysql) and once the replication is all sorted out for MySQL authentication I'll convert my CDB based systems over and stop maintenance of the .cdb version. If you plan to maintain using the .cdb format with MySQL relay, I can link to your pages. Matt -Original Message- From: javier [mailto:[EMAIL PROTECTED]] Sent: Saturday, June 09, 2001 8:23 PM To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: RE: Port of Matt SImerson's Mysql Relay + cdb auth patch tovpopm ail-4.9.10 Either am going crazy.. or the difference between your 4.9.8 versus the 4.9.10 patch is that with the 4.9.8 patch you could have the mysql open relay table, and have cdb auth. In your 4.9.10 patch, you can only have the mysql relay table IF you use mysql auth. Am I correct on the 4.9.10 patch? The ported patch allows for cdb + mysql relay on 4.9.10, versus your current 4.9.10 patch, which only allows mysql relay when people use mysql auth. am I goign crazy? ( more than I am already hehe ) ;) -Jav
RE: Error trying to compile + new mysql replication release
-Original Message- From: Ken Jones [mailto:[EMAIL PROTECTED]] Sent: Tuesday, June 05, 2001 3:01 AM To: Gabriel Ambuehl Cc: [EMAIL PROTECTED] Subject: Re: Error trying to compile + new mysql replication release I guess we could do that. It would be easier to impliment a list of servers and go down the list untill one is found that accepts the connection. That works for me, there's even a dbping function you can use to do that. You can see it used in my cdb-mysql patches. Which brings up another point. Where do the servers get defined. Currently they are compiled in via the vmysql.h file. Which is easy to implement. I prefer to have them defined in a configuration file. I don't worry about the fact that users could view the file because on a vpopmail system with each domain stored in a /etc/passwd user, it's pretty insecure to just allow users to interactively log into the system. I just plain don't allow it so I don't mind having a config file floating around with the password in it. Back in the day when I had to provide telnet/SSH logins I just created a seperate machine for that purpose and everyone else should take that advise to heart. Another option would be to put the list of servers in some configuration file that would get parsed. But I can't see a secure way to attempt to hide the passwords. That's because you can't. If you crypt them then you can't present the cleartext version for the challenge. The password must be stored clear text. Since domains can be stored under any /etc/passwd account, then any user on the system would need to have access to the file hence they could find the login information. Anyone have any thoughts about this? It would be great to be able to compile one binary and use it on multiple machines with different mysql server auth info. Ken Jones Which is what I do, and that necessitates having a config file. I suppose you could be really silly about it and hash the password using something like Cisco's level 7 that you can easily decrypt but if you can easily decrypt it, so can someone else. So, I say don't worry about and don't let users log into your mail server(s) interactively. Matt
RE: Vpopmail+Qmail -- pop3 has lost it's mind!
Make sure the clocks are syncronized on all the POP servers. Having incorrect or out of sync dates will cause such fun stuff. Matt -Original Message- From: David Gartner [mailto:[EMAIL PROTECTED]] Sent: Wednesday, May 30, 2001 10:28 AM To: [EMAIL PROTECTED] Subject: Vpopmail+Qmail -- pop3 has lost it's mind! Question! I've got a high availibilty cluster running qmail and vpopmail. Three computers mount /home from an nfs server. In the home directory is the vpopmail directory (all vpop apps and config's are in this dir). I have made /var/qmail/control and /var/qmail/users match on all nodes. When you pop from the nfs server, the users have mail. But if you hit one of the nodes, it authenticates fine, but says there's not mail. Anyone know how I broke this and how I can fix it? Here's what I do to test this-- This is on the NFS server Lists the mail fine... [root@mail(nfs1) /root]# telnet localhost 110 Trying 127.0.0.1... Connected to mail... Escape character is '^]'. +OK 4864.991244767@mail user me +OK pass test +OK list +OK 1 212 . This is on any of the nodes ... Where's my mail?!?! [root@mail(node1) /root]# telnet localhost 110 Trying 127.0.0.1... Connected to localhost.localdomain. Escape character is '^]'. +OK 735.991234046@mail user me +OK pass test +OK list +OK . Many thanks! David Gartner
RE: mysql high availability
I've been down this road and that's a terrible way to do it. Why? Because for every SMTP connection you have to initiate a connection to the database. That's expensive. What you really want to do is have the tcpserver program make a persistent connection to the database and query it for every SMTP session. I couldn't think of a better way to do it so I hacked up tcpserver and added MySQL support to it. It reads a control file (/var/qmail/control/sql) in the following format: server sql.domain.com port3306 databasevpopmail table relay userusername passsecret time1800 When/if the persistent connection drops it automatically retries. If it fails then it allows the SMTP session anyway but does not set RELAYCLIENT. It retries every time seconds. The patch is available on my web site: http://matt.simerson.net/computing/mail.shtml and is found under the link entitled tcpserver MySQL patch Matt -Original Message- From: Sean C Truman [mailto:[EMAIL PROTECTED]] Sent: Wednesday, May 30, 2001 6:50 AM To: Ken Jones; Dave 'Duke of URL' Weiner Cc: [EMAIL PROTECTED] Subject: Re: mysql high availability Ken, A good way of getting around the NFS locking problem is to insert another program inbetween tcpserver and qmail-smtpd that checks a database for a relay flag (Which can be set when you call open_smtp_relay()).and that program sets the RELAYCIENT. Sean - Original Message - From: Ken Jones [EMAIL PROTECTED] To: Dave 'Duke of URL' Weiner [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Sent: Wednesday, May 30, 2001 7:28 AM Subject: Re: mysql high availability Hi Dave, Dave 'Duke of URL' Weiner writes: This is probably worth talking about now, since we are working on the mysql module for the next several weeks. Using Matt's read/update patch as a starting point the mysql functions are broken up into read-only queries and update queries with separate definitions in vmysql.h for two mysql login connections. One for updates and one for read only. I don't have any clean ideas yet on how to handle the three cases most people are running into: 1) one mysql server for everything 2) replicated mysql system, with one read only server (slave) and one update server (master) 3) a primary server and a backup server Anyone have any ideas? Why yes, Ken, I do :) First off, a very good start. As you and I have discussed before, I was atempting a primary server with two slave servers replicating off the master. Using Ben's patch, everything seemed to work quite well, using the hostname of the master as the Write server, and localhost as the read-only server. In testing, when the master write server was shut down, users could not authenticate via pop, as the login routines were attempting to write the ip address and other info for the pop-before-smtp files. My only suggestion would be not to store the pop-before-smtp data in the MySQL database. That way, you end up with, IMO, a fault tolerant system for MySQL authentication. Yes, I was looking at that. The first thing that I ran into is trying to minimize database transactions during a pop access. The authentication part is easy, just read the user info from mysql. There are two optional transactions that do updates during an authentication 1) the roaming users --enable-roaming-users=y 2) authentication logging --enable-auth-log=y It seems pretty wasteful to have the authentication mechanism open two database connections, one for the read only auth and one for the two possible updates. But I really can not see a way around it. For the roaming users stuff to work, the IP addresses have to be centralized. Otherwise in a clustered setup once they pop, they may or may not be able to use smtp relaying. We have tried using a file based system with a NFS mount, but it runs into synchronization problems when multiple machines are updating the file at the same time, and locking doesn't help because NFS locks will fail. One thing we've been looking at is having fall back servers for both the read and update mysql connections. Meaning: if a connection to one fails, then it attempts a connection to the fallback machine. The other thing is the authentication needs to be compiled into the binaries. One other option is a patch from Ondrej that gets the mysql auth data from environment variables. Ken Jones
RE: mysql high availability
Oh trust me, I know that. :-) My point was that it's much less expensive to have tcpserver open up a persistent connnection to the MySQL server and keep it open rather than launching another (albeit small) program that connects to and queries the database. I'm only using MySQL for my SMTP relay (custom hacks) and I'm doing an average of 19 queries per second to the database over the last 60 days. Queries are only generated by SMTP sessions, updates are triggered by successful POP or IMAP authentications. Doing it my way, every incoming SMTP request only does a query. With your method you have to open a TCP connection, connect to the database, authenticate, query, and finally drop the connection for every SMTP session. I have seven of these mail toasters, the largest one handling an average of 1.7 million SMTP connections per day spread across 4 machines. That's the one I care about most and will refer to throughout the rest of this email. MySQL has no problems keeping up. It's running on a Quad Xeon 550 with 2GB of RAM and it only uses 5-10% of one CPU. When we have HA available for MySQL I'll move all the authentication from .cdb files into MySQL and the load on my MySQL server will rocket upward. It's to everyone's advantage to make every piece of the system as lightweight as possible. When that happens I'll probably also hack most of the functionality of vchkpw into tcpserver as well. Then all the POP3 connections (which represent the majority of mail retrieval) will trigger SQL updates via a persistent connection as well. At that time I'll probably try to twist Ken's arm into including the modified tcpserver as a compile time option to vpopmail. Why is persistent connections so important? Your system has a finite (normally tuneable) number of TCP connections that it can have open at a given time. Why waste a 100 of those connections to MySQL when you can have one persisent connnection? Why waste the CPU overhead necessary to open up and tear down an extra 100 TCP connections per second? Need more reasons? Read the very first tip given on MySQL's site about optimizing performance. It reads: Use persistent connections to the database to avoid the connection overhead. If you can't use persistent connections and you are doing a lot of new connections to the database, you may want to change the value of the thread_cache_size variable. See section 13.2.3 Tuning Server Parameters. It's also much easier to troubleshoot problems when you can look at your SQL servers processlist and only have 8 connections in there (two for each server (tcpserver SMTP tcpserver POP3)). It's cleaner, faster, and uses less resources. Another thought on this: What happens when you are using multiple SQL servers (for HA) and the primary is down? With having a small program that gets called by tcpserver, it's going to try to connect to the first SQL server and will have to timeout before it tries the second. That's painful, and every POP and SMTP session will then get a timeout length pause before their connection succeeds. When tcpserver opens the connection, if the persistent connection to the primary server drops, it'll try to reopen it. If it failes to reopen, I could add some logic to get it to connect to a secondary server. Then only the connections that arrive during the time tcpserver's connection is retrying have a pause, once it's failed over to the backup MySQL server(s) all connection resume as normal. This is also the way the RADIUS protocol handles failover. The client sends all requests and logging to the first configured server until that server stops answering. It then fails over to the next server and stays there until that server dies (at which time it'll try the first again). If we want connections to return to the first MySQL server we can have tcpserver periodically attempt to reconnect to the primary SQL server. Matt -Original Message- From: Sean C Truman [mailto:[EMAIL PROTECTED]] Sent: Wednesday, May 30, 2001 12:52 PM To: Matt Simerson; Ken Jones; Dave 'Duke of URL' Weiner Cc: [EMAIL PROTECTED] Subject: Re: mysql high availability Matt, I can provide you with load specifications. MySQL can handle the work load. I have one customer site running 5 Node Cluster, This does over 500,000 messages a day. Attached is a extended-status from the mysql database server, a show of load average on the machine and also shows the table structure and record count. Machine Specs. Pent III 700Mhz 512MB Ram. With this cluster we are checking the database for relay and or deny flag every single connection. Point being that this setup is not even stressing the database and MySQL is a champ!. Sean - Original Message - From: Matt Simerson [EMAIL PROTECTED] To: 'Sean C Truman' [EMAIL PROTECTED]; Ken Jones [EMAIL PROTECTED]; Dave 'Duke of URL' Weiner [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Sent: Wednesday, May 30, 2001 1:33 PM Subject: RE
Incomplete Feature (vmysql qmailadmin)
Hi Ken (et al),
I've found another place where the multiple UID/GID feature isn't quite
completely implemented. I've built another mail toaster with
qmail/vpopmail/qmailadmin and each domain is a unique UID/GID. The main
difference is that I am using MySQL authentication on this one. I've got the
server up and running happily but users can't change passwords in
qmailadmin. Whenever they attempt to they get a Failed to change password.
user domain.com newpass -22 error.
A little legwork tracked that back to error 140 in the html files. I tracked
that back to the function modusergo in qmailadmin/user.c which makes a call
to vpasswd (which of course is linked against the vpopmail library. So, back
in the vpopmail/vpopmail.c we find our vpasswd function which calles
vauth_vpasswd which is in the vmysql.c file. In there, we end the chase and
find the offending code:
myuid = geteuid();
if ( myuid != VPOPMAILUID myuid != 0 ) {
return(VA_BAD_UID);
}
Note of course that this does work just fine from the command line because
the commands are being run as root. However, when run from qmailadmin via
HTTPS we call qmailadmin (which is suid root.vchkpw) which ends up running
as the UID/GID of the domains owner. In this case, test.com is owned by user
test so qmailadmin is then running as test. We can confirm this by adding
the line: printf(my effective uid is: %d\n, myuid); to the C code and
recompiling. Sure enough, it adds a line to the HTML page reflecting the uid
of test. The end result is that anytime a user tries to change their
password, they get the error message I indicated earlier.
If I simply comment out the if statement, everything works just fine but I'm
wondering if that's a good idea or not. Is there some error and
authentication checking in there somewhere so that [EMAIL PROTECTED] can't
forge a HTTP request for [EMAIL PROTECTED] and change his password?
Matt
RE: Couple of problems..
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Thursday, April 26, 2001 1:53 PM To: [EMAIL PROTECTED] Subject: Couple of problems.. Well, this definitely didn't go over as smoothly as I had hoped..and of course, no one called to let me know that until I got back into the office. It's just so hard to find good help. First, one should not blame the software or hardware for the ineptitude of the system operator. Alright, problem 1. All of the users who were under the primary domain account, who only required a username (none of that @ or % stuff)cannot authenticate at this time. If, however, I add the primary domainname to the users account name..all works well. BUT, this is what I've been trying to avoid!! I do not want to have to go through and instruct a couple of hundred people on changing their authentication information. That would not be fun. Did you RTFM? Liar, go back and read the documenation that comes with the package. Pay special attention to the first question in the FAQ. Second problem, I have two people named fred..with different domains..who have received each others mail. How the hell does that happen?!?! It's your system, you figure it out. Qmail follows a strict set of delivery rules to determine which domain directory to deliver mail to. Vdelivermail obeys the vpasswd file. It's pretty darned simple to trace. I've spent WAY too much time on this. The old machine worked pretty flawlessly from day one, Why then did you touch it? and it's only about a year old (I cannot quote from memory at this time as to the versions..but, I know it's not what I was working with now). I'm pretty much having the feeling that I should have stuck with the versions I had before. I wouldn't have had to go through so much shit with wanting to simply setup a new server and port people over from the old one. This software definitely isn't made for such a feat. Why not? You're spouting FUD with absolutely no details to demonstrate what happened, how, configuratations or anything. This last week I converted a single machine vpopmail installation over to a clustered system running or 5 machines. I had the system offline for _1_ whole minute while I made the cutover with no loss of mail and an imperceptible amount of downtime. There are no limitations in the software that prevent this, just limitations imposed by the incompetence of the sysadmin. So, now, anyone have any suggestions on problems 1 and/or 2? Now that I have my bitching out of the way, I can safely get back to work. I don't think I'd agree with the safely part. Any help would be appreciated. Nobody has a chance of helping you. This post contains no relevent details about either system. How can we see what you screwed up? Matt
Items of interest
Hi all, RFC 2821 and 2822 are official now: ftp://ftp.isi.edu/in-notes/rfc2821.txt ftp://ftp.isi.edu/in-notes/rfc2822.txt They replace the 821 and 822 RFC's for SMTP mail servers and services and message formatting and make for some very interesting and insightful bedtime reading. The latest version of the vpopmail toaster recipe is online at http://matt.simerson.net/computing/qmail/qmail.toaster.shtml. New features include the creation of a archived and searchable mailing list for toaster recipe users. Updates include conversion from ApacheSSL to Apache modssl (better support, faster updates, etc..), using MySQL authentication. I'm getting dangerously close to being able to release my log processing scripts which feed cricket and make very pretty graphs. Matt
Bug fix and feature request.
Ken (et al), I have a couple modifications I'd like to see in vpopmail-4.9.11. I haven't seen any of this in the changelogs yet... First: Bug fix. Fix the vdelivermail crashing problem I described a while back: postmaster set's user account bob as default delivery. user deletes bob's pop account messages delivered by default rule (for bobs maildir) crash vdelivermail. If a directory is specified for mail delivery, vdelivermail should verify that it exists and if not, bounce the message (or something more useful than a core dump). Second: Feature request: Add a --enable-roaming=y-but-no-cdb option. If enable-roaming is set to "y-but-no-cdb" then go ahead and stuff the IP address and timestamp into the SQL database but don't update the open-smtp file or the tcp.cdb file. I've already hacked support into tcpserver (publicly available on my web site) to query the MySQL database directly so constantly updating those files is a waste of CPU and disk. Code changes to accomplish this are at: http://matt.simerson.net/computing/qmail/vpopmail-4.9.10-opensmtp-patch.txt Third: Feature Request: Enhanceme the vpopmail/relay table. Right now it only puts the ip_addr and timestamp in the database. I've added another field entitled "name" that stores the authenticated username. I've also written a few quick PHP scripts that make that table searchable by our tech support team. It's a very handy tool when a customer complains about not being able to relay: "Tech: What's your email address? (plugs it into the form) Oh, I see that you've logged in 4 times today but none during the last 1/2 hour. You need to POP your mail again and THEN you can send messages." Of course, it's useful for a lot more but it's much more convenient to search for than the IP address the use connected from. Since we're storing the info in a SQL table, we only clean out the table once a day. It's much faster to query than log files. :-) name char(65) default NULL is what I'm using and it's also in the patch file listed above.
RE: Converting to MySQL
I've written a perl script that goes through and counts every individual user in every vpassword file on the system (to see how many users exist). It would be pretty easy to parse through all the vpassword files and dump the contents into a MySQL table. I've been tempted to do this but myself (convert to MySQL) but I've opted not to. I have a large number of domains and the majority of the domains only have a small number of users. It's far more efficient for the system to read a tiny vpasswd.cdb file for every delivery and authentication than to make a database query. Matt -Original Message- From: Ben Beuchler [mailto:[EMAIL PROTECTED]] Sent: Wednesday, April 11, 2001 3:43 PM To: Vpopmail List Subject: Converting to MySQL Just curious: Has anyone come up with a spiffy way to continue to function off cdb while converting to sql? My only other option is to come in in the wee hours of the morning... Thanks, Ben -- Ben Beuchler There is no spoon. [EMAIL PROTECTED] -- The Matrix
RE: HELP, I'm on the ORBS list
Go to google, type in "qmail anti-spam" and hit enter. Learn the tools you use, it'll keep surprises from biting thee squarely in the Matt -Original Message- From: Calvin Crutcfield [mailto:[EMAIL PROTECTED]] Sent: Tuesday, April 10, 2001 12:30 AM To: vchkpwd Subject: HELP, I'm on the ORBS list I set up qmail/vpopmail a few months ago and the insatll went really well. I thought I remembered something about qmail only acceptimg outgoing mail from people who have already authenticated...thus elmintaing the problem for open relays. Ok I was naive and wrong. How do I investigate? How do I fix it!!! calvin
RE: Whats going on?
Ken, Did you install an intelligence filter. ;-) My messages get right through. Matt -Original Message- From: Daniel Hardaker [mailto:[EMAIL PROTECTED]] Sent: Monday, April 02, 2001 4:31 PM To: [EMAIL PROTECTED] Subject: Whats going on? Just wondered whats going on with this mailing list? Messages seem to be taking days to get through. I sent one 3 days ago and it has only just been sent out Has it got a really big queue or something? Or is it just some routing probelm between servers? Anyone else had any trouble with it? - Daniel --- www.iv2.co.uk
RE: vpopmail and smtp-after-pop
More importantly is the contents of your tcp.smtp file. If you've got a 0.0.0.0 in there, or just a default :RELAYCLIENT line, then you are allowing everyone to relay. I've got a hacked version of tcpserver that will check the SQL table directly for each SMTP session to determine if that IP is allowed to relay. I suggest using that. Check my web site for more info: http://matt.simerson.net/computing/qmail.toaster.shtml. Matt -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Monday, March 26, 2001 10:47 AM To: [EMAIL PROTECTED] Subject: Re: vpopmail and smtp-after-pop Can't somebody help me? It's very important! [EMAIL PROTECTED] wrote: Hello, I'm using qmail 1.03 in combination with vpopmail 4.9.8-1. So far it works good. The only problem is smtp-after-pop. I enabled the support with the --enable-roaming-users=y option. Remark: I'm running vpopmail with MySQL support. (--enable-mysql option) OS Is SuSE Linux 6.3 Kernel 2.2.13. The POP-Server and the SMTP-Server (qmail) are startet with tcpserver. ... env - PATH="/var/qmail/bin:/usr/local/bin" \ tcpserver -H -R -x /home/vpopmail/etc/tcp.smtp.cdb \ -c20 -u510 -g101 0 smtp \ /var/qmail/bin/qmail-smtpd 21 /dev/null env - PATH="/var/qmail/bin:/usr/local/bin" \ tcpserver 0 pop-3 /var/qmail/bin/qmail-popup pc.domain.com \ /home/vpopmail/bin/vchkpw /var/qmail/bin/qmail-pop3d Maildir When somebody gets mails with POP3 an entry with the current IP of the POP-User is added to the 'relay' MySQL table. The problem ist that EVERYBODY is able to send mails through my SMTP server even if his IP is NOT in the 'relay'-table. What's wrong? Regards Peter Baecker What's in your rcpthosts file? /var/qmail/control/rcpthosts is empty. Regards Peter -- GMX - Die Kommunikationsplattform im Internet. http://www.gmx.net
RE: How many Deliveries?
How many processes can your machine handle? How fast is your IDE drive, how much other crap are you running on the server. There is no single answer to your question other than "it depends" and it depends on a lot of stuff. With only 500 domains, your concurrent settings are probably just fine. I've got a server with 2500 domains, and I leave my local at 10 and remote is cranked up to 255. That is obscenely high and way more than necessary for most situations. Twenty is reasonable, 50 might be a little better depending on the activity levels and load of the machine. You seem to care about your mail server yet you use IDE drives. Why is that? Matt -Original Message-From: Sumith Ail [mailto:[EMAIL PROTECTED]]Sent: Saturday, March 24, 2001 2:58 AMTo: [EMAIL PROTECTED]Subject: How many Deliveries? How many concurrent local deliveries can Vpopmail handle ... Right now it is 10 local and 20 remote @40003abc7c1f041cc1dc status: local 1/10 remote 0/20@40003abc7c1f0514413c delivery 23701: success: did_2+0+0/@40003abc7c1f05145c94 status: local 0/10 remote 0/20@40003abc7c1f05146464 end msg 1376675 What would be the recommended number for local and remote deliveries on our qmail+vpopmail4.9.8-1(cdb) running on p3, 512 mb ram, ide hdd with around 500 domains. Regards Sumith
Mail toaster HOW-TO updated
Hi vpopmailers, I've updated my mail toaster recipe at http://matt.simerson.net/computing/qmail.toaster.shtml. The primary differences are: 1) includes code modifications to vpopmail to use MySQL for ONLY the open-smtp relay feature. 2) includes code modifications to tcpserver to query a MySQL database for relayclient information instead of the standard tcp.smtp.cdb file. 3) includes the maildomain Perl script I've written to manage my mail toaster. It's got all sorts of nifty features for allowing non-root users to add/edit/delete mail domains, check logs, fix problems, etc.. It's pretty cool. Someday soon I hope to release a few more scripts: 1) maillogs (processes log files and returns values to be stuffed into cricket/mrtg. I have an snmp agent installed on each of my mail servers. I then install maillogs, a custom snmp.conf and then have cricket poll each of the servers and log the results. https://vpopmail:[EMAIL PROTECTED]/admin/cricket/mail.html 2) logmonster - http log file processor I have a group of load balanced http servers. Logmonster collects the log files off each server, sorts them into cronological order, parses out the results for each virtual domain (if enabled) and feeds the results through your http log file analyzer. https://admin.mail.safeserver.com/admin/usage/ 3) update - clustered server syncronizer Uses ssh, rdist, and/or rsync to keep files constant among groups of servers. Also autogenerates httpd.conf and named.conf files but currently has too many dependencies to release on the public.
Qmailadmin allows deletion of default mail account
Currently qmailadmin will allow a user to specify a default (catch all) account and then delete that default mailbox. This of cource causes vdelivermail to crash when attempting to deliver messages. This could be fixed in one of two ways. a) don't allow a default account to be deleted b) if the default account get's deleted, set the default to bounce-no-mailbox Matt
RE: Vpopmail Pine
Pine works just fine as POP3 or IMAP client so have your users configure PINE on the shell server to access their email via POP or IMAP. How much easier can it be? Matt -Original Message- From: Tim Hunter [mailto:[EMAIL PROTECTED]] Sent: Monday, March 12, 2001 11:06 AM To: [EMAIL PROTECTED] Subject: RE: Vpopmail Pine It doesn't even make sense. vpopmail is for virtual users, not shell users. -Original Message- From: Jiøí Mikulá¹ [mailto:[EMAIL PROTECTED]] Sent: Monday, March 12, 2001 2:00 PM To: [EMAIL PROTECTED] Subject: Vpopmail Pine Hi all. I'm sorry for maybe OT Anybody using Vpopmail with PINE. Is there some guide or FAQ how to configure PINE for Vpopmail ? Preji pekny den Jiri Mikulas [EMAIL PROTECTED] ICQ: 108 36 60 66 Admin - anglictina.com
Feature request and error with vmoduser program
I have two issues: The first is a feature request for the vmoduser program. It appears that at present there is no way to disable qmailadmin. This would be a most welcome addition to the vmoduser program and hopefully more people than I would find it useful to be able to easily disable a domains qmailadmin capabilities. The second matter is the vmoduser command is supposed to be able to change the parameters for an entire domain. For example, I have test.com with two users. I run the following command: $vmoduser -p -i -w [EMAIL PROTECTED] and it correctly changes the gid bit in the vpasswd file to 14. When I try to do it for the entire domain: $vmoduser -p -i -w test.com It silently fails without doing anything. Matt
Sqwebmail and scaling beyond big to REALLY big.
Hi List, I have an interesting little story that all of you might find quite interesting. It has a happy ending but I'd also like a little feedback from the rest of you on other possible solutions to this. Six months ago I built, and pushed into production a mail server, running on 5 FreeBSD 4.1 servers, running qmail, vpopmail, sqwebmail, courier-imap, and all the trimmings. The original design intent was to develop a server that would support roughly a million email users. Scalability was, of course, of paramount importance in such a solution. The architecture is pretty standard for large shared environments. Once machine is a file server. It's got 300GB of RAID storage hanging off a scsi card and connected to the other 4 machines via a gigabit ethernet controller. That should last for quite some time I'm thinking. :-) Once I exceed that file servers ability I can slide up to 25 more file servers into the equation for nearly limitless storage and several T3's worth of mail bandwidth. That should be enough for a while. ;-) Anyway, since that time, the main problem I've been having has been the implementation of the pop before smtp authentication for relaying. The way it's implemented, by default, is pretty simple. A user POP auth's, and upon successful authentication we stuff their IP address into a file ~vpopmail/etc/open-smtp and compile that into the tcp.smtp.cdb database which tcpserver consults to determine if the IP is allowed to relay. Pretty simple stuff really. That all worked fine and dandy until somewhere around 1300 domains. I'm not sure how many users that equated to but I'll guess around 3,000. So, I had 4 mail servers, all configured identically, all sharing the same file system for local user mail spools (via NFS), and all sharing a common ~vpopmail/etc/tcp.smtp.cdb file to determine if a user is allowed to relay. At around 1300 domains we started seeing the ~vpopmail/etc/open-smtp file getting munged. At that time, each machine was seeing nearly one POP auth per second at peak times and, consequently, trying to update that file. As a result, the file got munged quite often during the middle of the day, users couldn't relay, and the phones in support started to ring. Since I already had 1300 vpasswd files strewn around the file system, the idea of converting entirely to MySQL wasn't really an appealing option. The solution then was to hack up vpopmail to use the pop-auth code that stuffed the IP's into a MySQL table. So, I quickly hacked up the code, recompiled vpopmail and shoved the new programs into production. Wahoo, the table got populated quite rapidly with hundreds of IP's and life was happy again, for a while. Two weeks ago I left work for France to spend a while with friends, drinking wine, eating well, and skiing in the Pyrenees. While I was gone, a new problem surfaced. While the IP table is being stored in MySQL, it still gets recompiled into the ~vpopmail/etc/tcp.smtp every time a POP session authenticates successfully. At this time I have some 2600 domains and over 10,000 users on the system (I wrote a perl script to figure that out by finding all the vpasswd files and adding up all the lines in the files :-)). Now that all four servers are seeing in excess of one POP auth per second, that file was getting written up to four times per second. Tcpserver would try to access the tcp.smtp.cdb file and get a stale NFS file handle and drop the connection. So, the phones started ringing because the SMTP server was intermittently dropping the connections. What to do? Well, we chose the most obvious solution. Hack up tcpserver to check our MySQL table directly instead of the .cdb file. I had one of our senior programmers tackle this and the results are great. The new enhanced tcpserver, when run with the -S flag, checks for /var/qmail/control/sql and open finding it, follows it's instructions for connecting to the sql server. Then, for every incoming SMTP connection, it checks the database for the IP and, if found, sets the RELAYCLIENT environment variable. It's pretty darned cool and works like a charm. Consequences? So far, so good. I've removed the -x tcp.smtp.cdb flag from tcpserver and only have it consult the database. The -x stuff still works, except that now I have to go back and hack up my hacked vpopmail so that it's stops rebuilding the tcp.smtp.cdb file. Shouldn't be a big deal. Then life should be good for a while. So, has anyone else run into a problem of this sort? How did you solve it?. I've emailed Dan to see if he might (not likely) like to include the SQL stuff in a released version of tcpserver but the odds of even getting a response are pretty slim. So, failing that I guess I'll release a custom version of tcpserver with SQL support. Other ideas? Matt
RE: vpopmail and sqwebmail running as root
I wasn't going to humor such a rude request but since Ken responded, I felt that I should at least point out that sqwebmail must run as root if vpopmail is configured to use multiple uid/gid. Matt -Original Message- From: Ken Jones [mailto:[EMAIL PROTECTED]] Sent: Friday, February 23, 2001 3:05 PM To: Dan Phoenix Cc: Matt Simerson; [EMAIL PROTECTED]; '[EMAIL PROTECTED]' Subject: Re: vpopmail and sqwebmail running as root Dan Phoenix wrote: Complete bullshit pardon the termsplease provide docs on not running as root. I think this should have been done after first sqwebmail remote exploit..thx. chown vpopmail sqwebmail chgrp vchkpw sqwebmail chmod ug+s sqwebmail done Need any more docs? Ken Jones
RE: Virtual User, symlink, Mutt...
Well, what I've done for one of my domains (my personal one) is use vpopmail's multiple uid/gid so that my mail domain is in my personal home directory. So, in this example I'm set up like this: /usr/home/matt/ (that me) /usr/home/matt/domains/simerson.net (one of my domains) /usr/home/matt/Maildir (symlinked to /usr/home/matt/domains/simerson.net/matt/Maildir Since I own all the files within the domain simerson.net, I can use pine to read the mail in that Maildir. I suppose Mutt could also do that if it reads Maildir's. Matt -Original Message- From: Kit Halsted [mailto:[EMAIL PROTECTED]] Sent: Friday, February 23, 2001 4:16 PM To: Matt Simerson; '[EMAIL PROTECTED]' Subject: RE: Virtual User, symlink, Mutt... At 1:40 PM -0700 2/23/01, Matt Simerson wrote: Does it have to be Mutt or can he use Pine? I'm not sure. Pine was pulled from the OpenBSD ports for security reasons, I respect that. OTOH, if Pine can do this I might be willing to take the risk. What can Pine do? -Kit Matt -Original Message- From: Kit Halsted [mailto:[EMAIL PROTECTED]] Sent: Friday, February 23, 2001 11:25 AM To: Ken Jones Cc: [EMAIL PROTECTED] Subject: Re: Virtual User, symlink, Mutt... Sorry, I really should have been more clear. Here's what I'm looking for: A user, let's call him Patrick, has to get POP mail at night when he gets home. He also has to be able to read mail via SSH Mutt during the day. Here's the kicker: it is strictly forbidden for users on this system to send shell passwords out in cleartext, so a .qmail file forwarding to his home dir is out. Use of APOP is also out for various reasons. Have I painted myself into a corner, or is there some permissions magic I can work on his vpopmail Maildir to pull this off? Thanks, snip
RE: Torture test Vpopmail + Qmail
While this isn't the "nicest" method of getting a server hammered, you can forge a bunch of emails with from and reply-to addresses of "[EMAIL PROTECTED]" and then use some spamming software to mail out a few hundred thousand messages. Within minutes you'll get 10's or hundreds of mail servers bouncing the undeliverable messages back to your server. I've never had to do this, I've had the help of a spammer or two, I just got to tune qmail while the server was getting hammered. With my setup, I can handle about 250 simultanous connections per box. :-) Matt -Original Message- From: Sumith Ail [mailto:[EMAIL PROTECTED]] Sent: Thursday, February 08, 2001 7:18 PM To: [EMAIL PROTECTED] Subject: Torture test Vpopmail + Qmail Hello, I have set up Qmail + tcpserver + vpopmail on our RH 6.2 p3 box. We'd like to torture test it. Is there anything more than sending bulk email from a group mail software that we can do. Please give me suggestions as to how can we we continously hit the server with a lot of mails. Regards Sumith __ Do You Yahoo!? Get personalized email addresses from Yahoo! Mail - only $35 a year! http://personal.mail.yahoo.com/
RE: mysql or not.
Well, I found the breaking point of vpopmail's open-smtp feature to be about 1500 domains (in an NFS environment). The actual breaking point isn't really related to the amount of domains/users but rather how many and how freqently your clients open POP/IMAP sessions. I was forced to move away from .cdb to MySQL for the open-smtp (which required a bit of hacking) portions of vpopmail about a month ago. I have yet to convert all my vpasswd files into a MySQL database. I like the fact that all the files are strewn about the file system, easily editable, easily viewable, and easily accessible from every mail server in a large NFS cluster. Having said that, I've found a few problems that stuffing authentication into a MySQL database would fix. Vpopmail's quota mechanism can be tempermental and in practice, I've had mixed results with it. I'm not going to say that this isn't due to some form of operator malfunction (my screwing up) but I know I'm not the only one that's had issues. Because of that, I use system quotas (I only need to enforce them for domains and each domain his it's own UID/GID) and I've had, on a couple occasions, the domains vpasswd file get nuked. I'm guessing it was because the domains mail was over the system quotas and it opened the file (destroying contents) and then couldn't write to it because of quota restrictions. This is just my theory on how it happened. Having the data stuffed into MySQL would prevent this. :-) I've also seen a few races in MySQL recently where my MySQL server just varies off into lala land. It still responds to queries but much more slowly and of course, the system load starts heading skyward. For some reason, I'm not able to kill the MySQL processes (using mysqladmin) that are causing the problems so I end up having to kill the MySQL process and restart it. I've just upgraded to the latest MySQL beta and we'll see if that doesn't help. I also want to play with the replication features. If I can get that working well I'll go ahead and put a MySQL instance on every server in my mail farm and use replication to keep all the data synced. That will be cool. :-) Matt -Original Message- From: Peter Brezny [mailto:[EMAIL PROTECTED]] Sent: Monday, January 22, 2001 1:02 PM To: [EMAIL PROTECTED] Subject: mysql or not. Why use mysql with vpopmail? What are the primary advantages/disadvantages. Is it something really needed for sites 2000 users? TIA Peter Brezny SysAdmin Services Inc.
RE: mysql or not.
I thought about it, read about it, and toyed with the LDAP idea and I think it's got a lot of potential for working well for authentication but for the POP-AUTH type stuff, I don't think it would work well enough on a large scale system handling hundreds or thousands of updates per minute across a dozen machines (as my POP servers are doing). Also, the LDAP support (per the documentation) for some of my other software is pretty beta and I don't know LDAP well enough to just jump in head first and hope I can solve any problems that crop up on a production server. With .cdb files or MySQL it's pretty easy for me to hack together tweaks that I need to make all the parts of my mail server play nicely together. Matt -Original Message- From: Chris Shenton [mailto:[EMAIL PROTECTED]] Sent: Monday, January 22, 2001 12:22 PM To: Matt Simerson Cc: '[EMAIL PROTECTED]' Subject: Re: mysql or not. "Matt Simerson" [EMAIL PROTECTED] writes: Well, I found the breaking point of vpopmail's open-smtp feature to be about 1500 domains (in an NFS environment). The actual breaking point isn't really related to the amount of domains/users but rather how many and how freqently your clients open POP/IMAP sessions. I was forced to move away from .cdb to MySQL for the open-smtp (which required a bit of hacking) portions of vpopmail about a month ago. Have you (or anyone else) tried using vpopmail's integration with LDAP? Putting users/passwords into an LDAP directory seems a better match -- SQL's optimized for read/write while LDAP is best at read-mostly.
RE: Symbolic links with aliasing of domains.
I for one really like that feature. All the accounts on my system are based upon a system UID/GID. Within that users home dir is the domains dir which has a list of all their domains. Since all their domains are symlinked there, it's a handy dir to glob for that info. I for one prefer it that way. Matt -Original Message- From: Brad Dameron [mailto:[EMAIL PROTECTED]] Sent: Monday, January 22, 2001 1:33 PM To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: Symbolic links with aliasing of domains. I am still curious as to why the authors of Qmailadmin and Vpopmail use symbolic links in the domains directory to add domain aliases. Doesn't the virtualdomains file tell qmail where the mail should go? Example being I have checkoutisle.net and checkoutisle.com which are the same. The virtualdomains file should be able to just do the following: checkoutisle.net:checkoutisle.com meaning that checkoutisle.net is also the same as checkoutisle.com. Every domain I add has a symbolic link because we also add mail.domainname to each virtual. With over 200 domains this get's quite ugly. There must be a better way to do this. --- Brad Dameron [EMAIL PROTECTED] Network Account Executive 877-663-4349 TSCNet Online Services www.tscnet.com --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.225 / Virus Database: 107 - Release Date: 12/22/2000
RE: vchkpw ssl addition
This is cool, and stunnel does work quite well (I use it elsewhere) but it's much easier to just install Courier IMAP which includes a qmail (Maildir) happy version of a POP3, POP3-SSL, IMAP, and IMAP-SSL server. What advantage is there to using qmail's POP3 daemon with stunnel over Courier IMAP's POP3-SSL? Matt -Original Message- From: James [mailto:[EMAIL PROTECTED]] Sent: Friday, January 19, 2001 11:21 AM To: [EMAIL PROTECTED] Subject: Re: vchkpw lacking authentication security Using stunnel: stunnel -d 995 -r localhost:pop3 -p /usr/local/etc/stunnel.pem Change the last argument to the path your private key/cert PEM file. Only downside is your pop3 logs now show the logins from 127.0.0.1. Ken Jones writes: Thanks. I'm reading the url now. I think I confused it with ssl tunnels. Does anyone have information on how to run vpopmail with ssl tunnels?
RE: vpopmail lacks authentication security
-Original Message- From: Tamer Hassan [mailto:[EMAIL PROTECTED]] Sent: Friday, January 19, 2001 11:46 AM To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: vpopmail lacks authentication security Sorry, I hate to do this. If you didn't hate to do it, you wouldn't have. So, not only are you rude and inconsiderate, but now I know that you're a liar too. I later posted to the list about the fact that vpopmail only uses DES. Matt Simerson said "it is silly to say that DES is insecure" and I disagreed. He then sent me a hashed password string betting me to crack it, and it turned out to be a BSD MD5 (what an iodiot). The dispute was not whether DES encryption was secure or not but rather whether DES was insecure because it only crypts based on the first 8 characters of the password. You claimed it was insecure, I said it wasn't. I sent you a 8 character MD5 encrypted password, stating that it was an 8 char or less password. If you had half an ounce of intelligence or the ability to run crack (and friends) against the encrypted password, you'd know that when using brute force techniques, it make no difference which algorithm is used to encrypt the 8 character password. Furthermore I use both MD5 and DES passwords on my FreeBSD machine(s) due to legacy issues with BSDI (which only uses DES). I'd be surprised if you knew the difference. If you were as smart as you think you are, you'd know how crypt works on your system and you'd also know how to change it. Maybe that would change the crypt libraries that vpopmail uses? Hmmm? Did you ever think of that? Of course not. You don't almost nothing about encryption and how your system uses it. The fact that two days later you still haven't been able to crack a simple 8 character crypted password proves my original point that because a password is limited to 8 characters it's not necessarily insecure. I could feed that password into my computing cluster and have the answer back in a matter of hours. That doesn't make it insecure. I have DES passwords in use on a lot of machines but the passwords are protected from brute force attacks which removes the "insecure" nature of DES passwords. This all goes to prove my first point that your concepts of security are very primitive. By itself, pretty much every piece of a security program is insecure. Running around like chicken little screaming that DES is insecure is foolishness. When used as part of a well designed security program, DES will never be the weakest link. Now, back to topic. IF ANYONE HAS SUCCESSFULLY USED MD5 WITH VPOPMAIL, POST TO THE FREAKING LIST. Excuse me! There is no excuse for that. You need a father with a leather belt to warm up your back side. Matt, I know you are going to trip out again. But, you seriously lack security insight. You cannot protect a box by disallowing pings to it. Who said that? Does someone have little voices talking to him? Security by obscurity is old fashioned. Rhetoric is lame. Them little voices must have been talking again. Same thing with using an 8 character password for your postmaster accounts (assuming that you do use the full 8 characters that DES allows you). Please, stop talking about your great inventions ssh'ing your pop server connections. If you administered hotmail or yahoo, would you do that? What did I invent using SSH? Now I'm almost certain you've got little voices whispering in your head. Do you smoke crack? You're a 19 year old (and immature at that) kid that's all ornery because I don't agree with you about 8 character passwords being insecure. You claim to be smart enough to make such a claim yet you don't know how to teach your sytem to crypt using libraries other than the defaults. Grow up child. Matt
RE: FreeBSD Cryptography 101 - WAS - RE: vchkpw lackingauthentic atio n security
One of the coolest things FreeBSD has done is include a little intelligence in their DES crypt libraries. When checking a password (against the system password files or MySQL) the DES libraries determine whether they're checking a MD5 or DES password. Unfortanately, the reverse is not true. The MD5 libraries cannot check a DES password. FreeBSD's default behavior is to use MD5 passwords (excluding DES encryption) so in order to authenticate against both, you simply change the symlinks to the crypt libraries from the MD5 crypts to the DES ones. For more information, read up on FreeBSD web site. Just do a search on DES from their main page and you'll find it. Matt -Original Message- From: Chris Shenton [mailto:[EMAIL PROTECTED]] Sent: Friday, January 19, 2001 11:50 AM To: Joe Modjeski Cc: [EMAIL PROTECTED] Subject: Re: FreeBSD Cryptography 101 - WAS - RE: vchkpw lacking authenticatio n security Joe Modjeski [EMAIL PROTECTED] writes: crypt( 'joeblow', '$1$qKMDvF5y$bcpzwp1mNbCQuTQYvkkeX.'); for MD5 On FreeBSD the DES libraries. libdescrypt is the DES+MD5 library. Currently I have vpopmail+mysql authenticating successfully for BOTH MD5 and DES passwords concurrently with no hitches. This is using the libdescrypt library. Thanks for pointing that out. Did you have to do mods to the code to get this to work, or tell it to link with libdescrypt instead of -- what, normal "libcrypt"? Again, thanks for your clear examples (and lack of flamebait :-)
RE: vchkpw lacking authentication security
I can't see how that could possibly be construed as a security drawback. POP is inherently insecure in the first place (sending clear text passwords across the net) and password sniffing is much more of a problem (and the easiest way to collect passwords) than people cracking passwords. So, unless you're exclusively using a) POP3-SSL or POP over SSH to prevent password sniffing, b) shadow passwords (who isn't?), c) MD5 (or blowfish) passwords on your current system (to utilize more than 8 char passwords), and d) forcing users to actually USE long passwords it's quite silly to say that using DES is a security drawback to using vpopmail. The risk of having a password cracked is minimal on a userless system. Matt -Original Message- From: Tim Hassan [mailto:[EMAIL PROTECTED]] Sent: Monday, January 15, 2001 10:09 PM To: [EMAIL PROTECTED]; [EMAIL PROTECTED] Subject: vchkpw lacking authentication security Dear Inter7 Developer: I recently discovered the following security drawback in vpopmail with vchkpw authentication: No matter how long you set the password to when adding a new user, only the first 8 characters of the password are used. So for example, if I do: ./vadduser [EMAIL PROTECTED] this-is-hard-to-guess-234234235-23423 and then I try to login to my email as user "test" and password "this-is-", it would let me in. As you may already know, any password below 8 characters is considered insecure, even if it was a combination of letters, numbers, and special characters. In other words, Standard DES crypto is used :( Best Regards, Tamer Hassan
RE: Secure SMTP/POP3
That program is named "stunnel". :-) Matt -Original Message- From: Ken Jones [mailto:[EMAIL PROTECTED]] Sent: Tuesday, January 09, 2001 7:44 AM To: Artem Koutchine Cc: [EMAIL PROTECTED] Subject: Re: Secure SMTP/POP3 Artem Koutchine wrote: Hi! I'd like to know how to setup some kind of encryption with qmail smtp and pop3 protocols. Particulary SSL would be very nice. Any patches? Setups? Config? Tutorials? Best regards, Artem Koutchine I saw a posting on the qmail mailing list where someone using vpopmail setup SSL tunnels for encrypted pop. No patches were needed. He ran some program like "tunnel" in the startup line. Try searching the qmail mailing list. If you do find the message, please send me the information so we can add it to the FAQ. Ken Jones
Don't sweat the small stuff.
Ken (et al), First, and foremost, why do you care what the script kiddies on #openbsd say about your code? Seriously. Most of the people that loiter around on irc channels are either extremely competent people who have gravy jobs and can sit around and talk about all the cool stuff they've done. These are the 1 to 2 people on the channel that really know their stuff and then you have the their throng of wanna be's soaking up the gospel from their cult hero or two. You can try all you want but you aren't going to change them. In time they'll begin to mature and begin the process of learning just how much they don't know. If that's not enough, you pick the openbsd channel where (not unlike that of most openbsd users) opinions of "code quality" is about as anal rententive as you'll find on the planet. I'm not in favor of Microsoft style code quality but there is a point of dimishing returns. I ofter throw things together "quick and dirty" because it doesn't matter if it's not done the best way. When I write a script that I run one time, I spend 5 minutes writing it (instead of 15) because it doesn't matter if it takes a minute to execute versus 10 seconds. You also need to remember that the overwhelming majority of people on IRC are kids. Literally, a bunch of little computer weenies running around with soggy ears and diapers. Not unlike most teenagers, they know everything and are more than willing to tell you all about it. Let them ramble on. What they say is almost assuredly out of ignorance and in time, they too will see the folly in their ways. If you want to measure the success of your project, ask people that use the software. Talk to guys like me, I'll tell you all about it. I've build more mail servers than there are script kiddies on that channel. I've used nearly everything MTA, MDA, and MUA that exists. I'm sure that within your ranks, I'm not the only very good, very smart, and highly clued systems architect/engineer/administrator. Talk to us about code quality if you want some productive feedback. Vpopmail users have a vested interest in the code. Some of us appreciate that it's open source. I've gone through quite a bit of effort to help document it. Have you seen my FreeBSD + Qmail + Vpopmail HOWTO? Take a look at it: http://matt.simerson.net/computing/qmail.toaster.shtml. Put a link to it on the vpopmail page. It's an excellent HOWTO and I know several guys have already built mail servers based on those instructions. That's a testimony to code quality. Even if it doesn't meet the yardstick of a bunch of script kiddies, the software does the job it's intended it to do. When bugs creep out we post them to the list. So far every issue I've had has been addressed. Granted sometimes I have to wait a week or two but that's nothing compared to the length of time I've waited for some other software companies to fix their bugs. That testifies that at some levels, the open source project is working. Last, pulling the source code is the last thing you should ever do. Either you release it or you don't. If you don't then you're pissing on us, your users, and those that have contributed time to making the software better without collecting a paycheck for doing so. You are punishing the wrong people. We're on your team and deserve better treatment than that. Matt On Thu, 7 Dec 2000, Ken Jones wrote: I'm sorry. The source code will be put back on the site within the next 24 hours. I apologize for the temporary lack of availablilty. Unfortunatly, i was conversing with people on #efnet on the openbsd channel, and they insulted me deeply. I can not begin to describe how deeply I was offeneded. It was not one thing or another thing, just the small attitudes of a small group of people. I would suggested you joing irc.east.gblx.net and go to the #openbsd channel and say at least something. The code will be put back up on the site soon. Better than ever. We are dedicated to providing open source software to the internet. It's just, some times, I get really ticked off when stupid kids tell me how bad my code is. And I know they don't understand. Sorry. If you really want to make a difference, do something to change something. I'm put the source back up in the next few days. It just makes me sad. It will be available. If you really need it.. just email me and i"ll send you the latest version. Ken Jones
RE: Bug in qmailadmin 0.39
-Original Message- From: Ken Jones [mailto:[EMAIL PROTECTED]] Sent: Tuesday, December 05, 2000 9:06 AM To: Matt Simerson Cc: 'Brumm Dominik'; '[EMAIL PROTECTED]' Subject: Re: Bug in qmailadmin 0.39 Matt Simerson wrote: Well, it's remotely possible that the problem is in the HTML but I have to believe it's not. I have many browers I test with (IE 5.5 on WinNT, Netscape 4.7 on WinNT, and Netscape 4.7 on FreeBSD, Netscape 4.7 on MacOS, IE 4.5 on MacOS) and I know it's not a browser specific problem. The HTML form works just fine for the domains that were added to the system very early on and doesn't work for any of the 500+ domains I've added since (as far as I can tell). So, I did some more testing and it appears the problems is with ALL domains that are created under a system username other than vpopmail. The reason I thought it was related the the inclusion in rcpthosts vs morercphosts was that early domains were in that file. The other common factor was that the early ones live in ~vpopmail/domains instead of ~username/domains/domain.com. Ahh.. If you are running a domain under a non vpopmail user name, make sure you change the permissions on qmailadmin to be root.root and setuid/setgid root. The standard distribution sets it to be setuid/gid vpopmail.vchkpw. Already done, that was one of the first things I checked. It needs the root setuid/gid to change ownership to the user who owns the domain. It gets the domain uid/gid information from the line in /var/qmail/users/cdb (assign) file. The actual solution came from another fellow on the list here who had already patched his vpopmail to work. The relavent lines were in the int vauth_vpasswd subroutine and required adding a couple declarations: int uid; int gid; and a few code tweaks. Check the list archives in this thread. Once I made the code changes, I recompiled vpopmail, reinstalled it, then recompiled and reinstalled sqwebmail and it's change password feature began working (I compile sqwebmail with --enable-webpass=vpopmail). Once I recompiled qmailadmin, it began working. Matt
RE: VpopMail + sPOP3
The other option here is to use the pop3s (and pop3) server that ships with Courier IMAP which authenticates using the vchkpw method and handles all that fun stuff for you. Matt -Original Message- From: Ken Jones [mailto:[EMAIL PROTECTED]] Sent: Tuesday, December 05, 2000 12:34 PM To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: Re: VpopMail + sPOP3 eric wrote: Just for kicks, I was wondering if anyone out there has anything hacked up to somehow enable roaming users to relay thru your qmail-smtpd AFTER authenticating via sPOP3. The problem is as follows... 1. End user connects to sPOP3 (995/tcp) with a client that supports sPOP3 and authenticates over an encrypted channel. This communication is performed by ``stunnel'' performing a man-in-the-middle tunnel to localhost:110. 2. End user happily reads their mail and decides that they indeed would like to send a reply. 3. End user composes message and hits "Send" 4. End user gets the standard "I just puked and can't send your email" message from the client. The reason that this happens is due to stunnel presenting the connection from localhost, rather than the remote-ip. Obviously when the email is to be sent, the smtpd cannot verify that this user has been authentication and errors back. As far as I can see it, here's two solutions to this. 1. Use ``stunnel'' to channel the user email to port localhost:25. This can present the problem of allowing anything that can talk SSL to relay mail off the machine. Ungood. 2. Somehow hackup either stunnel or vpopmail to keep a second database based on the stunnel logs, etc.. Difficult. 3. Bug Inter7 to build SSL support in natively to the pop3 server. Expensive. So I'd imagine that I'm pretty out of luck at this point, but any discussions on the matter would be great. Thanks. Eric If you could somehow get stunnel to go directly between the pop server port/ip and the client ip, you could get the clients IP and hand it over to vchkpw on the pop authentication. Then everything would work as if they were connecting to the pop3 port. Ken Jones
vpopmail system user feature breaks mail users ability tochange passwords
Using vpopmail 4.9.6, qmailadmin 0.39, sqwebmail 1.0.3, and courier-imap 1.2.2. Sqwebmail and Courier-IMAP are both compiled using the new preauthvchkpw.c file. I'm using vpopmail's feature so that each domain is created under a unique system user name. So, domain.com is owned by sytem user "domain" and domain2.com is owned by the system user "domain2", etc... Nearly everything works just great with just one notable exception. Nobody whose account exists under an account other than vpopmail.vchkpw is able to change their passwords. :-( They authenticate just fine, everything works, but when they log in via the qmailadmin interface or the sqwebmail interface and attempt to change the password for mailboxes, it fails. Sqwebmail fails silently, qmailadmin produces a -22 error. Matt
RE: Directory Structure.
There's a couple ways to do this and I opted for the easy way, I set up a user mapping in /home and wrote a quick perl script to create my a/aa/, a/ab/ tree structure. Then I use FreeBSD's pw utility to create the user and hand it my custom path name. Matt -Original Message- From: Chad [mailto:[EMAIL PROTECTED]] Sent: Friday, November 03, 2000 11:04 AM To: [EMAIL PROTECTED] Subject: Directory Structure. For very large sites I would like the directory structure to be like: /mail/a/apple.com/a/appleman/Maildir /mail/d/dittie.com/l/lapt/Maildir Would this be hard to implement? Thanks
vdelivermail behavior question
I would like to see a new option available to the vdelivermail program. Currently it can be given a parameter such as "bounce-no-mailbox" or a directory to deliver messages to. I'd like to be able to deliver messages that have no match in the vpasswd file to a filename (/dev/null in particular). Here's why: On Friday afternoon a domain that we host email for started getting pounded. This is an older system running sendmail and we had over 50 different SMTP servers connecting to this customers IP. We were seeing about 3 sendmail processes per second and sendmail, being the bloated pig that it is, brought the machine to it's knees when scores of sendmail instances were being opened. The machine was hammered so hard that we couldn't even get a couple of the messages through to see what was being delivered and causing the problem. My boss asked me what could be done about it. Since there wasn't any quick easy fix I had him update the DNS for the domain and point all the mail for that domain to my new, about to be implemented, mail system running qmail, vpopmail, and friends. I set up his domain, created the six email users he had and sat back and watched. Within minutes the SMTP traffic on my machines started picking up and within 5 minutes I was seeing 30 concurrent connections so my mail server. Since it wasn't sweating, I decided to see how high I could go with it and cranked up the limit to 1000 sessions. Within minutes my dual PIII650 machine was starting to get sluggish. I shut down qmail and changed the limit to 300 and fired it back up. I was handling 300 connections OK and I took a peek at the queue. In a few minutes I had accumulated over 13,000 messages in the queue. Rather than letting the messages accumulate in the queue (it was trying to bounce the message but there wasn't a valid reply-to) I created a "spam" account for the domain as the default delivery box. This worked quite nicely as it routed all the messages into the spam box and gave us a chance to take a look at what was bombarding us. However, it would only take a couple minutes for before "rm *" failed bacause there were too many files in the directory. Since I couldn't bounce the messages and we wanted this poor customers mail to work despite the DOS attack (of sorts), I decided to route all the messages to /dev/null. This way we're not queueing them or making a huge mess to clean up. My first attempt was to put a filename (/dev/null) in the .qmail-default file as a vdelivermail parameter. This caused vdelivermail to core dump so I speculated that that wasn't such a good idea. All I wanted to do was get the spam users mail routed to /dev/null instead of a mailbox. What I ended up doing was creating a new .qmail-user1, .qmail-user2, etc.. file for each of the users that had the same contents of the stock .qmail-default that calles vdelivermail. This got each of the users mail accounts working and then I created a new .qmail-default that contained the name of the file I wanted to deliver mail to "/dev/null" and my problem was worked around. However, I'd like some method of doing this that doesn't require a person to ssh into the server and manually create files in order to do this. Ultimately what was happening was a spammer sent a LOT of mail out using bogus reply-to addresses containing our customers domain. As the spam got bounced, it returned to his mail server and brought the thing to it's knees. So, we couldn't just deny the connections, RBL that I'm using can't catch them because the connections are from valid non-relaying mail servers, and I can't bounce them because they don't have a valid from/reply to header. :-( I'm going to patch qmail-smtpd to not accept emails that don't have a valid return address but the only other solution I can see is just accepting and routing to the bit bucket any emails known to be invalid. Comments? Ideas? Matt
RE: quota question on 4.9.5
I've installed the new vpopmail 4.9.5 and created a couple users under system users other than vpopmail and they work just fine for mail delivery and pop but neither sqwebmail nor courier-imap will authenticate users whose UID/GID is not vpopmail/vchkpw. Courier-imap has it's own authentication module for authenticating a vpopmail user. I had courier-imap 1.0 installed and it failed with an error message that indicated an authentication failure. Just for grins I decided to see if it was fixed in courier-imap 1.1 and installed it. It works just fine for domains installed user the vpopmail UID/GID but still won't authenticate for domains within another home dirctory. The error message returned is: Your account is temporarily unavailable (+t bit set on home directory). Configuration: Account: admin.mail.safeserver.com (1) Server: admin.mail.safeserver.com User name: [EMAIL PROTECTED] Protocol: IMAP Port: 993 Secure(SSL): 1 Code: 800cccdd It yields the same error message when connecting to either the IMAP or IMAP SSL ports. The same problem plagues sqwebmail at this time too. It won't authenticate users whose domains are not located within the ~vpopmail dir. I tried installing the newer 1.0.0.20001007 version but without success (so far, I'm still playing with it). Have I just missed something completely obvious or is this a valid limitation at this time? Matt -Original Message- From: listy-dyskusyjne Krzysztof Dabrowski [mailto:[EMAIL PROTECTED]] Sent: Sunday, October 15, 2000 2:42 PM To: [EMAIL PROTECTED] Subject: Re: quota question on 4.9.5 I don't mind doing some coding, but there seems to be more missing. I'll need to dig through the code some more. It would be nice if we can have the postmaster get an unlimited quota so they can be informed that their domain is over quota and that they should get the quota increased or have their people start cleaning up their mail. The postmaster may need to get mail on each bounce (not the message, but that a message from so-and-so bounced), and possibly which ID's are using up the space so it can be cleaned properly. Before putting some code together, we may want to put together a real design first and try to think it through. The domain per unix ID is good, but we may want to put the postmaster in a different place so they can still get notifications. What are your thoughts? What else did I miss? (I'm sure I've missed alot). My vargid patch (get it from my page, announced on the list separately) will let you choose any gid/uid for any domain. We can then use a GID per domain scheme. Postmaster can be given another uid (a special one) so it wont have the quota. I do not know 2 things at this point: a) how group user quotas mix together b) is it reasonable to have one quota-less account and give it to user... they will use this account to transfer big emials for sure and this is what we want to stop , using quotas. Kris
RE: courier-imap authentication and vpopmail 4.9.5
Follow up: I was partially in error when I stated that sqwebmail failed to authenticate against vpopmail users whose domains were in users home directory other than vpopmails. It does in fact work if you read the vpopmail web page and catch the new link to the patched preauthvchkpw.c. Works just peachy with sqwebmail 1.0 and domains within or out of vpopmails home dir. Now if only I had the patch to make courier-imap do that... BTW: I have written up the procedure I have used to build a mail toaster based around qmail, vpopmail, sqwebmail, courier-imap, and all the trimmings. It does everything I can think of having a mail server do except smtp-auth which I'll get around to implementing here someday. It's at http://matt.simerson.net/computing/qmail.toaster.shtml. Comments and suggestions are welcome. When I get some more time on my hands I'll so some really cool stuff like write up scripts that feed the log files through the processors and graph how many messages the servers are handling, cpu load, and network activity in something pretty like MRTG. Matt -Original Message- From: Matt Simerson [mailto:[EMAIL PROTECTED]] Sent: Sunday, October 15, 2000 5:44 PM To: Vpopmail (E-mail) Subject: RE: quota question on 4.9.5 I've installed the new vpopmail 4.9.5 and created a couple users under system users other than vpopmail and they work just fine for mail delivery and pop but neither sqwebmail nor courier-imap will authenticate users whose UID/GID is not vpopmail/vchkpw. Courier-imap has it's own authentication module for authenticating a vpopmail user. I had courier-imap 1.0 installed and it failed with an error message that indicated an authentication failure. Just for grins I decided to see if it was fixed in courier-imap 1.1 and installed it. It works just fine for domains installed user the vpopmail UID/GID but still won't authenticate for domains within another home dirctory. The error message returned is: Your account is temporarily unavailable (+t bit set on home directory). Configuration: Account: admin.mail.safeserver.com (1) Server: admin.mail.safeserver.com User name: [EMAIL PROTECTED] Protocol: IMAP Port: 993 Secure(SSL): 1 Code: 800cccdd It yields the same error message when connecting to either the IMAP or IMAP SSL ports. The same problem plagues sqwebmail at this time too. It won't authenticate users whose domains are not located within the ~vpopmail dir. I tried installing the newer 1.0.0.20001007 version but without success (so far, I'm still playing with it). Have I just missed something completely obvious or is this a valid limitation at this time? Matt -Original Message- From: listy-dyskusyjne Krzysztof Dabrowski [mailto:[EMAIL PROTECTED]] Sent: Sunday, October 15, 2000 2:42 PM To: [EMAIL PROTECTED] Subject: Re: quota question on 4.9.5 I don't mind doing some coding, but there seems to be more missing. I'll need to dig through the code some more. It would be nice if we can have the postmaster get an unlimited quota so they can be informed that their domain is over quota and that they should get the quota increased or have their people start cleaning up their mail. The postmaster may need to get mail on each bounce (not the message, but that a message from so-and-so bounced), and possibly which ID's are using up the space so it can be cleaned properly. Before putting some code together, we may want to put together a real design first and try to think it through. The domain per unix ID is good, but we may want to put the postmaster in a different place so they can still get notifications. What are your thoughts? What else did I miss? (I'm sure I've missed alot). My vargid patch (get it from my page, announced on the list separately) will let you choose any gid/uid for any domain. We can then use a GID per domain scheme. Postmaster can be given another uid (a special one) so it wont have the quota. I do not know 2 things at this point: a) how group user quotas mix together b) is it reasonable to have one quota-less account and give it to user... they will use this account to transfer big emials for sure and this is what we want to stop , using quotas. Kris
RE: Roaming users woes
Change the invocation of your SMTP server from "-x /home/vpopmail/etc/tcp.smtp.cdb" back to the default "-x /etc/tcp.smtp.cdb" and that should do it for you. This assumes that you did NOT override the default /etc/tcp.smtp database when you invoked "configure" while compiling vpopmail. When a pop user connects it adds their IP to the /home/vpopmail/etc/open-smtp and then adds the contents of that file to the /etc/tcp.smtp.cdb database and smtp connections from that IP work like a charm. Matt -Original Message-From: Michael French [mailto:[EMAIL PROTECTED]]Sent: Tuesday, October 03, 2000 2:39 PMTo: [EMAIL PROTECTED]Subject: Roaming users woes I saw a question about setting up roaming users on this list about twoweeks ago and Ken Jones gave a nice reply with detailed instructions. Iprinting out the instructions and followed them, but it is still not workingfor me. I can pop in and get my mail from home, but as soon as I try tosend something, I get that relay message because it has not added my ipaddress to the database. Here are some of my config files:from "qmail" file in /etc/rc.d/init.d/:tcpserver -v -R 0 pop3 /var/qmail/bin/qmail-popup ashevillemail.com\/home/vpopmail/bin/vchkpw /var/qmail/bin/qmail-pop3d Maildir 21 | \/var/qmail/bin/splogger pop3d from "run" file in /var/qmail/supervise/qmail-smtpd/:#!/bin/shQMAILDUID=`id -u qmaild`NOFILESGID=`id -g qmaild`exec /usr/local/bin/softlimit -m 300 \/usr/local/bin/tcpserver -v -H -R -x /home/vpopmail/etc/tcp.smtp.cdb \-c30 -u $QMAILDUID -g $NOFILESGID 0 smtp \/var/qmail/bin/qmail-smtpd 21crontab listing:# DO NOT EDIT THIS FILE - edit the master and reinstall.# (/tmp/crontab.4422 installed on Fri Sep 15 10:35:51 2000)# (Cron version -- $Id: crontab.c,v 2.13 1994/01/17 03:20:37 vixie Exp $)40 * * * * /home/vpopmail/bin/clearopensmtp 21 /dev/null10,25,40,55 * * * * /home/vpopmail/bin/clearopensmtpfrom "tcp.smtp" file in ~vpopmail/etc/:209.198.39.210:allow,RELAYCLIENT=""127.0.0.:allow,RELAYCLIENT="" Michael FrenchAsheville Citizen-TimesIT Dept.
RE: Stats
Well, you know... If you're using daemontools, it's pretty easy to have it trigger a script every time your log file reaches a predefined limit. This script can process the logs (using qmail's optional tools) and you could easily feed that output through whichever of the z* tools that spits out what you want to log. Daemontools are your friends. :-) Matt -Original Message- From: Ken Jones [mailto:[EMAIL PROTECTED]] Sent: Thursday, September 21, 2000 7:20 AM To: Robert J Adams Cc: [EMAIL PROTECTED] Subject: Re: Stats Robert J Adams wrote: Hello all, Has anyone written a stats program to gather number of pop/imap/smtp connections etc for vpopmail? Seems like it would be useful. -Jason If you are using tcpserver -v for pop and smtp, It would be fairly straight forward to write a file parsing program. What would be cool is if it could hook into mrtg graphs ;] http://vast.inter7.com/mrtg/messages.html Ken Jones
