Re: [vchkpw] How expensive is reloading the tcp.smtp.cdb?
On 2005-10-26, at 1805, ISP Lists wrote: Related to my earlier post, how expensive is it - resource-wise - to reload a tcp.smtp file of 100-1000 lines? if it's anything like mine, with currently 56,270 lines and more added every time some spammer sends a message to an email address in the delete.net domain, it takes almost no time at all. best of all, nothing has to be restarted afterward- while the processing is happening, any tcpserver processes will continue to use the old file as the new one is built... and when the processing is done, the new one is renamed over the old one, so there is no danger of any tcpserver process ever trying to use a halfway done file. If I have processing that is updating tcp.smtp every 5-10 minutes and I choose to reload the cdb from that tcp.smtp, is that a bad idea? i do this every one minute- i have a cron job which generates the text file, diffs it against the prior version, and rebuilds the cdb file only if the text file has changed. the overall procedure has no noticeable impact on the system- and the perl scripts which generate the text file eat ten times the CPU and memory that the tcprules command (which actually builds the cdb file) does. The qmailctl cdb command runs very fast for me now, but I don't have any idea what impact it has on any smtpd instances having to restart or re-read. Anyone know? no impact at all. nothing ever has to restart or re-read. qmail-smtpd doesn't know what a tcp.smtp file is, tcpserver does... and tcpserver re-opens the file for every incoming connection, so whatever the current file is at the time tcpserver calls open(), that's the file it reads. if tcprules happens to do the rename while tcpserver has the file open, it retains its existing handle on the old file (which now has no name pointing to it) until it calls close () after checking the remote IP address. nice thing about renaming a file... it's atomic, meaning that it happens all at once- there is no possible way for tcpserver to open the file in the middle of the rename() call... either it happens before rename() or after rename(). -- | John M. Simpson - KG4ZOW - Programmer At Large | | http://www.jms1.net/ [EMAIL PROTECTED] | -- | Mac OS X proves that it's easier to make UNIX | | pretty than it is to make Windows secure. | -- PGP.sig Description: This is a digitally signed message part
Re: [vchkpw] How to run multiple qmail-queue
On 2005-10-27, at 0404, Manish Jain wrote: I have installed netqmail+vpopmail+qmailadmin on RHEL 3.0 I have configured multile domains on the server. And using smtproutes in /var/qmail/control to route the mails of different domains to the clients mailsrevers in remote. I am facing problem, when one PPP link goes down to a customer all the incoming mails from the internet stucks in the qmail-queue as qmail not able to make smtp connection to the client server. In this case mail delivery to all other domain servers get delayed. To resolve the problem I want to run multiple qmai-queue for each domain so that one domain get unaffected from the another's queue. qmail was designed to have exactly one qmail-queue process running at a time. if you want to change this you will have to make some fairly major changes to qmail-queue itself. instead of trying to have two qmail-queue processes, you need to run more qmail-remote processes. raise your concurrencyremote value. read man qmail-queue if you don't know how this works (and add MANPATH /var/qmail/man to your /etc/man.config file if man qmail- queue doesn't work.) -- | John M. Simpson - KG4ZOW - Programmer At Large | | http://www.jms1.net/ [EMAIL PROTECTED] | -- | Mac OS X proves that it's easier to make UNIX | | pretty than it is to make Windows secure. | -- PGP.sig Description: This is a digitally signed message part
[vchkpw] [Fwd: [qmr] vpopmail with LDAP]
Hi, I'm trying to change vpopmail (currently with mysql backend) with LDAP support. vpopmail do all the authentication works for email transaction. Vpopmail version 5.4.9. Openldap 2.3.4 with sasl OS : FreeBSD 4.10 untar-ed the source and edited the vldap.h file. then : ./configure --enable-auth-module=ldap completed without problem. when make, i got these warnings and errors : vauth.c: In function `vauth_getpw': vauth.c:119: warning: implicit declaration of function `ldap_search_s' vauth.c:126: warning: implicit declaration of function `ldap_perror' vauth.c:153: warning: implicit declaration of function `ldap_get_values' vauth.c:153: warning: assignment makes pointer from integer without a cast vauth.c:186: warning: implicit declaration of function `ldap_value_free' vauth.c:229: warning: assignment makes pointer from integer without a cast vauth.c:242: warning: assignment makes pointer from integer without a cast vauth.c:259: warning: assignment makes pointer from integer without a cast vauth.c:271: warning: assignment makes pointer from integer without a cast vauth.c:283: warning: assignment makes pointer from integer without a cast vauth.c:292: warning: assignment makes pointer from integer without a cast vauth.c:301: warning: assignment makes pointer from integer without a cast vauth.c: In function `vauth_getall': vauth.c:373: warning: implicit declaration of function `ldap_sort_entries' vauth.c:388: warning: assignment makes pointer from integer without a cast vauth.c:410: warning: assignment makes pointer from integer without a cast vauth.c: In function `vauth_adduser': vauth.c:578: warning: implicit declaration of function `ldap_add_s' vauth.c: In function `vauth_deldomain': vauth.c:708: warning: implicit declaration of function `ldap_delete_s' vauth.c: In function `vauth_setpw': vauth.c:891: warning: implicit declaration of function `ldap_modify_s' vauth.c: In function `vclose': vauth.c:914: warning: implicit declaration of function `ldap_unbind_s' vauth.c: In function `ldap_connect': vauth.c:1339: warning: implicit declaration of function `ldap_init' vauth.c:1339: warning: assignment makes pointer from integer without a cast vauth.c:1349: warning: implicit declaration of function `ldap_simple_bind_s' gcc -I. -I/usr/local/include -I. -I. -I. -g -O2 -Wall -c `test -f 'file_lock.c' || echo './'`file_lock.c gcc -I. -I/usr/local/include -I. -I. -I. -g -O2 -Wall -c `test -f 'vpalias.c' || echo './'`vpalias.c gcc -I. -I/usr/local/include -I. -I. -I. -g -O2 -Wall -c `test -f 'seek.c' || echo './'`seek.c gcc -I. -I/usr/local/include -I. -I. -I. -g -O2 -Wall -c `test -f 'vlimits.c' || echo './'`vlimits.c gcc -I. -I/usr/local/include -I. -I. -I. -g -O2 -Wall -c `test -f 'maildirquota.c' || echo './'`maildirquota.c rm -f libvpopmail.a ar cru libvpopmail.a vpopmail.o md5.o bigdir.o vauth.o file_lock.o vpalias.o seek.o vlimits.o maildirquota.o cdb/*.o ranlib libvpopmail.a gcc -I. -I/usr/local/include -I. -I. -I. -g -O2 -Wall -c `test -f 'vchkpw.c' || echo './'`vchkpw.c gcc -I. -I/usr/local/include -I. -I. -I. -g -O2 -Wall -c `test -f 'hmac_md5.c' || echo './'`hmac_md5.c gcc -g -O2 -Wall -o vchkpw vchkpw.o md5.o hmac_md5.o libvpopmail.a -L/usr/local/lib -lldap -llber -lresolv -lcrypt /usr/libexec/elf/ld: cannot find -lresolv what is this lresolv? As i recall, there were no library called libresolv, isn't it? it only exists on Linux type OS, as opposed to *BSD. how would i go to fix this library error? also, is it safe to ignore those warnings? looks like those warnings might break vpopmail. any help will be greatly appreciated. thanks -- Edy Sulai IT Coordinator Bunda Mulia Australian International School Jl. Lodan Raya no. 2 Ancol, Jakarta Utara 14430 Indonesia Tel : +62-21-690-9742 Fax : +62-21-690-2768 Web : http://www.bmais.or.id Email: [EMAIL PROTECTED]
RE: [vchkpw] How to run multiple qmail-queue
Your could have many instances of qmail running in different ports. and with smtproutes deliver some domains to the qmails... Ingo Claro Gerente de Operaciones [EMAIL PROTECTED] (+56-2) 43 00 155 -Mensaje original- De: John Simpson [mailto:[EMAIL PROTECTED] Enviado el: Viernes, 28 de Octubre de 2005 3:15 Para: vchkpw@inter7.com Asunto: Re: [vchkpw] How to run multiple qmail-queue On 2005-10-27, at 0404, Manish Jain wrote: I have installed netqmail+vpopmail+qmailadmin on RHEL 3.0 I have configured multile domains on the server. And using smtproutes in /var/qmail/control to route the mails of different domains to the clients mailsrevers in remote. I am facing problem, when one PPP link goes down to a customer all the incoming mails from the internet stucks in the qmail-queue as qmail not able to make smtp connection to the client server. In this case mail delivery to all other domain servers get delayed. To resolve the problem I want to run multiple qmai-queue for each domain so that one domain get unaffected from the another's queue. qmail was designed to have exactly one qmail-queue process running at a time. if you want to change this you will have to make some fairly major changes to qmail-queue itself. instead of trying to have two qmail-queue processes, you need to run more qmail-remote processes. raise your concurrencyremote value. read man qmail-queue if you don't know how this works (and add MANPATH /var/qmail/man to your /etc/man.config file if man qmail- queue doesn't work.) -- | John M. Simpson - KG4ZOW - Programmer At Large | | http://www.jms1.net/ [EMAIL PROTECTED] | -- | Mac OS X proves that it's easier to make UNIX | | pretty than it is to make Windows secure. | --
RE: [vchkpw] How expensive is reloading the tcp.smtp.cdb?
John: what do you mean by delete.net domain ? you have a bogus virtualdomain to catch spammers? regards, Ingo Claro Gerente de Operaciones [EMAIL PROTECTED] (+56-2) 43 00 155 -Mensaje original- De: John Simpson [mailto:[EMAIL PROTECTED] Enviado el: Viernes, 28 de Octubre de 2005 3:09 Para: vchkpw@inter7.com Asunto: Re: [vchkpw] How expensive is reloading the tcp.smtp.cdb? On 2005-10-26, at 1805, ISP Lists wrote: Related to my earlier post, how expensive is it - resource-wise - to reload a tcp.smtp file of 100-1000 lines? if it's anything like mine, with currently 56,270 lines and more added every time some spammer sends a message to an email address in the delete.net domain, it takes almost no time at all. best of all, nothing has to be restarted afterward- while the processing is happening, any tcpserver processes will continue to use the old file as the new one is built... and when the processing is done, the new one is renamed over the old one, so there is no danger of any tcpserver process ever trying to use a halfway done file. If I have processing that is updating tcp.smtp every 5-10 minutes and I choose to reload the cdb from that tcp.smtp, is that a bad idea? i do this every one minute- i have a cron job which generates the text file, diffs it against the prior version, and rebuilds the cdb file only if the text file has changed. the overall procedure has no noticeable impact on the system- and the perl scripts which generate the text file eat ten times the CPU and memory that the tcprules command (which actually builds the cdb file) does. The qmailctl cdb command runs very fast for me now, but I don't have any idea what impact it has on any smtpd instances having to restart or re-read. Anyone know? no impact at all. nothing ever has to restart or re-read. qmail-smtpd doesn't know what a tcp.smtp file is, tcpserver does... and tcpserver re-opens the file for every incoming connection, so whatever the current file is at the time tcpserver calls open(), that's the file it reads. if tcprules happens to do the rename while tcpserver has the file open, it retains its existing handle on the old file (which now has no name pointing to it) until it calls close () after checking the remote IP address. nice thing about renaming a file... it's atomic, meaning that it happens all at once- there is no possible way for tcpserver to open the file in the middle of the rename() call... either it happens before rename() or after rename(). -- | John M. Simpson - KG4ZOW - Programmer At Large | | http://www.jms1.net/ [EMAIL PROTECTED] | -- | Mac OS X proves that it's easier to make UNIX | | pretty than it is to make Windows secure. | --
Re: [vchkpw] vpopmail upgrade == qmail recompile needed ?
On Thursday 27 October 2005 10:57 pm, John Simpson wrote: On 2005-10-26, at 1949, Rick Macdougall wrote: Domain Quotas are not going to happen anytime soon I think, the over head of calculating the quota for a domain of 20K users is just too much. If you really need domain quotas, use a separate user for each domain and use system quotas. Of course you will then have to run qmail-smtpd as root. or make the vchkpw binary setuid root, and let qmail-smtpd run as qmaild (as it was designed.) but that will break chkuser. -Jeremy -- Jeremy Kitchen ++ [EMAIL PROTECTED] In the beginning was The Word and The Word was Content-type: text/plain -- The Word of Bob. pgpph2zOv857t.pgp Description: PGP signature
[vchkpw] Migrating users
I've recently gone through a server migration of 20 or so domains, none of which had a ton of email in them. Here's what I did: 1. add domain to new server using vadddomain 2. stop qmail-send on old server 3. tar/gzip /home/vpopmail/domains/domaintomove.com directory 4. Move to new server and untar into place, overwriting existing vpasswd and postmaster files. 5. vdeldomain on the old server 6. add domain to rcpthosts on old server 7. add domain:newserver to smtproutes on old server 8. restart qmail-send 9. change DNS. This worked fine for my small server with few messages and few clients that would notice anything strange. However, I've been asked to do the same on another mail server with several hundred domains and 15 gig of mail. In order to transfer one domain with lots of mail, I could potentially have to have the stuff down for an extended period of time. I don't want people reading email while I'm tarring up their domain's email. My question is... is there a better way than the steps I described above? Rick
Re: [vchkpw] Migrating users
Rick Root wrote: I've recently gone through a server migration of 20 or so domains, none of which had a ton of email in them. Here's what I did: 1. add domain to new server using vadddomain 2. stop qmail-send on old server 3. tar/gzip /home/vpopmail/domains/domaintomove.com directory 4. Move to new server and untar into place, overwriting existing vpasswd and postmaster files. 5. vdeldomain on the old server 6. add domain to rcpthosts on old server 7. add domain:newserver to smtproutes on old server 8. restart qmail-send 9. change DNS. This worked fine for my small server with few messages and few clients that would notice anything strange. However, I've been asked to do the same on another mail server with several hundred domains and 15 gig of mail. In order to transfer one domain with lots of mail, I could potentially have to have the stuff down for an extended period of time. I don't want people reading email while I'm tarring up their domain's email. My question is... is there a better way than the steps I described above? Hi, I do exactly the same but I rsync instead of tar and try to do the rsync before shutting down the old servers qmail-smtpd/send, then the final rsync is quite quick. Regards, Rick
Re: [vchkpw] Migrating users
On Oct 28, 2005, at 12:27 PM, Rick Root wrote: My question is... is there a better way than the steps I described above? As Rick Macdougall said, rsync is your friend. You can run it once while everything is live, then stop qmail and run it again to sync up anything that changed since the first run. Very slick. I wrote some scripts to aid in migrating to a new server, and Paul Kremer recently prettied them up. Give them a shot, they should make the migration go fairly smoothly. They automate the procedure, and even include the multiple calls to rsync, along with vadddomain on the new server, etc. http://www.mail-archive.com/vchkpw@inter7.com/msg22410.html I recommend adding a dummy domain on your old server and use it as a dry run for the migration scripts. -- Tom Collins - [EMAIL PROTECTED] QmailAdmin: http://qmailadmin.sf.net/ Vpopmail: http://vpopmail.sf.net/ You don't need a laptop to troubleshoot high-speed Internet: sniffter.com
[vchkpw] Courier-imap+vpopmail problems
Hi guys!!! I´m new in this list and had subscribed here for some help. Yes, i read all msg of this list about my thougth. I have a Linux box Fedora core 3, mounted with Qmailrocks cake guide with vpopmail 5.4.9, qmailadmin 1.2.3, courier-authlib-0.57, courier-imap-4.0.6, squirrelmail-1.4.5, this configuration works well, but every day I had to reboot the machine because sundely the authentication stops, and nothing i do for restart the processes to recover this thing is valid. Simplily the authentications stop, and only rebooting the machine i get to recover the things to normal. No logging messages are register for this. Somebody had this problem too? Know the solution for this? Please excuse me my poor english, and i hope my problem is clear. thanks in advance. cmattos_58
[vchkpw] Migration using Rsync - Memory Problem
Hello List. I'm worried because my migration scheme is giving problems. My migration plan was use rsync for transfer all user's mailbox, but it don't work because my directory tree is very big. I have around 50K maildir directory (110 GB approx). I checked the rsync documentation and I note that it have memory problems with big directory tree (It use a lot of memory and it does shutdown my server). I need transfer all the maildirs without turn off the mail service or in any case, the time off of service should be short, for that reason I did think use rsync, because it could make incremental copy. Please, somebody know others alternative to rsync? Bye
Re: [vchkpw] Migration using Rsync - Memory Problem
Juan Enciso wrote: Hello List. I'm worried because my migration scheme is giving problems. My migration plan was use rsync for transfer all user's mailbox, but it don't work because my directory tree is very big. I have around 50K maildir directory (110 GB approx). I checked the rsync documentation and I note that it have memory problems with big directory tree (It use a lot of memory and it does shutdown my server). I need transfer all the maildirs without turn off the mail service or in any case, the time off of service should be short, for that reason I did think use rsync, because it could make incremental copy. Please, somebody know others alternative to rsync? Hi, Use rsync but with a script file that limits the number of files it does per pass. ie rsync vpopmail/domain.com/0 remote.example.com::vpopmail/domain.com/0 . . . rsync vpopmail/domain.com/Z remote.example.com::vpopmail/domain.com/Z Regards, Rick
Re: [vchkpw] Migrating users
Tom. But the rsync have problem when the directory tree is very big. What alternative could use against of rsync... Another technique for sync files? On 10/28/05, Tom Collins [EMAIL PROTECTED] wrote: On Oct 28, 2005, at 12:27 PM, Rick Root wrote: My question is... is there a better way than the steps I described above? As Rick Macdougall said, rsync is your friend. You can run it once while everything is live, then stop qmail and run it again to sync up anything that changed since the first run. Very slick. I wrote some scripts to aid in migrating to a new server, and Paul Kremer recently prettied them up. Give them a shot, they should make the migration go fairly smoothly. They automate the procedure, and even include the multiple calls to rsync, along with vadddomain on the new server, etc. http://www.mail-archive.com/vchkpw@inter7.com/msg22410.html I recommend adding a dummy domain on your old server and use it as a dry run for the migration scripts. -- Tom Collins - [EMAIL PROTECTED] QmailAdmin: http://qmailadmin.sf.net/ Vpopmail: http://vpopmail.sf.net/ You don't need a laptop to troubleshoot high-speed Internet: sniffter.com
Re: [vchkpw] Migrating users
On Friday 28 October 2005 05:03 pm, Juan Enciso wrote: Tom. But the rsync have problem when the directory tree is very big. What alternative could use against of rsync... Another technique for sync files? I've used rsync on ENORMOUS (read: 100+GB) mailstores. works fine. I couldn't possibly imagine migrating a large qmail/vpopmail system with something lesser than rsync. -Jeremy -- Jeremy Kitchen ++ [EMAIL PROTECTED] In the beginning was The Word and The Word was Content-type: text/plain -- The Word of Bob. pgp3682fpTdd9.pgp Description: PGP signature
Re: [vchkpw] Migrating users
Jeremy Kitchen wrote: On Friday 28 October 2005 05:03 pm, Juan Enciso wrote: Tom. But the rsync have problem when the directory tree is very big. What alternative could use against of rsync... Another technique for sync files? I've used rsync on ENORMOUS (read: 100+GB) mailstores. works fine. I couldn't possibly imagine migrating a large qmail/vpopmail system with something lesser than rsync. rsync on BSDi and perhaps other BSD systems do have this problem. I've run into it before. I've never had that problem with a Linux system though. Guess I might if I had less than 512 meg of ram or so though. It's not the size of the files, it is the number of files. Rick
