Re: [vchkpw] Migration from 5.4.26 to 5.4.27
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Harm van Tilborg wrote: > During the migration I've changed the hostname of the server, from > mail.example.com to pop1.mail.example.com and pop2.mail.example.com. I > can remember there were some problems from users by getting their mail > (through qmail-pop3d). At that time, I've changed the server's hostname > back to mail.example.com (at both servers), and every mail could be > reached smoothly again by our customers. The hostname of a system is not important, especially when dealing with protocols for retrieving mail. > Nowadays both servers are still called mail.example.com. However, I > finally want to get rid of those names. So I was wondering what could > have caused these problems? Unfortunately, you haven't provided enough detail to even know what the problems were, let alone enough to provide a possible solution. > vdelivermail generates the mail's new filename, based on the hostname, > process id, timestamp and perhaps the Maildir++'s 'S='. I was > wondering whether any of the delivery agents (such qmail-pop3d or > bincimap) makes use of the hostname to give out such mails. qmail-pop3d is not a delivery agent, nor is bincimap. > Or was this maybe a misbehavior of vpopmail around those version numbers. The Maildir specification does not specify the format for message filenames, it simply states they are unique. How you arrive at a unique filename is entirely up to the delivery process. Processes reading these files, like a POP3 or IMAP server, do not care what the filename is. Therefore, unless vpopmail was trying to re-create the same filenames over and over, there was no misbehavior in previous versions. - -- /* Matt BrookingsGnuPG Key D9414F70 Software developer Systems technician Inter7 Internet Technologies, Inc. (815)776-9465 */ -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAknvG6YACgkQ6QgvSNlBT3DkRwCfTGbuFypRhfpMNlbA/X9EhLpI S1cAoJASBFiuL1wvrjFzfjMBUUqhdcej =b8wW -END PGP SIGNATURE-
Re: [vchkpw] Migration using Rsync - Memory Problem
Juan Enciso wrote: Hello List. I'm worried because my migration scheme is giving problems. My migration plan was use rsync for transfer all user's mailbox, but it don't work because my directory tree is very big. I have around 50K maildir directory (110 GB approx). I checked the rsync documentation and I note that it have memory problems with big directory tree (It use a lot of memory and it does shutdown my server). I need transfer all the maildirs without turn off the mail service or in any case, the time off of service should be short, for that reason I did think use rsync, because it could make incremental copy. Please, somebody know others alternative to rsync? Hi, Use rsync but with a script file that limits the number of files it does per pass. ie rsync vpopmail/domain.com/0 remote.example.com::vpopmail/domain.com/0 . . . rsync vpopmail/domain.com/Z remote.example.com::vpopmail/domain.com/Z Regards, Rick
Re: Re: [vchkpw] migration
Slick! Wish I'd gotten this yesterday. D0H! > - Original Message - > Subject: Re: [vchkpw] migration > From: Jeremy Kitchen <[EMAIL PROTECTED]> > To: vchkpw@inter7.com > Date: 06-17-2005 8:46 am > > > On Thursday 16 June 2005 02:48 pm, Scott Gamble wrote: > > This obviously poses something of a problem in terms of migration. From a > > mysql install of vpopmail to a non-mysql install of vpopmail. > > > > First question - I'm going to have to recreate all these email accounts by > > hand aren't I...? > > nope, look at ~vpopmail/bin/vconvert > > > Second question - if we're going with a Non-Sql Integrated (NSI) version of > > vpopmail we're going to stick with it. How does one migrate data from one > > NSI-vpopmail server to another? How does it keep track of account > > information if not with mysql? > > it uses a passwd and passwd.cdb file in each domain's directory. > > -Jeremy > > -- > Jeremy Kitchen ++ Systems Administrator ++ Inter7 Internet Technologies, Inc. > [EMAIL PROTECTED] ++ inter7.com ++ 866.528.3530 ++ 815.776.9465 int'l > kitchen @ #qmail #gentoo on EFnet IRC ++ scriptkitchen.com/qmail > GnuPG Key ID: 481BF7E2 ++ jabber:[EMAIL PROTECTED]
Re: [vchkpw] migration
On Thursday 16 June 2005 02:48 pm, Scott Gamble wrote: > This obviously poses something of a problem in terms of migration. From a > mysql install of vpopmail to a non-mysql install of vpopmail. > > First question - I'm going to have to recreate all these email accounts by > hand aren't I...? nope, look at ~vpopmail/bin/vconvert > Second question - if we're going with a Non-Sql Integrated (NSI) version of > vpopmail we're going to stick with it. How does one migrate data from one > NSI-vpopmail server to another? How does it keep track of account > information if not with mysql? it uses a passwd and passwd.cdb file in each domain's directory. -Jeremy -- Jeremy Kitchen ++ Systems Administrator ++ Inter7 Internet Technologies, Inc. [EMAIL PROTECTED] ++ inter7.com ++ 866.528.3530 ++ 815.776.9465 int'l kitchen @ #qmail #gentoo on EFnet IRC ++ scriptkitchen.com/qmail GnuPG Key ID: 481BF7E2 ++ jabber:[EMAIL PROTECTED] pgpH18mPcUmya.pgp Description: PGP signature
Re: [vchkpw] migration from 4.9 to 5.4+mysql
but authentication fail, what's wrong? El Viernes, 1 de Octubre de 2004 17:32, Eduardo Cortés escribió: > vconvert is the solution, sorry > > El Viernes, 1 de Octubre de 2004 16:52, Eduardo Cortés escribió: > > Hi all, > > I want to migrate some domains from vpopmail 4.9.10 (with cdb) to latest > > 5.4 with mysql. On 5.2 versions, crypt passwords from 4.9 are valid on > > new installations with md5, but on mysql it seems to fail (I has replaced > > pw_passwd field with the encrypted password from vpasswd file and it fail > > to auth), how can I migrate passwords from 4.9 to mysql? All passwords > > are encrypted, any solution? > > Regards, > > Eduardo
Re: [vchkpw] migration from 4.9 to 5.4+mysql
vconvert is the solution, sorry El Viernes, 1 de Octubre de 2004 16:52, Eduardo Cortés escribió: > Hi all, > I want to migrate some domains from vpopmail 4.9.10 (with cdb) to latest > 5.4 with mysql. On 5.2 versions, crypt passwords from 4.9 are valid on new > installations with md5, but on mysql it seems to fail (I has replaced > pw_passwd field with the encrypted password from vpasswd file and it fail > to auth), how can I migrate passwords from 4.9 to mysql? All passwords are > encrypted, any solution? > Regards, > Eduardo
Re: [vchkpw] Migration
Wanted to thank everyone for their help. I tarred /home/vpopmail/domains updated the assign file and so far everything is working smoothly. I'll keep you posted on how it goes. Thanks again. Simon - Original Message - From: "Dave Kettmann" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Monday, August 23, 2004 3:57 PM Subject: RE: [vchkpw] Migration > -Original Message- > From: Jacob S. [mailto:[EMAIL PROTECTED] > Sent: Monday, August 23, 2004 2:44 PM > To: [EMAIL PROTECTED] > Subject: Re: [vchkpw] Migration > > > (quoting fixed) > > On Mon, 23 Aug 2004 14:35:27 -0500 > "Dave Kettmann" <[EMAIL PROTECTED]> wrote: > > > -Original Message- > > From: Simon [mailto:[EMAIL PROTECTED] > > Sent: Monday, August 23, 2004 2:08 PM > > To: [EMAIL PROTECTED] > > Subject: [vchkpw] Migration > > > > > > Hello everyone, > > > > I am presently running. vpopmail 5.2.1 with qmailadmin 1.0.6. I have > > just built a new box running vpopmail 5.4.0 and qmailadmin > 1.2.0. The > > new box is running without a problem. I would like to move > my current > > domains to the new box. I was planning on tarballing the domains > > folder and then untarballing it on the new machine. I > assume that its > > not this simple any help would be appreciated. > > I believe this can work, but make sure that the path to > ~vpopmail is the > same on both servers. Also don't forget to copy your changes in > /var/qmail/users/assign. > > > > It depends on how you are storing your passwords. I did a > move from > > > one machine to another (not sure what versions) and if qmailadmin > > > and vpopmail are working fine on the new box, I had no > problem just > > > tarring up the mail structure (/home/vpopmail/*) and putting it on > > > the new machine. Just dont forget to copy the passwords over :) I > > > may be wrong in this, but it did seem to work for me. If I'm wrong > > > .. someone please correct me. > > Don't you mean tarring the mail structure _/home/vpopmail/domains_? > Simply tarring /home/vpopmail/* will will defeat his upgrade, as the > vpopmail bin directory is part of what you just tarred. Yes, I did mean that, sorry to confuse anyone. :-) > > HTH, > Jacob > > -- > GnuPG Key: 1024D/16377135 > > Random .signature #57: > Windows NT encountered the following error: The operation completed > successfully. >
Re: [vchkpw] Migration
Are you using MySQL? We have done this twice, in the middle of the second migration now. If you have a lot of domains (over 100) getting the moved domains into the proper split is a concern. Example, you have 101 domains, domain 99.com is in /home/vpopmail/domains/99.com. But your new server already has 20 domains. When you add domain 99.com it will be in /home/vpopmail/domains/0/99.com. See the problem? All your paths in vpasswd, assign, possibly your .qmail files will be wrong. If you can script the edit process on assign, virtualdomains, morercpthosts, .qmail files, it is not too bad. This is what I have done both times. My recommended processes if you don't have that many users/domains is to simply add the domains and users to the new system, point your DNS there. Once you confirm the new system is working then move the mail files over to their new home. This can be done easily by mounting the domains directory from the old server to the new server and copying the Maildir contents from the old Maildir to the new Maildir. Keep in mind, vpopmail doesn't care what you do, the issue (if you don't use MySQL) is qmail. Keep thinking what you have to accomplish to make qmail happy and vpopmail will work just fine. I have found vuserinfo, vadduser, vadddomain, vmoduser, vpasswd to be very happy run from a script. DAve Simon wrote: Hello everyone, I am presently running. vpopmail 5.2.1 with qmailadmin 1.0.6. I have just built a new box running vpopmail 5.4.0 and qmailadmin 1.2.0. The new box is running without a problem. I would like to move my current domains to the new box. I was planning on tarballing the domains folder and then untarballing it on the new machine. I assume that its not this simple any help would be appreciated. Simon -- Systems Administrator http://www.tls.net Get rid of Unwanted Emails...get TLS Spam Blocker!
RE: [vchkpw] Migration
> -Original Message- > From: Jacob S. [mailto:[EMAIL PROTECTED] > Sent: Monday, August 23, 2004 2:44 PM > To: [EMAIL PROTECTED] > Subject: Re: [vchkpw] Migration > > > (quoting fixed) > > On Mon, 23 Aug 2004 14:35:27 -0500 > "Dave Kettmann" <[EMAIL PROTECTED]> wrote: > > > -Original Message- > > From: Simon [mailto:[EMAIL PROTECTED] > > Sent: Monday, August 23, 2004 2:08 PM > > To: [EMAIL PROTECTED] > > Subject: [vchkpw] Migration > > > > > > Hello everyone, > > > > I am presently running. vpopmail 5.2.1 with qmailadmin 1.0.6. I have > > just built a new box running vpopmail 5.4.0 and qmailadmin > 1.2.0. The > > new box is running without a problem. I would like to move > my current > > domains to the new box. I was planning on tarballing the domains > > folder and then untarballing it on the new machine. I > assume that its > > not this simple any help would be appreciated. > > I believe this can work, but make sure that the path to > ~vpopmail is the > same on both servers. Also don't forget to copy your changes in > /var/qmail/users/assign. > > > > It depends on how you are storing your passwords. I did a > move from > > > one machine to another (not sure what versions) and if qmailadmin > > > and vpopmail are working fine on the new box, I had no > problem just > > > tarring up the mail structure (/home/vpopmail/*) and putting it on > > > the new machine. Just dont forget to copy the passwords over :) I > > > may be wrong in this, but it did seem to work for me. If I'm wrong > > > .. someone please correct me. > > Don't you mean tarring the mail structure _/home/vpopmail/domains_? > Simply tarring /home/vpopmail/* will will defeat his upgrade, as the > vpopmail bin directory is part of what you just tarred. Yes, I did mean that, sorry to confuse anyone. :-) > > HTH, > Jacob > > -- > GnuPG Key: 1024D/16377135 > > Random .signature #57: > Windows NT encountered the following error: The operation completed > successfully. >
Re: [vchkpw] Migration
(quoting fixed) On Mon, 23 Aug 2004 14:35:27 -0500 "Dave Kettmann" <[EMAIL PROTECTED]> wrote: > -Original Message- > From: Simon [mailto:[EMAIL PROTECTED] > Sent: Monday, August 23, 2004 2:08 PM > To: [EMAIL PROTECTED] > Subject: [vchkpw] Migration > > > Hello everyone, > > I am presently running. vpopmail 5.2.1 with qmailadmin 1.0.6. I have > just built a new box running vpopmail 5.4.0 and qmailadmin 1.2.0. The > new box is running without a problem. I would like to move my current > domains to the new box. I was planning on tarballing the domains > folder and then untarballing it on the new machine. I assume that its > not this simple any help would be appreciated. I believe this can work, but make sure that the path to ~vpopmail is the same on both servers. Also don't forget to copy your changes in /var/qmail/users/assign. > > It depends on how you are storing your passwords. I did a move from > > one machine to another (not sure what versions) and if qmailadmin > > and vpopmail are working fine on the new box, I had no problem just > > tarring up the mail structure (/home/vpopmail/*) and putting it on > > the new machine. Just dont forget to copy the passwords over :) I > > may be wrong in this, but it did seem to work for me. If I'm wrong > > .. someone please correct me. Don't you mean tarring the mail structure _/home/vpopmail/domains_? Simply tarring /home/vpopmail/* will will defeat his upgrade, as the vpopmail bin directory is part of what you just tarred. HTH, Jacob -- GnuPG Key: 1024D/16377135 Random .signature #57: Windows NT encountered the following error: The operation completed successfully. pgpsJhb6X9uFi.pgp Description: PGP signature
RE: [vchkpw] Migration
Simon, It depends on how you are storing your passwords. I did a move from one machine to another (not sure what versions) and if qmailadmin and vpopmail are working fine on the new box, I had no problem just tarring up the mail structure (/home/vpopmail/*) and putting it on the new machine. Just dont forget to copy the passwords over :) I may be wrong in this, but it did seem to work for me. If I'm wrong .. someone please correct me. Dave Kettmann NetLogic 636-561-0680 -Original Message-From: Simon [mailto:[EMAIL PROTECTED]Sent: Monday, August 23, 2004 2:08 PMTo: [EMAIL PROTECTED]Subject: [vchkpw] Migration Hello everyone, I am presently running. vpopmail 5.2.1 with qmailadmin 1.0.6. I have just built a new box running vpopmail 5.4.0 and qmailadmin 1.2.0. The new box is running without a problem. I would like to move my current domains to the new box. I was planning on tarballing the domains folder and then untarballing it on the new machine. I assume that its not this simple any help would be appreciated. Simon
Re: [vchkpw] Migration from MySQL to CDB Files
On Jun 25, 2004, at 12:26 PM, Rick Macdougall wrote: How are you running both qscanq and qmail-spamc ? I modified qmail-spamc.c to run qmail-queue-real instead of qmail-queue. It's also modified to not run spamc if RELAYCLIENT is set -- this way, mail sent by my users is not scanned. I renamed /var/qmail/bin/qmail-queue to qmail-queue-real. I built qscanq, but I don't think I installed it. I have QMAILQUEUE set in my qmail-smtpd/run script: export QMAILQUEUE='/package/mail/qscanq/command/qscanq' So, qmail-smtpd runs qscanq (since QMAILQUEUE is set), qscanq runs /var/qmail/bin/qmail-queue (which is actually my qmail-spamc) which in turn runs spamc and the real qmail-queue. You can download my modified qmail-spamc.c here: http://tomlogic.com/qmail/qmail-spamc.c -- Tom Collins - [EMAIL PROTECTED] QmailAdmin: http://qmailadmin.sf.net/ Vpopmail: http://vpopmail.sf.net/ Info on the Sniffter hand-held Network Tester: http://sniffter.com/
Re: [vchkpw] Migration from MySQL to CDB Files
Tom Collins wrote: On Jun 25, 2004, at 11:54 AM, Jorge Valdes wrote: I would replace qmail-scanner 1.21 with qmail-spamc (for SpamAssassin processing) and qscanq (for ClamAV virus filtering). You would eliminate the Perl overhead of Qmail-Scanner and block incoming viruses at the qmail-smtpd level (cuts down on bounce message generation). I have been extremely happy with the results. Hi, How are you running both qscanq and qmail-spamc ? Regards, Rick
Re: [vchkpw] Migration from MySQL to CDB Files
On Jun 25, 2004, at 11:54 AM, Jorge Valdes wrote: I am running qmail-scanner 1.21, spamassassin 2.63(spamd), clamav 0.70, File::Scan 1.15, also I have patched tcpserver to deny connections if the load average is greater than 12 or more than 5 connections from the same IP address. This alleviated the problem temporarily. I would replace qmail-scanner 1.21 with qmail-spamc (for SpamAssassin processing) and qscanq (for ClamAV virus filtering). You would eliminate the Perl overhead of Qmail-Scanner and block incoming viruses at the qmail-smtpd level (cuts down on bounce message generation). I have been extremely happy with the results. My knowledge of process accounting is limited, otherwise I'd try to recommend something that could report on how much processor time and swapping was caused by different programs. If you don't have a lot of account adds/deletes/password-changes, then I'd think that going to CDB would be a good solution. You'll cut down on the memory overhead of MySQL connections, and looking up users in the CDB file should have a lower overhead. What's the maximum number of users you have in a domain? -- Tom Collins - [EMAIL PROTECTED] QmailAdmin: http://qmailadmin.sf.net/ Vpopmail: http://vpopmail.sf.net/ Info on the Sniffter hand-held Network Tester: http://sniffter.com/
Re: [vchkpw] Migration from MySQL to CDB Files
At 10:57 AM 25/06/2004, Greg Kopp wrote: I doubt this is a MySQL problem. We are using MySQL in several environments, including vpopmail. My guess is I don't have as many users as you, since I set my pop3 concurency at 30 with no problems. Are you running qmail-scanner or spamassassin? These added quite a bit of load to my mail server (dual 2.8 Xeons, 2GB RAM) but it still averages at around 0.3 for the 5min average. It will spike if Razor is slow to respond or one of our customers does a blast e-mail to thier client base. You can try and put MySQL on a seperate server and just have vpopmail connect to the external database and see if that solves any load problems. BTW, What versions are you running? I cannot add more RAM to the server.. :( I am running qmail-scanner 1.21, spamassassin 2.63(spamd), clamav 0.70, File::Scan 1.15, also I have patched tcpserver to deny connections if the load average is greater than 12 or more than 5 connections from the same IP address. This alleviated the problem temporarily. I have gathered the following average daily stats: 438 IMAP connections0 Rejects due to load/maxip 28708 POP3 connections 308 Rejects due to load/maxip 7355 SMTP connections 128 Rejects due to load/maxip Bloquing: 10 Emails due to policy reasons 2923 Emails due to Virus 595 Emails due to Spam > 15.0 pts. 39.4% spam/ham ratio Each instance of mysql Ver 11.18 Distrib 3.23.58, for pc-linux (i686) uses PID USER PR NI VIRT RES SHR S %CPU %MEM TIME COMMAND 1055 mysql 9 0 14500 9376 1152 S 0.0 1.8 4:28 mysqld So when there are several concurrent POP3 connections, another instance of mysql is spawned to query the DB. I think this is whats bringing the server to its knees... In any case, has anyone done a MySQL to CDB migration? -- Jorge Valdes IT Manager Intercom El Salvador [EMAIL PROTECTED] Tel. 503-278-5068 Tel. 503-265-7070 Fax. 503-265-7025
Re: [vchkpw] Migration from MySQL to CDB Files
I doubt this is a MySQL problem. We are using MySQL in several environments, including vpopmail. My guess is I don't have as many users as you, since I set my pop3 concurency at 30 with no problems. Are you running qmail-scanner or spamassassin? These added quite a bit of load to my mail server (dual 2.8 Xeons, 2GB RAM) but it still averages at around 0.3 for the 5min average. It will spike if Razor is slow to respond or one of our customers does a blast e-mail to thier client base. You can try and put MySQL on a seperate server and just have vpopmail connect to the external database and see if that solves any load problems. BTW, What versions are you running? On Fri, 25 Jun 2004 09:53:35 -0600, Jorge Valdes <[EMAIL PROTECTED]> wrote: > > Hi, > > I have been having problems with resouce utilization with one of my qmail > servers, which basically boils down to reaching the concurrencylimit of > tcpserver for my pop3 connections (60) and this brings the server to its > knees. The only recourse I have left is to reboot, since even console is > not responsive. > > I have checked my logs, and basically, for each pop3 connection, an > instance of mysql is generated to check password for a virtual domain user, > this generates swapping. The load average shoots up to 300 and I have seen > this go as high as 500. > > The server is running Linux RedHat 7.3 and has 512MB RAM and 1GB SWAP, for > about 4000 users in 70 virtual domains. It also runs Apache for qmailadmin. > Since there are not that many users per domain, less than 100 average, will > CDB Files be a better solution to this problem? > > Currently running vpopmail 5.3.19 and qmailadmin 1.0.15 > > > -- > Jorge Valdes > IT Manager > Intercom El Salvador > [EMAIL PROTECTED] > >
Re: [vchkpw] Migration from MySQL to CDB Files
On Jun 25, 2004, at 8:53 AM, Jorge Valdes wrote: I have been having problems with resouce utilization with one of my qmail servers, which basically boils down to reaching the concurrencylimit of tcpserver for my pop3 connections (60) and this brings the server to its knees. The only recourse I have left is to reboot, since even console is not responsive. What happens if you set concurrencylimit to 30? Are people checking their email too often? Is the problem that the POP3 users take a long time to download their email (on dialup)? I have checked my logs, and basically, for each pop3 connection, an instance of mysql is generated to check password for a virtual domain user, this generates swapping. The load average shoots up to 300 and I have seen this go as high as 500. Are you sure there isn't something else happening on the machine that pushes the load up? The server is running Linux RedHat 7.3 and has 512MB RAM and 1GB SWAP, for about 4000 users in 70 virtual domains. It also runs Apache for qmailadmin. Since there are not that many users per domain, less than 100 average, will CDB Files be a better solution to this problem? Can this machine work with more memory? You could add another 512M or even 1G of RAM and alleviate the problem. I'm sorry, but I don't know whether switching to CDB will help with your problem. -- Tom Collins - [EMAIL PROTECTED] QmailAdmin: http://qmailadmin.sf.net/ Vpopmail: http://vpopmail.sf.net/ Info on the Sniffter hand-held Network Tester: http://sniffter.com/
Re: [vchkpw] Migration of ~vpopmail/domains
On Wed, Jan 22, 2003 at 08:58:23AM -0600, [EMAIL PROTECTED] wrote: > I have been trying to move the ~vpopmail/domains directory to another > server. I did a tar --preserve-permissions -czf domains.tar.gz > ~vpopmail/domains to tar up all of the domains directory, then scp'd it > over to the other machine. I untar'd it in ~vpopmail I checked the > archives and it said you need to copy over /var/qmail/control/* and > /var/qmail/users/* which I have done. I can log into qmailadmin, but > once I'm logged in, I can't see any of the users. I tried vuserinfo on > several users, and they show up as not existing. Did I miss any files > in the transfer? I thought I have them all but I might be missing > something along the line. Check if the id/gid of vpopmail/vchkpw match on the two boxes, and if needed modify /var/qmail/users/asign with the correct one. > > -- > Thanks, > > Rob Burtelow > [EMAIL PROTECTED] > http://www.yewnicks.org > > > > Welcome. -- Kiril Todorov < -+- > +359 2 9712013 Bulgaria Online < -+- > http://home.online.bg /* waiting... dreaming... wishing... */
