Re: Mail servers for large numbers of users
On Sun, Dec 12, 1999 at 05:11:54AM +0800, Ronald Tin wrote: How could I deliver mails to accounts that don't really exist? (I can't allocate 10 uids on a single machine, right?) I have only read the FAQ and anatomy for postfix Shall I play with the mailbox transport option for local delivery, or do I create local users with same uid (and disable their login) ? Do the 2nd solution really work? But still it doesn't seem very efficient to store 10 files in a single mail spool directory? ... The MTA won't give you any problems - they typically don't care about the password database. However, here's another reason why you don't want to But the MTA have to lookup the local user from the password database.. and mails are stored under owner's uid. That's the case for sendmail and exim at least. qmail can handle an unlimited number of users in an unlimited number of domains using the likes of vchkpw. Check out: http://www.qmail.org qmail is highly efficient and designed to be secure. It does have its quirks though -- William Burrow -- New Brunswick, Canada A 'box' is something that accomplishes a task -- you feed in input and out comes the output, just as God and Larry Wall intended. -- brian moore
Re: Mail servers for large numbers of users
On Sat, Dec 11, 1999 at 06:27:58PM -0800, George Bonser wrote: On Sat, 11 Dec 1999, William Burrow wrote: qmail can handle an unlimited number of users in an unlimited number of domains using the likes of vchkpw. Check out: So can exim. The problem is not delivering email via smtp ... the problem is accessing it via IMAP. If you have 64K users, how do you set permissions so that one user can not deduce the path to another user's directory and potentially read their mail? Oops, missed the IMAP bit, was looking at POP3. Anyway, it sounds like some kind of problem with the IMAP server. Perhaps you can elaborate in private email the particular problem that an IMAP server has in this regard. Then it will become possible to search out a solution. The problem will be corrected soon with the raising of UIDs from 16 bits to 32 bits but until then, how do you make sure that each user can only access their own directory from anywhere on the planet if you have more users than UIDs? You design your server to separate the paths that users are permitted to access in a consistent, logical manner. The fact that a path exists to the user does not mean it maps directly to any shared path on the server. Think of virtual domains and web sites. UIDs are irrelevant. For reading email, I cannot see any reason to allow an external user to peruse the entire system directory structure. It doesn't make sense. Excuse my ignorance on how this service is implemented, but I can't see this being a problem in a properly designed system. -- William Burrow -- New Brunswick, Canada A 'box' is something that accomplishes a task -- you feed in input and out comes the output, just as God and Larry Wall intended. -- brian moore
Re: Mail servers for large numbers of users
On Sat, 11 Dec 1999, George Bonser wrote: NOT want to be able to type the path to another user's directory and see their mail (or they, mine). My mail folders need to be unreadable by any other user except me and the mail delivery program. That is why you integrate the proper security measures into your IMAP daemon - for instance you could have to chroot to the proper user directory so that it cannot access files ouside that area then have it run as not-root so it cannot chroot out. There are many other possible schemes that don't require uids, but they all need special code. Jason
Re: Mail servers for large numbers of users
On Sat, Dec 11, 1999 at 06:27:58PM -0800, George Bonser wrote: So can exim. The problem is not delivering email via smtp ... the problem is accessing it via IMAP. If you have 64K users, how do you set permissions so that one user can not deduce the path to another user's directory and potentially read their mail? Whew, I did some investigation on the issue. Looked at Courier-IMAP, an alternative IMAP server to the UW-IMAP server. It clearly explains in its documentation how you can setup thousands of virtual accounts without requiring any more than just *one* UID. Courier-IMAP has a link on the qmail web page, and seems to be written independent of any particular mailer, so long as Maildir format is supported (see safecat for adding Maildir format mailboxes to any mailer). In the meantime, the EASIEST method is to simply use more than one server for READING email and using something to auto-direct the user to the proper server to read their mail. The EASIEST method is to understand your IMAP server or get one that works properly. -- William Burrow -- New Brunswick, Canada A 'box' is something that accomplishes a task -- you feed in input and out comes the output, just as God and Larry Wall intended. -- brian moore
Re: Mail servers for large numbers of users
On Sat, Dec 11, 1999 at 07:21:49PM -0800, George Bonser wrote: On Sat, 11 Dec 1999, William Burrow wrote: You design your server to separate the paths that users are permitted to access in a consistent, logical manner. The fact that a path exists to the user does not mean it maps directly to any shared path on the server. Think of virtual domains and web sites. UIDs are irrelevant. For reading email, I cannot see any reason to allow an external user to peruse the entire system directory structure. It doesn't make sense. Excuse my ignorance on how this service is implemented, but I can't see this being a problem in a properly designed system. You can not equate http and imap in this manner. http serves files. You Why not. An email resides in a file. In a Maildir setup, exactly one message resides in one file. With Unix mailbox format, several messages exist in one file, big deal. can create a different path for each virtual domain. If I type in the full path for a different domain, I see a different site. Not a big deal. You can't get from http://www.virtualOne.com to http://www.virtualTwo.com from from http://www.virtualOne.com by typing paths (other than a link directly to virtualTwo's site). It is impossible. Same deal with IMAP servers. IMAP does not transfer mail to the user. It allows a remote user to access their mail on a local filesystem. That is the point of IMAP. You can check I am aware of the purpose of IMAP, I have not seen the implementation. The point is that other users on the system MUST be prevented from reading my mail files. This is done with ownership permissions on the directory. Do that with ONE UID. Courier-IMAP does this. All users must access their mail through IMAP. It makes sense. It works. It is the way it is done. You don't own that directory according to the database in the IMAP server, you aren't allowed to enter it. Your request is turned away for entering a bad path. Just like entering a path that doesn't exist for a particular virtual domain on a web server. Otherwise, I could log in as me but tell IMAP to use someone elses directory and read their mail. This is why it breaks when you have more users than you have bits to assign unique user ID's. You can't do this because the path you specify is not associated with your IMAP login ID. The database tells the server what is the acceptable base path. The path is logically constructed, so it is easy to tell apart illegal paths from legal paths. The scenario you present is all in your head. POP3 is no big deal, SMTP is no big deal. IMAP is a big deal because it is a direct read/write file access to a file that remains persistant on the server. Persistency is not a big deal, email can be made to persist on POP3 servers as well, it is just not often done (and seems to be often discouraged with small quotas). I am just a little bothered by statements that implementations of Internet services are broken and useless by design. Particularly when RTFM shows otherwise. I hope my outburst is not taken the wrong way. -- William Burrow -- New Brunswick, Canada A 'box' is something that accomplishes a task -- you feed in input and out comes the output, just as God and Larry Wall intended. -- brian moore
Re: Mail servers for large numbers of users
On Sat, 11 Dec 1999, Ronald Tin wrote: I am about to build a (group of?) e-mail server for a large number of users... more than 65536. Using Debian of course. :) It should include SMTP and IMAP. Users do not need to have login accounts. Probably I will be using Potato. What should I start with? Are there good open sourced MTA and IMAP servers that do not operate on system password accounts? (so I don't have to create 6+ accounts) For now I will be trying to distribute the accounts on more than 1 server, set-up the DNS to have more than 1 MX, and forward the mails internally through SMTP for non-existent local users. Which MTA is good for this purpose? For ease of configuration I may be choosing exim, because this is what I have been using at home (it's the default MTA for Debian.. :) I see that exim can match user names with dbm/ldap lookups as well.. which may prove useful. What about qmail or postfix? Are there other precautions that I have to take care? Are there any other precautions? -- Unsubscribe? mail -s unsubscribe [EMAIL PROTECTED] /dev/null Hi Ronald, I'm doing an SMTP / POP3 / Finger server that run on Linux M$ WinNT that support an unlimited number of users and that is not linked to system user accounts. It handle multiple domains, mailing lists, anti-spam, relay-checking, pop3 mailbox syncronization with external users POP3 accounts, SMTP mail exchanger overrides. It is written in C++ using gcc in Linux and Visual C++ in M$-Win and is full GPL license. This server is running in my Co. ( under NT ) from 6 months without give any problems to us. It works fine also on HPUX 9.x 10.x. I've implemented a file that contain all system dependent code to localize portability. I'm having a semaphore weird in Linux in which seems that if a task A creates tasks B C, C creates another task D, if B go to sleep on a semaphore and D release the semaphore, B don't wakeup. As soon as I've solved this problem I'll release the code ( say beta 0.1 ) on FreshMeat. Anyway if You 're interested I can make a snapshot for You. Cheers, Davide. -- Debian, the freedom in freedom.
Re: Mail servers for large numbers of users
In article [EMAIL PROTECTED], Davide Libenzi [EMAIL PROTECTED] wrote: I'm doing an SMTP / POP3 / Finger server that run on Linux M$ WinNT that support an unlimited number of users and that is not linked to system user accounts. It is written in C++ using gcc in Linux and Visual C++ in M$-Win and is full GPL license. That sounds great! I'm having a semaphore weird in Linux in which seems that if a task A creates tasks B C, C creates another task D, if B go to sleep on a semaphore and D release the semaphore, B don't wakeup. As soon as I've solved this problem I'll release the code ( say beta 0.1 ) on FreshMeat. Why? Remember what they say release early, release often. If you make an alpha release now, and put this clearly in the BUGS file, chances are someone will find the bug and solve if for you. That's what open source is all about. Leave a note here when you put it up on freshmeat Mike.
Mail servers for large numbers of users
I am about to build a (group of?) e-mail server for a large number of users... more than 65536. Using Debian of course. :) It should include SMTP and IMAP. Users do not need to have login accounts. Probably I will be using Potato. What should I start with? Are there good open sourced MTA and IMAP servers that do not operate on system password accounts? (so I don't have to create 6+ accounts) For now I will be trying to distribute the accounts on more than 1 server, set-up the DNS to have more than 1 MX, and forward the mails internally through SMTP for non-existent local users. Which MTA is good for this purpose? For ease of configuration I may be choosing exim, because this is what I have been using at home (it's the default MTA for Debian.. :) I see that exim can match user names with dbm/ldap lookups as well.. which may prove useful. What about qmail or postfix? Are there other precautions that I have to take care? Are there any other precautions?
Re: Mail servers for large numbers of users
A long time ago, in a galaxy far, far way, someone said... I am about to build a (group of?) e-mail server for a large number of users... more than 65536. Ooh... that's quite a few :) Using Debian of course. :) A noble cause! It should include SMTP and IMAP. Users do not need to have login accounts. Probably I will be using Potato. What should I start with? Either exim or postfix, definitely - they're very easy to configure. I've not had any experience with anything else. Are there good open sourced MTA and IMAP servers that do not operate on system password accounts? (so I don't have to create 6+ accounts) The MTA won't give you any problems - they typically don't care about the password database. However, here's another reason why you don't want to base the accounts on the password database: Linux's password database can't (as it is in Debian) can't deal with more than slightly less than 65536 users. You'd do good to look into something that can use LDAP (or something like that) for authentication. For now I will be trying to distribute the accounts on more than 1 server, set-up the DNS to have more than 1 MX, and forward the mails internally through SMTP for non-existent local users. It might be easier to use round-robin DNS for mail.yourdomain.com. Which MTA is good for this purpose? Any of them will do. [...] -- -- Phil Brutsche [EMAIL PROTECTED] There are two things that are infinite; Human stupidity and the universe. And I'm not sure about the universe. - Albert Einstein
Re: Mail servers for large numbers of users
A long time ago, in a galaxy far, far way, someone said... It should include SMTP and IMAP. Users do not need to have login accounts. Probably I will be using Potato. What should I start with? Either exim or postfix, definitely - they're very easy to configure. I've not had any experience with anything else. I've seen setups like these being mentioned in the Postfix ML (more than one person there claimed more than 10 users under Cyrus IMAPD+Postfix). Cyrus IMAPD seems to be the way to go for that many accounts, but beware their non-dfsg license. They like ISPs, though ;-) Cyrus is not available as a debian package, though there's a pending ITP. Are there good open sourced MTA and IMAP servers that do not operate on system password accounts? (so I don't have to create 6+ accounts) The MTA won't give you any problems - they typically don't care about the password database. However, here's another reason why you don't want to The MTA will give you problems if too many users try to bang on your server at the same time, though :-) You'll need a paralelized-delivery MTA (i.e.: not sendmail). Postfix seems to be able to handle this very well, and so does qmail, I'm told. I don't know about exim. Postfix is very easy to setup, very light on the cpu and is engineered for security and speed (runs in a chroot jail in debian's default configuration, for example), but it is still beta. Exim is even easier to setup IMHO, but forget the 'engineered for security' part. -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh
Re: Mail servers for large numbers of users
On Sat, Dec 11, 1999 at 05:07:53PM -0200, Henrique M Holschuh wrote: A long time ago, in a galaxy far, far way, someone said... It should include SMTP and IMAP. Users do not need to have login accounts. Probably I will be using Potato. What should I start with? Either exim or postfix, definitely - they're very easy to configure. I've not had any experience with anything else. I've seen setups like these being mentioned in the Postfix ML (more than one person there claimed more than 10 users under Cyrus IMAPD+Postfix). Sorry if this question looks dumb... How could I deliver mails to accounts that don't really exist? (I can't allocate 10 uids on a single machine, right?) I have only read the FAQ and anatomy for postfix Shall I play with the mailbox transport option for local delivery, or do I create local users with same uid (and disable their login) ? Do the 2nd solution really work? But still it doesn't seem very efficient to store 10 files in a single mail spool directory? Cyrus IMAPD seems to be the way to go for that many accounts, but beware their non-dfsg license. They like ISPs, though ;-) Cyrus is not available as a debian package, though there's a pending ITP. Are there good open sourced MTA and IMAP servers that do not operate on system password accounts? (so I don't have to create 6+ accounts) The MTA won't give you any problems - they typically don't care about the password database. However, here's another reason why you don't want to But the MTA have to lookup the local user from the password database.. and mails are stored under owner's uid. That's the case for sendmail and exim at least. And, thanks for all the replies. :)
Re: Mail servers for large numbers of users
On Sat, 11 Dec 1999, Henrique M Holschuh wrote: Postfix is very easy to setup, very light on the cpu and is engineered for security and speed (runs in a chroot jail in debian's default configuration, Hmm... my bad here. Debian's postfix can be very easily set to run chrooted (just change a bunch of 'n' to 'y' in /etc/postfix/master.cf), but doesn't by default. -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh
Re: Mail servers for large numbers of users
On Sun, 12 Dec 1999, Ronald Tin wrote: On Sat, Dec 11, 1999 at 05:07:53PM -0200, Henrique M Holschuh wrote: I've seen setups like these being mentioned in the Postfix ML (more than one person there claimed more than 10 users under Cyrus IMAPD+Postfix). Sorry if this question looks dumb... No, it's not dumb. But you could have found the answer by going through the Cyrus WWW site ;-) How could I deliver mails to accounts that don't really exist? (I can't allocate 10 uids on a single machine, right?) You deliver them to a database of some sort. All the 'mail user account' concept is kept by your program. As far as the OS is concerned, these users do not exist. Accounts, bookeeping and permissions are all managed by the program. For Cyrus, this means you have to call its special 'delivery agent' to store the mail to the users 'mailbox', and also that the only way to access this email is through the pop3 and imap components of Cyrus. You'll probably need LDAP to keep the user's information, but that's something you'll need to read the Cyrus docs to find out for sure. I have only read the FAQ and anatomy for postfix Shall I play with the mailbox transport option for local delivery, or do I create local users with same uid (and disable their login) ? Do the 2nd solution really work? I have not tried this with postfix, but it should be simple enough: read all the docs (this ought to be a faq, I think). I recall you need to uncomment the transport option which says 'cyrus'. This makes postfix use the 'cyrus' line in master.cf as the transport, and I believe that in Debian, this line is already there. You'll need to do more than that to get it to work fine, though. For one, you _will_ have to fine-tune master.cf, as the default cannot handle a very busy system. You'll need to limit the max number of incoming smtpd processes, as well as the number of outgoing smtp processes, etc. to avoid swapping. Lots and lots of RAM will help :-) But still it doesn't seem very efficient to store 10 files in a single mail spool directory? I don't think Cyrus keeps this in a single mail spool directory, probably it uses database backends. You *will* need decent IO for this many email users in a single machine, as well as lots of disk space and a lot of RAM. This much I'm sure. The MTA won't give you any problems - they typically don't care about the password database. However, here's another reason why you don't want to But the MTA have to lookup the local user from the password database.. and mails are stored under owner's uid. That's the case for sendmail and exim at least. I believe exim can be configured to look up the users somewhere else. As for postfix, I *think* you will have to add all users to a database (db, dbm or a text file which is hashed by the 'postmap' program included with postfix, and I believe LDAP is possible as well with a patch) and tell it to use that. I'm really only repeating whatever I can remember from lurking at the postfix ml. Go after their archives, and you will find quite a lot of threads about this. -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh
