Re: Let's write a system admin friendly mail server packaging system
Thomas Goirand wrote: > Mario 'BitKoenig' Holbe wrote: >> Thomas Goirand wrote: >>> Mario 'BitKoenig' Holbe wrote: So far this is independent of third packages which is IMHO fine and desirable. So far, this could be solved by a postfix-conf.d-snippet shipped with the amavis package. >>> Quite not. You also need to configure the incoming and outgoing ports of >>> amavis the correct way. >> Which of course will also be done by the amavis package itself. Still, >> no dependency on third packages so far. > I think you quite don't get it. Here's 3 configuration: > - postfix with dkimproxy dkimproxy listens on localhost:10121, and ships a postfix-conf.d-snippet which instructs postfix to send mail to localhost:10121 and receive it back on localhost:10122. > - postfix with amavis amavis listens on localhost:10211, and ships a postfix-conf.d-snippet which instructs postfix to send mail to localhost:10211 and receive it back on localhost:10212. > - postfix with dkimproxy + amavis Bot postfix-conf.d-snippets are concatenated, that's it. postfix ships to localhost:10121 first, gets it back on 10122, ships it to localhost:10211, gets it back on 10212, and delivers it. Everything that package-maintainers had to do is to make sure their port-pairs don't collide with other packages - and to agree on some useful conf.d-snippet ordering, of course. As I said, I don't know if this would be possible with current postfix at all even if we'd implement conf.d-snippets on our own. sendmail milters can be and are combined that way, one "just" have to make sure to merge the milter-macro definitions correctly. > Cleaner for using it, of course not for the packaging. There's no reason > why you should load postfix more, and have the message go 3 times by it, > if it can go there only twice. That's exactly the small performance drop as price paid for a straight and clean configuration. regards Mario -- There are 10 types of people in the world: Those who understand binary, and those who don't... -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/slrni06nfi.m0q.mario.ho...@darkside.dyn.samba-tng.org
Re: Let's write a system admin friendly mail server packaging system
On Wed, 26 May 2010 18:42:50 +0200, Michael Banck wrote: >Eh, Debian can patch upstream software if it thinks it is necessary for >inter-operation, that's the one of the major points of having a >distribution. This is not always liked by the upstream communities though. For example, the exim communities hates us for integrating exim with debconf, and they don't even think a second whenever a Debian user shows up on their mailing lists and direct them directly to us for user support - even if it's a valid question that the community could easily answer. We even take care to make our config as similiar as possible to upstreams (we even follow changes in the _comments_ of their example config), but to no avail. Greetings Marc -- -- !! No courtesy copies, please !! - Marc Haber | " Questions are the | Mailadresse im Header Mannheim, Germany | Beginning of Wisdom " | http://www.zugschlus.de/ Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fon: *49 621 72739834 -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/e1oinnw-0004kv...@swivel.zugschlus.de
Re: Let's write a system admin friendly mail server packaging system
Mario 'BitKoenig' Holbe wrote: > Thomas Goirand wrote: >> Mario 'BitKoenig' Holbe wrote: >>> So far this is independent of third packages which is IMHO fine and >>> desirable. So far, this could be solved by a postfix-conf.d-snippet >>> shipped with the amavis package. >> Quite not. You also need to configure the incoming and outgoing ports of >> amavis the correct way. > > Which of course will also be done by the amavis package itself. Still, > no dependency on third packages so far. I think you quite don't get it. Here's 3 configuration: - postfix with dkimproxy - postfix with amavis - postfix with dkimproxy + amavis In these 3 cases, you have different configuration for the ports for both amavis, and the others. >>> * postfix ships to amavis >>> * amavis ships back to postfix >>> * postfix ships to dkimproxy >>> * dkimproxy ships back to postfix >> No, it's not any cleaner, and it's slower. As I wrote previously, we > > Cleaner from a dependency-perspective. Why isn't it? Cleaner for using it, of course not for the packaging. There's no reason why you should load postfix more, and have the message go 3 times by it, if it can go there only twice. > This way you are able to configure the whole integration within one > package (i.e. amavis ships the amavis-postfix integration, dkimproxy > ships the dkimproxy-postfix integration, etc. pp), you can just avoid > any complex chaining-magic. > > And slower? What are we talking about? Whole two percent? Well, both you and me are doing wild guesses here! We wouldn't be able to tell unless we bench it. But you must be right that the overhead wouldn't be so big, considering how much CPU amavis with spamassassin and clamav is taking. >> Are you trying to say that we shall ship a not performing configuration >> by default, because big ISP are capable of reconfiguring? If you are, > > I'm trying to say that I would prefer a straight, clean dependency- > minimizing approach over one that does and/or requires complex magic. > And I'm aware that clean approaches are usually somewhat less performant > than optimized setups, which, in turn, tend to be more complex. > > And I think that a clean and simple approach would help more users than > one that tries to squeeze the last cpu cycles out of your system for the > price of being hard to understand. > Don't get me wrong: I totally agree with you that what we have now is > neither the one nor the other. Cool. Then we agree! :) Just one remark (which doesn't invalidate your point): you named few percents of cpu cycle, but do not forget to also consider I/O here, as postfix will also use your HDD when managing its queue. I believe that the I/O wait is a bigger concern than any CPU load issue in this case. >> I think we shall try to release the best distribution as we can, with >> correct, performing values by default, and even, if possible, have a >> default configuration that you never even need to edit, because it's >> just right by default. We should at least have this goal in mind, > > "those goals" - you named three: correctness, performance and useful > defaults. Choose two of them :) Here, you are proposing to trade a little bit of performance, so that we have a correct, useful default, and with not too complex maintainer scripts. If we don't trade too much performance, then I don't mind. When having a server on the edge of collapsing because of the load, a bit more or less wont mater: you need a faster server and that is it. > I don't think it's bad to be idealistic. We just have different ideals > probably :) Well, most likely we just weigh conflicting goals different. In an ideal world, we wouldn't need such complex setup, as there wouldn't be any SPAM or viruses! :) Thomas -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4c012711.60...@goirand.fr
Re: Let's write a system admin friendly mail server packaging system
Thomas Goirand wrote: > Mario 'BitKoenig' Holbe wrote: >> So far this is independent of third packages which is IMHO fine and >> desirable. So far, this could be solved by a postfix-conf.d-snippet >> shipped with the amavis package. > Quite not. You also need to configure the incoming and outgoing ports of > amavis the correct way. Which of course will also be done by the amavis package itself. Still, no dependency on third packages so far. >> * postfix ships to amavis >> * amavis ships back to postfix >> * postfix ships to dkimproxy >> * dkimproxy ships back to postfix > No, it's not any cleaner, and it's slower. As I wrote previously, we Cleaner from a dependency-perspective. Why isn't it? This way you are able to configure the whole integration within one package (i.e. amavis ships the amavis-postfix integration, dkimproxy ships the dkimproxy-postfix integration, etc. pp), you can just avoid any complex chaining-magic. And slower? What are we talking about? Whole two percent? > Are you trying to say that we shall ship a not performing configuration > by default, because big ISP are capable of reconfiguring? If you are, I'm trying to say that I would prefer a straight, clean dependency- minimizing approach over one that does and/or requires complex magic. And I'm aware that clean approaches are usually somewhat less performant than optimized setups, which, in turn, tend to be more complex. And I think that a clean and simple approach would help more users than one that tries to squeeze the last cpu cycles out of your system for the price of being hard to understand. Don't get me wrong: I totally agree with you that what we have now is neither the one nor the other. And yes, I think that somebody who tries to optimize a system for highest performance will write his own configurations anyways. Hey, if you really like to squeeze cpu cycles the first thing you do is to build architecture-specific binaries. Writing some optimized config- files doesn't matter then anymore. > I think we shall try to release the best distribution as we can, with > correct, performing values by default, and even, if possible, have a > default configuration that you never even need to edit, because it's > just right by default. We should at least have this goal in mind, "those goals" - you named three: correctness, performance and useful defaults. Choose two of them :) There are way more btw. - not only for users but also for maintainers, etc. > Maybe I'm being too idealistic here. What's your opinion? I don't think it's bad to be idealistic. We just have different ideals probably :) Well, most likely we just weigh conflicting goals different. regards Mario -- Um mit einem Mann gluecklich zu werden, muss man ihn sehr gut verstehen und ihn ein bisschen lieben. Um mit einer Frau gluecklich zu werden, muss man sie sehr lieben und darf erst gar nicht versuchen, sie zu verstehen. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/slrnhvurj0.m0q.mario.ho...@darkside.dyn.samba-tng.org
Re: Let's write a system admin friendly mail server packaging system
Mario 'BitKoenig' Holbe wrote: > So far this is independent of third packages which is IMHO fine and > desirable. So far, this could be solved by a postfix-conf.d-snippet > shipped with the amavis package. Quite not. You also need to configure the incoming and outgoing ports of amavis the correct way. > That's pain, indeed, and should IMHO be avoided at all. > A clean way to conf this would be > * postfix ships to amavis > * amavis ships back to postfix > * postfix ships to dkimproxy > * dkimproxy ships back to postfix > I don't know if this is possible with postfix yet. The sendmail milter > approach is way cleaner regarding such stuff. No, it's not any cleaner, and it's slower. As I wrote previously, we really don't want to have lower performances on a mail server if we want to do things seriously. And by the way, you wrote: postfix -> amavis -> dkimproxy -> postfix when what we really want to do is: postfix -> dkimproxy -> amavis -> postfix for obvious performance reasons. >> This is a LOT less trivial than what you pretend. > > That's not just less trivial, it's horrible :) > > And this is probably one of the reasons why newer amavis is now able to > perform DKIM signing on it's own. So, this specific chaining should be > historic sooner or later. I do not agree, for a very simple reason. Amavis is a dog, and you might want to remove it completely (depending on your setup/needs/mood). As much as I could see, each amavis instance is taking about 80MB of RAM! Back running sarge, it was a way smaller. I am quite stunned about the fact that, under Lenny, running amavis + clamav + spamassassin is taking about 6/700 MB of RAM when the server is idle. I quite don't understand what happened between Sarge and Lenny (or even, between Etch and Lenny). We used to run these 3 plus apache, mysql and others with just under 200MB of RAM + same amount of swap, on small footprint VMs. Currently, 512MB of RAM + same amount of swap is hardly enough. Also, running amavis is slow, very slow. So that's the one you want to run the last, after all other checks have been performed. To what I could see, dkimproxy performs very well. Others reported that it ran very well under such heavy load as 100k+ email a day (which is the type of traffic we always should keep in mind). > Do you have another example where such a chaining is unavoidable? Sure. clamsmtp for example. That makes 3 software that are used as SMTP proxies, and that could be chained. All of them would need dynamic configuration depending what is installed on the system. And this is only the one that I know/use. There must be more than this in the archive. The current situation in Debian, is that not even the default incoming and relaying ports are set correctly so the components can work together. This is quite a real mess. >> OF COURSE we do care about the performances of a mail server. Some ISP >> are running servers that are managing 100s of thousands of mail a day. > > And of course they use distribution-default configured mail servers for > that :) scnr. Are you trying to say that we shall ship a not performing configuration by default, because big ISP are capable of reconfiguring? If you are, sorry, I don't agree. I think we shall try to release the best distribution as we can, with correct, performing values by default, and even, if possible, have a default configuration that you never even need to edit, because it's just right by default. We should at least have this goal in mind, otherwise it slowly leads to an unusable default system, which is really not wanted here (and which I believe is the current state of many of the mail components in Debian, and that is the reason of my original post). Maybe I'm being too idealistic here. What's your opinion? Thomas -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4bfeec27.1070...@goirand.fr
Re: Let's write a system admin friendly mail server packaging system
Thomas Goirand wrote: > Mario 'BitKoenig' Holbe wrote: >> Why would you like to go another way with mail servers? > Because upstream doesn't want a conf.d folder, unfortunately, and that Well, you can have something equal without upstream support by concatenating conf.d snippets into one huge conf-file, like modutils did (Andreas did describe this for exim already), and today we can also trigger this on package upgrades. > If you setup postfix + amavis, then postfix must relay emails to the > incoming port of amavis, and amavis got to give the email back to > postfix on another port. Both postfix and amavis have to be configured > so they can talk to each other. So far this is independent of third packages which is IMHO fine and desirable. So far, this could be solved by a postfix-conf.d-snippet shipped with the amavis package. > Now, add dkimproxy in the loop. You have to configure dkimproxy to > receive from postfix, relay to amavis, and amavis forwards to postfix. That's pain, indeed, and should IMHO be avoided at all. A clean way to conf this would be * postfix ships to amavis * amavis ships back to postfix * postfix ships to dkimproxy * dkimproxy ships back to postfix I don't know if this is possible with postfix yet. The sendmail milter approach is way cleaner regarding such stuff. > This is a LOT less trivial than what you pretend. That's not just less trivial, it's horrible :) And this is probably one of the reasons why newer amavis is now able to perform DKIM signing on it's own. So, this specific chaining should be historic sooner or later. Do you have another example where such a chaining is unavoidable? > OF COURSE we do care about the performances of a mail server. Some ISP > are running servers that are managing 100s of thousands of mail a day. And of course they use distribution-default configured mail servers for that :) scnr. regards Mario -- () Ascii Ribbon Campaign /\ Support plain text e-mail -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/slrnhvss71.m0q.mario.ho...@darkside.dyn.samba-tng.org
Re: Let's write a system admin friendly mail server packaging system
"Jaldhar H. Vyas" wrote: >Scott Kitterman kitterman.com> writes: > >> >> Additionally, Ubuntu ships a distro specific binary called dovecot-postfix >> that implements part of this >> vision already. We'd love to see it in Debian if the dovecot maintainer is >> interested. >> > >The dovecot maintainer might be interested if someone told him about it. Like >with a wishlist bug to the BTS maybe? > Excellent. It's only been recently it's gotten to the point that I think it might be worth looking at for Debian. Scott K -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/91519a0b-55e1-4c06-b3b0-4f48c6efa...@email.android.com
Re: Let's write a system admin friendly mail server packaging system
Scott Kitterman kitterman.com> writes: > > Additionally, Ubuntu ships a distro specific binary called dovecot-postfix > that implements part of this > vision already. We'd love to see it in Debian if the dovecot maintainer is > interested. > The dovecot maintainer might be interested if someone told him about it. Like with a wishlist bug to the BTS maybe? -- Jaldhar H. Vyas -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/loom.20100527t030124-...@post.gmane.org
Re: Let's write a system admin friendly mail server packaging system
"Roberto C. Sánchez" wrote: >On Wed, May 26, 2010 at 06:42:50PM +0200, Michael Banck wrote: >> On Wed, May 26, 2010 at 11:58:26PM +0800, Thomas Goirand wrote: >> > Mario 'BitKoenig' Holbe wrote: >> > > I'm installing apache2 and have a web server - more or less working, >> > > I'm installing dhelp and ... magic, magic ... it extends the running >> > > web-server to serve the dhelp content as well. I'm installing smb2www >> > > and it extends the running web-server to act as smb client as well. >> > > How do they do this? There is some conf.d directory which contains >> > > config snippets for each of the packages. >> > >> > Yes, which feature I requested from the upstream of postfix. I got a >> > stunning reply that it was a stupid idea, that it would be slow to >> > parse, and that postconf wouldn't work anymore. So forget about having >> > this in postfix, we must find another way. >> >> Eh, Debian can patch upstream software if it thinks it is necessary for >> inter-operation, that's the one of the major points of having a >> distribution. >> >That is true. However, it must be balanced with making the software >different than it is on every other platform. I'm not saying that it >cannot be done. Rather, there needs be a discussion as to whether that >is something that Debian wants to do. It is not as simple as just >patching a high profile package like postfix. > FWIW, dovecot supports config.d too. For postfix, as I understand the design, it would be as non-trivial effort to move to a similar cascading config directory approach. Fortunately it's not needed. In postfix you need to be able to programmatically modify both main.cf and master.cf. I believe that the upstream supported postconf tool supports making all needed changes to main.cf. The Debian package also ships some Debian (and derivative) specific scripts to allow smtp filters and policy services to be added to master.cf by other packages in a policy compliant way. They are simplistic, but should serve this use case (it's the one I wrote them for). Additionally, Ubuntu ships a distro specific binary called dovecot-postfix that implements part of this vision already. We'd love to see it in Debian if the dovecot maintainer is interested. Scott K -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/aa1a35fd-a1de-4e9f-b4ac-a8795379b...@email.android.com
Re: Let's write a system admin friendly mail server packaging system
]] Michael Banck | On Wed, May 26, 2010 at 11:58:26PM +0800, Thomas Goirand wrote: | > Mario 'BitKoenig' Holbe wrote: | > > I'm installing apache2 and have a web server - more or less working, | > > I'm installing dhelp and ... magic, magic ... it extends the running | > > web-server to serve the dhelp content as well. I'm installing smb2www | > > and it extends the running web-server to act as smb client as well. | > > How do they do this? There is some conf.d directory which contains | > > config snippets for each of the packages. | > | > Yes, which feature I requested from the upstream of postfix. I got a | > stunning reply that it was a stupid idea, that it would be slow to | > parse, and that postconf wouldn't work anymore. So forget about having | > this in postfix, we must find another way. | | Eh, Debian can patch upstream software if it thinks it is necessary for | inter-operation, that's the one of the major points of having a | distribution. Wouldn't it be easier to generate a configuration directory in /var from snippets in /etc/postfix, if that was what you desired? The init script could do that easily enough. -- Tollef Fog Heen UNIX is user friendly, it's just picky about who its friends are -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87fx1el4xf@qurzaw.linpro.no
Re: Let's write a system admin friendly mail server packaging system
On 05/26/2010 02:29 PM, Thomas Goirand wrote: [snip] Anyway, postfix is NOT the only package that we shall consider modifying here. As per my original post, there's loads of other components that are to configure as well. The question is: is there a will to do this job by other maintainers. I am myself strongly motivated for this, but I wont be able to do it alone. As was mentioned earlier, this can't be a Debian-only task. It's just too big and complicated. Upstream of all the relevant packages must buy in. -- Dissent is patriotic, remember? -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4bfd86c6.7080...@cox.net
Re: Let's write a system admin friendly mail server packaging system
Ron Johnson wrote: > On 05/26/2010 11:42 AM, Michael Banck wrote: >> On Wed, May 26, 2010 at 11:58:26PM +0800, Thomas Goirand wrote: >>> Mario 'BitKoenig' Holbe wrote: I'm installing apache2 and have a web server - more or less working, I'm installing dhelp and ... magic, magic ... it extends the running web-server to serve the dhelp content as well. I'm installing smb2www and it extends the running web-server to act as smb client as well. How do they do this? There is some conf.d directory which contains config snippets for each of the packages. >>> >>> Yes, which feature I requested from the upstream of postfix. I got a >>> stunning reply that it was a stupid idea, that it would be slow to >>> parse, and that postconf wouldn't work anymore. So forget about having >>> this in postfix, we must find another way. >> >> Eh, Debian can patch upstream software if it thinks it is necessary for >> inter-operation, that's the one of the major points of having a >> distribution. >> > > That would be some *serious* patching. > > Maybe, though, LaMont Jones (the Postfix DD) has a better relationship > with upstream and could convince them that conf.d is a good idea. I have to admit that I didn't insist so much, given the type of response I had when I asked. I also agree that it would be much better if upstream were happily accepting such patch, even if one of us is doing the job. I didn't really dive into the postfix code to know how hard it would be to add the feature, and I hope that LaMont Jones can talk about it. Anyway, postfix is NOT the only package that we shall consider modifying here. As per my original post, there's loads of other components that are to configure as well. The question is: is there a will to do this job by other maintainers. I am myself strongly motivated for this, but I wont be able to do it alone. Thomas -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4bfd7686.4030...@goirand.fr
Re: Let's write a system admin friendly mail server packaging system
Thomas: Sorry for the private mail, I hit the wrong reply button. On Wed, May 26, 2010 at 11:58:26PM +0800, Thomas Goirand wrote: > Mario 'BitKoenig' Holbe wrote: > > I'm installing apache2 and have a web server - more or less working, > > I'm installing dhelp and ... magic, magic ... it extends the running > > web-server to serve the dhelp content as well. I'm installing smb2www > > and it extends the running web-server to act as smb client as well. > > How do they do this? There is some conf.d directory which contains > > config snippets for each of the packages. > > Yes, which feature I requested from the upstream of postfix. I got a > stunning reply that it was a stupid idea, that it would be slow to > parse, and that postconf wouldn't work anymore. So forget about having > this in postfix, we must find another way. The packaging of Exim optionally supports a setup in which the config file is split into several conf.d-like directories. Whenever Exim is restarted all files in those directories are concatenated into a single file somewhere under /var which is then fed to the Exim daemon. Couldn't the postfix package be modified to do something similiar? Andreas signature.asc Description: Digital signature
Re: Let's write a system admin friendly mail server packaging system
On 05/26/2010 11:42 AM, Michael Banck wrote: On Wed, May 26, 2010 at 11:58:26PM +0800, Thomas Goirand wrote: Mario 'BitKoenig' Holbe wrote: I'm installing apache2 and have a web server - more or less working, I'm installing dhelp and ... magic, magic ... it extends the running web-server to serve the dhelp content as well. I'm installing smb2www and it extends the running web-server to act as smb client as well. How do they do this? There is some conf.d directory which contains config snippets for each of the packages. Yes, which feature I requested from the upstream of postfix. I got a stunning reply that it was a stupid idea, that it would be slow to parse, and that postconf wouldn't work anymore. So forget about having this in postfix, we must find another way. Eh, Debian can patch upstream software if it thinks it is necessary for inter-operation, that's the one of the major points of having a distribution. That would be some *serious* patching. Maybe, though, LaMont Jones (the Postfix DD) has a better relationship with upstream and could convince them that conf.d is a good idea. -- Dissent is patriotic, remember? -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4bfd5ff6.8020...@cox.net
Re: Let's write a system admin friendly mail server packaging system
On Wed, May 26, 2010 at 06:42:50PM +0200, Michael Banck wrote: > On Wed, May 26, 2010 at 11:58:26PM +0800, Thomas Goirand wrote: > > Mario 'BitKoenig' Holbe wrote: > > > I'm installing apache2 and have a web server - more or less working, > > > I'm installing dhelp and ... magic, magic ... it extends the running > > > web-server to serve the dhelp content as well. I'm installing smb2www > > > and it extends the running web-server to act as smb client as well. > > > How do they do this? There is some conf.d directory which contains > > > config snippets for each of the packages. > > > > Yes, which feature I requested from the upstream of postfix. I got a > > stunning reply that it was a stupid idea, that it would be slow to > > parse, and that postconf wouldn't work anymore. So forget about having > > this in postfix, we must find another way. > > Eh, Debian can patch upstream software if it thinks it is necessary for > inter-operation, that's the one of the major points of having a > distribution. > That is true. However, it must be balanced with making the software different than it is on every other platform. I'm not saying that it cannot be done. Rather, there needs be a discussion as to whether that is something that Debian wants to do. It is not as simple as just patching a high profile package like postfix. Regards, -Roberto -- Roberto C. Sánchez http://people.connexer.com/~roberto http://www.connexer.com signature.asc Description: Digital signature
Re: Let's write a system admin friendly mail server packaging system
On Wed, May 26, 2010 at 11:58:26PM +0800, Thomas Goirand wrote: > Mario 'BitKoenig' Holbe wrote: > > I'm installing apache2 and have a web server - more or less working, > > I'm installing dhelp and ... magic, magic ... it extends the running > > web-server to serve the dhelp content as well. I'm installing smb2www > > and it extends the running web-server to act as smb client as well. > > How do they do this? There is some conf.d directory which contains > > config snippets for each of the packages. > > Yes, which feature I requested from the upstream of postfix. I got a > stunning reply that it was a stupid idea, that it would be slow to > parse, and that postconf wouldn't work anymore. So forget about having > this in postfix, we must find another way. Eh, Debian can patch upstream software if it thinks it is necessary for inter-operation, that's the one of the major points of having a distribution. Michael -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100526164250.gf2...@nighthawk.chemicalconnection.dyndns.org
Re: Let's write a system admin friendly mail server packaging system
Mario 'BitKoenig' Holbe wrote: > I'm installing apache2 and have a web server - more or less working, > I'm installing dhelp and ... magic, magic ... it extends the running > web-server to serve the dhelp content as well. I'm installing smb2www > and it extends the running web-server to act as smb client as well. > How do they do this? There is some conf.d directory which contains > config snippets for each of the packages. Yes, which feature I requested from the upstream of postfix. I got a stunning reply that it was a stupid idea, that it would be slow to parse, and that postconf wouldn't work anymore. So forget about having this in postfix, we must find another way. > Why would you like to go another way with mail servers? Because upstream doesn't want a conf.d folder, unfortunately, and that the issue is NOT ONLY with postfix, but with so many package that have to understand each other. Let me give an example. If you setup postfix + amavis, then postfix must relay emails to the incoming port of amavis, and amavis got to give the email back to postfix on another port. Both postfix and amavis have to be configured so they can talk to each other. Now, add dkimproxy in the loop. You have to configure dkimproxy to receive from postfix, relay to amavis, and amavis forwards to postfix. This is a LOT less trivial than what you pretend. > Get maintainer support for such conf.d directories, maybe get upstream > support for such conf.d mimics (sendmail most likely won't need it - > some m4 magic and triggers will just do it, dunno how it is for the less > flexible ones like postfix, exim, whatever), change the depending > packages to put their config snippets in there, everything is fine. > >> argument a list of packages that it should use. But the MTA is not the >> only one to modify here, for example we might have to change the listen >> and relay port of dkimproxy and amavis, depending if each others are >> present on the system or not. I am quite in the favor of this system, > > I don't think this is really necessary. It would probably be a bit more > efficient to have one component forwarding the stuff it processed to the > next one, but I'm sure there is a less efficient but more generic way to > just bounce it back to the MTA which continues forwarding it to the next > components. ... And who cares about efficiency in generic approaches. OF COURSE we do care about the performances of a mail server. Some ISP are running servers that are managing 100s of thousands of mail a day. But anyway, this was just an example, there's many use case where you must configure one of the elements of your mail server. Thomas -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4bfd4522.4080...@goirand.fr
Re: Let's write a system admin friendly mail server packaging system
On Wednesday 26 May 2010 15:58:51 Mario 'BitKoenig' Holbe wrote: > Why would you like to go another way with mail servers? > Get maintainer support for such conf.d directories, maybe get upstream > support for such conf.d mimics (sendmail most likely won't need it - > some m4 magic and triggers will just do it, dunno how it is for the less > flexible ones like postfix, exim, whatever), change the depending > packages to put their config snippets in there, everything is fine. Amavis already has conf.d it's a start :) Bye -- Salvo Tomaselli signature.asc Description: This is a digitally signed message part.
Re: Let's write a system admin friendly mail server packaging system
Thomas Goirand wrote: > What happens here is that, if you take a normal Debian system, then > install postfix, then let's say amavis, they don't talk to each other. ... > much spams. It is also totally unrealistic to say that it's up to the > system administrator to configure everything by hand. If, like me, you > do at least one setup a day, it takes too much time for no reason, and > it has to be automated in some way. There are lots of examples out there where already works what you are requesting for mail servers. Let's have a look at web servers (well, apache... okay) and how they deal with it. I'm installing apache2 and have a web server - more or less working, I'm installing dhelp and ... magic, magic ... it extends the running web-server to serve the dhelp content as well. I'm installing smb2www and it extends the running web-server to act as smb client as well. How do they do this? There is some conf.d directory which contains config snippets for each of the packages. Let's have a look at DHCP and how they deal with it. I'm installing dhcp3-client and my machine's network settings are configured automagically. I'm installing resolvconf and my name servers become configured automagically via DHCP. I'm installing samba and it's also getting configured automagically via DHCP. I'm installing ntp and my ntp-server is configured automagically via DHCP. How do they do this? There is some conf.d directory which contains config snippets for each of the packages. Let's have a look at SysV boot mimics and how they deal with it. I'm installing the sysvinit packages ... well, you can imagine the rest: ... There is some conf.d directory which contains config snippets for each of the packages. Why would you like to go another way with mail servers? Get maintainer support for such conf.d directories, maybe get upstream support for such conf.d mimics (sendmail most likely won't need it - some m4 magic and triggers will just do it, dunno how it is for the less flexible ones like postfix, exim, whatever), change the depending packages to put their config snippets in there, everything is fine. > argument a list of packages that it should use. But the MTA is not the > only one to modify here, for example we might have to change the listen > and relay port of dkimproxy and amavis, depending if each others are > present on the system or not. I am quite in the favor of this system, I don't think this is really necessary. It would probably be a bit more efficient to have one component forwarding the stuff it processed to the next one, but I'm sure there is a less efficient but more generic way to just bounce it back to the MTA which continues forwarding it to the next components. ... And who cares about efficiency in generic approaches. regards Mario -- who | grep -i blonde | talk; cd; wine; talk; touch; unzip; touch; strip; gasp; finger; gasp; mount; fsck; more; true; gasp; umount; make clean; sleep -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/slrnhvqa8s.m0q.mario.ho...@darkside.dyn.samba-tng.org
Let's write a system admin friendly mail server packaging system
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Dear everyone, 1/ Briefly, who am I My first Debian package was for the web hosting control panel (a web interface) that my company released in open source. I'm the main programmer of it. The first time I tried to have it enter in Debian, it created a huge controversy, with (I heard) a 70 post thread in -private after it got sponsored. The reason was that my package goal is to have an over-simplified system, so that the user of it doesn't have to touch anything to the system configuration, everything has to be automated (which is the goal of my app). In Debian, by policy, a package cannot touch another package's configuration file. While I believe this policy is a good one, but it prevented me from having my postinst to do a successful setup without breaking the policy. The result is that what should have been sent to the postinst of my package has then been sent to a userland script (with often, users not starting the script an complaining about it in my forum). It doesn't make this script less ugly if running in userland rather than in the maintainer scripts (it is REALLY an ugly script, and I'm quite not proud of it), but at least it respects the policy. As I am soon to become a Debian Developer (if the DAM accepts me, after my AM wrote his report), I believe it is now time to get even more involved in Debian, and try to solve that issue once and for all. Even if for a reason or another, I'm rejected (which I don't think will happen), I still want to start the below discussion. 2/ The problematic == What happens here is that, if you take a normal Debian system, then install postfix, then let's say amavis, they don't talk to each other. In the same way, if you add dkimproxy (that I maintain), or clamsmtp, or tumgreyspf (that I maintain as well), you end up with a system that is not configured at all. None of these mail server components are aware of each other, and a system administrator has to spend a great deal of time to make it work. Truth is, in today's world, it is totally unrealistic to believe that just postfix is enough for setting up a mail system. There's just too much spams. It is also totally unrealistic to say that it's up to the system administrator to configure everything by hand. If, like me, you do at least one setup a day, it takes too much time for no reason, and it has to be automated in some way. There's loads of howtos available in many places that describe in 10 pages or more how to have a successful setup. This is really a pain. This is the reason why I'm writing this today: I want to (with the help of other maintainer of the concerned packages, if they agree) change that fact. 3/ Goal description === In the ideal world, a command like this: apt-get install postfix-mysql clamav clamav-daemon clamav-freshclam spamassassin tumgreyspf would create a mail toaster with postfix and all the above apps configured correctly so that the mail system would do like this: 1- postfix gets a mail, does some basic domain checkings (domain MX existance, etc.) 2- postfix asks tumgreyspf to check for SPF and greylisting 3- (see later) 4- postfix forwards the email to amavis 5- amavis does clamav and spamassassin checks with header tagging 6- amavis forwards the email to postfix 7- postfix sends the email to maildrop for delivery Let's say now that I add dkimproxy, I would do: apt-get install dkimproxy and then the sequence would become: 3- postfix sends the email to dkimproxy for DKIM signature checkings 4- dkimproxy forwards the email to amavis I don't see any reason why it shouldn't be as easy to use as what I wrote above. The complexity of this kind of setup MUST be done on our side, and not rely on the system administrator knowledge. The above is what we currently use, but of course, this could be extended to DSPAM (I read it's better than spamassassin), clamsmtp, some milter checks, some alternative MDA, etc. And of course, this could be extended as well to other mail servers (exim4 anyone?). That's for the problematic. Now, how to achieve this, I'm not sure how to do it yet, but I have couples of ideas. 4/ Few ideas, and what I believe should happen == First thing we could do, would be a special postfix package that would have the above packages as dependency. Let's say we call it postfix-toaster, and it would have the configuration already made so that it would be already configured for using other packages. But that's not really idealistic, because of so many possibilities that we have. The second idea would be to have some kind of triggers, a bit like we have for generating the mandb and others. The trigger would ask the MTA scripts to do the reconfiguration process, for example, giving it as argument a list of packages that it should use. But the MTA is not the only one to modify here, for example we might have to change the lis