Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On Sun, Aug 30, 2020 at 06:00:11PM +0100, Wols Lists wrote: > (I've never seen an smtp:/ or imap:/ url, but I guess they could easily > be formulated.) They are sometimes used, usually in addition to a port specifier, in order to express a protocol univocally. For example, here's part of my Mutt configuration file (the first line forces STARTTLS, hence the omission of `s` in `smtp://`): set ssl_force_tls = yes set smtp_url = "smtp://a...@mail.suugaku.co.uk:587" set folder = "imaps://mail.suugaku.co.uk:993/" -- Ashley Dixon suugaku.co.uk 2A9A 4117 DA96 D18A 8A7B B0D2 A30E BF25 F290 A8AA signature.asc Description: PGP signature
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
‐‐‐ Original Message ‐‐‐ On Friday, August 28, 2020 11:27 PM, antlists wrote: > On 26/08/2020 21:21, Grant Taylor wrote: > > > > so basically total expected number of protocols/layers used in the > > > universe, per second, will be much less if we, on planet earth, use a > > > mail system that uses HTTP* instead of RESXCH_*. > > > > I obviously disagree. > > Exactly. You now need a protocol/layer that says you're running "mail > over http" as opposed to "web". HTTP is tcp/80 that means web. As soon > as you start using it for something (anything) else you've just added > another protocol/layer. you know there is this almost neat concept called url? rumours say that urls can identify various web applications, ranging from websites, rss, games, video, and, guess what? mails. all over http/https/h2 over same tcp 80/443. hard to believe, but this magic is known since early 1990s. are you saying [1] won't work unless we have a new tcp port for it? [1] https://github.com/al-caveman/hillarymail (work in progress, incomplete) i don't want to repeat. re-read this sub-tread, and search for "resource exchange layer". you really don't know what's http*. also not going to respond to you in this sub-thread any more (ignore list is growing...). side note: i seriously suspect that we got GPT-4 bots in the list.
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 30/08/20 14:06, Caveman Al Toraboran wrote: > you know there is this almost neat concept called > url? > > rumours say that urls can identify various web > applications, ranging from websites, rss, games, > video, and, guess what? mails. all over > http/https/h2 over same tcp 80/443. hard to > believe, but this magic is known since early > 1990s. Have you looked at urls lately? http:/example.org/blah/blah https:/example.org/foo/bar git:/github.com/my/wonderful/project smb:/mypc/username/documents The first bit of the url is the protocol - guess what - it explicitly chooses between http / hhtps / git / whatever. It runs above tcp/udp, not tcp80/443. (I've never seen an smtp:/ or imap:/ url, but I guess they could easily be formulated.) Cheers, Wol
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 2020-08-28 20:29, Grant Taylor wrote: > On 8/28/20 6:10 PM, Michael Orlitzky wrote: >> I think I see where we're diverging: I'm assuming that the employees of >> the VPS provider can hop onto any running system with root privileges. > > Perhaps I'm woefully ignorant, but my current working understanding > is that ...They still need to connect to a terminal (be it console or > serial or ssh or other), log in (with credentials that they should > not have) and access things that way. > > I'm actually not encrypting the full VM. I have an encrypted disk. The > VM boots like normal, I log in, unlock the encrypted disk, mount it, and > start services. If /etc/passwd, /etc/shadow, and friends aren't encrypted, they can get in pretty easily without credentials. The VPS admins have physical access to the disk -- they could swap out your root password for theirs temporarily, or create a secondary privileged account. And keep in mind that your shell and all of the executables used to decrypt/mount the disk are themselves unencrypted and can be replaced by malware. A lazy attack would be to reboot in single-user mode (bypasses the root password) and then replace your utilities with keyloggers before rebooting again. This might look suspicious to you, but would you really avoid logging into the system ever again because it rebooted once?
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 8/28/20 3:54 PM, Poison BL. wrote: On Mon, Aug 17, 2020 at 12:51 AM Caveman Al Toraboran wrote: hi. context: 1. tinfoil hat is on. 2. i feel disrespected when someone does things to my stuff without getting my approval. 3. vps admin is not trusty and their sys admin may read my emails, and laugh at me! 4. whole thing is not worth much money. so not welling to pay more than the price of a cheap vps. moving to dedicated hardware for me is not worth it. my goal is to make it annoying enough that cheap-vps's admins find it a bad idea for them to allocate their time to mingle with my stuff. thoughts on how to maximally satisfy these requirements? rgrds, cm. I'm rather late to the game with this, but at the end of the day, mail coming *into* a mail server isn't typically encrypted (and even that is only the body, the headers can still reveal a great deal, and are necessary for the server to work with it). A packet dump at the switch will turn over every piece of mail you receive along the way. Email's not designed for end to end security by default. Secondly, any hosting on hardware you don't control is impossible to fully secure, if the services on that end have to operate on the data at all. You can encrypt the drive, encrypt the mail stores themselves, etc, but all of those things will result in the encryption key being loaded into ram while the VPS is running, and dumping ram from the hypervisor layer destroys every illusion of security you had. Dedicated hardware in a locked cabinet is as close as you get to preventing physical attacks when you're hosting in someone else's DC, and that's not nearly in the same market segment, price-wise, as a cheap VPS. At best, if you have sensitive email that you're sending or receiving, work with the other end of the communication and then encrypt the contents properly. Even better, go with a larger scale, paid, solution in which your email isn't even remotely worth the effort to tamper with for the hosting company's employees, and hope the contractual obligations are sufficient to protect you. If you have any sort of controlled data going in and out of your email, step up to a plan that adheres to the regulatory frameworks you're required to adhere to and make very sure the contracts for it obligate the vendor to secure things properly on their end (aws, azure/o365/etc mostly all have offerings for, at least, US Gov level requirements). Hm. How about paying for codes the US F. Feds do not have, like Real Random. Supposedly, they are legally pissing of the F. Feds. Do your own evaluation. A US corp in good standing the F. Feds do not want anyone to know. About. Why? For the F. Feds to challenge what they do, they have to PUBLICLY disclose their p. p. https://www.realrandom.co/wp/ yes it's commercial. But for Gentoo, I'd push for a deep discount. They have totally awesome technology, and I know a sales guy there. Any solution, should have open source codes, and options for non-publish commercial codes. Are there back doors? Dunno. Ask. Make your own decision. But rumors are the F. Feds are pissed at these guys, cause they have real technology solutions right now. Not bullshit-AI jibberish. Sure, by executive order Trump could single action them out of existence, but rumor has it, he has already decided NO, on that pathway. My postulate is US Citizens, in good legal standing, with NO felony convictions, have superior rights to privacy, than the F. Feds. It's constitutionally bake in by our for fathers. We just need to stand up and demand this. F. these scumbag lawyers, judges and corrupt (sold out) politicians. The rest of the work is on their own. But, if we organize and stand up, we can put this 'demon' back into the darkness (abyss). I have no fear of the F. Feds. Others would be wise to self examine, before joining up with such an effort. James Horton, pe
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 8/28/20 6:10 PM, Michael Orlitzky wrote: I think I see where we're diverging: I'm assuming that the employees of the VPS provider can hop onto any running system with root privileges. Perhaps I'm woefully ignorant, but my current working understanding is that no virtual machine hypervisor solution provides a way for someone at the hypervisor level to access a guest VM as if they were root. They still need to connect to a terminal (be it console or serial or ssh or other), log in (with credentials that they should not have) and access things that way. I see little difference in the full (fat) VM compared to a stand alone server. Safe for the fact that there are ways to cross access memory. Though I think those types of things are decidedly atypical. My mental security model probably completely fails for containers. I suppose you can make that pretty annoying to do. If you're willing to encrypt everything, then you can even put /boot on the encrypted disk, unlocking it in (say) grub. The VPS provider can still replace grub with something that faxes them your password, but it's not totally trivial. (How are you accessing the console at boot time? Is it using software from the VPS provider? It's turtles all the way to hell.) I'm actually not encrypting the full VM. I have an encrypted disk. The VM boots like normal, I log in, unlock the encrypted disk, mount it, and start services. So, I feel like I've done the things that I reasonably can do to protect my email. Or said another way, I'm not sure what else I could do that would not also apply to a co-lo server. My VPS provider does offer the ability to access a console so I could use full encrypted system. -- Grant. . . . unix || die
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 8/28/20 5:00 PM, Grant Taylor wrote: On 8/28/20 1:18 PM, antlists wrote: The main reason other applications use "TCP over HTTP(S)" is because stupid network operators block everything else! I agree that filtering is a problem. I also think that it's something that most people can overcome when they control the firewall between the private LAN and the Internet.� (Your typical SOHO NATing gateway.) The few times that I have run into filtering, it has been for uninitiated inbound connections.� I've almost always been able to initiate outbound connections to / from odd ports.� The few times that I could not do so in the last 20 years were resolved by engaging the ISP and ... politely ... getting them to knock it off.� Inbound can be more tricky.� But even inbound HTTP(S) was subject to the same problems. Actually, inbound HTTP(S) was more of a problem than other ports. The more I hear, the more the FEDS and judges need to get involve. From what I read, it's and education problem. This is commonly referred to as 'racketeering' and is extremely illegal, meaning the FEDS fine them super high amounts of money, and judges rubber stamp the fines. You want to play and compete in the US? A fair economic playground is constitutionally required. There is so much case law, that it is ridiculous. What case law does is establish precedence. And that precedence of relevant case builds a HUGH argument that most judges will not ignore. Combine that with the fact that the general public, will side with us? SlameDunk, in legal parlance. Anyone with access to legal precedence setting case law, can research this out. I had no idea what a pile of Horse S. this has become. It is totally illegal. Even if I were to loose, on Gentoo's behalf, a legal battle, going public will destroy their pierce strings. All of this should be codified in RFCs, or labeled as optional. WE do need to get organized, before seeking legal moguls to assist this public effort. The more I dig, the more I realize it is way past time to fix this illegal activity. My guess is a well documented and organized effort is the first step. Then a few mavericks educating politicians and filing briefs with the court systems will get the ball rolling. I kicked Verizon's Ass, back when they were call GTE, so it's routine for me to kick some big corps ass. If we get, just 0.1 % of their subscribers onboard, it's a done deal. Cell phone criminal activity by the big telco operators is 'part and parcel' to this HS too. My question is have a few technical devs had enough? There are plenty of tech-savie High School kids that need jobs. Think of it as a jobs program. Started here in the US, but easily expandable to most countries. https://foreignpolicy.com/2020/08/27/china-tech-facebook-google/ James
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 8/28/20 4:45 PM, james wrote: If we can get these codes running on arm64 (R.P.4) surely running them on AMD or intel is trivial? I will be flabbergasted if something would run on the Raspberry Pi that won't run on x86 (Intel / AMD). Presuming that it's complied from common source code. Perhaps a read on "Intel cripple AMD functions is in order? https://www.agner.org/forum/viewtopic.php?f=1=6 I don't believe this is germane to the primary topic of this thread. (2) identical R.Pi.4 8gig rams systems, running gentoo. Okay. (1) dns resolver codes emails service codes etc (1) dns resolver codes, webserver to support email services etc. So each Raspberry Pi is performing a different function. Okay. I was wanting to make sure that you weren't wanting to try to do some sort of clustering where each Raspberry Pi could stand in for the other. As that's a considerably more complex configuration. I'm open to the stack (list) of codes necessary to securely run 1. embedded gentoo on R.P.4 (other hardware can be funded by others). 2. Any number of robust email servers-systems (open) I've recently shared what I have used for email. 3. a DNS servers to provide "primary dns services" a total of (2). More than 2 would be great. Please elaborate on what you are proposing network connectivity to be? Are you thinking the Pi's have globally routed IPs? As such, primary DNS could be 192.0.2.1 and secondary DNS could be at 192.0.2.2? Note: It is best practice to have primary and secondary DNS servers in different /24 (or larger) networks. If you are thinking two globally routed IPs, I believe that significantly, if not artificially, narrows the number of people that could participate as getting multiple IPs on a SOHO Internet connection can be challenging and almost always requires additional monthly fees. Conversely, a single IP with proper network magic is much simpler entry point. 4. A companion ngnix(?) web server just to complement the project. The ideas is each email services collective could have their own web pages explaining their email and related services. Okay. You can run the web server on the same system. But if you want to run it on a separate system, that's fine too. I'm somewhat confused by your choice of the word "collective". My anticipation is that many of the people that would be doing this, would be doing so for their own person reasons. Much like I have my domain name for my own reasons. I don't anticipate that people will be offering services to more than a few friends and / or family members (if that). 5. On these (3) projects, I'd be open to other, complementary experimentation, as long as it is published. Grant Taylor, do not let it go to your head, but I agree with most of what you write in Gentoo User. Me? I'm just an idiot on the Internet with some things to say. Sometimes they happen to be true. Ideally, you know (or learn) enough to tell which is which. ;-) But, thank you. :-) 6. (2) Rpi4 (8 gig) systems and extras are 2-3 hundred dollars. So it's total less than $900 USD dollars. NOT a bid deal for my little corp. Actually, if I get what I need, then it's the most inexpensive && robust way for my little corp to get exactly what I need. My own small email servers and dns resolvers supporting those email services. Based on some back of the envelope math Sure. I'm not funding somebody else's idea. I'm funding what *I* want, open to input. That seems reasonable. Though, I think that some of your requirements are still a bit too undefined. Even independent of what software is used and how it's configured, there are still questions: - Are IP addresses globally routed or not? - Are said IP addresses static or dynamic? - What sort of client's will be accessing this? - Where will they be accessing from; LAN and / or Internet? With this effort others benefit from the project. The ultimate goals is for hundreds of email services to be setup, gentoo centric. OK, great. FUND what you want. Run things as you see fit I have been. My intention is to see if there is a way that I can contribute to your community project without consuming any funds so that other people might be able to benefit from your generosity. Show me a concise, easy to follow set of codes and docs, and I'll just build (2) R.P.4 servers and share my docs 100%. There is more to setting up and running an email server off of a SOHO internet connection than just how the email stack is configured. Forget the fact, for now, that all static IPs Frontier has, are blocked by this same group of higher and higher standards. Really, I'm kinda shocked NeddySeagoon, or others have not already fixed this, via 100% gentoo codes, complete with ample documentation. That's an example of the type of problem that will need to be overcome which is independent of the email server stack. Just add the email, dns, ngnix, security
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 2020-08-28 19:43, Grant Taylor wrote: > > The only way to get the key is to extract it out of the running VPS's > memory. Something that I think is beyond the capability of many, but > definitely not all, people. > > ... > > As long as STARTTLS is used (and validated) between the MTAs and the VPS > provider doesn't have a way to get the keys (because they are on an > encrypted disk), then the contents of the transmission should be fairly > secure. I think I see where we're diverging: I'm assuming that the employees of the VPS provider can hop onto any running system with root privileges. I suppose you can make that pretty annoying to do. If you're willing to encrypt everything, then you can even put /boot on the encrypted disk, unlocking it in (say) grub. The VPS provider can still replace grub with something that faxes them your password, but it's not totally trivial. (How are you accessing the console at boot time? Is it using software from the VPS provider? It's turtles all the way to hell.)
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 8/28/20 4:26 PM, Michael Orlitzky wrote:> The contents of the disk are unencrypted while the server is powered on, or at least while the server is receiving email (while it's reading from and writing to that disk). In practice that will be all the time -- you can't log in and type the disk-encryption password every time an email arrives. You don't need to enter a password every time that an email comes in. I have a VPS with an encrypted file system. I enter the password at the time that it boots. The disk and file system(s) therein are encrypted all the time. So a clone of the disk will require the passphrase to unlock the key. The only way to get the key is to extract it out of the running VPS's memory. Something that I think is beyond the capability of many, but definitely not all, people. A clone of the VPS will effectively present the same security posture as the running system. I've been running like this for five (or more) years without any problems. I think it works great. I shouldn't have used the word "secret." Pre-established or out-of-band authentication would have been more accurate. Okay. Poor choice of words happen. Unfortunately I mistook your statement to be referencing symmetric encryption with a shared secret. With GPG, the trust is between you and I, and the VPS provider acts as the eavesdropper. All three parties are distinct, and the security can work. With TLS between MTAs, the trust is established on-the-fly between the other MTA and the VPS provider, but the VPS provider still also plays the the role of the eavesdropper. When the eavesdropper is trusted, you're in trouble. As long as STARTTLS is used (and validated) between the MTAs and the VPS provider doesn't have a way to get the keys (because they are on an encrypted disk), then the contents of the transmission should be fairly secure. -- Grant. . . . unix || die
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 8/28/20 4:56 PM, Grant Taylor wrote: On 8/28/20 1:55 PM, james wrote: I'm proposing, via a small corp I own, to purchase up to (3) dual Rasp.pi 4 setups of (2) R.Pi.4 8gig ram setups and send them to the devs WE all decide on. A few points. 1)� I don't think that 8 GB of RAM is required.� --� My email server is a VPS with 2 GB of RAM and is running just fine.� So, maybe smaller systems would work.� And maybe that would mean that more of them could be acquired for the same funding. OK, but I like the R.P.8gram quite a lots. so my money is with prototyping on via (3) innovators each with (2) R. P. 4 8g ram. Others can use what they want. Surely others can propose and use other embedded (low2 power) boards. 2)� I don't know that a Raspberry Pi is strictly required for the testing.� I think that anything that will run Gentoo can be used to prove out the software stack.� --� Sure, there will /eventually/ need to be /some/ testing on Raspberry Pis.� But I think that testing will be later in the game and more of a confirmation after the fact. Great idea. Fund that pathway yourself. I LOVE R.PI.4 boards, with 8gig of ram. ymmv. If we can get these codes running on arm64 (R.P.4) surely running them on AMD or intel is trivial? Perhaps a read on "Intel cripple AMD functions is in order? https://www.agner.org/forum/viewtopic.php?f=1=6 3)� I'm not sure what you mean by "dual ... setups". (2) identical R.Pi.4 8gig rams systems, running gentoo. (1) dns resolver codes emails service codes etc (1) dns resolver codes, webserver to support email services etc. What are the two systems (be it Raspberry Pis or VPSs or VMs or something else) supposed to do?� -� Are you wanting primary and backup (as in MX) or some sort of cluster with shared file system or something else? Well establish, albeit, in long postings on this gentoo-user list. It's just (2) R.P.4 systems, running gentoo. I'm open to the stack (list) of codes necessary to securely run 1. embedded gentoo on R.P.4 (other hardware can be funded by others). 2. Any number of robust email servers-systems (open) 3. a DNS servers to provide "primary dns services" a total of (2). More than 2 would be great. 4. A companion ngnix(?) web server just to complement the project. The ideas is each email services collective could have their own web pages explaining their email and related services. 5. On these (3) projects, I'd be open to other, complementary experimentation, as long as it is published. Grant Taylor, do not let it go to your head, but I agree with most of what you write in Gentoo User. 6. (2) Rpi4 (8 gig) systems and extras are 2-3 hundred dollars. So it's total less than $900 USD dollars. NOT a bid deal for my little corp. Actually, if I get what I need, then it's the most inexpensive && robust way for my little corp to get exactly what I need. My own small email servers and dns resolvers supporting those email services. Let's us start compiling up the codes, keep it simple (for now) and implement them with gentoo-users as the testers of the email services. These discussions should be continued to everyone's benefit. However there are way more than (3) folks on these threads who are most capable to do this community prototyping. I think the idea of using VPSs or VMs means that a lot more people can participate using the same funding. I'm not funding somebody else's idea. I'm funding what *I* want, open to input. With this effort others benefit from the project. The ultimate goals is for hundreds of email services to be setup, gentoo centric. If WE do not act and get hundreds of these deployed, email, as we know it via RFCS and other standards may just disappear, or be relegated to the far reaches of the Internet.� What I have read, is standards based email services, particularly by small organizations, are under extreme pressure by large corporations to be marginalized out of existence. I think I disagree with that. OK, great. FUND what you want. Run things as you see fit Many of the big email operators are enforcing higher and higher standards.� But the standards /are/ /open/ and /can/ /be/ /implemented/ /by/ /anyone/ who wants to do so. Show me a concise, easy to follow set of codes and docs, and I'll just build (2) R.P.4 servers and share my docs 100%. Forget the fact, for now, that all static IPs Frontier has, are blocked by this same group of higher and higher standards. Really, I'm kinda shocked NeddySeagoon, or others have not already fixed this, via 100% gentoo codes, complete with ample documentation. https://wiki.gentoo.org/wiki/Raspberry_Pi4_64_Bit_Install Just add the email, dns, ngnix, security setup codes to this doc? I have been researching and reading, for over (3) weeks and have yet been able to formulate a pathway to get a mail server up. Granted the industry black-balling Frontier, is a bit of a shocker
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 2020-08-28 17:53, Grant Taylor wrote: > On 8/28/20 3:33 PM, Michael Orlitzky wrote: >> TLS only secures the channel; what comes out at the end is a plain-text >> message that can be read with minimal effort by the VPS provider, >> no skullduggery needed. > > I agree that STARTTLS only protects the email while it's in flight > between servers. > > Though I do think that it's going to somewhat difficult for a VPS > provider to read the contents of the message if it's stored on an > encrypted disk. The contents of the disk are unencrypted while the server is powered on, or at least while the server is receiving email (while it's reading from and writing to that disk). In practice that will be all the time -- you can't log in and type the disk-encryption password every time an email arrives. >> Unless the sender and recipient have some pre-shared secret (like GPG >> assumes), > > I *REALLY* thought that PGP (GPG) was based on public & private key > pairs, much like S/MIME and TLS. > > As such, Alice and Bob can encrypt messages to each other, even through > an untrusted medium such as a questionable email server. > > Yes, that still leaves the bootstraping issue of how do Alice and Bob > get each other's public key. -- I defer to my recent comments about > publishing keys in DNS and relying on DNSSEC. > GPG is based on public keys, but you've anticipated my response: public-key encryption still requires you to verify that "my" public key does in fact belong to *me* somehow. If you believe in the web of trust, then someone you know (or someone someone you know knows...) has to have met me in person and signed my key before it means anything to you. I shouldn't have used the word "secret." Pre-established or out-of-band authentication would have been more accurate. With GPG, the trust is between you and I, and the VPS provider acts as the eavesdropper. All three parties are distinct, and the security can work. With TLS between MTAs, the trust is established on-the-fly between the other MTA and the VPS provider, but the VPS provider still also plays the the role of the eavesdropper. When the eavesdropper is trusted, you're in trouble.
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 8/28/20 3:33 PM, Michael Orlitzky wrote: TLS only secures the channel; what comes out at the end is a plain-text message that can be read with minimal effort by the VPS provider, no skullduggery needed. I agree that STARTTLS only protects the email while it's in flight between servers. Though I do think that it's going to somewhat difficult for a VPS provider to read the contents of the message if it's stored on an encrypted disk. I think that taking a snapshot of a running VPS / VM with the disk encryption keys in memory and accessing it qualifies as skullduggery. Plus, they will still need to content with the authentication requirements of the running snapshot, just like they would with the running VPS / VM. So things like LUKS definitely raises the bar and makes a VPS provider work a fair bit harder to access what's on the encrypted disk. (And the private key for each TLS session is generated on-the-fly by the VPS anyway, so they could snoop on the channel too if they wanted to.) Harvesting keys (TLS and / or LUKS) out of memory definitely qualifies as skullduggery. You can only protect against so much. You have to find what is acceptable risk. Unless the sender and recipient have some pre-shared secret (like GPG assumes), I *REALLY* thought that PGP (GPG) was based on public & private key pairs, much like S/MIME and TLS. As such, Alice and Bob can encrypt messages to each other, even through an untrusted medium such as a questionable email server. Yes, that still leaves the bootstraping issue of how do Alice and Bob get each other's public key. -- I defer to my recent comments about publishing keys in DNS and relying on DNSSEC. you're going to fall into the same trap that DRM falls into. The technology provides a way for Alice and Bob to communicate securely in the presence of Eve, but only when Alice, Bob, and Eve are three distinct people. If the VPS is playing the part of both Bob and Eve, an off-the-shelf encryption model isn't going to work. I see no need for either Alice nor Bob to be on the VPS. I would expect that they are their own independent (smart) devices accessing their respective email servers. Don't put any unencrypted sensitive data on the central server(s). Decrypting the emails in any capacity on the central server means that the gig is up and anyone with access, OS level or more nefarious, can access things. -- Grant. . . . unix || die
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 2020-08-28 17:12, Grant Taylor wrote: > On 8/28/20 1:54 PM, Poison BL. wrote: >> I'm rather late to the game with this, but at the end of the day, >> mail coming *into* a mail server isn't typically encrypted (and even >> that is only the body, the headers can still reveal a great deal, >> and are necessary for the server to work with it). > > You seem to be referring to S/MIME and / or PGP encryption. You are > correct that S/MIME and PGP don't offer protection for headers. > > However, STARTTLS provides an encrypted channel to protect all of the > SMTP traffic. Thus, even the headers of email are encrypted while in > flight between servers. > >> A packet dump at the switch will turn over every piece of mail you >> receive along the way. > > When STARTTLS is in use, the only thing that you will see is the initial > EHLO and STARTTLS commands. Everything after that will be encrypted > traffic. > TLS only secures the channel; what comes out at the end is a plain-text message that can be read with minimal effort by the VPS provider, no skullduggery needed. (And the private key for each TLS session is generated on-the-fly by the VPS anyway, so they could snoop on the channel too if they wanted to.) Unless the sender and recipient have some pre-shared secret (like GPG assumes), you're going to fall into the same trap that DRM falls into. The technology provides a way for Alice and Bob to communicate securely in the presence of Eve, but only when Alice, Bob, and Eve are three distinct people. If the VPS is playing the part of both Bob and Eve, an off-the-shelf encryption model isn't going to work.
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 8/28/20 1:54 PM, Poison BL. wrote: I'm rather late to the game with this, but at the end of the day, mail coming *into* a mail server isn't typically encrypted (and even that is only the body, the headers can still reveal a great deal, and are necessary for the server to work with it). You seem to be referring to S/MIME and / or PGP encryption. You are correct that S/MIME and PGP don't offer protection for headers. However, STARTTLS provides an encrypted channel to protect all of the SMTP traffic. Thus, even the headers of email are encrypted while in flight between servers. A packet dump at the switch will turn over every piece of mail you receive along the way. When STARTTLS is in use, the only thing that you will see is the initial EHLO and STARTTLS commands. Everything after that will be encrypted traffic. Email's not designed for end to end security by default. Encryption for anything other than military use didn't really exist when email was developed. Since then, things like STARTTLS (or SMTPS if you choose to abuse it for B2B connections) and VPNs have become a realistic option. Most contemporary MTAs will opportunistically use STARTTLS if it is an option. We only need to worry about things that try to prevent STARTTLS from working. Thankfully, this is mostly authoritarian regimes in some parts of the world. Secondly, any hosting on hardware you don't control is impossible to fully secure, if the services on that end have to operate on the data at all. You can encrypt the drive, encrypt the mail stores themselves, etc, but all of those things will result in the encryption key being loaded into ram while the VPS is running, and dumping ram from the hypervisor layer destroys every illusion of security you had. I agree those are all valid concerns. However, we shouldn't let the inability to realistically have perfect prevent us from having something better than no encryption. Dedicated hardware in a locked cabinet is as close as you get to preventing physical attacks when you're hosting in someone else's DC, and that's not nearly in the same market segment, price-wise, as a cheap VPS. At best, if you have sensitive email that you're sending or receiving, work with the other end of the communication and then encrypt the contents properly. Even better, go with a larger scale, paid, solution in which your email isn't even remotely worth the effort to tamper with for the hosting company's employees, and hope the contractual obligations are sufficient to protect you. I'm not aware of any place that contracts will protect you against local court orders / government involvement. If you have any sort of controlled data going in and out of your email, step up to a plan that adheres to the regulatory frameworks you're required to adhere to and make very sure the contracts for it obligate the vendor to secure things properly on their end (aws, azure/o365/etc mostly all have offerings for, at least, US Gov level requirements). Yep. -- Grant. . . . unix || die
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
james wrote: > On 8/21/20 4:10 PM, Grant Taylor wrote: >> On 8/21/20 11:01 AM, Caveman Al Toraboran wrote: >>> yes, i do consider re-inventing octagonal wheels. >> >> I think that it's occasionally a good thing to have a thought >> experiment about how $THING might be made better. >> >> It's probably good to have discussions around green feel types of >> replacements. >> >> But it's important to eventually assess the pros and cons of the old >> (as it exists), the new (as from green field), and the transition >> between the two. >> >> Sometimes the new doesn't warrant the transition, but it does provide >> an option that might be worth augmenting into the old. >> >> If nothing else, it's good to have the discussions and be able to >> answer why something was done or chosen to remain the same. >> >>> here, i'm just "asking" to see what makes the "safely stored" >>> guarantee. >> >> MTAs are supposed to be written such that they commit the message to >> persistent storage medium (disk) before returning an OK message to >> the sending server. >> >> There is some nebulous area around what that actually means.� But >> the idea is that the receiving server believes, in good faith, that >> it has committed the message to persistent storage.� Usually this >> involves writing the message to disk, probably via a buffered >> channel, and then issued system calls to ask the OS to flush the >> buffer to disk. >> >> Is there room for error?� Probably. >> >> Had the server made (more than) reasonable effort to commit the >> message to disk?� Yes. >> >> The point being, don't acknowledge receipt of the message while the >> message is only in the MTA's memory buffer.� Take some steps to >> commit it to persistent storage. >> >> That being said, there are some questionable servers / configurations >> that will bypass this safety step in the name of performance.� And >> they /do/ /loose/ /email/ as a negative side effect if (when) they do >> crash. This is a non-default behavior that has been explicitly chosen >> by the administrators to violate the SMTP specification.� Some MTAs >> will log a warning that they are configured to violate RFCs. >> >>> got any specific definition of what makes a storage "guaranteed"? >>> e.g. what kind of tests does the mail server do in order to say >>> "yup, i can now guarantee this is stored safely!"? >> >> It's more that they do something safe (write the message to disk) >> instead of risky (only store it in memory). >> >> Everything can fail at some point.� It's a matter of what and how >> many reasonable steps did you take to be safe.� Read: Don't cut >> corners and do something risky. >> >>> i guess you think that i meant that a relay should be mandatory? >> >> Sending / outbound SMTP servers /are/ a relay for any messages not >> destined to the local server. >> >> There is almost always at least two SMTP servers involved in any >> given email delivery.� All but the final receiving system is a relay. >> >>> (yes, a relay doesn't have to be used.� i'm just describing some >>> uses of relays that i think make sense.� (1) indicate trust >>> hierarchy, (2) offload mail delivery so that i can close my laptop >>> and let the relay have fun with the retries.� not sure there is >>> any other use.� anyone?) >> >> There are many uses for email relays.� A common one, and best >> practice, is to have an /inbound/ relay, commonly known as a backup >> email server. The idea being it can receive inbound messages while >> the primary email server is down (presumably for maintenance). >> >> Many SaaS Email Service Providers (ESPs) /are/ relay servers.� They >> receive email from someone and send it to someone else. >> >> A number of email hygiene appliances function as an email relay >> between the world and your ultimate internal email server.� >> Services that filter inbound email qualify here too. >> >> It is common, and I think it's best practice, to have web >> applications send email via localhost, which is usually a relay to a >> more capable hub email server which sends outbound email.� Both of >> these layers are relays. >> >> A relay is the same thing for email that a router is for a network. > > WOW do I love these discussions, but let me 'cut to the chase'. > > I'm proposing, via a small corp I own, to purchase up to (3) dual > Rasp.pi 4 setups of (2) R.Pi.4 8gig ram setups > and send them to the devs WE all decide on. Let's us start compiling > up the codes, keep it simple (for now) and implement them with > gentoo-users as the testers of the email services. > > > These discussions should be continued to everyone's benefit. However > there are way more than (3) folks on these threads who are most > capable to do this community prototyping. If WE do not act and get > hundreds of these deployed, email, as we know it via RFCS and other > standards may just disappeaar, or be relegated to the far reaches of > the Internet. What I have read, is standards based
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 8/28/20 1:18 PM, antlists wrote: The main reason other applications use "TCP over HTTP(S)" is because stupid network operators block everything else! I agree that filtering is a problem. I also think that it's something that most people can overcome when they control the firewall between the private LAN and the Internet. (Your typical SOHO NATing gateway.) The few times that I have run into filtering, it has been for uninitiated inbound connections. I've almost always been able to initiate outbound connections to / from odd ports. The few times that I could not do so in the last 20 years were resolved by engaging the ISP and ... politely ... getting them to knock it off. Inbound can be more tricky. But even inbound HTTP(S) was subject to the same problems. Actually, inbound HTTP(S) was more of a problem than other ports. -- Grant. . . . unix || die
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 8/28/20 1:55 PM, james wrote: I'm proposing, via a small corp I own, to purchase up to (3) dual Rasp.pi 4 setups of (2) R.Pi.4 8gig ram setups and send them to the devs WE all decide on. A few points. 1) I don't think that 8 GB of RAM is required. -- My email server is a VPS with 2 GB of RAM and is running just fine. So, maybe smaller systems would work. And maybe that would mean that more of them could be acquired for the same funding. 2) I don't know that a Raspberry Pi is strictly required for the testing. I think that anything that will run Gentoo can be used to prove out the software stack. -- Sure, there will /eventually/ need to be /some/ testing on Raspberry Pis. But I think that testing will be later in the game and more of a confirmation after the fact. 3) I'm not sure what you mean by "dual ... setups". What are the two systems (be it Raspberry Pis or VPSs or VMs or something else) supposed to do? - Are you wanting primary and backup (as in MX) or some sort of cluster with shared file system or something else? Let's us start compiling up the codes, keep it simple (for now) and implement them with gentoo-users as the testers of the email services. These discussions should be continued to everyone's benefit. However there are way more than (3) folks on these threads who are most capable to do this community prototyping. I think the idea of using VPSs or VMs means that a lot more people can participate using the same funding. If WE do not act and get hundreds of these deployed, email, as we know it via RFCS and other standards may just disappeaar, or be relegated to the far reaches of the Internet. What I have read, is standards based email services, particularly by small organizations, are under extreme pressure by large corporations to be marginalized out of existence. I think I disagree with that. Many of the big email operators are enforcing higher and higher standards. But the standards /are/ /open/ and /can/ /be/ /implemented/ /by/ /anyone/ who wants to do so. The /only/ thing that I've seen that is somewhat of a closed system that small players -- like myself -- have no real hop of is getting people like Google to trust our ARC (not DMARC) signatures. Though this is probably more a shortcoming in the ARC specification as it doesn't tackle how to get providers to trust your signature as a small operator. So any of the folks in these treads can announce publically, or send me private email as to your concerns. Public is best, but, I understand the needs for private communications sometimes. So yea, I'll personally finaces, at least 6 months of (3) projects. I'll take all input, but will make my (funding) decision, in a focus, quick strategy. I'm happy to participate. My preference would be to use a VPS / VM (which I can provide) and allow others to take advantage of the Pis that are on offer. -- Grant. . . . unix || die
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 28/08/2020 20:34, J. Roeleveld wrote: Cheers, Wol I think you meant that Caveman doesn't understand what TCP (and UDP) actually is. Grant does seem to know what he is talking about. Sorry yes I did. I got rather confused ... not surprising really :-) Cheers, Wol
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 8/21/20 4:10 PM, Grant Taylor wrote: On 8/21/20 11:01 AM, Caveman Al Toraboran wrote: yes, i do consider re-inventing octagonal wheels. I think that it's occasionally a good thing to have a thought experiment about how $THING might be made better. It's probably good to have discussions around green feel types of replacements. But it's important to eventually assess the pros and cons of the old (as it exists), the new (as from green field), and the transition between the two. Sometimes the new doesn't warrant the transition, but it does provide an option that might be worth augmenting into the old. If nothing else, it's good to have the discussions and be able to answer why something was done or chosen to remain the same. here, i'm just "asking" to see what makes the "safely stored" guarantee. MTAs are supposed to be written such that they commit the message to persistent storage medium (disk) before returning an OK message to the sending server. There is some nebulous area around what that actually means.� But the idea is that the receiving server believes, in good faith, that it has committed the message to persistent storage.� Usually this involves writing the message to disk, probably via a buffered channel, and then issued system calls to ask the OS to flush the buffer to disk. Is there room for error?� Probably. Had the server made (more than) reasonable effort to commit the message to disk?� Yes. The point being, don't acknowledge receipt of the message while the message is only in the MTA's memory buffer.� Take some steps to commit it to persistent storage. That being said, there are some questionable servers / configurations that will bypass this safety step in the name of performance.� And they /do/ /loose/ /email/ as a negative side effect if (when) they do crash. This is a non-default behavior that has been explicitly chosen by the administrators to violate the SMTP specification.� Some MTAs will log a warning that they are configured to violate RFCs. got any specific definition of what makes a storage "guaranteed"? e.g. what kind of tests does the mail server do in order to say "yup, i can now guarantee this is stored safely!"? It's more that they do something safe (write the message to disk) instead of risky (only store it in memory). Everything can fail at some point.� It's a matter of what and how many reasonable steps did you take to be safe.� Read: Don't cut corners and do something risky. i guess you think that i meant that a relay should be mandatory? Sending / outbound SMTP servers /are/ a relay for any messages not destined to the local server. There is almost always at least two SMTP servers involved in any given email delivery.� All but the final receiving system is a relay. (yes, a relay doesn't have to be used.� i'm just describing some uses of relays that i think make sense.� (1) indicate trust hierarchy, (2) offload mail delivery so that i can close my laptop and let the relay have fun with the retries.� not sure there is any other use.� anyone?) There are many uses for email relays.� A common one, and best practice, is to have an /inbound/ relay, commonly known as a backup email server. The idea being it can receive inbound messages while the primary email server is down (presumably for maintenance). Many SaaS Email Service Providers (ESPs) /are/ relay servers.� They receive email from someone and send it to someone else. A number of email hygiene appliances function as an email relay between the world and your ultimate internal email server.� Services that filter inbound email qualify here too. It is common, and I think it's best practice, to have web applications send email via localhost, which is usually a relay to a more capable hub email server which sends outbound email.� Both of these layers are relays. A relay is the same thing for email that a router is for a network. WOW do I love these discussions, but let me 'cut to the chase'. I'm proposing, via a small corp I own, to purchase up to (3) dual Rasp.pi 4 setups of (2) R.Pi.4 8gig ram setups and send them to the devs WE all decide on. Let's us start compiling up the codes, keep it simple (for now) and implement them with gentoo-users as the testers of the email services. These discussions should be continued to everyone's benefit. However there are way more than (3) folks on these threads who are most capable to do this community prototyping. If WE do not act and get hundreds of these deployed, email, as we know it via RFCS and other standards may just disappeaar, or be relegated to the far reaches of the Internet. What I have read, is standards based email services, particularly by small organizations, are under extreme pressure by large corporations to be marginalized out of existence. So any of the folks in these treads can announce publically, or send me private email
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On Mon, Aug 17, 2020 at 12:51 AM Caveman Al Toraboran wrote: > > hi. context: > > 1. tinfoil hat is on. > 2. i feel disrespected when someone does things to >my stuff without getting my approval. > 3. vps admin is not trusty and their sys admin may >read my emails, and laugh at me! > 4. whole thing is not worth much money. so not >welling to pay more than the price of a cheap >vps. moving to dedicated hardware for me is >not worth it. my goal is to make it annoying >enough that cheap-vps's admins find it a bad >idea for them to allocate their time to mingle >with my stuff. > > thoughts on how to maximally satisfy these > requirements? > > rgrds, > cm. > I'm rather late to the game with this, but at the end of the day, mail coming *into* a mail server isn't typically encrypted (and even that is only the body, the headers can still reveal a great deal, and are necessary for the server to work with it). A packet dump at the switch will turn over every piece of mail you receive along the way. Email's not designed for end to end security by default. Secondly, any hosting on hardware you don't control is impossible to fully secure, if the services on that end have to operate on the data at all. You can encrypt the drive, encrypt the mail stores themselves, etc, but all of those things will result in the encryption key being loaded into ram while the VPS is running, and dumping ram from the hypervisor layer destroys every illusion of security you had. Dedicated hardware in a locked cabinet is as close as you get to preventing physical attacks when you're hosting in someone else's DC, and that's not nearly in the same market segment, price-wise, as a cheap VPS. At best, if you have sensitive email that you're sending or receiving, work with the other end of the communication and then encrypt the contents properly. Even better, go with a larger scale, paid, solution in which your email isn't even remotely worth the effort to tamper with for the hosting company's employees, and hope the contractual obligations are sufficient to protect you. If you have any sort of controlled data going in and out of your email, step up to a plan that adheres to the regulatory frameworks you're required to adhere to and make very sure the contracts for it obligate the vendor to secure things properly on their end (aws, azure/o365/etc mostly all have offerings for, at least, US Gov level requirements). -- Poison [BLX] Joshua M. Murphy
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 28 August 2020 21:27:52 CEST, antlists wrote: >On 26/08/2020 21:21, Grant Taylor wrote: >>> so basically total expected number of protocols/layers used in the >>> universe, per second, will be much less if we, on planet earth, use >a >>> mail system that uses HTTP* instead of RESXCH_*. >> >> I obviously disagree. > >Exactly. You now need a protocol/layer that says you're running "mail >over http" as opposed to "web". HTTP is tcp/80 that *means* web. As >soon >as you start using it for something (anything) else you've just added >another protocol/layer. > >I get the distinct impression that Grant doesn't actually understand >what TCP is ... (hint - it has port numbers that are meant (if they're >not abused) to indicate what is going over the connection, like SMTP, >or >HTTP, or POP, or IMAP, etc etc). > >Cheers, >Wol I think you meant that Caveman doesn't understand what TCP (and UDP) actually is. Grant does seem to know what he is talking about. -- Joost -- Sent from my Android device with K-9 Mail. Please excuse my brevity.
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 26/08/2020 21:21, Grant Taylor wrote: so basically total expected number of protocols/layers used in the universe, per second, will be much less if we, on planet earth, use a mail system that uses HTTP* instead of RESXCH_*. I obviously disagree. Exactly. You now need a protocol/layer that says you're running "mail over http" as opposed to "web". HTTP is tcp/80 that *means* web. As soon as you start using it for something (anything) else you've just added another protocol/layer. I get the distinct impression that Grant doesn't actually understand what TCP is ... (hint - it has port numbers that are meant (if they're not abused) to indicate what is going over the connection, like SMTP, or HTTP, or POP, or IMAP, etc etc). Cheers, Wol
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 26/08/2020 19:51, Grant Taylor wrote: Just because it's possible to force something to use HTTP(S) does not mean that it's a good idea to do so. The main reason other applications use "TCP over HTTP(S)" is because stupid network operators block everything else! Cheers, Wol
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 26/08/2020 18:40, Grant Taylor wrote: On 8/21/20 10:15 PM, Caveman Al Toraboran wrote: just to double check i got you right. due to flushing the buffer to disk, this would mean that mail's throughput is limited by disk i/o? Yes. This speed limitation is viewed as a necessary limitation for the safety of email passing through the system. Nothing states that it must be a single disk (block device). It's entirely possible that a fancy MTA can rotate through many disks (block devices), using a different one for each SMTP connection. Thus in theory allowing some to operate in close lock step with each other without depending on / being blocked by any given disk (block device). Thank you for the detailed explanation Ashley. Or think back to the old days - network was slow and disks were relatively fast. The disk was more than capable of keeping up with the network, and simple winchesters didn't lie about saving to the rotating rust ... As Ashley explained, some MTAs trust the kernel. I've heard of others issuing a sync after the write. But that is up to each MTA's developers. They have all taken reasonable steps to ensure the safety of email. Some have taken more-than-reasonable steps. Depends on the filesystem. "sync after write" was an EXTREMELY daft idea on ext4 in the early days - that would really have killed system response. Nowadays you could stick an SSD cache in front of a raid array ... Cheers, Wol
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
‐‐‐ Original Message ‐‐‐ On Friday, August 28, 2020 2:35 AM, Ashley Dixon wrote: > On Thu, Aug 27, 2020 at 09:07:03PM +, Caveman Al Toraboran wrote: > > > anyway i'm out of this. massive waste of time. i > > could've finished server-side hillarymail by it. > > Oh, come on. People on this list have decades of experience managing and > implementing e-mail protocols, and you call their (free) help a "massive waste > of time"? Stop being silly and realise that no initial proposal is completely > flawless. it's not against "people on the list". it's rather for them. because continuing talking to grant (and soon you) is fueling a useless conversation that is effectively vandalising the mailboxes of 100s of people on this list. now you're posting this yet another useless drama message trying to make it sound as if it's against "people on the list" or as if i'm too defensive of hillarymail. so now i'll also stop talking to you in this sub-thread (in addition to grant taylor). nothing personal. we may talk in other sub-threads. it's just that talking to you 2 in these late threads became a fuel to vandalise others' mailboxes. > As I keep urging you, define some goals (and as Grant said, start with > defining > the current problem), finish an initial standards document, and begin writing > a > reference implementation. Or just define some of the core algorithms with > pseudocode. I can almost-guarantee that you will start realising things that > need changing almost immediately upon doing so. nothing new. we already discussed this in the other sub-thread and, as i said there, i am already planning to write an implementation. and i'm already refining the draft. i don't know why you keep repeating non-new things over and over (zero information content). that sub-thread has also became very useless thanks to you and grant for talking about margaret thatcher, LaTeX and other unrelated things. zero actual comments about technical aspects. > Perhaps it is just me with my English sense of over-politeness, but I find > your > conduct to be remarkably audacious (and frankly rude) considering all the time > people are spending to help you. ... And if you don't want this sort of > on-line > discourse, why did you post on the list at all? is your "English" sense of "over-politeness" capable of sensing vandalism caused by having you post texts with low information content, or irrelevant info, to people's inboxes? (rhetorical)
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
‐‐‐ Original Message ‐‐‐ On Thursday, August 27, 2020 8:15 PM, Grant Taylor wrote: > On 8/27/20 7:00 AM, Caveman Al Toraboran wrote: > > > but i this way of looking at protocols (despite being common) is wrong. > > Why do you think that it is wrong? > > What is not factually correct about it? it depends on how you use it: - if you use it for what it is made for (making speech about protocols easier), then that's fine. not perfect, but not wrong either. - but if you use it to do some complexity analysis as you did earlier by counting such layers, then, you're wrong. because even though smtp appears as a single layer in such common diagrams, it is functionally 2 layers (one being a resource exchange layer overlapping with http). > > i also disagree with the network layering proposed by osi or the > > other ones commonly published in books. i specially disagree with > > using such layering for studying the complexity of protocols. > > If you're going to make such a statement, which is fine to do, you must > provide information ~> evidence as to why you are doing so and why you > think what you think. see above or previous emails. you're basically abusing such diagrams to perform protocol complexity analysis. i was trying to be indirect by blaming the common protocol layering for leading you to this mistake. what's happening is that you're simply abusing them to do what they are not made for. for details you can re-read my previous email(s) on how smtp is functionally at least 2 layers. > > so i suggest that if we want to study the complexity of messaging > > systems, we better not count SMTP as a single thing (like how it is > > normally done in books and talks), but instead talk about it based on > > the fundamental tasks that it actually does. this way, SMTP becomes > > at least 2 layers: > > I think that I see part of a problem. > > RFC 822 - Standard for the format of ARPA Internet Text Message - is > what defines what I was referring to as the opaque blob sent between > systems. > > I will argue that the content of the opaque blob that SMTP transfers is > independent of SMTP itself. > > > 1. "resource exchange" layer where binaries are made into a single > > giant text file by base64 encoding and then partitioned by rfc822. > > this part overlaps with http* and is much less efficient (rightfully, > > since email had to be backwards compatible as it is critical). > > > > SMTP* does not support binary in any (original) capacity. As such, > email service, which /rides/ /on/ /top/ /of/ SMTP, is where the encoding > ""hack was placed. This /encoding/ and / or /formatting/ is completely > independent of the SMTP protocol used to exchange opaque blobs between > mail servers. i'm amazed how you skipped the real point that i'm making about your incorrect layer-based protocol complexity analysis, and —instead— moved to talk about how email's inefficient binary encoding is due to rfc822 and not rfc821. it's irrelevant at several levels: - doesn't justify your layer-based complexity analysis earlier, either way. - no one discussed which rfc is the reason why smtp is being used for inefficient binary transfer. - the fact that attachments are inefficiently sent over smtp as per rfc822 is by itself due to bad historical design decisions in smtp that lead people to commonly use rfc822 with smtp. the several next paragraphs that you wrote are simply talking about whether smtp (rfc821) was born with a horrible binary encoding, or was it born retarded enough to push people to end up adding rfc822 to it in order to minimise suffering. which is irrelevant at many levels, so i'm skipping over them to save space/time. > > this way, if we ignore the problem of maintaining backwards > > compatibility, > > That is a HUGE if. One that I do not accept at all. You absolutely > MUST have backwards compatibility in some way. Even if that > compatibility is something that acts as an edge gateway between SMTP and > your new method. You MUST have backward compatibility in some way. also irrelevant. yes, that hypothetical "if" statement is indeed "huge". so? it's a hypothetical if statement to show another point: that your complexity analysis by counting layers is wrong. i'm once again amazed how you skipped the main point, and went on to write about how HUGE that hypothetical "if" statement is. (for the record i'm not suggesting to drop smtp's backwards compatibility, nor suggesting it would be easy.) > > then having http* in the "resource exchange" layer would be more > > efficient and simpler as there will be less protocols doing the > > "resource exchange" task (instead of having each do its own). > > So you want to take two completely different protocols that serve two > completely different functions and force them into a third completely > different protocol that
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On Thu, Aug 27, 2020 at 09:07:03PM +, Caveman Al Toraboran wrote: > anyway i'm out of this. massive waste of time. i > could've finished server-side hillarymail by it. Oh, come on. People on this list have decades of experience managing and implementing e-mail protocols, and you call their (free) help a "massive waste of time"? Stop being silly and realise that no initial proposal is completely flawless. As I keep urging you, define some goals (and as Grant said, start with defining the current problem), finish an initial standards document, and begin writing a reference implementation. Or just define some of the core algorithms with pseudocode. I can almost-guarantee that you will start realising things that need changing almost immediately upon doing so. Perhaps it is just me with my English sense of over-politeness, but I find your conduct to be remarkably audacious (and frankly rude) considering all the time people are spending to help you. ... And if you don't want this sort of on-line discourse, why did you post on the list at all? -- Ashley Dixon suugaku.co.uk 2A9A 4117 DA96 D18A 8A7B B0D2 A30E BF25 F290 A8AA signature.asc Description: PGP signature
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
‐‐‐ Original Message ‐‐‐ On Thursday, August 27, 2020 12:21 AM, Grant Taylor wrote: > email emailemail > SMTP SMTP POP3S/IMAPS > A) [1]---(TCP)---[2]---(TCP)---[3]---(TCP)---[4] > > Now what you are proposing: > > email email email > TBD TBD TBD > HTTPS HTTPS HTTPS > A) [1]---(TCP)---[2]---(TCP)---[3]---(TCP)---[4] > > The number of layers has increased from three to four. that's true if "SMTP" is a single layer. which is also how networking engineers including those highly skilled ones in standard bodies commonly commonly talk about protocols (e.g. based on layers of that sort). so i see why it makes sense that you did it this way. but i this way of looking at protocols (despite being common) is wrong. i also disagree with the network layering proposed by osi or the other ones commonly published in books. i specially disagree with using such layering for studying the complexity of protocols. so i suggest that if we want to study the complexity of messaging systems, we better not count SMTP as a single thing (like how it is normally done in books and talks), but instead talk about it based on the fundamental tasks that it actually does. this way, SMTP becomes at least 2 layers: 1. "resource exchange" layer where binaries are made into a single giant text file by base64 encoding and then partitioned by rfc822. this part overlaps with http* and is much less efficient (rightfully, since email had to be backwards compatible as it is critical). 2. "resource use" where the mail server parses such exchanged resources (e.g. email bodies, attachments, etc) and then acts upon them (e.g. forward them, discard them, etc). and so will pop* and imap. this way, if we ignore the problem of maintaining backwards compatibility, then having http* in the "resource exchange" layer would be more efficient and simpler as there will be less protocols doing the "resource exchange" task (instead of having each do its own). i also think that the kind of resource that email exchanges is fundamentally identical to a subset of resources that are natively exchanged in the web. so i think the only reason that smtp/pop/imap have different resource exchange protocols is purely due to backwards compatibility due to how things evolved historically. - i suspect that we actually agree on everything, but speak different languages (possibly due to how books commonly talk about protocols and layering), or assume things beyond what's written. e.g. we agree that: 1. smtp/pop*/imap make the best messaging system today, and is not going away any time soon, thanks to its wide spread. most likely i'll be dead and still have multiple active smtp/imap/pop account. 2. smtp/imap/pop are imperfect and have many shortcomings that are "rightfully" not solved "cleanly" due to historical reasons and its critical nature which imposed on us the constraint of having to maintaining its backwards compatibility. 3. trying new protocols is fine. and is also fine to have sub-communities that use different messaging protocols if they find it more fitting. e.g. i'll probably end up using smtp/imap for talking to people in general, and use hillarymail [1] for talking to a closer nerdy community. [1] https://github.com/al-caveman/hillarymail
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 8/27/20 7:00 AM, Caveman Al Toraboran wrote: but i this way of looking at protocols (despite being common) is wrong. Why do you think that it is wrong? What is not factually correct about it? i also disagree with the network layering proposed by osi or the other ones commonly published in books. i specially disagree with using such layering for studying the complexity of protocols. If you're going to make such a statement, which is fine to do, you must provide information ~> evidence as to why you are doing so and why you think what you think. so i suggest that if we want to study the complexity of messaging systems, we better not count SMTP as a single thing (like how it is normally done in books and talks), but instead talk about it based on the fundamental tasks that it actually does. this way, SMTP becomes at least 2 layers: I think that I see part of a problem. RFC 822 - Standard for the format of ARPA Internet Text Message - is what defines what I was referring to as the opaque blob sent between systems. I will argue that the content of the opaque blob that SMTP transfers is independent of SMTP itself. 1. "resource exchange" layer where binaries are made into a single giant text file by base64 encoding and then partitioned by rfc822. this part overlaps with http* and is much less efficient (rightfully, since email had to be backwards compatible as it is critical). SMTP* does not support binary in any (original) capacity. As such, email service, which /rides/ /on/ /top/ /of/ SMTP, is where the encoding ""hack was placed. This /encoding/ and / or /formatting/ is completely independent of the SMTP protocol used to exchange opaque blobs between mail servers. *I did elide some more modern SMTP extensions to simplify the previous statement. To whit: It is conceptually possible that there could be an SMTP exchange consisting of the following: S: Hello... C: EHLO S: Nice to meet you... C: MAIL FROM: S: Okay. Continue C: RCPT TO: S: Okay. Continue C: DATA S: Okay. Continue C: . S: Okay. Thank you. C: QUIT S: Goodbye. The XXX...XXX content is /OUTSIDE/ of the SMTP specification. That content could conceptually be anything that you want it to be. The only limitation is that the communications channel must safely support the bit pattern and there must not be any way to cause confusion for the protocol outside of it. SMTP doesn't care /what/ the contents of the XXX...XXX is. SMTP's ob is to exchange the XXX...XXX between servers based on the envelope from and recipients. Some, if not many, email servers have instituted sanity checks to make sure that the XXX...XXX has some specific content (headers) and is well formatted. But this sanity checking is outside of the SMTP protocol. So, your "where binaries are made into a single giant text file by base64 encoding and then partitioned by rfc822" is /NOT/ about SMTP. Instead it is about RFC 822 - Internet Message Format. SMTP is RFC 821 - Simple Mail Transfer Protocol 2. "resource use" where the mail server parses such exchanged resources (e.g. email bodies, attachments, etc) and then acts upon them (e.g. forward them, discard them, etc). Again, this is outside of the SMTP protocol. It has nothing to do with how the opaque blob is transferred between servers. and so will pop* and imap. I'm guessing that you are making a similar mistake with POP3 and IMAP. Both POP3 and IMAP are also about transferring an opaque blob between the email server and the client. The base POP3 and IMAP protocol do not care what the contents of the opaque blob are. Their goal is to get said opaque blob of XXX...XXX from the email server to the client. I say base protocol because I think there are some - what I'll call - fancier POP3 and / or IMAP commands that can be issued to search and / or retrieve messages based on content in the blob. At their base, SMTP and POP3 and IMAP are about transferring opaque blobs between servers. In fact, they are the most recent replacement for alternative methods for exchanging the same opaque blobs. Historically, SMTP replaced FTP and UUCP for exchanging the same opaque blobs. Something could, and probably
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 8/27/20 6:07 AM, Victor Ivanov wrote: I have been quietly following this discussion and I've seen SRS being mentioned a number of times. Welcome to an active part in the conversation. :-) Now, I know what SRS _does_ (perhaps not fully?) to prevent unintended rejection by a receiving MTA due to SPF policy and I know it's highly recommended if not mandatory for any sensible MTA set-up (allegedly). I don't think that SRS qualifies as recommended, much less mandatory. More specifically, I think some of the email industry discourages it. I know that Google wants people to not use SRS and instead would prefer that people forward things without using SRS. I think that Google wants to see the original SMTP envelope sender and make judgement calls on that, even if doing so would violate the purported sending domain's SPF records. Note: SRS means that the domain that is used in the re-written sender domain becomes associated with the content. So if you are forwarding and re-writing bad content (virus, spam, or malware) your domain becomes associated with said bad content. Aside: One of the most important things when forwarding email is to *ONLY* forward believed -> known to be clean and good email. Do *NOT* forward spam, virus, malware, or otherwise questionable email. I've heard / read about many people talking down / bad about SRS. But, I personally have used SRS for 10-15 years and have not knowingly had any ill side effects from doing so. I do conditionally apply SRS to outgoing messages from domains that my server is not responsible for (MX for inbound email). However, I'm less aware of the actual use cases where it is absolutely necessary to prevent above issues with SPF. Is SRS only relevant for MTAs that do any sort of forwarding/open relay? In short, "yes" and "correct". You only /need/ (for however much value you give "need") SRS when you are sending an email that would otherwise violate the SMTP envelope from domain's SPF record. So you /might/ be able to conditionally apply SRS to any outgoing email where there is an SPF policy that you would be violating. This is most likely going to happen when you are forwarding email. As in someone has a .forward file. I think that an open relay is a bad thing. Despite the fact that it would benefit from things like SRS for the above mentioned reasons, I think an open relay should be avoided. As such, I don't give it much more thought. Oddly enough, I find some of the explanations I came across (including Wikipedia) fairly atrocious compared to SPF, DKIM, and DMARC which are perfectly explained and their use-cases are very clear. Though frankly, these three are fairly self explanatory themselves for anyone with technical background. I think there are those in the email industry that have an adverse reaction to SPF in general and view SRS as a nasty hack (which might be fair) that is only needed in response to SPF. They seem to really dislike SPF and hate SRS. I have been running my own MTA for just under a decade without SRS (but with the rest of the above) and never had issues with delivery - be that local or remote. However, I have explicitly disabled open relay and do not make use of any forwarding maps. That sounds reasonable. It also sounds like you have little to no need for SRS. I have a few friends that I forward email for. As such, I have need for SRS. It makes me wonder - am I missing something and have I just been lucky? I don't think you're missing anything. Perhaps you have been lucky in that your user base doesn't need forwarding. But that is perfectly fine and perfectly legitimate use case. -- Grant. . . . unix || die
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 27/08/2020 02:31, Grant Taylor wrote: > - SRS (mail-filter/libsrs2) [] > - emerge -a mail-filter/libsrs2 > I have been quietly following this discussion and I've seen SRS being mentioned a number of times. Now, I know what SRS _does_ (perhaps not fully?) to prevent unintended rejection by a receiving MTA due to SPF policy and I know it's highly recommended if not mandatory for any sensible MTA set-up (allegedly). However, I'm less aware of the actual use cases where it is absolutely necessary to prevent above issues with SPF. Is SRS only relevant for MTAs that do any sort of forwarding/open relay? Oddly enough, I find some of the explanations I came across (including Wikipedia) fairly atrocious compared to SPF, DKIM, and DMARC which are perfectly explained and their use-cases are very clear. Though frankly, these three are fairly self explanatory themselves for anyone with technical background. I have been running my own MTA for just under a decade without SRS (but with the rest of the above) and never had issues with delivery - be that local or remote. However, I have explicitly disabled open relay and do not make use of any forwarding maps. It makes me wonder - am I missing something and have I just been lucky? - V signature.asc Description: OpenPGP digital signature
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 8/18/20 6:44 PM, Grant Taylor wrote:
I will have to collect a list and get back to you.
Here are part of some crude notes that I created for myself to use to
build a Gentoo mail server about three years ago. This is the email
specific parts. The rest were for other non-email aspects.
Note: This just gets things installed. It doesn't do anything for how
to configure things. (I was copying config files between systems.)
The square brackets at the end of the line are dependencies. (I use the
word loosely.)
- eMail
- Sendmail (mail-mta/sendmail) [ipv6 sasl ssl]
- emerge -a mail-mta/sendmail
- chmod g+w /var/spool/mqueue
- rc-update add sendmail default
- {mc,cf} file
- certificates
- cp /etc/pam.d/{pop,smtp}
- touch /etc/mail/access
- makemap hash /etc/mail/access < /etc/mail/access
- touch /etc/mail/genericstable
- makemap hash /etc/mail/genericstable < /etc/mail/genericstable
- Cyrus-SASL (pulled in)
- rc-update add saslauthd default
- Nail (mail-client/nail) []
- emerge -a mail-client/nail
- Procmail (mail-filter/procmail) []
- emerge -a mail-filter/procmail
- Mutt (mail-client/mutt) []
- emerge -a mail-client/mutt
- SpamAssassin (mail-filter/spamassassin) [cron ipv6 ssl]
- emerge -a mail-filter/spamassassin
- rc-update add spamd default
- sa-update
- SpamAssassin Milter (mail-filter/spamass-milter) []
- emerge -a mail-filter/spamass-milter
- rc-update add spamass-milter default
- ClamAV (app-antivirus/clamav) [ipv6 milter]
- emerge -a app-antivirus/clamav
- rc-update add clamd default
- /etc/conf.d/clamd
- START_MILTER=yes
- SPF (mail-filter/libspf2) []
- emerge -a mail-filter/libspf2
- SRS (mail-filter/libsrs2) []
- emerge -a mail-filter/libsrs2
- OpenDKIM (mail-filter/opendkim) [ssl]
- echo "mail-filter/opendkim ldap" >
/etc/portage/package.use/opendkim
- emerge -a mail-filter/opendkim
- rc-update add XXX default
- OpenDMARC (mail-filter/opendmarc) []
- emerge -a mail-filter/opendmarc
- rc-update add XXX default
- CourierIMAP (net-mail/courier-imap) [ipv6]
- emerge -a net-mail/courier-imap
- /etc/courier/conf files
- mkimapcert
- mkdhparams
- rc-update add courier-imapd-ssl default
- Install valid SSL cert.
--
Grant. . . .
unix || die
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 8/21/20 10:11 PM, Caveman Al Toraboran wrote: not a major point but just to clarify a thing. i think it's unfair to look at SMTP as a single thing that compares against HTTP*. because while HTTP* is a single-ish thing, SMTP is several things. i.e. SMTP is at least 2 parts: Fair point. As I see it, we have two things; 1) what is transferred (email message) and 2) how it is transferred (SMTP, UUCP, etc.). 1. resource exchange layer where people are defined as some kind of URL (e.g. n...@dom.zone) and attachments are base64-ed text balls referred to by some numbers in RFC822. That sounds like the "what is transferred" to me. As in the opaque value of the sender / recipient / body / attachments. It does not matter what the contents of those opaque values are. They make up what is transferred. This part overlaps with HTTP*. let's call this "RESXCH_SERVER". HTTP(S) can overlap with the "how" part of the transfer. But I believe that the HTTP(S) "how" part is independent of the opaque blob part of "what". 2. the part where it defines how to process the exchanged resources (e.g. safe storage, routing, etc). this part is beyond HTTP*'s scope, and is the "web app" scope. let's call this "RESUSE_SERVER" Okay. of course, email still doesn't work with those 2 parts, because you need a way to get mails to your email client, so you end up using POP or IMAP. now, this --itself-- is also two parts: That is highly dependent on your email client. Many traditional Unix email clients read email directly off the file system of the email server. In fact, most POP3(S) and IMAP(S) implementations read the same data off of the email server. The only difference being that the allow the email client to be remote from the email server. I'd like to point out that thus far this thread has been about how to get email between email servers. As such, I think client access protocols are somewhat outside of the existing scope. 1. resource exchange layer to send resources to users. which also overlaps with HTTP* (again). let's call this "RESXCH_CLIENT". The "how". 2. the part where it defines how the mail client to treat the resources. let's call this "RESUSE_CLIENT". See above comment about scope. This is also a "how". But "how" clients get email from servers is decidedly different than "how" servers exchange email. They can both use any underlying transport, be it TCP (like POP3 & IMAP) or HTTP(S). i disagree. i think this is more like it about the current email system: RESXCH_SERVER / RESUSE_SERVER / RESXCH_CLIENT / RESUSE_CLIENT I'm having trouble unpacking that. So I'll provide my version of what I think you're trying to say. Since all of this rides on top of TCP, I'm not going to call it out. 1) The sender (who-sender) sends their email (what) to their outbound email server (who-recipient) with SMTP (how). 2) (The sender's) outbound server (who-sender) sends (the sender's) email (what) to the (the recipient's) inbound server (who-recipient) with SMTP (how). 3) The recipient (who-recipient) pulls email (what) from their inbound server (who-sender) with POP3S / IMAPS (how). Notice how we have a what, who-sender, who-recipient, and how in each of these. The who-sender and who-recipient are the parties of each discrete conversation; 1, 2, or 3. Notice how the what (email) is an opaque blob on each leg of the transmission journey. There are four distinct points with three distinct paths in between. A) [1]---(SMTP)---[2]---(SMTP)---[3]---(POP3S/IMAPS)---[4] 1) Person sending the email. 2) #1's email server. 4) Person receiving the email. 3) #4's email server. (Yes I know those are not in numerical order. #3 and #4 are in dependency order.) The only way to remove a point and transmission path from this process end-to-end process is for #2 and #3 to be the same. B) [1]---(SMTP)---[2/3]---(POP3S/IMAPS)---[4] Other than mega email providers; Google, Microsoft, Yahoo, et al., you are going to have scenario A above. Each step of the way you have what is transferred (the message) and how it is transferred (the protocol) between two points. The what (email) is still an opaque blob on each leg of the transfer. it's 4 different layers to exchange mail between people. We apparently aren't using layers the same way. You seem to be using layers in place of hops. That is both inaccurate and unfair for a number of reasons. There can be any number of additional hops in between # 2 & 3 above for any number of reasons; legal, technical, political, etc. but if we plug HTTP* in the mix, it because only 3 different layers: HTTP* / RESUSE_SERVER / HTTP* / RESUSE_CLIENT You also seem to be describing scenario B above. Here is a variation on scenario A above that is completely realistic if not probable. C) [1]---(SMTP)---[2]---(SMTP)---[3]---(SMTP)---[4]---(SMTP)---[5]---(SMTP)---[6]---(POP3S/IMAPS)---[7] 1) The
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On Wed, Aug 26, 2020 at 2:51 PM Grant Taylor wrote: > > On 8/21/20 5:58 PM, Rich Freeman wrote: > > It is what just about every other modern application in existence uses. > > VoIP does not. Yes, but VoIP isn't just implementing a simple data-exchange API. It is a streaming protocol. > No RDBMSs that I'm aware of use it as their primary protocol. (Some may > be able to use HTTP(S) as an alternative.) These pre-date webservices in general. They aren't modern. > Outlook to Exchange does (did?) not use it. Again, not modern. > I'm not aware of any self hosted enterprise grade remote desktop > solution that uses HTTP(S) as it's native transport. Also streaming. > > DNSSEC is: > > > > 1. Still poorly supported even where it makes sense. > > Yet another example of ignorance and / or laziness. > > I have found DNSSEC to be relatively easy to implement, and trivial to > enable on my recursive resolvers. > > The ignorance portion is relatively easy to resolve if people want to. > I highly recommend DNSSEC Mastery by Michael W. Lucas. That $20 (?) > book and moderate amounts of motivation is all anybody that wants to > implement DNSSEC /needs/. Well, maybe check back when you get everybody who sends email to read that book. You can start with my mother. -- Rich
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 8/21/20 5:58 PM, Rich Freeman wrote: It is what just about every other modern application in existence uses. VoIP does not. No RDBMSs that I'm aware of use it as their primary protocol. (Some may be able to use HTTP(S) as an alternative.) Outlook to Exchange does (did?) not use it. It does have a sub-optimal RPC-over-HTTP(S) option for things like mobile clients. But you still get much better service using native non-HTTP(S) protocols. I'm not aware of any self hosted enterprise grade remote desktop solution that uses HTTP(S) as it's native transport. Just because it's possible to force something to use HTTP(S) does not mean that it's a good idea to do so. I'm sure there are hundreds of articles on the reasons for this that are far better than anything I'd come up with here. Probably. And they don't work for anything OTHER than SMTP. There are /other/ libraries that work for /other/ things. Having a general thing that can be abused for almost all things is seldom, if ever, the optimal way to achieve the goal. A library for JSON/webservices/whatever works for half the applications being written today. I choose to believe that even that 50% is significantly sub-optimal and that they have been pressed into that role for questionable reasons. This is simple. This is how just about EVERYBODY does it these days. I disagree. Yes, a lot of people do. But I think it's still a far cry from "just about everybody". http works as a transport mechanism. Simply working is not always enough. Dial up modems "worked" in that data was transferred between two endpoints. Yet we aren't using them today. Frequently, we want optimal or at least the best solution that we can get. That is the beauty of standards like this - somebody else figured out SSL on top of HTTP so we don't need an email-specific reimplementation of that. I think that you are closer than you realize or may be comfortable with. "somebody else figured out" meaning that "someone else has already done the work or the hard part". Meaning that it's possible to ride people's coat tails. HTTP(S) as a protocol has some very specific semantics that make it far from ideal for many things. Things like the server initiating traffic to clients. Some, if not many, of these semantics impose artificial limitations on services. I mean, why use TCP? For starters, TCP ensures that your data arrives at the other end (or notifies you if it doesn't), that it's in order, and that it's not duplicated. There are multiple other protocols that you can use. UDP is a prominent one. Why not use something more tailored to email? Now you're comparing an application layer protocol (email(SMTP)) to a network transport protocol (TCP / UDP / etc.). What transport layer protocol would you suggest using? Be careful to trace higher layer abstraction protocols, e.g. QUIC, back down to the transport layer protocol (UDP in QUIC's case). TCP probably has a dozen optional features that nobody uses with email, so why implement all that just to send email? What contemporary operating system does not have a TCP/IP stack already? How are applications on said operating system speaking traditional / non-QUIC based HTTP(S) /without/ TCP? Even exclusively relying on QUIC still uses UDP. The answer is that it makes WAY more sense to recycle a standard protocol than to invent one. You're still inventing a protocol. It's just at a higher layer. You still have to have some way / method / rules / dialect, more commonly known as a protocol, on whatever communications transport you use. Even your web based application needs to know how to communicate different things about what it's doing. Is it specifying the sender or the recipient or the subject or something else. Protocols are what define that. They are going to exist. It's just a question of where they exist. You're still inventing a protocol. Do you want your protocol to run on top of a taller more complex stack of dependencies? Or would you prefer a shorter simpler stack of dependencies? You're still inventing a protocol. You're just choosing where to put it. And you seem to be in favor of the taller more complex stack. Conversely I am in favor of the shorter simpler stack. If SMTP didn't exist, and we needed a way to post a bunch of data to a remote server, you'd use HTTP, because it already works. No. /I/ would not. HTTP(S) is inherently good at /retrieving/ data. We have used and abused HTTP(S) to make it push data. Depending what the data was and the interactive nature of it, I would consider a bulk file transfer protocol FTPS / SFTP / SCP or even NAS protocols. Things that are well understood and well tested. If I had some reason that they couldn't do what I needed or as efficiently as I needed, I would develop my own protocol. I would certainly do it on top of a transport protocol that ensured my data
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 8/21/20 10:15 PM, Caveman Al Toraboran wrote: just to double check i got you right. due to flushing the buffer to disk, this would mean that mail's throughput is limited by disk i/o? Yes. This speed limitation is viewed as a necessary limitation for the safety of email passing through the system. Nothing states that it must be a single disk (block device). It's entirely possible that a fancy MTA can rotate through many disks (block devices), using a different one for each SMTP connection. Thus in theory allowing some to operate in close lock step with each other without depending on / being blocked by any given disk (block device). Thank you for the detailed explanation Ashley. or did i misunderstand? You understand correctly. i sort of feel it may suffice to only save to disk, and close fd. then let the kernel choose when to actually store it in disk. As Ashley explained, some MTAs trust the kernel. I've heard of others issuing a sync after the write. But that is up to each MTA's developers. They have all taken reasonable steps to ensure the safety of email. Some have taken more-than-reasonable steps. -- Grant. . . . unix || die
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On Sat, Aug 22, 2020 at 09:17:56PM +0100, Ashley Dixon wrote: > On Sat, Aug 22, 2020 at 04:15:38AM +, Caveman Al Toraboran wrote: > > just to double check i got you right. due to > > flushing the buffer to disk, this would mean that > > mail's throughput is limited by disk i/o? [...] > When an M.T.A. encounters mail, the content of the mail will first exist in > the > M.T.A.'s local memory, in a buffer. Before sending an "OK" to the > sending > server, it should first make an attempt to write it to disk, through an > fwrite > (stdio) or write (POSIX) call. At that point, it is, in theory, the > kernel's > choice if and when it is _actually_ written to disk, but if one of > the > aforementioned functions return a success code, the M.T.A. has done its bit, > and > can consider the message "safely stored". true and yes given a sink willing to accept your throughput, an mta is generally disk i/o bound -- Eray
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On Sat, Aug 22, 2020 at 04:15:38AM +, Caveman Al Toraboran wrote: > ‐‐‐ Original Message ‐‐‐ > On Saturday, August 22, 2020 12:10 AM, Grant Taylor > wrote: > > > There is some nebulous area around what that actually means. But the > > idea is that the receiving server believes, in good faith, that it has > > committed the message to persistent storage. Usually this involves > > writing the message to disk, probably via a buffered channel, and then > > issued system calls to ask the OS to flush the buffer to disk. > > just to double check i got you right. due to > flushing the buffer to disk, this would mean that > mail's throughput is limited by disk i/o? > > or did i misunderstand? > > i sort of feel it may suffice to only save to > disk, and close fd. then let the kernel choose > when to actually store it in disk. When an M.T.A. encounters mail, the content of the mail will first exist in the M.T.A.'s local memory, in a buffer. Before sending an "OK" to the sending server, it should first make an attempt to write it to disk, through an fwrite (stdio) or write (POSIX) call. At that point, it is, in theory, the kernel's choice if and when it is _actually_ written to disk, but if one of the aforementioned functions return a success code, the M.T.A. has done its bit, and can consider the message "safely stored". -- Ashley Dixon suugaku.co.uk 2A9A 4117 DA96 D18A 8A7B B0D2 A30E BF25 F290 A8AA signature.asc Description: PGP signature
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
‐‐‐ Original Message ‐‐‐ On Saturday, August 22, 2020 12:19 AM, Grant Taylor wrote: > > i was thinking (and still) if such relay-by-relay delivery increases > > probability of error by a factor of n (n = number of relays in the > > middle). e.g. probability of accidental silent mail loss is if one, > > or more, accidentally said "yes got it!" but actually didn't. i.e.: > > It definitely won't be a factor of n, where n is the number of relays. why? since relays are in series, and since each relay trusts next relay's "yup got it!", then error rate should add up for every extra step. no?
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
‐‐‐ Original Message ‐‐‐ On Saturday, August 22, 2020 12:10 AM, Grant Taylor wrote: > There is some nebulous area around what that actually means. But the > idea is that the receiving server believes, in good faith, that it has > committed the message to persistent storage. Usually this involves > writing the message to disk, probably via a buffered channel, and then > issued system calls to ask the OS to flush the buffer to disk. just to double check i got you right. due to flushing the buffer to disk, this would mean that mail's throughput is limited by disk i/o? or did i misunderstand? i sort of feel it may suffice to only save to disk, and close fd. then let the kernel choose when to actually store it in disk.
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
‐‐‐ Original Message ‐‐‐ On Friday, August 21, 2020 11:37 PM, Grant Taylor wrote: > SMTP may not be the best, but I do think that it has some merits. > Merits that the previously mentioned HTTP/2 alternative misses. not a major point but just to clarify a thing. i think it's unfair to look at SMTP as a single thing that compares against HTTP*. because while HTTP* is a single-ish thing, SMTP is several things. i.e. SMTP is at least 2 parts: 1. resource exchange layer where people are defined as some kind of URL (e.g. n...@dom.zone) and attachments are base64-ed text balls referred to by some numbers in RFC822. This part overlaps with HTTP*. let's call this "RESXCH_SERVER". 2. the part where it defines how to process the exchanged resources (e.g. safe storage, routing, etc). this part is beyond HTTP*'s scope, and is the "web app" scope. let's call this "RESUSE_SERVER" of course, email still doesn't work with those 2 parts, because you need a way to get mails to your email client, so you end up using POP or IMAP. now, this --itself-- is also two parts: 1. resource exchange layer to send resources to users. which also overlaps with HTTP* (again). let's call this "RESXCH_CLIENT". 2. the part where it defines how the mail client to treat the resources. let's call this "RESUSE_CLIENT". > Why add an additional protocol to the stack? > > TCP / SMTP is two layers. > > TCP / HTTP / $Email-protocol-de-jure is three layers. > > UDP / HTTP / $Email-protocol-de-jusre is three layers. > > Why introduce an additional layer? i disagree. i think this is more like it about the current email system: RESXCH_SERVER / RESUSE_SERVER / RESXCH_CLIENT / RESUSE_CLIENT it's 4 different layers to exchange mail between people. but if we plug HTTP* in the mix, it because only 3 different layers: HTTP* / RESUSE_SERVER / HTTP* / RESUSE_CLIENT and it is even nicer for when HTTP* is plugged, because it is also the protocol used for most of internet's traffic (web browsing). so basically total expected number of protocols/layers used in the universe, per second, will be much less if we, on planet earth, use a mail system that uses HTTP* instead of RESXCH_*.
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On Fri, Aug 21, 2020 at 3:37 PM Grant Taylor wrote: > > On 8/21/20 6:37 AM, Rich Freeman wrote: > > This stuff can be interesting to discuss, but email is SO entrenched > > that I don't see any of this changing because of all the legacy issues. > > You would need to offer something that is both easier and better to > > use to such a degree that people are willing to change. > > "something that is /both/ *easier* and *better*". > > That's a VERY tall bar to clear. Hence I'm going to basically reply to only a few lines of your email - this isn't going to go anywhere so I really don't want to get into an endless argument about why people use webservices/etc. > What does JSON / XML / etc. on top of HTTP(S) provide that SMTP doesn't > provide? It is what just about every other modern application in existence uses. I'm sure there are hundreds of articles on the reasons for this that are far better than anything I'd come up with here. > > ... use more standard libraries/etc. > > There are many libraries to interface with SMTP. And they don't work for anything OTHER than SMTP. A library for JSON/webservices/whatever works for half the applications being written today. > > Stop focusing on HTTP(S) for a few minutes. > > If you were to create a new protocol from the ground up, why would you > introduce additional layers into the application stack — which represent > additional dependencies and complications — if you didn't have to? > > Not everything is a web server. Not everything can benefit from being > forced through a web server. This is simple. This is how just about EVERYBODY does it these days. http works as a transport mechanism. And obviously when I say http I mean http/https with the latter being preferred. That is the beauty of standards like this - somebody else figured out SSL on top of HTTP so we don't need an email-specific reimplementation of that. I mean, why use TCP? Why not use something more tailored to email? TCP probably has a dozen optional features that nobody uses with email, so why implement all that just to send email? The answer is that it makes WAY more sense to recycle a standard protocol than to invent one. If SMTP didn't exist, and we needed a way to post a bunch of data to a remote server, you'd use HTTP, because it already works. > > Most of the benefits only accrue if you adopt PKI as well, which is > > a problem that is hard to scale out to the masses. > > PKI is relatively easy to scale. (For a given value.) Hence the > thriving CA industry. > > Doing so in a trust worthy manner is the problem. Especially if you > want to avoid a single point of control / failure / GEO political / etc. It isn't just the trust problem. It is also the ease of use problem. Let's assume you trust all the CAs out there. Why isn't anybody using SSL for email? It is built into half the email clients out there. The problem is that exchanging certificates at an individual level is hard. Oh, for an organization with an IT department it isn't a problem. The IT department can just issue a certificate to everybody, generating keys for them. Then it can stick all the certs in some kind of directory that their email clients all use. Getting your cousin to use SSL certs for their email is a whole different matter. Heck, just using it between two different companies, both supported by professional IT depts, is a pain. You could use TLS for the transport itself and have the email servers configured to only use TLS for those domains, but actually doing E2E encryption is hard with email because there aren't a lot of standards for key exchange across unrelated orgs. > If we throw all the trusted CAs out the window, how do you /bootstrap/ > encrypted communications? - My personal opinion is DNS. > DNSSEC is: 1. Still poorly supported even where it makes sense. 2. A great hypothetical solution for the 0.002% of email users who own a domain name, like you and me. -- Rich
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
thanks. highly appreciate your time. to save
space i'll skip parts where i fully agree
with/happily-learned.
(e.g. loop detection; good reminder, i wasn't
thinking about it. plus didn't know of acronyms
DSN, MDNs, etc; nice keywords for further
googing).
‐‐‐ Original Message ‐‐‐
On Friday, August 21, 2020 8:59 PM, Grant Taylor
wrote:
> On 8/20/20 7:39 PM, Caveman Al Toraboran wrote:
>
> > 1. receipt by final mail server (mandatory).
> >
>
> You're missing the point that each and every single server along the
> path between the original submission server and the final destination
> server is on the hook for delivery of the message -or- notification of
> it's failure back to the purported sender address. So "final mail
> server" is not sufficient.
i was thinking (and still) if such relay-by-relay
delivery increases probability of error by a
factor of n (n = number of relays in the middle).
e.g. probability of accidental silent mail loss is
if one, or more, accidentally said "yes got it!"
but actually didn't. i.e.:
Pr(silent loss) =
sum_{k=1}^n
{n choose k}
* Pr(mistake)**k
* Pr(no mistake)**{n-k}
n = number of relays in the middle.
* = mult.
** = exponent.
i wonder if it would be better if only the entry
relay aims at the confirmation from the terminal
server? this way we won't need to assume that
relays in the middle are honouring their guarantees,
hence the probability above would be smaller since
k is limited up to 2 despite n's growth.
> Of course, there are servers that go against the RFC "MUST" directives
> and either don't safely commit messages to disk /before/ saying
> "Okay..." and / or don't deliver failure messages.
care to point part of the rfc that defines "safe"
commit to disk? e.g. how far does the rfc expect
us to go? should we execute `sync`'s equivalent
to ensure that data is actually written on disk
and is not in operating system's file system write
buffer?
> Signing will be of somewhat limited value as it will quite likely be
> subject to the same problem that DMARC / ARC suffer from now. Mail
> servers can sign what they receive. But in doing so, they alter what is
> sent to include their signature. As such, the data that the next server
> receives is different. The real problem is working backwards. Down
> stream servers don't have a reliable way to undo what upstream servers
> have done to be able to get back to the original message to validate
> signatures.
onion signatures? e.g. message is wrapped around
several layers of signatures for every relay in
the path?
> > this way we can have group-level rules.
>
> I'm not quite sure what you mean by group-level rules in this context.
e.g. whitelisting, tagging, spam filtration,
prioritizing, etc, based on entities that
onion-signed the message.
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
‐‐‐ Original Message ‐‐‐ On Friday, August 21, 2020 4:28 PM, Wols Lists wrote: > You're re-inventing the wheel. yes, i do consider re-inventing octagonal wheels. though this wasn't my point here. here, i'm just "asking" to see what makes the "safely stored" guarantee. perhaps i should've asked more directly (and yes, i know these are not new features). > > 1. receipt by final mail server (mandatory). > > > > This is part of SMTP already, in that each server (post office) > acknowledges that the message has been received AND SAFELY STORED. > Without that last guarantee, "receipt by the server" isn't worth > diddley-squat. got any specific definition of what makes a storage "guaranteed"? e.g. what kind of tests does the mail server do in order to say "yup, i can now guarantee this is stored safely!"? > > the job of a relay would be to optionally add some > > metadata (e.g. maybe describing sender's role) and > > sign the whole thing (e.g. by company's private > > key). this way we can have group-level rules. > > Except that SMTP allows for the fact that a message may (or may not) > pass through several post-offices on the way. The old internet thing of > "don't assume any computer will survive a nuclear attack - take whatever > route you can find ..." so there is no guarantee that a relay going in > one direction will even see a message going back in the other. so? not sure how this relates to what i said. i guess you think that i meant that a relay should be mandatory? or maybe i'm misunderstanding your point? (yes, a relay doesn't have to be used. i'm just describing some uses of relays that i think make sense. (1) indicate trust hierarchy, (2) offload mail delivery so that i can close my laptop and let the relay have fun with the retries. not sure there is any other use. anyone?)
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 8/21/20 11:54 AM, Caveman Al Toraboran wrote: thanks. highly appreciate your time. to save space i'll skip parts where i fully agree with/happily-learned. You're welcome. (e.g. loop detection; good reminder, i wasn't thinking about it. plus didn't know of acronyms DSN, MDNs, etc; nice keywords for further googing). ;-) i was thinking (and still) if such relay-by-relay delivery increases probability of error by a factor of n (n = number of relays in the middle). e.g. probability of accidental silent mail loss is if one, or more, accidentally said "yes got it!" but actually didn't. i.e.: It definitely won't be a factor of n, where n is the number of relays. i wonder if it would be better if only the entry relay aims at the confirmation from the terminal server? this way we won't need to assume that relays in the middle are honouring their guarantees, hence the probability above would be smaller since k is limited up to 2 despite n's growth. Nope. Each and every server MUST behave the same way. care to point part of the rfc that defines "safe" commit to disk? e.g. how far does the rfc expect us to go? should we execute `sync`'s equivalent to ensure that data is actually written on disk and is not in operating system's file system write buffer? TL;DR: Yes on all accounts. See the recent reply about guarantee and relays for more details. onion signatures? e.g. message is wrapped around several layers of signatures for every relay in the path? That doesn't help the problem. We sort of have an onion already. It's quite likely possible to validate the signature of the immediate sending server. But how does the receiving server know how to undo any modifications that the immediate sending server made to be able to validate the previous sending server's signature? Rinse, later, repeat out multiple levels. What if some of the servers signs what they send vs what they receive? These are the types of problems that don't currently have a solution. e.g. whitelisting, tagging, spam filtration, prioritizing, etc, based on entities that onion-signed the message. How is doing those things based on signature different than doing them based on the system sending to you? The only thing that comes to mind is trusting an upstream signature, but not the immediate sending system. But, you have to unwrap the onion to be able to validate the signature. But to do that you need to know what the server(s) downstream of the signature being validated did to the message. Some of this is a one way trap door without any indication of what each trap door did to the message. -- Grant. . . . unix || die
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 8/21/20 11:01 AM, Caveman Al Toraboran wrote: yes, i do consider re-inventing octagonal wheels. I think that it's occasionally a good thing to have a thought experiment about how $THING might be made better. It's probably good to have discussions around green feel types of replacements. But it's important to eventually assess the pros and cons of the old (as it exists), the new (as from green field), and the transition between the two. Sometimes the new doesn't warrant the transition, but it does provide an option that might be worth augmenting into the old. If nothing else, it's good to have the discussions and be able to answer why something was done or chosen to remain the same. here, i'm just "asking" to see what makes the "safely stored" guarantee. MTAs are supposed to be written such that they commit the message to persistent storage medium (disk) before returning an OK message to the sending server. There is some nebulous area around what that actually means. But the idea is that the receiving server believes, in good faith, that it has committed the message to persistent storage. Usually this involves writing the message to disk, probably via a buffered channel, and then issued system calls to ask the OS to flush the buffer to disk. Is there room for error? Probably. Had the server made (more than) reasonable effort to commit the message to disk? Yes. The point being, don't acknowledge receipt of the message while the message is only in the MTA's memory buffer. Take some steps to commit it to persistent storage. That being said, there are some questionable servers / configurations that will bypass this safety step in the name of performance. And they /do/ /loose/ /email/ as a negative side effect if (when) they do crash. This is a non-default behavior that has been explicitly chosen by the administrators to violate the SMTP specification. Some MTAs will log a warning that they are configured to violate RFCs. got any specific definition of what makes a storage "guaranteed"? e.g. what kind of tests does the mail server do in order to say "yup, i can now guarantee this is stored safely!"? It's more that they do something safe (write the message to disk) instead of risky (only store it in memory). Everything can fail at some point. It's a matter of what and how many reasonable steps did you take to be safe. Read: Don't cut corners and do something risky. i guess you think that i meant that a relay should be mandatory? Sending / outbound SMTP servers /are/ a relay for any messages not destined to the local server. There is almost always at least two SMTP servers involved in any given email delivery. All but the final receiving system is a relay. (yes, a relay doesn't have to be used. i'm just describing some uses of relays that i think make sense. (1) indicate trust hierarchy, (2) offload mail delivery so that i can close my laptop and let the relay have fun with the retries. not sure there is any other use. anyone?) There are many uses for email relays. A common one, and best practice, is to have an /inbound/ relay, commonly known as a backup email server. The idea being it can receive inbound messages while the primary email server is down (presumably for maintenance). Many SaaS Email Service Providers (ESPs) /are/ relay servers. They receive email from someone and send it to someone else. A number of email hygiene appliances function as an email relay between the world and your ultimate internal email server. Services that filter inbound email qualify here too. It is common, and I think it's best practice, to have web applications send email via localhost, which is usually a relay to a more capable hub email server which sends outbound email. Both of these layers are relays. A relay is the same thing for email that a router is for a network. -- Grant. . . . unix || die
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 8/21/20 6:37 AM, Rich Freeman wrote: This stuff can be interesting to discuss, but email is SO entrenched that I don't see any of this changing because of all the legacy issues. You would need to offer something that is both easier and better to use to such a degree that people are willing to change. "something that is /both/ *easier* and *better*". That's a VERY tall bar to clear. 1. Modernizing the actual data exchange using http/etc. I don't think you'll get much argument that SMTP isn't the way somebody would do things if they were designing everything from scratch. SMTP may not be the best, but I do think that it has some merits. Merits that the previously mentioned HTTP/2 alternative misses. Remember, SMTP is a protocol, or a set of rules, for how two email servers are supposed to exchange email. These rules include the following: 1) Proper server greeting, used to identify the features that can be used. (HELO for SMTP and EHLO for ESMTP.) 2) Identifying the sender. 3) Identifying the recipient(s). 4) Sending the message payload. Each of these discrete steps is a bi-directional communication. Communication that can alter all subsequent aspects of a given SMTP exchange. The server could be an older server that only speaks SMTP and as such doesn't support STARTTLS. Thus causing the client to /not/ try to use STARTTLS in accordance with current best practices. Other ESMTP features come into play here like message size, 8-bit, notification features, etc. The receiving server has the ability to assess the sending server's behavior at each point to use that as an indication to know if the sender is a legitimate email server or something more nefarious and undesirable. -- There is a surprising amount of email hygiene based on sending server's behavior. The receiving server may carte blanch reject messages from specific senders / sending domains and / or specific recipients, particularly non-existent recipients. SMTP has low overhead in that the message body is not transferred from the sending server to the receiving server if any part of steps 1-3 aren't satisfactory to the receiving server. I'm not an API expert but I imagine that just about any of the modern alternatives would be an improvement. Maybe. Maybe not. What is the actual problem with SMTP as it is? What /need/ to be improved? What could benefit from improvement even if it's not /needed/? http would be a pretty likely protocol to handle the transportation, Why HTTP? Did you mean to imply HTTPS? Why add an additional protocol to the stack? TCP / SMTP is two layers. TCP / HTTP / $Email-protocol-de-jure is three layers. UDP / HTTP / $Email-protocol-de-jusre is three layers. Why introduce an additional layer? Why introduce an additional dependency in the application stack? Why co-mingle email with other non-email traffic? -- Or are you wanting to run HTTP(S) on TCP port 25? likely with something like JSON/xml/etc carrying the payload. Why add an additional encapsulation on top of the additional layer? What does JSON / XML / etc. on top of HTTP(S) provide that SMTP doesn't provide? What is the gain? Is the gain worth the cost of doing so? You might want to tweak things slightly so that recipient validity can be tested prior to transferring data. Definitely. (See above for more details as to why.) But now you are doing multiple exchanges. If we extrapolate this out to also include sender validation, and probably hello messages, you are back to four or more exchanges. How is this better than SMTP on TCP port 25? What is the benefit? What does it cost to get this benefit? A mail server doesn't want to accept a 20MB encrypted payload if it can bounce the whole message based on the recipient or a non-authenticated sender or whatever. Which is one of the many reasons why there are multiple exchanges back and forth. However, in principle there are plenty of modern ways to build a transport protocol that would be an improvement on SMTP ... Please elaborate. Please include what network layer (3) protocol(s) you want to use and why. Please include what application layer (7) protocol(s) you want to use and why. Please include any encoding / encapsulation you want to use and why. ... use more standard libraries/etc. There are many libraries to interface with SMTP. Also, handing a message off to an SMTP server alleviates the application from having to deal with failed deliveries, queuing, and retrying. Why add that complexity into each and every application? Especially if you don't have to? Note: Pulling it in via a library is still indirectly adding it to each and every application. How is SMTP /not/ standard? Also: https://xkcd.com/927/ -- Standards I think this is actually the part of your proposal that might be more likely to take off. You could have a new type of MX field in DNS ... So yet more complexity in
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 8/20/20 7:39 PM, Caveman Al Toraboran wrote: 1. receipt by final mail server (mandatory). You're missing the point that each and every single server along the path between the original submission server and the final destination server is on the hook for delivery of the message -or- notification of it's failure back to the purported sender address. So "final mail server" is not sufficient. Each receiving server in the chain tells the sending server a definitive message that means "Okay, I've received the message and I will dutifully pass it on to the next server -or- report a problem." The RFCs defining SMTP don't consider a server crashing / power failing / etc. after saying "Okay..." as sufficient reason to fail to perform the job. Even in the event of a crash / power fail, the receiving server MUST process the email when it recovers. Of course, there are servers that go against the RFC "MUST" directives and either don't safely commit messages to disk /before/ saying "Okay..." and / or don't deliver failure messages. There are still other servers that don't accept such incoming failure notices. Both of which are against RFC "MUST" directives and as such violating the SMTP specification and thereby breaking interoperability. Particularly the "trust" part there of. These failure notifications have standardized on Delivery Status Notification, a.k.a. DSN, and historically called a "bounce". There has been evolution from many disparate formats of a bounce to an industry standard DSN. Said industry standard DSN is defined in RFCs. DSNs MUST be non-optional for failure. The only exception is if the sending system uses an extra option to specify that a DSN should /not/ be sent in the event of a failure. Receiving systems are compelled to send DSNs in the absence of said option. 2. receipt by end user(s) (optional). 3. opening by end user(s) (optional). These notifications are generally considered to be Message Disposition Notifications, and are optional. Further, the client is what sends MDNs /if/ it chooses to do so. MDNs are even more unreliable than DSNs. (1) is required by the server, else mail will be retransmitted from source relay(s) (or client if done directly). (2) is optional by final server, (3) is optional by end user's client. #1 is generated by RFC compliant servers. Not all servers are RFC compliant. #2 and #3 are generated by end user clients. Not all clients are willing to to do so. the job of a relay would be to optionally add some metadata This really isn't optional. *SOMETHING* /standard/ *MUST* be added. If for no other reason than to detect and prevent loops. (e.g. maybe describing sender's role) and sign the whole thing (e.g. by company's private key). I would suggest a /server's/ private key, and *NOT* the company's private key. There is a HUGE difference in potential for private key exposure. If one server is compromised, the entire company could be crippled if the /company's/ private key is used. Conversely, if the /server's/ private key is used, then /only/ /the/ /server/ is compromised. It is quite possible to have the company's public key published such that the company's internal CA can issue individual keys to each server. Signing will be of somewhat limited value as it will quite likely be subject to the same problem that DMARC / ARC suffer from now. Mail servers can sign what they receive. But in doing so, they alter what is sent to include their signature. As such, the data that the next server receives is different. The real problem is working backwards. Down stream servers don't have a reliable way to undo what upstream servers have done to be able to get back to the original message to validate signatures. There is also the fact that various fields of the email are allowed to have specific trivial modifications made to them, such a line wrapping or character encoding conversion. Such changes don't alter the content of the message, but they do alter the bit pattern of the message. These types of changes are even more difficult to detect and unroll as part of signature validation. this way we can have group-level rules. I'm not quite sure what you mean by group-level rules in this context. -- Grant. . . . unix || die
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On Thu, Aug 20, 2020 at 9:39 PM Caveman Al Toraboran wrote: > > ‐‐‐ Original Message ‐‐‐ > On Thursday, August 20, 2020 11:41 AM, antlists > wrote: > > > Will that python script allow for the situation that the message is > > received, but the message was NOT safely stored for onwards transmission > > before the receiver crashed, and as such the message has not been > > SUCCESSFULLY received? > > > > SMTP has lots of things specifically meant to ensure messages survive > > the internet jungle on their journey ... > > thanks for the point. would it suffice if we have > these notifications: > > 1. receipt by final mail server (mandatory). > 2. receipt by end user(s) (optional). > 3. opening by end user(s) (optional). > I don't really want to get into the gory details and have only skimmed the discussion, but my sense is that you're talking about a new way to deliver mail that makes two sorts of changes: 1. Modernizing the actual data exchange using http/etc. 2. Changing the model around email moving to something more PKI-based/etc and redefining what an email address is. This stuff can be interesting to discuss, but email is SO entrenched that I don't see any of this changing because of all the legacy issues. You would need to offer something that is both easier and better to use to such a degree that people are willing to change. However, I'll comment on both of these issues: 1. Modernizing the actual data exchange using http/etc. I don't think you'll get much argument that SMTP isn't the way somebody would do things if they were designing everything from scratch. I'm not an API expert but I imagine that just about any of the modern alternatives would be an improvement. http would be a pretty likely protocol to handle the transportation, likely with something like JSON/xml/etc carrying the payload. You might want to tweak things slightly so that recipient validity can be tested prior to transferring data. A mail server doesn't want to accept a 20MB encrypted payload if it can bounce the whole message based on the recipient or a non-authenticated sender or whatever. However, in principle there are plenty of modern ways to build a transport protocol that would be an improvement on SMTP and use more standard libraries/etc. I think this is actually the part of your proposal that might be more likely to take off. You could have a new type of MX field in DNS and if it is set then it defines servers that will use the new protocol, and you could have backup SMTP servers for legacy transition. You could change the transport side of things without redefining how email works, so the same messages are interoperable with both systems. On the flip side, while I think this is an easier change to make, I don't think it necessarily offers huge improvements. SMTP still gets the job done, and the new system wouldn't really be a major improvement from a sysadmin standpoint, so I don't see everybody running out to change. Just changing the transport protocols doesn't provide any anti-spam/etc benefits. SMTP already supports TLS so it isn't any more secure. No doubt if we were starting from scratch this is how it would be done, but it isn't enough of a reason to change. That said, if there were even modest benefits for large mail providers I could see it taking off simply because it could leverage a lot of existing libraries and standards, and could have legacy compatibility. 2. Changing the model around email moving to something more PKI-based/etc and redefining what an email address is. This is where you could have revolutionary improvements for privacy/spam/etc. However, it completely breaks everything that exists with email today, so it is a hard change. Most of the benefits only accrue if you adopt PKI as well, which is a problem that is hard to scale out to the masses. If somebody comes up with a way to make PKI take off in a real way, then I could see something like this taking off eventually. Otherwise, this stuff is only going to be of interest to the sorts who use gpg to sign their email, and nobody else. -- Rich
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 21/08/20 02:39, Caveman Al Toraboran wrote: > ‐‐‐ Original Message ‐‐‐ > On Thursday, August 20, 2020 11:41 AM, antlists > wrote: > >> Will that python script allow for the situation that the message is >> received, but the message was NOT safely stored for onwards transmission >> before the receiver crashed, and as such the message has not been >> SUCCESSFULLY received? >> >> SMTP has lots of things specifically meant to ensure messages survive >> the internet jungle on their journey ... > > thanks for the point. would it suffice if we have > these notifications: You're re-inventing the wheel. > > 1. receipt by final mail server (mandatory). This is part of SMTP already, in that each server (post office) acknowledges that the message has been received AND SAFELY STORED. Without that last guarantee, "receipt by the server" isn't worth diddley-squat. > 2. receipt by end user(s) (optional). This is part of current mail protocol - dunno what it's called but I can switch on a flag in Thunderbird, and I will get a message back saying my email is in the recipient's inbox. > 3. opening by end user(s) (optional). Likewise, I will get a notification that the email has been "read". > > ? > > > > (1) is required by the server, else mail will be > retransmitted from source relay(s) (or client if > done directly). (2) is optional by final server, > (3) is optional by end user's client. > > the job of a relay would be to optionally add some > metadata (e.g. maybe describing sender's role) and > sign the whole thing (e.g. by company's private > key). this way we can have group-level rules. > Except that SMTP allows for the fact that a message may (or may not) pass through several post-offices on the way. The old internet thing of "don't assume any computer will survive a nuclear attack - take whatever route you can find ..." so there is no guarantee that a relay going in one direction will even see a message going back in the other. And as an example of how hard this is, look at what a mess the telcos have made of SMS, which is basically the same thing! How often on New Year's Eve do (or did) the system fall over so all the "Happy New Year" messages either disappeared, or arrived several days late ... Cheers, Wol
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
‐‐‐ Original Message ‐‐‐ On Thursday, August 20, 2020 11:41 AM, antlists wrote: > Will that python script allow for the situation that the message is > received, but the message was NOT safely stored for onwards transmission > before the receiver crashed, and as such the message has not been > SUCCESSFULLY received? > > SMTP has lots of things specifically meant to ensure messages survive > the internet jungle on their journey ... thanks for the point. would it suffice if we have these notifications: 1. receipt by final mail server (mandatory). 2. receipt by end user(s) (optional). 3. opening by end user(s) (optional). ? (1) is required by the server, else mail will be retransmitted from source relay(s) (or client if done directly). (2) is optional by final server, (3) is optional by end user's client. the job of a relay would be to optionally add some metadata (e.g. maybe describing sender's role) and sign the whole thing (e.g. by company's private key). this way we can have group-level rules.
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 19/08/2020 16:19, Caveman Al Toraboran wrote: ‐‐‐ Original Message ‐‐‐ On Wednesday, August 19, 2020 7:10 PM, Grant Taylor wrote: Per protocol specification, SMTP is EXTREMELY robust. It will retry delivery, nominally once an hour, for up to five (or seven) days. That's 120-168 delivery attempts. Further, SMTP implementations MUST (RFC sense of the word) deliver a notification back to the sender if the implementation was unable to delivery a message. this queue re-transmission, and failure notification, can be done with a small python script. Will that python script allow for the situation that the message is received, but the message was NOT safely stored for onwards transmission before the receiver crashed, and as such the message has not been SUCCESSFULLY received? SMTP has lots of things specifically meant to ensure messages survive the internet jungle on their journey ... Cheers, Wol
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
‐‐‐ Original Message ‐‐‐ On Wednesday, August 19, 2020 7:10 PM, Grant Taylor wrote: > Per protocol specification, SMTP is EXTREMELY robust. > > It will retry delivery, nominally once an hour, for up to five (or > seven) days. That's 120-168 delivery attempts. > > Further, SMTP implementations MUST (RFC sense of the word) deliver a > notification back to the sender if the implementation was unable to > delivery a message. this queue re-transmission, and failure notification, can be done with a small python script.
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
‐‐‐ Original Message ‐‐‐ On Wednesday, August 19, 2020 12:25 PM, Ashley Dixon wrote: > I don't think you fully understand Grant's point. Whilst HTTP(/2) may be more > featureful for serving web pages, it makes absolutely no sense to use for > anything but. Protocol age absolutely is not irrelevant: SMTP has been > ubiquitous in mail transportation for many years, and thus, every single mail > client supports it pretty close to the RFC. Moreover, as Grant mentioned in > the > previous message, it is the only reliable method of reliably transferring > messages to and fro systems which, in most cases, differ quite vastly in every > element except their understanding of SMTP. there are two aspects: (1) backwards compatibility: sure, email is better if the goal is to deal with a large audience. but this is not necessarily my goal because i don't talk to everyone. and for rare cases when i need to send an archaic email, i can just open gmail.com, protonmail.com, etc, and use their web gui. (2) technically irrespective of backwards compatibility: there is no doubt that a http/2-based mail system will be much more efficient than smtp's archaic format where all attachments are base64-ed into giant mono text balls. the only reason we're using smtp's archaic text base64-ed balls is pure history. but, fundamentally, contents of emails are in the same scope as of web pages. so emails' contents is not alien to http/2. the only reason we don't have http/2-based mail is pure history, and that people resist change. > Interoperability is the entire point of protocol standardisation in the first > place, and if you're going to suggest a revision, or complete overhaul, of a > standard as well-understood as SMTP, you need to provide extremely compelling > evidence which supports your proposed replacement. So far, you haven't done > that. SMTP can be tricky and unwieldy to configure on certain (most) > implementations, but that does not indicate a lack of features. The complete > opposite, in fact. but i'm not proposing a standard for "everyone". it's about my case of using cheap vps with untrusty admins. so i don't "need" to present any compelling evidence, because i don't care about the approval of these standardization organizations. worst case scenario i can shove an smtp-client leg into gmail and call it a day, and thrive with only 1 listening tcp port (for https). in fact, if possible, even if we wanted to go as far as changing a protocol, we better create our own standards free from them, specially with the likes of w3c which have absolutely no respect for us (they slapped us with drm despite our cries, simply because netflex/google paid enough). currently we're being treated like sheep and get told which disgusting protocols to use.
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
First, well said. On 8/19/20 2:27 AM, Ashley Dixon wrote: Apologies for my unintended verbosity. My subconscious _really_ wanted to point out that SMTP is (generally) RELIABLE. ;-) Second, this is an understatement. Per protocol specification, SMTP is EXTREMELY robust. It will retry delivery, nominally once an hour, for up to five (or seven) days. That's 120-168 delivery attempts. Further, SMTP implementations MUST (RFC sense of the word) deliver a notification back to the sender if the implementation was unable to delivery a message. SMTP's ability to deliver email without end-to-end connectivity is almost unmatched. UUCP and NNTP are there with it. -- Grant. . . . unix || die
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On Wed, Aug 19, 2020 at 09:25:11AM +0100, Ashley Dixon wrote: > reliable method of reliably transferring Apologies for my unintended verbosity. My subconscious _really_ wanted to point out that SMTP is (generally) RELIABLE. ;-) -- Ashley Dixon suugaku.co.uk 2A9A 4117 DA96 D18A 8A7B B0D2 A30E BF25 F290 A8AA signature.asc Description: PGP signature
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On Wed, Aug 19, 2020 at 01:34:46AM +, Caveman Al Toraboran wrote: > sure, smtp is older, but protocol age is > irrelevant. > > right now http/2 is more developed and much more > efficient (e.g. compressed binary, pipelining, > single connection multiplexing, encryption by > default). even http1.4 was a more efficient > replacement. I don't think you fully understand Grant's point. Whilst HTTP(/2) may be more featureful for serving web pages, it makes absolutely no sense to use for anything but. Protocol age absolutely _is not_ irrelevant: SMTP has been ubiquitous in mail transportation for many years, and thus, every single mail client supports it pretty close to the RFC. Moreover, as Grant mentioned in the previous message, it is the only reliable method of reliably transferring messages to and fro systems which, in most cases, differ quite vastly in every element _except_ their understanding of SMTP. Interoperability is the entire point of protocol standardisation in the first place, and if you're going to suggest a revision, or complete overhaul, of a standard as well-understood as SMTP, you need to provide extremely compelling evidence which supports your proposed replacement. So far, you haven't done that. SMTP can be tricky and unwieldy to configure on certain (most) implementations, but that does not indicate a lack of features. The complete opposite, in fact. With that said, if you'd like to propose a standard and write a reference implementation, that is how most new ideas in computing arise. I don't think anyone would attempt to stop you from doing that; we're merely suggesting that your current idea of the HTTP-for-mail protocol is flawed. -- Ashley Dixon suugaku.co.uk 2A9A 4117 DA96 D18A 8A7B B0D2 A30E BF25 F290 A8AA signature.asc Description: PGP signature
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
> So you want to change from a ubiquitous protocol that is supported by > many Many MANY devices to niche protocol that has a non-trivial > installation / configuration curve. 1st half is "yes", 2nd half is "no" (mine is simpler). > > then, verify messages by mailing their supplied email a confirmation > > message. > > And then you want to take what people send you, turn around and send > unsolicited messages based on it — this is the icing on the cake — using > the protocol that you are trying to avoid. > > It's only a matter of time before someone uses your Tor hidden service > as a vector to send spam. — Joe Job comes to mind. this was just a quick thought. maybe adding a captcha is enough in the contact-us html submission form. this is not a permanent element. just a temporary solution to get messages from the lagging wold. > > redundant as in containing concepts already done in other protocols, > > so smtp has many re-invented wheels that are already invented in > > existing protocols. > > Please elaborate. Please be careful to provide information about /when/ > the protocols that SMTP is supposedly redundant of were developed. > > I suspect that you will quickly find that SMTP predates the protocols > that you are stating it's redundant of. I further suspect that you will > find that SMTP predates them by 10, or more likely 20, if not 30 years. > > Here's a hint. SMTP was ~82. HTTP (1.0) was ~89. We couldn't post > thing in HTTP 1.0. HTTP 2.0 was ~15. sure, smtp is older, but protocol age is irrelevant. right now http/2 is more developed and much more efficient (e.g. compressed binary, pipelining, single connection multiplexing, encryption by default). even http1.4 was a more efficient replacement. > > imo, smtp should be a much-higher level protocol defined purely on > > top of how dns and http/2. > > How do you get any higher layer than the application layer? it's a matter of definition. if we define http/2 as an application layer protocol, and we define "depends on" as "on layer below", then mail is necessarily above the application layer. anyway, this whole osi/internet model is not accurate and many protocols ignore it. i propose this model (fireball model?): 6. app layer(usual drill..) 5. resource layer (exch. by res.; http/2) 4. socket layer (socke ids; tcp/udp/etc ports) 3. end-to-end layer (inter-lan; e.g. ip) 2. hop layer(intra-lan; e.g. mac addr.) 1. physical layer (electromagnetic fluctuations) http/2 is morphing into general "resource layer" where data is exchanged between difference resources. email is just a special case of this inter-resource communication where some resources are humans. > > e.g. for mail submission, there is no need for a separate > > application-layer protocol as we can simply use http/2. because the > > concept of mail submission is a special case of data submission, > > which is already in http/2. > > HTTP /now/ has a way to submit data. HTTP didn't exist when SMTP was > developed. Further, HTTP didn't have the ability to submit data for a > while. true, but that's history. now http/2 is better for resource exchange than smtp. > If you look at multiple layers of the network stack, HTTP and SMTP are > both at the application layer. Now you are suggesting moving equal > peers so that mail is subservient of / dependent on web? yes. > Does HTTP or the web servers have the ability to queue messages to send > between systems? How many web servers handle routing of incoming > messages to send to other servers? How dynamic is this web server > configuration to allow servers for two people who have never exchanged > email to do so? > > This routing, queuing, and many more features are baked into the email > ecosystem. Features that I find decidedly lacking in the web ecosystem. of course. it's called web application; it can do all fancy queueing and routing you want. basically the only part of current "email system" that is not redundant is the part where it is a "mail web app". every other part (e.g. protocol for data exchange) is redundant and inferior to what exists (e.g. http/2). i am considering to make an uwsgi ptyhon script for my personal use. there is absolutely nothing really challenging about the concept of mail routing and queueing. > > here is a more complete example of what i mean: > > > > 1. we lookup MX records to identify smtp servers to submit mails to. > > 2. from the response to that lookup we get a domain name, say, > > mail.dom.com. > > #1 and 2 are par for what we have today. No improvement. yes. dns is ok for now. i never said dns is redundant. > > 3. then, the standard defines a http/2 request format to submit > > the mail. > > Given how things never die on the Internet, you're going to need both > SMTP /and/ HTTP /on/ /the/ /email/ /server/ to be able to send & receive > email with people on the Internet. no, but that's how most of today's mail servers are. e.g. they
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 8/18/20 4:25 PM, james wrote: I find all of this *fascinating*. ;-) So I have threads from 7/28 and others that attempt to discover the (gentoo) packages necessary to run my own email services. I have (2) R.Pi4 (8Gram) and (2) more on order to build out complete mail/DNS/security for a small/moderate number of folks to use. Just me to start/test/debug. I expect that, other than CPU speed, the four systems that you have are probably overkill. The CPUs may, or may not, be slow depending on the number of messages you want to handle a day. They are probably quite adequate to start with for personal email. I'd like to build out Grant(Taylor) and Ashley's solution for further learning and testing, on Rpi4 based gentoo systems. robust security and reasonable straightforward (gentoo) admin, is my goal. Sorry to be pedantic, but please list out what you mean by "robust security". I ask more as an exercise for you to think about, and — more importantly — document goals that you'd like to achieve. This documentation may seem somewhat silly, but as has been mentioned multiple times in this thread, there are a LOT of options. So, documenting your desires helps reduce compatible options and makes some choices for you. Don't worry if you find that your previous choice limits you. That will happen. You then need to decide if you want to live with the choice -or- go back a few steps and change your choice. Note: Changing your choice is perfectly fine. Call it what it is, a change, and deal with it. The documentation you're creating is sort of a proto / alpha checklist of goals that you want to achieve. Can either or both of you concisely list what I'd need (the ebuild list) to implement a basic, but complete, secure email system, as delineated in your recent posts? I'd be willing to document both the build and running tests, for the greater good of the gentoo community. I will have to collect a list and get back to you. Note: My list will be biased towards my choices. Given that I do things differently than many email admins, my list is likely to be considerably different than others. If there is interests in the tests and results. I think that quality documentation is always a laudable goal. Remember, I started this some months ago, cause Frontier does not even offer basic email services. I hate all thing cloud (deep desire to be 100% independent of the cloud) and want the ability to remotely retrieve mails and send emails through *my email systems*. I am certainly not alone, as some have sent me private email, with similar desires. Fair enough. The big corporations are trying to destroy and remove standards based email from the internet. I haven't seen much where the big players are trying to actively destroy standards based protocols. I have seen where the big players are requiring higher and higher standards than they did 5 / 10 / 15 years ago. Note: This is neither breaking nor removing standards. If anything, it's adding new public standards and making people adhere to them. Analogy: Some states in the U.S.A. aren't removing old vehicles from the road. They are however introducing requirements for vehicles to adhere to more strict emission standards -or- register as historic vehicles which imposes some restrictions. For me, it is my most useful, important and most desired feature of the internet. I find email (SMTP(S) & IMAPS) and Usenet news (NNTP(S)) to be two of the most critical Internet services to me. The web (HTTP(S)) is extremely convenient. But I could live without the web, admittedly reluctantly. I'm ordering up (6) static IPs from Frontier. Will this be an 8-block (/29) of globally routed IPs? Or is it going to be 6 random IPs in a larger co-mingled IP network? Start inquiring of Frontier about how to configure Reverse DNS. Chances are good that Frontier will be familiar with RFC 2317 — Classless IN-ADDR.ARPA delegation. — If you're not familiar with it, I suggest you read RFC 2317. I'd also suggest starting inquiries of Frontier if they Shared Whois Project (SWIP) and / or RWhois. — My VPS provider doesn't offer SWIP or RWhois, and I wish that they did. — SWIP and / or RWhois are quite nice to have when it comes to making your IP(s) / block(s) stand out from other IP(s) / block(s) near yours. (Think in the same /24). Note: Many things on the Internet prefer for name servers to be in different /24 networks. So, having multiple on different IPs in the same /24 doesn't count to many people. At some point, I'll put another primary bandwidth provider under this, I would encourage you to start with a bandwidth provider that you plan to stick with for a number of years. (I know, things change. Do the best you can with the information you have at hand now, and deal with change if / when it comes.) I say this because it takes a fair bit of effort to turn up a mail
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 8/18/20 4:36 PM, Grant Taylor wrote: On 8/18/20 5:59 AM, Caveman Al Toraboran wrote: redundant as in containing concepts already done in other protocols, so smtp has many re-invented wheels that are already invented in existing protocols. Please elaborate.� Please be careful to provide information about /when/ the protocols that SMTP is supposedly redundant of were developed. I suspect that you will quickly find that SMTP predates the protocols that you are stating it's redundant of.� I further suspect that you will find that SMTP predates them by 10, or more likely 20, if not 30 years. Here's a hint.� SMTP was ~82.� HTTP (1.0) was ~89.� We couldn't post thing in HTTP 1.0.� HTTP 2.0 was ~15. basically smtp, as an application-layer protocol, is needless. We are all entitled to our own opinion. imo, smtp should be a much-higher level protocol defined purely on top of how dns and http/2. How do you get any higher layer than the application layer? e.g. for mail submission, there is no need for a separate application-layer protocol as we can simply use http/2.� because the concept of mail submission is a special case of data submission, which is already in http/2. HTTP /now/ has a way to submit data.� HTTP didn't exist when SMTP was developed.� Further, HTTP didn't have the ability to submit data for a while. If you look at multiple layers of the network stack, HTTP and SMTP are both at the application layer.� Now you are suggesting moving equal peers so that mail is subservient of / dependent on web? Does HTTP or the web servers have the ability to queue messages to send between systems?� How many web servers handle routing of incoming messages to send to other servers?� How dynamic is this web server configuration to allow servers for two people who have never exchanged email to do so? This routing, queuing, and many more features are baked into the email ecosystem.� Features that I find decidedly lacking in the web ecosystem. here is a more complete example of what i mean: 1. we lookup MX records to identify smtp servers to submit mails to. 2. from the response to that lookup we get a domain name, say, mail.dom.com. #1 and 2 are par for what we have today.� No improvement. 3. then, the standard defines a http/2 request format to submit the mail. Given how things never die on the Internet, you're going to need both SMTP /and/ HTTP /on/ /the/ /email/ /server/ to be able to send & receive email with people on the Internet. So you now have a HUGE net negative in that you have the existing service plus a new service.� You're in many ways doubling the exposure and security risk of email servers. an example of step (3) could be this: ���� https://mail.dom.com/from=...=...=...\ ���� =...=...=...=...\ ���� =... If you were to do this, you would NOT do it via GETs with URL parameters.� You would do it as POSTs. You will also have to find a way to deal with all the aspects of SMTP and RFC 822 email + mime.� I suspect that you will find this to be extremely tricky.� Especially if you try to avoid SMTP / RFC 822 semantics b/c SMTP is apparently a bad thing. How does your example scheme account for the fact that the SMTP envelope from address frequently diff from the RFC 822 From: header?� Remember, this very feature is used thousands of times per day.� So you have to have it. There's also many Many MANY other features of SMTP that are also used thousands of times a day that I'm seeing no evidence of. ���� i don't know how http/2 works.� do they have ���� POST requests?� if so maybe fields attach1, ���� attach2, ..., attachn can be submitted as file ���� uploads using POST. further, if we modify steps (1) and (2), we can generalise this concept into tor services.� e.g.� an email address simply becomes an onion address.� e.g. if vagzgdrh747aei0q.onion is the hidden service address of your mail server, then your email address could be written as (for convenience): ���� remco@vagzgdrh747aei0q.onion I ... I don't have the words.� Go run that idea past an SEO expert. Go ask people to drop their domain name in favor of a hash. I'm not going to hold my breath. How are you going to handle the billions of email clients that exist today, many of which will never be updated, to handle ToR?� You're going to have to have something to gateway old and new. That means that you're still going to have steps #1 and #2.� You can't get away from them without everybody and everything migrating to the new system.� Even then, chances are still extremely good that you're /still/ going to have #2. and when a "mail" client tries to submit you an email, it submits it by this url: ���� https://vagzgdrh747aei0q.onion/to=remco&...etc. I haven't the words. then, in order to authenticate a source, we simply use
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 8/18/20 4:30 AM, Ashley Dixon wrote: but nothing can replace it in terms of interoperability and convenience. That is an EXTREMELY important point. SMTP is a protocol that completely independent implementations can use to exchange messages with each other. You can set up gateways to enable different forms of messages to go to and come from SMTP. Things like this allow you to send a fax to an email gateway to an MMS gateway to receive a picture on your phone. There are /SO/ /MANY/ to / from email gateways (which effectively means SMTP) that are used each and every day to run the world. Perhaps the younger generation ... would almost-unanimously disagree, but your proposed solution doesn't really provide greater ease for you, or the people e-mailing you! mail(fire)ball provides something, but I'm quite sure "ease (of use)" is decidedly /NOT/ one of the things that it provides. -- Grant. . . . unix || die
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 8/18/20 5:59 AM, Caveman Al Toraboran wrote: redundant as in containing concepts already done in other protocols, so smtp has many re-invented wheels that are already invented in existing protocols. Please elaborate. Please be careful to provide information about /when/ the protocols that SMTP is supposedly redundant of were developed. I suspect that you will quickly find that SMTP predates the protocols that you are stating it's redundant of. I further suspect that you will find that SMTP predates them by 10, or more likely 20, if not 30 years. Here's a hint. SMTP was ~82. HTTP (1.0) was ~89. We couldn't post thing in HTTP 1.0. HTTP 2.0 was ~15. basically smtp, as an application-layer protocol, is needless. We are all entitled to our own opinion. imo, smtp should be a much-higher level protocol defined purely on top of how dns and http/2. How do you get any higher layer than the application layer? e.g. for mail submission, there is no need for a separate application-layer protocol as we can simply use http/2. because the concept of mail submission is a special case of data submission, which is already in http/2. HTTP /now/ has a way to submit data. HTTP didn't exist when SMTP was developed. Further, HTTP didn't have the ability to submit data for a while. If you look at multiple layers of the network stack, HTTP and SMTP are both at the application layer. Now you are suggesting moving equal peers so that mail is subservient of / dependent on web? Does HTTP or the web servers have the ability to queue messages to send between systems? How many web servers handle routing of incoming messages to send to other servers? How dynamic is this web server configuration to allow servers for two people who have never exchanged email to do so? This routing, queuing, and many more features are baked into the email ecosystem. Features that I find decidedly lacking in the web ecosystem. here is a more complete example of what i mean: 1. we lookup MX records to identify smtp servers to submit mails to. 2. from the response to that lookup we get a domain name, say, mail.dom.com. #1 and 2 are par for what we have today. No improvement. 3. then, the standard defines a http/2 request format to submit the mail. Given how things never die on the Internet, you're going to need both SMTP /and/ HTTP /on/ /the/ /email/ /server/ to be able to send & receive email with people on the Internet. So you now have a HUGE net negative in that you have the existing service plus a new service. You're in many ways doubling the exposure and security risk of email servers. an example of step (3) could be this: https://mail.dom.com/from=...=...=...\ =...=...=...=...\ =... If you were to do this, you would NOT do it via GETs with URL parameters. You would do it as POSTs. You will also have to find a way to deal with all the aspects of SMTP and RFC 822 email + mime. I suspect that you will find this to be extremely tricky. Especially if you try to avoid SMTP / RFC 822 semantics b/c SMTP is apparently a bad thing. How does your example scheme account for the fact that the SMTP envelope from address frequently diff from the RFC 822 From: header? Remember, this very feature is used thousands of times per day. So you have to have it. There's also many Many MANY other features of SMTP that are also used thousands of times a day that I'm seeing no evidence of. i don't know how http/2 works. do they have POST requests? if so maybe fields attach1, attach2, ..., attachn can be submitted as file uploads using POST. further, if we modify steps (1) and (2), we can generalise this concept into tor services. e.g. an email address simply becomes an onion address. e.g. if vagzgdrh747aei0q.onion is the hidden service address of your mail server, then your email address could be written as (for convenience): remco@vagzgdrh747aei0q.onion I ... I don't have the words. Go run that idea past an SEO expert. Go ask people to drop their domain name in favor of a hash. I'm not going to hold my breath. How are you going to handle the billions of email clients that exist today, many of which will never be updated, to handle ToR? You're going to have to have something to gateway old and new. That means that you're still going to have steps #1 and #2. You can't get away from them without everybody and everything migrating to the new system. Even then, chances are still extremely good that you're /still/ going to have #2. and when a "mail" client tries to submit you an email, it submits it by this url: https://vagzgdrh747aei0q.onion/to=remco&...etc. I haven't the words. then, in order to authenticate a source, we simply use public-private keys ... Because that has worked out so well and with so few problems in the past 25 years. ... to sign messages. Even /more/ unlikely to be
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 8/18/20 1:00 AM, Caveman Al Toraboran wrote: not specifically with a mail provider, but with other i.t. services, yes. and since they're all humans, then the simplest model that explains this is that this is about humans in general, and same past experience would extend to mail provider's admins. To each their own. yes. smtp is nasty, and also redundant. I disagree. Simple Mail Transfer /Protocol/, as in the application layer language spoken between mail servers is fairly elegant. What is done on top of that and with the data that goes back and forth is far more iffy. I also disagree that SMTP is redundant. I'm not aware of anything else nearly as ubiquitous as SMTP for getting messages between systems in a fault tolerant manner. makes me wonder if i should just create me a hidden tor service that is just a normal website, and give its url to people (instead of email) who want to message me by telling them ``submit your messages to me''. So you want to change from a ubiquitous protocol that is supported by many Many MANY devices to niche protocol that has a non-trivial installation / configuration curve. then, verify messages by mailing their supplied email a confirmation message. And then you want to take what people send you, turn around and send unsolicited messages based on it — this is the icing on the cake — using the protocol that you are trying to avoid. It's only a matter of time before someone uses your Tor hidden service as a vector to send spam. — Joe Job comes to mind. -- Grant. . . . unix || die
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 8/18/20 12:43 AM, Caveman Al Toraboran wrote: would i get blacklisted for simply not using spf/dkim/etc? even if no other user is using the mail service other than me and i'm not mass mailing? I don't think it's that you would be black listed per say. Rather, I think it's that nothing would cause your email to stand out / up and appear more legitimate than the general background noise. You want to stand out from the background noise so that you aren't subject to the almost default block of a lot of background noise. It also provides something for positive (and negative) reputation to be associated with. Think "some random person said" vs "Caveman said". Which will mean more in the circles you travel in? -- Grant. . . . unix || die
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
‐‐‐ Original Message ‐‐‐ On Tuesday, August 18, 2020 2:21 PM, Remco Rijnders wrote: > On Tue, Aug 18, 2020 at 07:00:52AM +, Caveman wrote in > > > yes. smtp is nasty, and also redundant. > > How is it redundant? redundant as in containing concepts already done in other protocols, so smtp has many re-invented wheels that are already invented in existing protocols. basically smtp, as an application-layer protocol, is needless. imo, smtp should be a much-higher level protocol defined purely on top of how dns and http/2. e.g. for mail submission, there is no need for a separate application-layer protocol as we can simply use http/2. because the concept of mail submission is a special case of data submission, which is already in http/2. here is a more complete example of what i mean: 1. we lookup MX records to identify smtp servers to submit mails to. 2. from the response to that lookup we get a domain name, say, mail.dom.com. 3. then, the standard defines a http/2 request format to submit the mail. an example of step (3) could be this: https://mail.dom.com/from=...=...=...\ =...=...=...=...\ =... i don't know how http/2 works. do they have POST requests? if so maybe fields attach1, attach2, ..., attachn can be submitted as file uploads using POST. further, if we modify steps (1) and (2), we can generalise this concept into tor services. e.g. an email address simply becomes an onion address. e.g. if vagzgdrh747aei0q.onion is the hidden service address of your mail server, then your email address could be written as (for convenience): remco@vagzgdrh747aei0q.onion and when a "mail" client tries to submit you an email, it submits it by this url: https://vagzgdrh747aei0q.onion/to=remco&...etc. then, in order to authenticate a source, we simply use public-private keys to sign messages. basically, our public keys become our user identifiers. this will also solve the problem of the case when an onion address changes. i call this protocol mailball for the purpose of making speech this mail thread a bit easier. of course, we can pick better names, and refine the mechanics. > > makes me wonder if i should just create me a > > hidden tor service that is just a normal website, > > and give its url to people (instead of email) who > > want to message me by telling them ``submit your > > messages to me''. then, verify messages by > > mailing their supplied email a confirmation > > message. > > Ah, the "Don't spam us, we'll spam you approach?" for people who use the deprecated smtp protocol, yes, it will be "don't spam us, we'll spam you". however, that's not our fault. they are using a deprecated protocol, and we are just kind enough to allow them an opportunity to talk to us over the superior mailball protocol. basically, they are using deprecated identifiers (email ids) instead of public keys, and we're kind enough to give them a temporary api so that we confirm their emails. on the other hand, people who use mailball will not have this problem. why? because ids are public keys anyway, and their messages are signed by their private keys (the usual drill, won't insult your intelligence).
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 18-Aug-20 8:43, Caveman Al Toraboran wrote: would i get blacklisted for simply not using spf/dkim/etc? even if no other user is using the mail service other than me and i'm not mass mailing? Well, hear my story: I too was running simple mail-server. Just a few users I trust, no public relaying, so what could possibly go wrong? As it turned out later: everything! For a few months all was running as expected, but then some time later all valid email sent by my mail-server was suddenly flagged as spam and rejected. It took me some time to investigate but finally I found my domain (not IP) was on Spamhaus' DBL (domain block list). How did it get there? It seems that someone has created faked spf-record for my domain (I was not using dnssec at that time) and somehow spread it out (maybe using dns cache-poisoning?) to many public dn-resolvers. With that spf-record he authorised many spam-sending hosts to send email with sender field pointing to my domain. And that was even bigger problem, because one can easily switch to different vps/IP if it gets blacklisted, but I did not want to abandon my domain. It took me quite long time to fix everything. So short answer is yes! Even if you are not mass-mailing, you can still get blacklisted, if you do not secure your IP, domain and mail-server properly... Jarry -- ___ This mailbox accepts e-mails only from selected mailing-lists! Everything else is considered to be spam and therefore deleted.
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On Tue, Aug 18, 2020 at 2:43 AM Caveman Al Toraboran wrote: > ‐‐‐ Original Message ‐‐‐ > On Monday, August 17, 2020 3:48 PM, Jarry wrote: > > > Rent VPS and be your own admin. But running properly configured > > mail-server is not so easy. Setting up postfix/exim/sendmail > > is just a beginning. If you mean it seriously and do not want > > your IP to land on blacklists (and you vps suspended), there is > > much more to do, i.e. spf, dkim, dmarc, dnssec, etc... > > would i get blacklisted for simply not using > spf/dkim/etc? even if no other user is using the > mail service other than me and i'm not mass > mailing? It is up to the individual recipient's email admin, but increasingly the answer is yes. Your biggest issue will be IP reputation, however. IPs that are assigned to consumers are almost always blacklisted regardless of what you're doing on your end, and the're blacklisted before you even attempt to send your first message. Personally I run my own server for reception, but all my outgoing mail either goes through Gmail or Amazon SES, depending on whether Gmail was used as the MUA. Sure, Amazon isn't free, but it is REALLY cheap ($0.10 for 1000 emails, plus $0.12 per GB). I don't send that many emails or much in attachments, so the bill is tiny. Gmail is free and you can send outgoing messages from any email address that you control. Receiving email isn't a big deal though managing spam can be painful. Sending it has become increasingly difficult because of others managing spam, and IMO it isn't worth trying to deal with directly unless you're a large concern. -- Rich
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
‐‐‐ Original Message ‐‐‐ On Monday, August 17, 2020 8:00 PM, Grant Taylor wrote: > On 8/16/20 10:50 PM, Caveman Al Toraboran wrote: > > 3. vps admin is not trusty and their sys admin may read my emails, > > and laugh at me! > > Do you have any (anecdotal) evidence that this has actually happened? not specifically with a mail provider, but with other i.t. services, yes. and since they're all humans, then the simplest model that explains this is that this is about humans in general, and same past experience would extend to mail provider's admins. > Well, seeing as how you're talking about email, the biggest elephant in > the room is SMTP's default of unencrypted communications path. It's > realtively easy to add support for encryption, but more systems than I'm > comfortable with don't avail themselves of the optional encryption for > some reason. Sure, it's possible to configure many receiving SMTP > servesr to require it from specific sending systems and / or sending > domains. But this is effort you have to expend to enact these restrictions. yes. smtp is nasty, and also redundant. makes me wonder if i should just create me a hidden tor service that is just a normal website, and give its url to people (instead of email) who want to message me by telling them ``submit your messages to me''. then, verify messages by mailing their supplied email a confirmation message.
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
‐‐‐ Original Message ‐‐‐ On Monday, August 17, 2020 3:48 PM, Jarry wrote: > Rent VPS and be your own admin. But running properly configured > mail-server is not so easy. Setting up postfix/exim/sendmail > is just a beginning. If you mean it seriously and do not want > your IP to land on blacklists (and you vps suspended), there is > much more to do, i.e. spf, dkim, dmarc, dnssec, etc... would i get blacklisted for simply not using spf/dkim/etc? even if no other user is using the mail service other than me and i'm not mass mailing?
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
‐‐‐ Original Message ‐‐‐ On Monday, August 17, 2020 3:33 PM, Ashley Dixon wrote: > How many concurrent users will be connected to the mail server? How much > traffic > will the S.M.T.P. server receive (read: how many e-mails arrive on a daily > basis)? If you really don't trust your V.P.S. provider, and your mail server > is > small-ish, you could just skip all the trust issues and buy a cheap Raspberry > Pi > for £20 or so. 1 user (me). about 2 real daily mails. maybe 10 in peak times. that, plus gentoo's users list, plus spam. but i don't see much spammers in protonmail's spambox. so i guess my spam is low. > Running a mail server over a domestic connection presents some issues, such as > dynamic I.P. ranges appearing in the Spamhaus blocklist, or some > tyrannicalesque > I.S.P.s blocking outbound port 25 (S.M.T.P. submission port), but it is > possible > to have a smooth, self-administered mail server, providing you can put in the > time and effort. I have been doing it myself for a few years with Courier and > Postfix (although I wouldn't recommend Courier; Dovecot is far superior). > > What do you think? interesting. do you have reverse ptr records for your domain name pointing to your home's ip? did you pay extra fees for this ptr to your isp? i wonder if price-wise, and uptime-wise, that would beat a cheap vps at 20 bucks/year.
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On Tue, Aug 18, 2020 at 07:00:52AM +, Caveman Al Toraboran wrote: > yes. smtp is nasty, and also redundant. S.M.T.P. is extremely well-established, and has always been a reasonably pleasant experience for me. Which part of the protocol makes you think of it as "nasty"? > makes me wonder if i should just create me a > hidden tor service that is just a normal website, > and give its url to people (instead of email) who > want to message me by telling them ``submit your > messages to me''. then, verify messages by > mailing their supplied email a confirmation > message. That doesn't make any sense. Setting up e-mail is a fair challenge, but nothing can replace it in terms of interoperability and convenience. Perhaps the younger generation (of whom I am unfortunately a part) would almost-unanimously disagree, but your proposed solution doesn't really provide greater ease for you, or the people e-mailing you! -- Ashley Dixon suugaku.co.uk 2A9A 4117 DA96 D18A 8A7B B0D2 A30E BF25 F290 A8AA signature.asc Description: PGP signature
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On Tue, Aug 18, 2020 at 07:00:52AM +, Caveman wrote in <2gbde2AzVUH4P9HuGPvBvJpZBjaeFBqUfOPfrojvSXcRQgqYcNuLDa0BZbdtS1QrvKymDpPLozphS0JtKDgRXX4WE1rh439hq_VnseMCJZo=@protonmail.com>: yes. smtp is nasty, and also redundant. How is it redundant? makes me wonder if i should just create me a hidden tor service that is just a normal website, and give its url to people (instead of email) who want to message me by telling them ``submit your messages to me''. then, verify messages by mailing their supplied email a confirmation message. Ah, the "Don't spam us, we'll spam you approach?"
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 8/17/20 6:10 AM, Wols Lists wrote: Yup. If you've got mail DNS records pointing at your home server, incoming mail shouldn't be a problem and your vps admin can't snoop :-) True. But the ISP can still sniff the traffic and you can be subject to DPI. Can't you tell your server to forward all outgoing mail to your ISP's SMTP server? That way, you don't have to worry about all the spam issues, and it *should* just pass through. That can start to run afoul of some SPF configurations. Or you must allow your ISP's SMTP server to send email as you. Which means that other ISP users can also send email as you. You are also beholden to the ISP's SMTP infrastructure not changing, lest a change on their end breaking your SPF configuration. I would probably recommend an ESP's SMTP service over your ISP's SMTP service as the ESP will have more experience with this because it's part of their business model. "Should" is the operative word. There is also the fact that your outbound email will now potentially, if not likely, sit in the ISP's SMTP server queue, thus re-introducing an opportunity for it to be scrutinized. The main worry for snooping is inbound mail waiting for collection - outbound requires a dedicated eavesdropping solution and if they're going to do that they can always snoop ANY outgoing SMTP. It depends what you mean by "dedicated eavesdropping solution". General network sniffing and / or DPI does not fall under many definitions of dedicated. Carte blanch redirecting / intercepting SMTP traffic through one of their hosts is also possible. Your local / residential ISP can't do anything if you tunnel your outbound SMTP through an encrypted connection to a VPS. But that re-introduces other complications of VPSs. -- Grant. . . . unix || die
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 8/17/20 5:33 AM, Ashley Dixon wrote: How many concurrent users will be connected to the mail server? How much traffic will the S.M.T.P. server receive (read: how many e-mails arrive on a daily basis)? My main VPS has a single digit number of clients and processes anywhere between 50,000 and 200,000 emails per day. It does so without any problem. If you really don't trust your V.P.S. provider, and your mail server is small-ish, you could just skip all the trust issues and buy a cheap Raspberry Pi for £20 or so. The VPS includes a globally routed IP, something that a Raspberry Pi doesn't inherently include. The connectivity, including reverse DNS, is a big issue for running an email server. Running a mail server over a domestic connection presents some issues, such as dynamic I.P. ranges appearing in the Spamhaus blocklist, or some tyrannicalesque I.S.P.s blocking outbound port 25 (S.M.T.P. submission port), Nitpick: SMTP's /submission/ port is TCP 587. "Submission" is a very specific term in SMTP nomenclature. Specifically client's /submitting/ email into the SMTP ecosystem. Server to server happens over the SMTP port. I believe you mean the regular SMTP port, TCP 25. but it is possible to have a smooth, self-administered mail server, providing you can put in the time and effort. Agreed. ProTip: Running an email server is about more than just SMTP. You really should have a good working understanding of the basics of multiple protocols and technologies that are part of the email ecosystem: - SMTP protocol - DNS protocol - POP3 and / or IMAP client access protocols - MTA - LDA - Virus filtering - Spam filtering - SPF - DKIM - DMARC - RBLs - RWLs - Client operations - email ecosystem nomenclature That's just the short list. When I say "have a good working understanding", I mean that you should be able to provide a 101 level 30-90 second description of each of those items. Actual understanding, not just wrote memorization. I have been doing it myself for a few years with Courier and Postfix I've been doing it for 20+ years with multiple MTAs, multiple client MUAs, multiple 3rd part as a service providers. None of any of the components is difficult itself. The annoying thing comes when you try to get multiple to interact well with each other. (although I wouldn't recommend Courier; Dovecot is far superior). To each their own. I chose Courier because it could do things that Dovecot couldn't (at the time I made the decision) and fit my needs considerably better. Some of the things that you need to make decisions about are learned about with experience, usually unfavorable experience. As in "crap, I don't like the way that works". Thus you make a new decision. There is (or used to be) much debate about should email accounts be real and have backing Unix (OS) level accounts, or should they be virtual and fall under the auspice of one single Unix (OS) level account that the client access protocol daemon(s) run as. From a purely email perspective, this might not matter. But it really starts to matter if you want friends that have email with you to also be able to host a web site with you and need to connect in to manage their site, thus needing a Unix (OS) level account to do so. What do you think? There are MANY different ways that you can combine the things I listed above. It is usually a personal choice. Some things that work out well in one configuration are completely non-applicable or even detrimental in another configuration. There are many recopies to get started. You really need to start somewhere, learn as you go, and make your own choices. -- Grant. . . . unix || die
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 8/16/20 10:50 PM, Caveman Al Toraboran wrote: hi. Hi context: 1. tinfoil hat is on. Okay. 2. i feel disrespected when someone does things to my stuff without getting my approval. Sure. 3. vps admin is not trusty and their sys admin may read my emails, and laugh at me! Do you have any (anecdotal) evidence that this has actually happened? Hanlon's razor comes to mind: Never attribute to malice that which is adequately explained by stupidity. My experience supports Hanlon's razor. This doesn't mean that there aren't malicious admins out there. Many in our industry have fun with the B.O.F.H. and P.F.Y. But I think that's more what we want to do -- if there were no repercussions -- and not what we actually do. *MANY* people talk a big game. I've seen few follow through on the boasting. 4. whole thing is not worth much money. so not welling to pay more than the price of a cheap vps. That is your choice. I personally find that my email / DNS / website is worth ~$240 a year. I could probably do it for ~$120 a year if I wanted to drop redundancy. I could theoretically do it for $60 a year if I wanted to lower functionality. moving to dedicated hardware for me is not worth it. Fair enough and to each their own. I used to have dedicated hardware in my house, and then migrated to VPS based solutions as part of a cross country move without a static IP on the destination end. my goal is to make it annoying enough that cheap-vps's admins find it a bad idea for them to allocate their time to mingle with my stuff. I'd like to hear any (anecdotal) evidence of this happening that you have. If there is anything, I'd suspect that it's bulk Deep Packet Inspection monitoring things. I doubt that actual malicious involvement is common. thoughts on how to maximally satisfy these requirements? Well, seeing as how you're talking about email, the biggest elephant in the room is SMTP's default of unencrypted communications path. It's realtively easy to add support for encryption, but more systems than I'm comfortable with don't avail themselves of the optional encryption for some reason. Sure, it's possible to configure many receiving SMTP servesr to require it from specific sending systems and / or sending domains. But this is effort you have to expend to enact these restrictions. Actual encrypted email; S/MIME, PGP, etc. help in this regard. -- Grant. . . . unix || die
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 17/08/20 12:33, Ashley Dixon wrote: > On Mon, Aug 17, 2020 at 04:50:43AM +, Caveman Al Toraboran wrote: >> thoughts on how to maximally satisfy these >> requirements? > > How many concurrent users will be connected to the mail server? How much > traffic > will the S.M.T.P. server receive (read: how many e-mails arrive on a > daily > basis)? If you really don't trust your V.P.S. provider, and your mail server > is > small-ish, you could just skip all the trust issues and buy a cheap Raspberry > Pi > for £20 or so. Yup. If you've got mail DNS records pointing at your home server, incoming mail shouldn't be a problem and your vps admin can't snoop :-) > > Running a mail server over a domestic connection presents some issues, such > as > dynamic I.P. ranges appearing in the Spamhaus blocklist, or some > tyrannicalesque > I.S.P.s blocking outbound port 25 (S.M.T.P. submission port), but it is > possible > to have a smooth, self-administered mail server, providing you can put in > the > time and effort. I have been doing it myself for a few years with Courier > and > Postfix (although I wouldn't recommend Courier; Dovecot is far superior). > Can't you tell your server to forward all outgoing mail to your ISP's SMTP server? That way, you don't have to worry about all the spam issues, and it *should* just pass through. The main worry for snooping is inbound mail waiting for collection - outbound requires a dedicated eavesdropping solution and if they're going to do that they can always snoop ANY outgoing SMTP. Cheers, Wol
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On 17-Aug-20 6:50, Caveman Al Toraboran wrote: hi. context: 1. tinfoil hat is on. 2. i feel disrespected when someone does things to my stuff without getting my approval. 3. vps admin is not trusty and their sys admin may read my emails, and laugh at me! 4. whole thing is not worth much money. so not welling to pay more than the price of a cheap vps. moving to dedicated hardware for me is not worth it. my goal is to make it annoying enough that cheap-vps's admins find it a bad idea for them to allocate their time to mingle with my stuff. thoughts on how to maximally satisfy these requirements? Rent VPS and be your own admin. But running properly configured mail-server is not so easy. Setting up postfix/exim/sendmail is just a beginning. If you mean it seriously and do not want your IP to land on blacklists (and you vps suspended), there is much more to do, i.e. spf, dkim, dmarc, dnssec, etc... Jarry -- ___ This mailbox accepts e-mails only from selected mailing-lists! Everything else is considered to be spam and therefore deleted.
Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
On Mon, Aug 17, 2020 at 04:50:43AM +, Caveman Al Toraboran wrote: > thoughts on how to maximally satisfy these > requirements? How many concurrent users will be connected to the mail server? How much traffic will the S.M.T.P. server receive (read: how many e-mails arrive on a daily basis)? If you really don't trust your V.P.S. provider, and your mail server is small-ish, you could just skip all the trust issues and buy a cheap Raspberry Pi for £20 or so. Running a mail server over a domestic connection presents some issues, such as dynamic I.P. ranges appearing in the Spamhaus blocklist, or some tyrannicalesque I.S.P.s blocking outbound port 25 (S.M.T.P. submission port), but it is possible to have a smooth, self-administered mail server, providing you can put in the time and effort. I have been doing it myself for a few years with Courier and Postfix (although I wouldn't recommend Courier; Dovecot is far superior). What do you think? -- Ashley Dixon suugaku.co.uk 2A9A 4117 DA96 D18A 8A7B B0D2 A30E BF25 F290 A8AA signature.asc Description: PGP signature
[gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?
hi. context: 1. tinfoil hat is on. 2. i feel disrespected when someone does things to my stuff without getting my approval. 3. vps admin is not trusty and their sys admin may read my emails, and laugh at me! 4. whole thing is not worth much money. so not welling to pay more than the price of a cheap vps. moving to dedicated hardware for me is not worth it. my goal is to make it annoying enough that cheap-vps's admins find it a bad idea for them to allocate their time to mingle with my stuff. thoughts on how to maximally satisfy these requirements? rgrds, cm.
