Re: [mailop] current state of multi-protocol mail forwarding - Summary
Dnia 25.11.2024 o godz. 05:55:57 Miles Fidelman via mailop pisze: > I've been noodling with the notion of building a complete enterprise > environment in "virtual space" - i.e, the Web3 space defined by > protocols supported by LibP2P, naming & addressing by IPNS & CIDs, > and a computing environment of Files & Processes defined by IPFS & > IPVMs. Partially as a design experiment, but ultimately because I'm > thinking of doing that for a current project. Just out of curiosity: could you give me link to any document that presents in an easy-to-understand form the concept of those processes running on IPVMs? Because all documentation I could find is pretty vague, and from what I have read - I might be wrong of course, so feel free to correct me - I have the impression that there is nothing permanent, "solid" in this infrastructure that might be suitable for running long-term processes, which are - to my understanding - quasi-necessary for a mail service. All processing seems to be a kind of microservices that just appear and disappear, get some data and return some result. This might be suitable for most typical web applications that usually do little more than fetch/transform/store data in a single transaction, might be suitable for some types of numerical computations (of course! - that's what distributed computing systems were initially invented for), but I don't think it's suitable for a MTA, for which keeping the state is a crucial thing. If anybody wants to build a MTA on top of such infrastructure, it would probably need to have a totally different concept and design than the MTAs we are using today. As this seems to be largely off-topic for this list, feel free to reply to me directly if you wish. -- Regards, Jaroslaw Rafa r...@rafa.eu.org -- "In a million years, when kids go to school, they're gonna know: once there was a Hushpuppy, and she lived with her daddy in the Bathtub." ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] current state of multi-protocol mail forwarding - Summary
Dave Crocker via mailop wrote: This thread has touched on several topic that seem unrelated. Given your Subject line and reference to gateway, I had assumed you were asking about translating mail between two different syntactic/semantic environment. That has nothing to do with file systems or shared access, which seems to be where the thread went. And Jaroslaw Rafa via mailop wrote: Dnia 25.11.2024 o godz. 04:04:13 Dave Crocker via mailop pisze: This thread has touched on several topic that seem unrelated. IMHO, the problem has not been clearly stated from the very beginning. In the first email of previous related thread, the OP is saying that "I've been thinking of migrating our mail infrastructure to a virtual server, running in the Web3 IPFS cloud - without having a physical IP address attached to it." This statement already brings up several issues. To be clear (maybe): I've been noodling with the notion of building a complete enterprise environment in "virtual space" - i.e, the Web3 space defined by protocols supported by LibP2P, naming & addressing by IPNS & CIDs, and a computing environment of Files & Processes defined by IPFS & IPVMs. Partially as a design experiment, but ultimately because I'm thinking of doing that for a current project. The core question is how integrate into the broader environment of DNS naming, SMTP Mail, HTTP for pretty much everything else - via some kind of gateways between an environment where things are ultimately addressed by IP address. How do I present standard NS & MX records that have no stable IP address to resolve to? The underlying question becomes how to run virtual processes that present a WebSocket or WebTransport interface to the world, addressed by CID - and then to implement a collection of gateways that present IP addresses to the net. It's looking like the answer becomes a combination of utilizing a VPN that supports anycast routing, and a distributed process manager like BOINC or Bacalhu. Anyway - I think I have the architectural approach I've been looking for. Now it's a matter of assembling the pieces. Thanks All, Miles -- In theory, there is no difference between theory and practice. In practice, there is. Yogi Berra Theory is when you know everything but nothing works. Practice is when everything works but no one knows why. In our lab, theory and practice are combined: nothing works and no one knows why. ... unknown ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] current state of multi-protocol mail forwarding
Dnia 25.11.2024 o godz. 04:04:13 Dave Crocker via mailop pisze: > This thread has touched on several topic that seem unrelated. IMHO, the problem has not been clearly stated from the very beginning. In the first email of previous related thread, the OP is saying that "I've been thinking of migrating our mail infrastructure to a virtual server, running in the Web3 IPFS cloud - without having a physical IP address attached to it." This statement already brings up several issues. While it is quite clear what IPFS is - it's just a distributed file storage mechanism - it's not clear what "Web3 IPFS cloud" would be. Use of the term "cloud" suggests some computing infrastructure built around IPFS (as storage backend probably?) that can provide computing power. I'm not aware of existence of any such infrastructure. But if it exists, and you can run virtual servers on it, and these virtual servers are able to use IPFS as storage, then it's just an internal implementation detail. From outside, such mail server would look like a regular mail server, but instead of storing messages in the internal file system, it would store them on the IPFS. Nobody sending or receiving mail from such a server would need to know it, as it's a detail internal to that server. For the outsiders, it's just a regular mail server. Which means, it obviously has to have an IP address. As others have stated, I can't imagine how you can ever run a mail server without having an IP address. What is more unclear to me, is why the OP wants to store messages in the IPFS. What advantages does it have over storing them in a regular storage? What does the OP want to do with these messages that would be facilitated by having them in IPFS? If it should be just a proof of concept that it can be done, then of course, go on and do it, but without more information, it seems as useful to me as the famous experiment with TCP/IP connection with bongo drums used as the physical layer [1]. Of course, it demonstrates that it is possible and can be done, but I see no use for it. [1] http://web.archive.org/web/20130724230518/http://eagle.auc.ca/~dreid/ -- Regards, Jaroslaw Rafa r...@rafa.eu.org -- "In a million years, when kids go to school, they're gonna know: once there was a Hushpuppy, and she lived with her daddy in the Bathtub." ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] current state of multi-protocol mail forwarding
On 11/22/2024 11:37 AM, Miles Fidelman via mailop wrote: So, one answer to my problem is just to set up a copy of sendmail with local delivery to an IPFS file. But, that leads me to wonder if anybody is offering mail-gateways as a service - be they free gateways or paid? This thread has touched on several topic that seem unrelated. Given your Subject line and reference to gateway, I had assumed you were asking about translating mail between two different syntactic/semantic environment. That has nothing to do with file systems or shared access, which seems to be where the thread went. At the least, it might help to distinguish the what from the how. File system is an example of how. Email 'delivery' into a file system is an example of what. A basic test of whether data exchange is simple email relaying -- or posting or delivering -- or is actually gatewaying, is to look for what information is lost in the transition. If the two mail services really are different -- and hence need gatewaying rather than just relaying -- some information will be lost in one or both directions. d/ -- Dave Crocker Brandenburg InternetWorking bbiw.net *** bluesky: @dcrocker.bsky.social *** mast: @dcrocker@mastodon.social ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] current state of multi-protocol mail forwarding
Miles, please let us all know if a *reliable* (tm) SMTP inbound service has been set up that way. And a rather detailed How-To, if you would, of how you have accomplished this eventually. Thanks & best, -C. On 23.11.2024 18:04, Miles Fidelman via mailop wrote: One can set up a file as a CRDT-style log file. Local delivery adds to the log, the new messages propagate across the cloud (essentially as a torrent). VPN, with an anycast address for mail receipt would probably work just fine. Spin up a few VMs on different hosts as listeners, and it would all seem to work! Thanks for the various comments folks! Miles ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] current state of multi-protocol mail forwarding
It appears that Miles Fidelman via mailop said: >Andy Smith via mailop wrote: >> Hi, >> >> On Sat, Nov 23, 2024 at 12:16:55AM +0100, Philipp Kern via mailop wrote: >>> On 11/22/24 8:37 PM, Miles Fidelman via mailop wrote: So, one answer to my problem is just to set up a copy of sendmail with local delivery to an IPFS file. But, that leads me to wonder if anybody is offering mail-gateways as a service - be they free gateways or paid? >>> Isn't that everyone who also offers sending email as a service? >>> >>> Amazon SES allows receiving mail and publishing it on an SNS topic. Some >>> worker could fetch it from there and store it somewhere else. > >Ahh... that's good to know I suppose that might be useful if you're planning to receive and dispatch hundreds of thousands of messages a day to fixed addresses in a single domain. (SES sets very strict address limits to avoid being used to send or relay spam.) If you know enough to spin up a linux VM with a copy of sendmail or Postfix, and you're anticipating a normal amount of mail. that's going to be a lot easier and cheaper than trying to figure out SES and SNS and all the other stuff you have to do to run on AWS. I still don't understand why you want to permaently store all your spam in IPFS, though. R's, John ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] current state of multi-protocol mail forwarding
Andy Smith via mailop wrote: Hi, On Sat, Nov 23, 2024 at 12:16:55AM +0100, Philipp Kern via mailop wrote: On 11/22/24 8:37 PM, Miles Fidelman via mailop wrote: So, one answer to my problem is just to set up a copy of sendmail with local delivery to an IPFS file. But, that leads me to wonder if anybody is offering mail-gateways as a service - be they free gateways or paid? Isn't that everyone who also offers sending email as a service? Amazon SES allows receiving mail and publishing it on an SNS topic. Some worker could fetch it from there and store it somewhere else. Ahh... that's good to know I get the impression that OP is looking for pre-existing work around interfacing IPFS and SMTP, but the problem here is that we know about SMTP and not IPFS. IPFS being far and away the more niche topic, I think OP would be better off asking these questions in the IPFS community. Personally speaking my only exposure to IPFS is receiving phishing mails with links to ipfs.io… Oh.. I have.. the thing is that the folks there are rather ignorant about Mail (And DNS) infrastructure. There don't seem to be a lot of folks who understand both. But... in fairness.. I do have to say that the folks developing for IPFS seem to have some serious technical chops, and their approach to organizing backbone services as community governed DAOs (Distributed Autonomous Organizations), seems to be working pretty well. Basically a step beyond the IETF, ICANN, and the Apache Software Foundation, and the Free Software Foundation. Both the infrastructure and the ecosystem are rather impressive. (This from someone who's a bit of a student of governance models - back to the days that I wrote the Network Management Architecture for the Defense Data Network, and played in the Military Simulation & Open Mapping Communities). Thanks Again, Folks -- In theory, there is no difference between theory and practice. In practice, there is. Yogi Berra Theory is when you know everything but nothing works. Practice is when everything works but no one knows why. In our lab, theory and practice are combined: nothing works and no one knows why. ... unknown ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] current state of multi-protocol mail forwarding
One can set up a file as a CRDT-style log file. Local delivery adds to the log, the new messages propagate across the cloud (essentially as a torrent). VPN, with an anycast address for mail receipt would probably work just fine. Spin up a few VMs on different hosts as listeners, and it would all seem to work! Thanks for the various comments folks! Miles Eric Tykwinski via mailop wrote: Here’s the thing that confuses me, and perhaps because I don’t know Interplanetary File System as much as I should. You have /var/spool/mail/user which changes every time you receive/delete a message, and that changes the hash/CID which I’m assuming will replicate to other distributed systems on IPFS. Is there a read/write lock so you don’t overwrite changes on server a vs server b. I’m guessing you could create a database in IPFS since it’s not just a filesystem which would allow locks, but that’s getting above my pay grade. On Nov 22, 2024, at 4:10 PM, Grant Taylor via mailop wrote: On 11/22/24 1:37 PM, Miles Fidelman via mailop wrote: I'm reminded of the days of MMDF, and mail-uucp gateways and such. chuckle So, one answer to my problem is just to set up a copy of sendmail with local delivery to an IPFS file. But, that leads me to wonder if anybody is offering mail-gateways as a service - be they free gateways or paid? I don't know about commercial companies. But I've run various gateway sin the past for hobbyists / retro computing enthusiasts. Feel free to email me directly if you'd like to chat more about specifics. -- Grant. . . . unix || die ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop -- In theory, there is no difference between theory and practice. In practice, there is. Yogi Berra Theory is when you know everything but nothing works. Practice is when everything works but no one knows why. In our lab, theory and practice are combined: nothing works and no one knows why. ... unknown ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] current state of multi-protocol mail forwarding
On 22Nov24, John Levine apparently wrote: > You can make the problem a lot easier by putting each message in a different > file like Maildir For the mail payload sure, Maildir offers a likely unique ID for storage, but it doesn't really help much with metadata or for syncing and resolving mailbox operations and conflict-resolution such as moves initiated from two difference instances. Maildir and their ilk, solve the easy part of the problem. Distributed file-system sync is a subset of distributed mailbox sync so you'll have to layer something on top of your storage system regardless of what type it is. Mark. ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] current state of multi-protocol mail forwarding
I was actually thinking more in line with LiberationTech from Stanford: https://lists.ghserv.net/mailman/listinfo/lt It’s not a prime concern of MailOps that’s for sure. It’s definitely going to come up again and again. Ladar Levison’s Dark Mail sort of disappeared as well, so I’m not sure what’s next: https://darkmail.info/ Something may eventually take precedent, right now it’s definitely up in the air. > On Nov 22, 2024, at 7:08 PM, Andy Smith via mailop wrote: > > Hi, > > On Sat, Nov 23, 2024 at 12:16:55AM +0100, Philipp Kern via mailop wrote: >> On 11/22/24 8:37 PM, Miles Fidelman via mailop wrote: >>> So, one answer to my problem is just to set up a copy of sendmail with >>> local delivery to an IPFS file. But, that leads me to wonder if anybody >>> is offering mail-gateways as a service - be they free gateways or paid? >> >> Isn't that everyone who also offers sending email as a service? >> >> Amazon SES allows receiving mail and publishing it on an SNS topic. Some >> worker could fetch it from there and store it somewhere else. > > I get the impression that OP is looking for pre-existing work around > interfacing IPFS and SMTP, but the problem here is that we know about > SMTP and not IPFS. > > IPFS being far and away the more niche topic, I think OP would be better > off asking these questions in the IPFS community. > > Personally speaking my only exposure to IPFS is receiving phishing mails > with links to ipfs.io… > > Thanks, > Andy > > -- > https://bitfolk.com/ -- No-nonsense VPS hosting > ___ > mailop mailing list > mailop@mailop.org > https://list.mailop.org/listinfo/mailop ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] current state of multi-protocol mail forwarding
Hi, On Sat, Nov 23, 2024 at 12:16:55AM +0100, Philipp Kern via mailop wrote: > On 11/22/24 8:37 PM, Miles Fidelman via mailop wrote: > > So, one answer to my problem is just to set up a copy of sendmail with > > local delivery to an IPFS file. But, that leads me to wonder if anybody > > is offering mail-gateways as a service - be they free gateways or paid? > > Isn't that everyone who also offers sending email as a service? > > Amazon SES allows receiving mail and publishing it on an SNS topic. Some > worker could fetch it from there and store it somewhere else. I get the impression that OP is looking for pre-existing work around interfacing IPFS and SMTP, but the problem here is that we know about SMTP and not IPFS. IPFS being far and away the more niche topic, I think OP would be better off asking these questions in the IPFS community. Personally speaking my only exposure to IPFS is receiving phishing mails with links to ipfs.io… Thanks, Andy -- https://bitfolk.com/ -- No-nonsense VPS hosting ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] current state of multi-protocol mail forwarding
It appears that Mark Delany via mailop said: >On 22Nov24, Eric Tykwinski via mailop apparently wrote: >> Here’s the thing that confuses me, and perhaps because I don’t know >> Interplanetary File System as much as I should. >> You have /var/spool/mail/user which changes every time you receive/delete a >> message, and that changes the hash/CID which I’m assuming will >replicate to other distributed systems on IPFS. Is there a read/write lock so >you don’t overwrite changes on server a vs server b. > >This has been a challenge of distributed mail stores since day one. Same >problem arose >when people started putting /var/spool mail on NFS servers in the 80s. You can make the problem a lot easier by putting each message in a different file like Maildir does, but you still need some sort of locking or you'll have race conditions when two agents try to work in the same file. Maildir is sort of cheating there since you get the locks for free on atomic operations like rename() or unlink(). It's not immediately clear to me what kind of mail you'd want to put on a write-once file system. The vast majority of the mail I get is not worth saving. R's, John ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] current state of multi-protocol mail forwarding
On 11/22/24 8:37 PM, Miles Fidelman via mailop wrote: > So, one answer to my problem is just to set up a copy of sendmail with > local delivery to an IPFS file. But, that leads me to wonder if anybody > is offering mail-gateways as a service - be they free gateways or paid? Isn't that everyone who also offers sending email as a service? Amazon SES allows receiving mail and publishing it on an SNS topic. Some worker could fetch it from there and store it somewhere else. In the end you just need some service that converts email into an RPC and some code that receives the RPC and stores the result somewhere. Kind regards Philipp Kern ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] current state of multi-protocol mail forwarding
On 22Nov24, Eric Tykwinski via mailop apparently wrote: > Here’s the thing that confuses me, and perhaps because I don’t know > Interplanetary File System as much as I should. > You have /var/spool/mail/user which changes every time you receive/delete a > message, and that changes the hash/CID which I’m assuming will replicate to > other distributed systems on IPFS. Is there a read/write lock so you don’t > overwrite changes on server a vs server b. This has been a challenge of distributed mail stores since day one. Same problem arose when people started putting /var/spool mail on NFS servers in the 80s. You also have to consider the distributed state of a mailbox on changes due to retrieval (read flags, deleted, moved, etc) not just on initial storage. It's a fun trip, enjoy. Mark. ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] current state of multi-protocol mail forwarding
Here’s the thing that confuses me, and perhaps because I don’t know Interplanetary File System as much as I should. You have /var/spool/mail/user which changes every time you receive/delete a message, and that changes the hash/CID which I’m assuming will replicate to other distributed systems on IPFS. Is there a read/write lock so you don’t overwrite changes on server a vs server b. I’m guessing you could create a database in IPFS since it’s not just a filesystem which would allow locks, but that’s getting above my pay grade. > On Nov 22, 2024, at 4:10 PM, Grant Taylor via mailop > wrote: > > On 11/22/24 1:37 PM, Miles Fidelman via mailop wrote: >> I'm reminded of the days of MMDF, and mail-uucp gateways and such. > > chuckle > >> So, one answer to my problem is just to set up a copy of sendmail with local >> delivery to an IPFS file. But, that leads me to wonder if anybody is >> offering mail-gateways as a service - be they free gateways or paid? > > I don't know about commercial companies. But I've run various gateway sin > the past for hobbyists / retro computing enthusiasts. Feel free to email me > directly if you'd like to chat more about specifics. > > > > -- > Grant. . . . > unix || die > > ___ > mailop mailing list > mailop@mailop.org > https://list.mailop.org/listinfo/mailop ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] current state of multi-protocol mail forwarding
Do you need a gateway to receive email? Send email? Both? Le ven. 22 nov. 2024 à 20:42, Miles Fidelman via mailop a écrit : > Hi Folks, > > Apropos my earlier question of funneling mail to a server that doesn't > have a fixed IP address. It occurs to me that this is simply another > way to say "proxy server." > > I'm reminded of the days of MMDF, and mail-uucp gateways and such. > > So, one answer to my problem is just to set up a copy of sendmail with > local delivery to an IPFS file. But, that leads me to wonder if anybody > is offering mail-gateways as a service - be they free gateways or paid? > > Thanks, > > Miles Fidelman > > -- > In theory, there is no difference between theory and practice. > In practice, there is. Yogi Berra > > Theory is when you know everything but nothing works. > Practice is when everything works but no one knows why. > In our lab, theory and practice are combined: > nothing works and no one knows why. ... unknown > > ___ > mailop mailing list > mailop@mailop.org > https://list.mailop.org/listinfo/mailop > ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] current state of multi-protocol mail forwarding
Miles, If you are just trying to send/receive on a dynamic ip range, I would purchase a cheap VPS and run something like Proxmox Mail Gateway, or a home-brew deal of the same with a static IP, valid reverse dns, et al. Setup a WireGuard connection server on the mail gateway, and then just proxy through the gateway from your internal IP address through wireguard. > On Nov 22, 2024, at 2:37 PM, Miles Fidelman via mailop > wrote: > > Hi Folks, > > Apropos my earlier question of funneling mail to a server that doesn't have a > fixed IP address. It occurs to me that this is simply another way to say > "proxy server." > > I'm reminded of the days of MMDF, and mail-uucp gateways and such. > > So, one answer to my problem is just to set up a copy of sendmail with local > delivery to an IPFS file. But, that leads me to wonder if anybody is > offering mail-gateways as a service - be they free gateways or paid? > > Thanks, > > Miles Fidelman > > -- > In theory, there is no difference between theory and practice. > In practice, there is. Yogi Berra > > Theory is when you know everything but nothing works. > Practice is when everything works but no one knows why. > In our lab, theory and practice are combined: > nothing works and no one knows why. ... unknown > > ___ > mailop mailing list > mailop@mailop.org > https://list.mailop.org/listinfo/mailop ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] current state of multi-protocol mail forwarding
On 11/22/24 1:37 PM, Miles Fidelman via mailop wrote: I'm reminded of the days of MMDF, and mail-uucp gateways and such. chuckle So, one answer to my problem is just to set up a copy of sendmail with local delivery to an IPFS file. But, that leads me to wonder if anybody is offering mail-gateways as a service - be they free gateways or paid? I don't know about commercial companies. But I've run various gateway sin the past for hobbyists / retro computing enthusiasts. Feel free to email me directly if you'd like to chat more about specifics. -- Grant. . . . unix || die ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] current state of multi-protocol mail forwarding
Am 22.11.2024 um 14:37:23 Uhr schrieb Miles Fidelman via mailop: > So, one answer to my problem is just to set up a copy of sendmail > with local delivery to an IPFS file. sendmail can save mail to a local file with the local mailer. This file includes all mail of one user. Is that what you want? > But, that leads me to wonder if anybody is offering mail-gateways as > a service - be they free gateways or paid? Such companies exist, but I recommend renting a VPS for some bucks per months and set it up yourself, so you have full control. -- Gruß Marco Send unsolicited bulk mail to 1732282643mu...@cartoonies.org ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] current state of multi-protocol mail forwarding
It appears that Miles Fidelman via mailop said: >So, one answer to my problem is just to set up a copy of sendmail with >local delivery to an IPFS file. But, that leads me to wonder if anybody >is offering mail-gateways as a service - be they free gateways or paid? Gateways to what? The way you pass mail from one kind of transport to another depends a lot on what's at each end. Compare for example, the way you do a fax-to-email gateway to an email-to-SMS one. Setting up an MTA to drop the messages into a folder is easy. (These days we usually use postfix rather than sendmail, but whatever.) The hard part is what you do once it's in the folder. R's, John ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
