Re: How to limit Apple Mail (desktop)?
El 31/10/17 a las 15:30, Robert Schetterer escribió: Am 31.10.2017 um 14:06 schrieb María Arrea: If you use Linux you can use netfilter (iptables) mark feature to track individual connections from a given IP, and above a threshold you can shape the bandwitdh based on those iptables mark. For example, if an IMAP connection from ip X exceed 100 megabytes of data, bandwitdh for that ip would be limited to 100 KB/sec (or whatever number). First 100 megabytes, top speed, above that they get slower. dont think this will work tons of cons can come from a nat ip so you would punish everyone behind that ip ,not only apple mail Not true. You set a connection tracking using ip+port, not only IP. i see no real solution for the problem, cause if the server works out some slow down this would only lead to longer download times, but will no solve the orig wanted task. It will all be workarounds for a "buggy" client Original post talked about "unbearable server load". Shapping network traffic will definitely help with that. María.
Re: How to limit Apple Mail (desktop)?
"@lbutlr"writes: (Are you the OP, or have I mistakenly atributed this to Rupert Gallagher?) So what the composition of all this traffic? Are you saying the mail client is ultra dumb and repeatedly downloading entire messages, read and unread, attachment and all (i.e. you're truly bandwidth limited?) It most certainly does not do that. If this is the case for the OP, it's back to square one to find log entries, and/or nfsstat/iostat/vmstat/etc output to see what is "killing" the server. File locking issues, mail_max_userip_connections and process limit values could also cause service flakiness. Apple mail readers seems to gobble up IMAP connections like candy. Joseph Tam
Re: How to limit Apple Mail (desktop)?
On Oct 31, 2017, at 5:01 PM, Joseph Tamwrote: > Rupert Gallagher writes: > >> By default, Apple Mail downloads all e-mails from? server's account. >> Previous versions of this client allowed to opt-out. The latest two >> versions? however, only allow to opt-out from downloading the >> attachments. The stress on the server is unbearable. We cannot ask users to >> be >> considerate: this is the default behaviour of Apple Mail. > > (I'm coming into this thread late.) > > I'm having a hard time visualizing the cause of your problem. Most big > messages have the bulk of their data as attachments (which you state the mail > reader can defer downloading). Also, readers usually cache local copies, so > they're downloaded once. Yep. > So what the composition of all this traffic? Are you saying the mail client > is ultra dumb and repeatedly downloading entire messages, read and unread, > attachment and all (i.e. you're truly bandwidth limited?) It most certainly does not do that. > Do your users typically have many concurrent readers for the same mailbox? Most of my users have 3 devices, a desktop, an iPhone, and an iPad. It doesn't seem to cause any issues. -- Apple broke AppleScripting signatures in Mail.app, so no random signatures.
Re: How to limit Apple Mail (desktop)?
Rupert Gallagher writes: By default, Apple Mail downloads all e-mails from? server's account. Previous versions of this client allowed to opt-out. The latest two versions? however, only allow to opt-out from downloading the attachments. The stress on the server is unbearable. We cannot ask users to be considerate: this is the default behaviour of Apple Mail. (I'm coming into this thread late.) I'm having a hard time visualizing the cause of your problem. Most big messages have the bulk of their data as attachments (which you state the mail reader can defer downloading). Also, readers usually cache local copies, so they're downloaded once. So what the composition of all this traffic? Are you saying the mail client is ultra dumb and repeatedly downloading entire messages, read and unread, attachment and all (i.e. you're truly bandwidth limited?) Do your users typically have many concurrent readers for the same mailbox? What kind of session numbers are you supporting? 100s? 1000s? 10K+? Your data outflow has an incoming counterpart: if you're having a hard time serving it out, is your MTA also having a hard time with the incoming volume? Can it simply be the case you've underscaled your remote mail service? Joseph Tam
Re: How to limit Apple Mail (desktop)?
I think I am not mistaken in saying that the original purpose of IMAP was to access a remote database, vs POP's approach to download it. When the IMAP client opens a folder, the server uploads the *index* of its content. When the client requests a specific item, the server uploads the item while keeping the original. If the client renames a folder, the server just renames the folder. Apple Mail behaves like a POP client that wants to download everything while keeping the db on the server. Further down Apple's madness, if the client renames a folder, Apple mail asks the server to upload the full content of the "new" folder. Batshit crazy! The very best therapy here is, in my opinion, to serve Apple Mail's request for mass download as if it were a regular IMAP index request, if technically feasible. If it is not feasible, then the alternative is not to rate limit the connection, but to ban Apple Mail entirely. Sent from ProtonMail Mobile On Tue, Oct 31, 2017 at 2:46 PM, Rupert Gallagherwrote: > Aki, the IMAP client can receive the e-mails with an empty body without any > damage. This is how IMAP works normally. The full body is queried again by > the client when reading the e-mail for real. > > On Tue, Oct 31, 2017 at 1:52 PM, Aki Tuomi wrote: > What's in your mind as solution? > >>> When dovecot receives many full body downloads from a client, it could >>> respond by sending the header only. > >> This sounds rather dangerous. Client is expecting full body download, not >> headers. Aki
Re: How to limit Apple Mail (desktop)?
Am 31.10.2017 um 14:06 schrieb María Arrea: > > If you use Linux you can use netfilter (iptables) mark feature to > track individual connections from a given IP, and above a threshold you > can shape the bandwitdh based on those iptables mark. For example, if an > IMAP connection from ip X exceed 100 megabytes of data, bandwitdh for > that ip would be limited to 100 KB/sec (or whatever number). First 100 > megabytes, top speed, above that they get slower. dont think this will work tons of cons can come from a nat ip so you would punish everyone behind that ip ,not only apple mail i see no real solution for the problem, cause if the server works out some slow down this would only lead to longer download times, but will no solve the orig wanted task. It will all be workarounds for a "buggy" client > > Regards > > María > > > El 30/10/17 a las 10:38, Rupert Gallagher escribió: >> By default, Apple Mail downloads all e-mails from server's account. >> Previous versions of this client allowed to opt-out. The latest two >> versions? however, only allow to opt-out from downloading the >> attachments. >> >> The stress on the server is unbearable. We cannot ask users to be >> considerate: this is the default behaviour of Apple Mail. >> >> We need a server-side solution to the problem. >> >> Please share your ideas. Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Schleißheimer Straße 26/MG, 80333 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
Re: How to limit Apple Mail (desktop)?
Aki, the IMAP client can receive the e-mails with an empty body without any damage. This is how IMAP works normally. The full body is queried again by the client when reading the e-mail for real. On Tue, Oct 31, 2017 at 1:52 PM, Aki Tuomiwrote: >>> What's in your mind as solution? >> When dovecot receives many full body downloads from a client, it could >> respond by sending the header only. > This sounds rather dangerous. Client is expecting full body download, not > headers. Aki
Re: How to limit Apple Mail (desktop)?
If you use Linux you can use netfilter (iptables) mark feature to track individual connections from a given IP, and above a threshold you can shape the bandwitdh based on those iptables mark. For example, if an IMAP connection from ip X exceed 100 megabytes of data, bandwitdh for that ip would be limited to 100 KB/sec (or whatever number). First 100 megabytes, top speed, above that they get slower. Regards María El 30/10/17 a las 10:38, Rupert Gallagher escribió: By default, Apple Mail downloads all e-mails from server's account. Previous versions of this client allowed to opt-out. The latest two versions? however, only allow to opt-out from downloading the attachments. The stress on the server is unbearable. We cannot ask users to be considerate: this is the default behaviour of Apple Mail. We need a server-side solution to the problem. Please share your ideas.
Re: How to limit Apple Mail (desktop)?
> On 31 Oct 2017, at 14.44, Rupert Gallagherwrote: > >> What's in your mind as solution? > > When dovecot receives many full body downloads from a client, it could > respond by sending the header only. That would not be nice. Breaking IMAP protocol that way. Some rate limiting would be better solution. Sami
Re: How to limit Apple Mail (desktop)?
On 31.10.2017 14:44, Rupert Gallagher wrote: >> What's in your mind as solution? > When dovecot receives many full body downloads from a client, it could > respond by sending the header only. This sounds rather dangerous. Client is expecting full body download, not headers. Aki > Sent from ProtonMail Mobile > > On Tue, Oct 31, 2017 at 11:29 AM,wrote: > >> Hi, On 30.10.2017 10:38, Rupert Gallagher wrote: > We need a server-side >> solution to the problem. more powerfull hardware? What's in your mind as >> solution? Don't tell the client about the mails -> user will miss old mails. >> Stop him from fetching body with a temporary "UNAVAILABLE" failure? -> Don't >> know how apple mail will react, but probably present some error to the user >> for every mail. The only possibility that would make sense to me is some >> form of rate limiting to reduce the server load. Give them some tens of >> Megabytes w/o any restriction, then reduce. Regards Moritz
Re: How to limit Apple Mail (desktop)?
> What's in your mind as solution? When dovecot receives many full body downloads from a client, it could respond by sending the header only. Sent from ProtonMail Mobile On Tue, Oct 31, 2017 at 11:29 AM,wrote: > Hi, On 30.10.2017 10:38, Rupert Gallagher wrote: > We need a server-side > solution to the problem. more powerfull hardware? What's in your mind as > solution? Don't tell the client about the mails -> user will miss old mails. > Stop him from fetching body with a temporary "UNAVAILABLE" failure? -> Don't > know how apple mail will react, but probably present some error to the user > for every mail. The only possibility that would make sense to me is some form > of rate limiting to reduce the server load. Give them some tens of Megabytes > w/o any restriction, then reduce. Regards Moritz
Re: How to limit Apple Mail (desktop)?
Hi, On 30.10.2017 10:38, Rupert Gallagher wrote: We need a server-side solution to the problem. more powerfull hardware? What's in your mind as solution? Don't tell the client about the mails -> user will miss old mails. Stop him from fetching body with a temporary "UNAVAILABLE" failure? -> Don't know how apple mail will react, but probably present some error to the user for every mail. The only possibility that would make sense to me is some form of rate limiting to reduce the server load. Give them some tens of Megabytes w/o any restriction, then reduce. Regards Moritz
Re: How to limit Apple Mail (desktop)?
> On 31 Oct 2017, at 8.42, Rupert Gallagherwrote: > > When Apple Mail connects to an IMAP account for the very first time, it > downloads all e-mails to build a local mirror. > > When the user changes the name of a folder, Apple Mail downloads the whole > subtree and erases the old one. > > We have Apple Mail users with >20GB worth of e-mails, downloaded multiple > times (horrified emoticon here). > > Sent from ProtonMail Mobile I can’t think of any easy way to rate limit Apple mail. Only thing that crossed my mind is to enable ID response logging and then with some logwatch process use some traffic shaping solution to limit the bandwith of that particular connection after some period of time. Sami
Re: How to limit Apple Mail (desktop)?
When Apple Mail connects to an IMAP account for the very first time, it downloads all e-mails to build a local mirror. When the user changes the name of a folder, Apple Mail downloads the whole subtree and erases the old one. We have Apple Mail users with >20GB worth of e-mails, downloaded multiple times (horrified emoticon here). Sent from ProtonMail Mobile On Mon, Oct 30, 2017 at 10:38 AM, Rupert Gallagherwrote: > By default, Apple Mail downloads all e-mails from server's account. Previous > versions of this client allowed to opt-out. The latest two versions? however, > only allow to opt-out from downloading the attachments. > > The stress on the server is unbearable. We cannot ask users to be > considerate: this is the default behaviour of Apple Mail. > > We need a server-side solution to the problem. > > Please share your ideas.
Re: How to limit Apple Mail (desktop)?
Am 30.10.2017 um 17:50 schrieb Robert Schetterer: > Am 30.10.2017 um 10:38 schrieb Rupert Gallagher: >> By default, Apple Mail downloads all e-mails from server's account. >> Previous versions of this client allowed to opt-out. The latest two >> versions? however, only allow to opt-out from downloading the attachments. >> >> The stress on the server is unbearable. We cannot ask users to be >> considerate: this is the default behaviour of Apple Mail. >> >> We need a server-side solution to the problem. >> >> Please share your ideas. >> > > first check if you can ident Apple Mail versions related, i recent have > none in my log , but i.e Android does > > ---log > ID sent: name=com.samsung.android.email.provider, os=android, > os-version=7.0; NRD90M, vendor=samsung, x-android-device-model=SM-G930F > - > > then you need a procedure for limiti have no idea which one > > Best Regards > MfG Robert Schetterer > I dont know Apple Mail very good, but as workaround you might use sieve to presort mails on the server at incomming in (sub)folders which arent synced at default i.e in a date named folder, for sure users ( you should know your apple mail users ) then need configure an extra subscribe on these folders. Perhaps a combi with virtual folders may usefull , for now not better idea , iam nearly sure this is not what you you expected and/or wanted seems google has a feature to "hide" mails see https://www.guidingtech.com/44581/prevent-mail-app-space-mac/ however if feel very strange with this Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Schleißheimer Straße 26/MG, 80333 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
Re: How to limit Apple Mail (desktop)?
Am 30.10.2017 um 10:38 schrieb Rupert Gallagher: > By default, Apple Mail downloads all e-mails from server's account. Previous > versions of this client allowed to opt-out. The latest two versions? however, > only allow to opt-out from downloading the attachments. > > The stress on the server is unbearable. We cannot ask users to be > considerate: this is the default behaviour of Apple Mail. > > We need a server-side solution to the problem. > > Please share your ideas. > first check if you can ident Apple Mail versions related, i recent have none in my log , but i.e Android does ---log ID sent: name=com.samsung.android.email.provider, os=android, os-version=7.0; NRD90M, vendor=samsung, x-android-device-model=SM-G930F - then you need a procedure for limiti have no idea which one Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Schleißheimer Straße 26/MG, 80333 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
How to limit Apple Mail (desktop)?
By default, Apple Mail downloads all e-mails from server's account. Previous versions of this client allowed to opt-out. The latest two versions? however, only allow to opt-out from downloading the attachments. The stress on the server is unbearable. We cannot ask users to be considerate: this is the default behaviour of Apple Mail. We need a server-side solution to the problem. Please share your ideas.