Re: [cyrus 3.0] 20 delayed mailbox deleted limit?

2016-06-10 Thread Andre Felipe Machado via Info-cyrus 
Bron Gondwana via Info-cyrus  wrote ..
> On Fri, Jun 10, 2016, at 09:41, Jason L Tibbitts III wrote:
> > > "BG" == Bron Gondwana  writes:
> > 
> > BG> Just to be really clear what this is.  It's per mailbox name - if
> > BG> you create and delete the SAME mailbox more 20 times, it only keeps
> > BG> the most recent 20 of that mailbox.
> > 
> > Hmm.  That's much less problematic, but it still allows someone to force
> > something to be deleted if they really want it to be deleted.  That's
> > not really an issue for me because my users wouldn't figure it out, but
> > I can imagine that someone using delayed expiry to easily implement some
> > sort of legal requirement might be unhappy.  But that's somewhat of a
> > stretch.
> 
> Yep.
> 
> Anyway, magic numbers are bad, so I will make this configurable.  It's easy to
> do, and if people with different systems need it changed, then that's fine.
> 
> With uniqueid based storage it will all be nicer anyway :)
> 
> Bron.
> 
> 
> -- 
>   Bron Gondwana
>   br...@fastmail.fm
> 

Cheers, Bron.
configurable on imapd.conf .
But I guess it is still not enogh to protect against the DoS / waste space you 
cited.
Your ideas of 2 quotas and having means to also control individual total quota 
is better suited for these tasks.
Are there better ideas?
Regards.
Andre Felipe


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: [cyrus 3.0] 20 delayed mailbox deleted limit?

2016-06-09 Thread Bron Gondwana via Info-cyrus 
On Fri, Jun 10, 2016, at 09:41, Jason L Tibbitts III wrote:
> > "BG" == Bron Gondwana  writes:
> 
> BG> Just to be really clear what this is.  It's per mailbox name - if
> BG> you create and delete the SAME mailbox more 20 times, it only keeps
> BG> the most recent 20 of that mailbox.
> 
> Hmm.  That's much less problematic, but it still allows someone to force
> something to be deleted if they really want it to be deleted.  That's
> not really an issue for me because my users wouldn't figure it out, but
> I can imagine that someone using delayed expiry to easily implement some
> sort of legal requirement might be unhappy.  But that's somewhat of a
> stretch.

Yep.

Anyway, magic numbers are bad, so I will make this configurable.  It's easy to
do, and if people with different systems need it changed, then that's fine.

With uniqueid based storage it will all be nicer anyway :)

Bron.


-- 
  Bron Gondwana
  br...@fastmail.fm

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: [cyrus 3.0] 20 delayed mailbox deleted limit?

2016-06-09 Thread Jason L Tibbitts III via Info-cyrus 
> "BG" == Bron Gondwana  writes:

BG> Just to be really clear what this is.  It's per mailbox name - if
BG> you create and delete the SAME mailbox more 20 times, it only keeps
BG> the most recent 20 of that mailbox.

Hmm.  That's much less problematic, but it still allows someone to force
something to be deleted if they really want it to be deleted.  That's
not really an issue for me because my users wouldn't figure it out, but
I can imagine that someone using delayed expiry to easily implement some
sort of legal requirement might be unhappy.  But that's somewhat of a
stretch.

 - J<

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: [cyrus 3.0] 20 delayed mailbox deleted limit?

2016-06-09 Thread Bron Gondwana via Info-cyrus 
On Fri, Jun 10, 2016, at 04:38, Jason L Tibbitts III wrote:
> > "BG" == Bron Gondwana via Info-cyrus  
> > writes:
> 
> BG> How would you suggest we protect against exploiting delayed delete
> BG> to fill the server without going over quota?
> 
> Well, I don't even run quotas.  But I do keep deleted messages around
> for 12 weeks, and even if I didn't, I do delete accounts occasionally.
> Deleting one account would go over the limit, and though I suck the
> messages out to mbox format for the final archiving, an instant nuke of
> older mailboxes would prevent an "easy" restore.
> 
> BG> Maybe a new quota
> BG> field for "total mailbox usage including deleted stuff" that can be
> BG> set to a high enough value that no reasonable user will ever hit it?
> 
> As long as I can just set it to 'unlimited', I don't care.  Disk is
> cheap and I don't have enough users to worry about it.  But I've had
> people delete all 100+ of their mailboxes before, and come screaming.

Just to be really clear what this is.  It's per mailbox name - if you create 
and delete
the SAME mailbox more 20 times, it only keeps the most recent 20 of that 
mailbox.

If you accidentally delete 100 mailboxes, they'll all still be there.

And it doesn't stop you deleting mailboxes or anything - it just immediately 
cleans
up the 21st one when you delete the mailbox again.

Bron.

-- 
  Bron Gondwana
  br...@fastmail.fm

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: [cyrus 3.0] 20 delayed mailbox deleted limit?

2016-06-09 Thread Jason L Tibbitts III via Info-cyrus 
> "BG" == Bron Gondwana via Info-cyrus  
> writes:

BG> How would you suggest we protect against exploiting delayed delete
BG> to fill the server without going over quota?

Well, I don't even run quotas.  But I do keep deleted messages around
for 12 weeks, and even if I didn't, I do delete accounts occasionally.
Deleting one account would go over the limit, and though I suck the
messages out to mbox format for the final archiving, an instant nuke of
older mailboxes would prevent an "easy" restore.

BG> Maybe a new quota
BG> field for "total mailbox usage including deleted stuff" that can be
BG> set to a high enough value that no reasonable user will ever hit it?

As long as I can just set it to 'unlimited', I don't care.  Disk is
cheap and I don't have enough users to worry about it.  But I've had
people delete all 100+ of their mailboxes before, and come screaming.

 - J<

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: [cyrus 3.0] 20 delayed mailbox deleted limit?

2016-06-09 Thread Andre Felipe Machado via Info-cyrus 
Andrew Morgan  wrote ..
> On Thu, 9 Jun 2016, Andre Felipe Machado via Info-cyrus wrote:
> 
> > Bron Gondwana via Info-cyrus  wrote ..
> >> On Thu, Jun 9, 2016, at 03:02, Andre Felipe Machado via Info-cyrus wrote:
> >>> Hello,
> >>> At future release notes I read
> >>> "Under delete_mode: delayed, only the 20 most recently deleted mailboxes 
> >>> are
> >> kept for any given name."
> >>> https://cyrusimap.org/imap/release-notes/3.0/x/3.0.0-beta2.html
> >>> Is there any configuration parameter to increase this limit?
> >>> Why this limit is needed?
> >>
> >> denial of service / space wastage protection.  There's no config option 
> >> available
> >> right now.  I could be convinced to change it.
> >>
> >> How would you suggest we protect against exploiting delayed delete to fill 
> >> the
> >> server without going over quota?  Maybe a new quota field for "total 
> >> mailbox
> usage
> >> including deleted stuff" that can be set to a high enough value that no 
> >> reasonable
> >> user will ever hit it?
> >>
> >> Bron.
> >>
> >> --
> >>   Bron Gondwana
> >>   br...@fastmail.fm
> >> 
> >
> > Hello, Bron
> > I understand the problem.
> > But at a corporate scenario, it is a rare event, because of jobs at stake, 
> > tracked
> user accounts,  antispam measures, etc.
> > It is more likely a "rogue" client,  bug/misconfiguration on a smartphone 
> > causing
> such problems.
> > We stay with official debian repositories versions as long as we could, 
> > receiving
> security patches.
> > So, mantaining an unofficial patch will be a big problem.
> > The sysadmin configurable parameters will be a more elegant solution.
> > Having configurations at sysadmin control will mantain cyrus flexible for 
> > use
> at different usage scenarios.
> > For the DoS / waste space problems, the 2 quota limits configurations are 
> > more
> suitable than counting folders quantity.
> > What if each folder contains 1 TB deleted messages?
> > Maybe a reasonable default (10 times user quota?) for those not wanting to 
> > configure
> is good idea.
> > Even better to have also a way to control individual accounts total quotas, 
> > for
> those corporate accounts like "sa...@foo.bar" that  receive lots of legitimate
> emails and have to
> > delete them after processing.
> > We have zabbix monitoring space at our cyrus backends, and need unlimited  
> > or
> configurable delayed expunge limits for recovering messages and folders for 
> years
> at corporate
> > scenario.
> > Thanks .
> > Andre Felipe
> 
> Remember, this is a limit on the number of deleted *mailboxes* kept, not 
> messages.
> 
> Bron, this could impact Pine/Alpine users that frequently postpone 
> messages.  Pine creates a folder named "postponed-msgs" to store drafts. 
> The folder is created when a draft is saved and deleted when all drafts 
> have been deleted/sent.
> 
> Here is my personal deleted folders list, right now:
> 
> DELETED.user.morgan.postponed-msgs.5755CF0C 0 p2 morgan lrswipkxtecda
> DELETED.user.morgan.postponed-msgs.5755F446 0 p2 morgan lrswipkxtecda
> DELETED.user.morgan.postponed-msgs.5755F486 0 p2 morgan lrswipkxtecda
> DELETED.user.morgan.postponed-msgs.5755F4D1 0 p2 morgan lrswipkxtecda
> DELETED.user.morgan.postponed-msgs.5755F4E4 0 p2 morgan lrswipkxtecda
> DELETED.user.morgan.postponed-msgs.5755F50E 0 p2 morgan lrswipkxtecda
> DELETED.user.morgan.postponed-msgs.5755F65F 0 p2 morgan lrswipkxtecda
> DELETED.user.morgan.postponed-msgs.5755F844 0 p2 morgan lrswipkxtecda
> DELETED.user.morgan.postponed-msgs.5756ECFC 0 p2 morgan lrswipkxtecda
> DELETED.user.morgan.postponed-msgs.5756F602 0 p2 morgan lrswipkxtecda
> DELETED.user.morgan.postponed-msgs.575706F8 0 p2 morgan lrswipkxtecda
> DELETED.user.morgan.postponed-msgs.57585C5D 0 p2 morgan lrswipkxtecda
> DELETED.user.morgan.postponed-msgs.57587FE1 0 p2 morgan lrswipkxtecda
> 
> We are removing deleted mailboxes after 7 days:
> 
> delprune  cmd="/usr/local/cyrus/bin/cyr_expire -E 1 -X 7 -D 7" at=0100
> 
> 
> I don't know if other IMAP clients have similar quirky behavior, but I 
> could see myself running into this limit.  However, I certainly don't care 
> about recovering my old postponed-msgs mailboxes.
> 
> Hmmm, is this a limit per-mailbox (user.morgan.postponed-msgs) or per-user 
> (all mailboxes under user.morgan)?
> 
> Thanks,
>   Andy


Hello, Andrew
Yes, I am aware of being mailboxes limit. This causes it to be even less 
suitable for the intended DoS/waste space control than the 2 quotas idea and 
less yet at corporate 
scenario. And there are the individual total quota idea to evaluate.
We observed that there are corporate users that organize their inboxes at 
extreme levels, containing dozens of folders, classified by project, by 
department, by date, by subject, by 
sender, etc. 
Sometimes their reorganize / delete many of

Re: [cyrus 3.0] 20 delayed mailbox deleted limit?

2016-06-09 Thread Andrew Morgan via Info-cyrus 

On Thu, 9 Jun 2016, Andre Felipe Machado via Info-cyrus wrote:


Bron Gondwana via Info-cyrus  wrote ..

On Thu, Jun 9, 2016, at 03:02, Andre Felipe Machado via Info-cyrus wrote:

Hello,
At future release notes I read
"Under delete_mode: delayed, only the 20 most recently deleted mailboxes are

kept for any given name."

https://cyrusimap.org/imap/release-notes/3.0/x/3.0.0-beta2.html
Is there any configuration parameter to increase this limit?
Why this limit is needed?


denial of service / space wastage protection.  There's no config option 
available
right now.  I could be convinced to change it.

How would you suggest we protect against exploiting delayed delete to fill the
server without going over quota?  Maybe a new quota field for "total mailbox 
usage
including deleted stuff" that can be set to a high enough value that no 
reasonable
user will ever hit it?

Bron.

--
  Bron Gondwana
  br...@fastmail.fm



Hello, Bron
I understand the problem.
But at a corporate scenario, it is a rare event, because of jobs at stake, 
tracked user accounts,  antispam measures, etc.
It is more likely a "rogue" client,  bug/misconfiguration on a smartphone 
causing such problems.
We stay with official debian repositories versions as long as we could, 
receiving security patches.
So, mantaining an unofficial patch will be a big problem.
The sysadmin configurable parameters will be a more elegant solution.
Having configurations at sysadmin control will mantain cyrus flexible for use 
at different usage scenarios.
For the DoS / waste space problems, the 2 quota limits configurations are more 
suitable than counting folders quantity.
What if each folder contains 1 TB deleted messages?
Maybe a reasonable default (10 times user quota?) for those not wanting to 
configure is good idea.
Even better to have also a way to control individual accounts total quotas, for those 
corporate accounts like "sa...@foo.bar" that  receive lots of legitimate emails 
and have to
delete them after processing.
We have zabbix monitoring space at our cyrus backends, and need unlimited  or 
configurable delayed expunge limits for recovering messages and folders for 
years at corporate
scenario.
Thanks .
Andre Felipe


Remember, this is a limit on the number of deleted *mailboxes* kept, not 
messages.


Bron, this could impact Pine/Alpine users that frequently postpone 
messages.  Pine creates a folder named "postponed-msgs" to store drafts. 
The folder is created when a draft is saved and deleted when all drafts 
have been deleted/sent.


Here is my personal deleted folders list, right now:

DELETED.user.morgan.postponed-msgs.5755CF0C 0 p2 morgan lrswipkxtecda
DELETED.user.morgan.postponed-msgs.5755F446 0 p2 morgan lrswipkxtecda
DELETED.user.morgan.postponed-msgs.5755F486 0 p2 morgan lrswipkxtecda
DELETED.user.morgan.postponed-msgs.5755F4D1 0 p2 morgan lrswipkxtecda
DELETED.user.morgan.postponed-msgs.5755F4E4 0 p2 morgan lrswipkxtecda
DELETED.user.morgan.postponed-msgs.5755F50E 0 p2 morgan lrswipkxtecda
DELETED.user.morgan.postponed-msgs.5755F65F 0 p2 morgan lrswipkxtecda
DELETED.user.morgan.postponed-msgs.5755F844 0 p2 morgan lrswipkxtecda
DELETED.user.morgan.postponed-msgs.5756ECFC 0 p2 morgan lrswipkxtecda
DELETED.user.morgan.postponed-msgs.5756F602 0 p2 morgan lrswipkxtecda
DELETED.user.morgan.postponed-msgs.575706F8 0 p2 morgan lrswipkxtecda
DELETED.user.morgan.postponed-msgs.57585C5D 0 p2 morgan lrswipkxtecda
DELETED.user.morgan.postponed-msgs.57587FE1 0 p2 morgan lrswipkxtecda

We are removing deleted mailboxes after 7 days:

delprune  cmd="/usr/local/cyrus/bin/cyr_expire -E 1 -X 7 -D 7" at=0100


I don't know if other IMAP clients have similar quirky behavior, but I 
could see myself running into this limit.  However, I certainly don't care 
about recovering my old postponed-msgs mailboxes.


Hmmm, is this a limit per-mailbox (user.morgan.postponed-msgs) or per-user 
(all mailboxes under user.morgan)?


Thanks,
Andy

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: [cyrus 3.0] 20 delayed mailbox deleted limit?

2016-06-09 Thread Andre Felipe Machado via Info-cyrus 
Bron Gondwana via Info-cyrus  wrote ..
> On Thu, Jun 9, 2016, at 03:02, Andre Felipe Machado via Info-cyrus wrote:
> > Hello,
> > At future release notes I read
> > "Under delete_mode: delayed, only the 20 most recently deleted mailboxes are
> kept for any given name."
> > https://cyrusimap.org/imap/release-notes/3.0/x/3.0.0-beta2.html
> > Is there any configuration parameter to increase this limit?
> > Why this limit is needed?
> 
> denial of service / space wastage protection.  There's no config option 
> available
> right now.  I could be convinced to change it.
> 
> How would you suggest we protect against exploiting delayed delete to fill the
> server without going over quota?  Maybe a new quota field for "total mailbox 
> usage
> including deleted stuff" that can be set to a high enough value that no 
> reasonable
> user will ever hit it?
> 
> Bron.
> 
> -- 
>   Bron Gondwana
>   br...@fastmail.fm
> 

Hello, Bron
I understand the problem.
But at a corporate scenario, it is a rare event, because of jobs at stake, 
tracked user accounts,  antispam measures, etc.
It is more likely a "rogue" client,  bug/misconfiguration on a smartphone 
causing such problems.
We stay with official debian repositories versions as long as we could, 
receiving security patches.
So, mantaining an unofficial patch will be a big problem.
The sysadmin configurable parameters will be a more elegant solution.
Having configurations at sysadmin control will mantain cyrus flexible for use 
at different usage scenarios.
For the DoS / waste space problems, the 2 quota limits configurations are more 
suitable than counting folders quantity.
What if each folder contains 1 TB deleted messages?
Maybe a reasonable default (10 times user quota?) for those not wanting to 
configure is good idea.
Even better to have also a way to control individual accounts total quotas, for 
those corporate accounts like "sa...@foo.bar" that  receive lots of legitimate 
emails and have to 
delete them after processing.
We have zabbix monitoring space at our cyrus backends, and need unlimited  or 
configurable delayed expunge limits for recovering messages and folders for 
years at corporate 
scenario.
Thanks .
Andre Felipe

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: [cyrus 3.0] 20 delayed mailbox deleted limit?

2016-06-08 Thread John Capo via Info-cyrus 
On Wed, June 8, 2016 20:23, Bron Gondwana via Info-cyrus wrote:
> On Thu, Jun 9, 2016, at 03:02, Andre Felipe Machado via Info-cyrus wrote:
>
>> Hello,
>> At future release notes I read
>> "Under delete_mode: delayed, only the 20 most recently deleted mailboxes are 
>> kept for
>> any given name." 
>> https://cyrusimap.org/imap/release-notes/3.0/x/3.0.0-beta2.html
>> Is there any configuration parameter to increase this limit?
>> Why this limit is needed?
>>
>
> denial of service / space wastage protection.  There's no config option 
> available right
> now.  I could be convinced to change it.
>
> How would you suggest we protect against exploiting delayed delete to fill 
> the server
> without going over quota?  Maybe a new quota field for "total mailbox usage 
> including
> deleted stuff" that can be set to a high enough value that no reasonable user 
> will ever
> hit it?

Fastmail needs to protect against malicious users but barring an account being
compromised, the business world does not have that problem.  A config option 
would be
nice or very isolated code that can be easily patched out.

John Capo

>
> Bron.
>
>
> --
> Bron Gondwana
> br...@fastmail.fm 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
>
>



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: [cyrus 3.0] 20 delayed mailbox deleted limit?

2016-06-08 Thread Bron Gondwana via Info-cyrus 
On Thu, Jun 9, 2016, at 03:02, Andre Felipe Machado via Info-cyrus wrote:
> Hello,
> At future release notes I read
> "Under delete_mode: delayed, only the 20 most recently deleted mailboxes are 
> kept for any given name."
> https://cyrusimap.org/imap/release-notes/3.0/x/3.0.0-beta2.html
> Is there any configuration parameter to increase this limit?
> Why this limit is needed?

denial of service / space wastage protection.  There's no config option 
available right now.  I could be convinced to change it.

How would you suggest we protect against exploiting delayed delete to fill the 
server without going over quota?  Maybe a new quota field for "total mailbox 
usage including deleted stuff" that can be set to a high enough value that no 
reasonable user will ever hit it?

Bron.

-- 
  Bron Gondwana
  br...@fastmail.fm

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

[cyrus 3.0] 20 delayed mailbox deleted limit?

2016-06-08 Thread Andre Felipe Machado via Info-cyrus 
Hello,
At future release notes I read
"Under delete_mode: delayed, only the 20 most recently deleted mailboxes are 
kept for any given name."
https://cyrusimap.org/imap/release-notes/3.0/x/3.0.0-beta2.html
Is there any configuration parameter to increase this limit?
Why this limit is needed?
Regards.
Andre Felipe

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus