Re: [Dovecot] Issues with SIS and Backups - was Re: v2.1.0 status

2012-02-13 Thread Timo Sirainen
On 13.2.2012, at 9.49, Jan-Frode Myklebust wrote:

> On Sun, Feb 12, 2012 at 05:58:20PM +0200, Timo Sirainen wrote:
>> 
>> doveadm backup -u user@domain backup:
>> 
>> And it would output the user's messages to stdout (or to some file). So it 
>> would be similar to e.g. PostgreSQL's pg_dump.
> 
> So only full backups, no incremental backups?

I know people are doing incremental backups with PostgreSQL, so it should be 
possible with Dovecot too. I'm not sure about the details yet tough.

> Then what's the benefit over just copying the files (of a snapshot)?

If SIS is enabled, it's a little difficult to restore specific user's mails 
from a filesystem based backup.



Re: [Dovecot] Issues with SIS and Backups - was Re: v2.1.0 status

2012-02-12 Thread Jan-Frode Myklebust
On Sun, Feb 12, 2012 at 05:58:20PM +0200, Timo Sirainen wrote:
> 
> doveadm backup -u user@domain backup:
> 
> And it would output the user's messages to stdout (or to some file). So it 
> would be similar to e.g. PostgreSQL's pg_dump.

So only full backups, no incremental backups? Then what's the benefit
over just copying the files (of a snapshot)?


  -jf


Re: [Dovecot] Issues with SIS and Backups - was Re: v2.1.0 status

2012-02-12 Thread Charles Marcus

On 2012-02-12 10:58 AM, Timo Sirainen  wrote:

On 12.2.2012, at 15.43, Charles Marcus wrote:



I'm planning on creating a new "backup" mail storage backend, where
you could basically run:

doveadm backup -u user@domain backup:

And it would output the user's messages to stdout (or to some file).
So it would be similar to e.g. PostgreSQL's pg_dump. Then for
restoring you can use the same input for "doveadm import". So in the
backups there would be no deduplication, unless implemented by the
backup software.


Hmmm, ok thanks... loss of dedupe on the backup server may or may not be 
an issue I'll be concerned about, but good to know that there is an 
option at least.



It looks like using mimedefang would eliminate the problem with
backups, and would provide the added benefit of eliminating large
file attachments for external recipients as well, however I'm
concerned about security - ie, how do you limit access to these
attachments to only the recipients of the message with the
hyperlink in it?



As long as the URL is random enough it shouldn't be possible to guess
the links. But if they're HTML mails and such then hyperlinks would
send the referrer-url to the next site. I guess you could also add
HTTP authentication.


Thanks for that too... I'll have to a bit of research on  this, because 
the more I think about it, the more I like the idea, *if* it can be done 
securely enough...


--

Best regards,

Charles


Re: [Dovecot] Issues with SIS and Backups - was Re: v2.1.0 status

2012-02-12 Thread Timo Sirainen
On 12.2.2012, at 15.43, Charles Marcus wrote:

> One thing that concerns me right now about my main clients site that I will 
> soon be migrating from courier-imap to dovecot is the message/thread a while 
> back about the problems with SIS and backups.

I'm planning on creating a new "backup" mail storage backend, where you could 
basically run:

doveadm backup -u user@domain backup:

And it would output the user's messages to stdout (or to some file). So it 
would be similar to e.g. PostgreSQL's pg_dump. Then for restoring you can use 
the same input for "doveadm import". So in the backups there would be no 
deduplication, unless implemented by the backup software.

> I am trying to decide between using dovecot's SIS, or using mimedefang's 
> ability to replace attachments with hyperlinks to them (storing them on a 
> webserver - still not clear on exactly how it works or the security 
> implications of doing so). It looks like using mimedefang would eliminate the 
> problem with backups, and would provide the added benefit of eliminating 
> large file attachments for external recipients as well, however I'm concerned 
> about security - ie, how do you limit access to these attachments to only the 
> recipients of the message with the hyperlink in it?

As long as the URL is random enough it shouldn't be possible to guess the 
links. But if they're HTML mails and such then hyperlinks would send the 
referrer-url to the next site. I guess you could also add HTTP authentication.

[Dovecot] Issues with SIS and Backups - was Re: v2.1.0 status

2012-02-12 Thread Charles Marcus

Hi Timo,

One thing that concerns me right now about my main clients site that I 
will soon be migrating from courier-imap to dovecot is the 
message/thread a while back about the problems with SIS and backups. I 
am trying to decide between using dovecot's SIS, or using mimedefang's 
ability to replace attachments with hyperlinks to them (storing them on 
a webserver - still not clear on exactly how it works or the security 
implications of doing so). It looks like using mimedefang would 
eliminate the problem with backups, and would provide the added benefit 
of eliminating large file attachments for external recipients as well, 
however I'm concerned about security - ie, how do you limit access to 
these attachments to only the recipients of the message with the 
hyperlink in it?


Anyway here is the thread outlining the problem with dovecot's SIS and 
backups:


http://www.dovecot.org/list/dovecot/2011-December/062340.html

Have you given this any more thought?

--

Best regards,

Charles