Re: Questions about hardlinks, alternate storage and compression]

[Gionatan Danti]

OK, I see. Let me attack the problem from another side: there is any 
method to use some sort of message filtering (not necessarily Sieve) 
without destroying hardlinking capability? Or are you all running 
dovecot without hardlinking?


Thanks.

Il 08-06-2016 02:13 Timo Sirainen ha scritto:

It's a rather difficult change and also pretty low priority right now.
I'd be happy to explain the details to whoever wants to develop this,
but I wouldn't be surprised if it was still unimplemented a few years
from now.


On 07 Jun 2016, at 15:59, Gionatan Danti  wrote:

Hi list,
any news regarding hardlinks + sieve/pigeonhole setup?

Hardlinking seems a very important feature to me, and I would really 
like to get it working.


Thanks.

On 30/11/2015 14:23, Timo Sirainen wrote:

On 30 Nov 2015, at 10:21, Gionatan Danti  wrote:


So, let me do a straigth question: is someone using dovecot/LMTP 
with hardlinking? To me, this seems a _very_ important feature, and 
I wonder if I am doing something wrong or if the feature 
(hardlink+sieve) simply does not exists.


Hardlink+Sieve has never worked. The fix is a bit complicated. Here's 
my TODO entry about it:


 - remove mail_deliver_session after all, do all the stuff 
transparently

   by hooking into mailbox_copy().
 - use this hook also to do the mail deduplication: 1) sort all 
destination
   users, 2) create mail_user only once for each user, 3) 
remember in
   src_mail the previously copied mail, 4) use that for 
mailbox_copy()ing

   to following recipients
 - make sure this removes duplicate dbox mails when sieve saves 
mail to

   multiple mailboxes



--
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.da...@assyoma.it - i...@assyoma.it
GPG public key ID: FF5F32A8


--
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.da...@assyoma.it - i...@assyoma.it
GPG public key ID: FF5F32A8

Re: Questions about hardlinks, alternate storage and compression]

[Timo Sirainen]

It's a rather difficult change and also pretty low priority right now. I'd be 
happy to explain the details to whoever wants to develop this, but I wouldn't 
be surprised if it was still unimplemented a few years from now.

> On 07 Jun 2016, at 15:59, Gionatan Danti  wrote:
> 
> Hi list,
> any news regarding hardlinks + sieve/pigeonhole setup?
> 
> Hardlinking seems a very important feature to me, and I would really like to 
> get it working.
> 
> Thanks.
> 
> On 30/11/2015 14:23, Timo Sirainen wrote:
>> On 30 Nov 2015, at 10:21, Gionatan Danti  wrote:
>>> 
>>> So, let me do a straigth question: is someone using dovecot/LMTP with 
>>> hardlinking? To me, this seems a _very_ important feature, and I wonder if 
>>> I am doing something wrong or if the feature (hardlink+sieve) simply does 
>>> not exists.
>> 
>> Hardlink+Sieve has never worked. The fix is a bit complicated. Here's my 
>> TODO entry about it:
>> 
>>  - remove mail_deliver_session after all, do all the stuff transparently
>>by hooking into mailbox_copy().
>>  - use this hook also to do the mail deduplication: 1) sort all 
>> destination
>>users, 2) create mail_user only once for each user, 3) remember in
>>src_mail the previously copied mail, 4) use that for mailbox_copy()ing
>>to following recipients
>>  - make sure this removes duplicate dbox mails when sieve saves mail to
>>multiple mailboxes
>> 
> 
> -- 
> Danti Gionatan
> Supporto Tecnico
> Assyoma S.r.l. - www.assyoma.it
> email: g.da...@assyoma.it - i...@assyoma.it
> GPG public key ID: FF5F32A8

Re: Questions about hardlinks, alternate storage and compression]

[Gionatan Danti]

Hi list,
any news regarding hardlinks + sieve/pigeonhole setup?

Hardlinking seems a very important feature to me, and I would really 
like to get it working.


Thanks.

On 30/11/2015 14:23, Timo Sirainen wrote:

On 30 Nov 2015, at 10:21, Gionatan Danti  wrote:


So, let me do a straigth question: is someone using dovecot/LMTP with 
hardlinking? To me, this seems a _very_ important feature, and I wonder if I am 
doing something wrong or if the feature (hardlink+sieve) simply does not exists.


Hardlink+Sieve has never worked. The fix is a bit complicated. Here's my TODO 
entry about it:

  - remove mail_deliver_session after all, do all the stuff transparently
by hooking into mailbox_copy().
  - use this hook also to do the mail deduplication: 1) sort all destination
users, 2) create mail_user only once for each user, 3) remember in
src_mail the previously copied mail, 4) use that for mailbox_copy()ing
to following recipients
  - make sure this removes duplicate dbox mails when sieve saves mail to
multiple mailboxes



--
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.da...@assyoma.it - i...@assyoma.it
GPG public key ID: FF5F32A8

Re: Questions about hardlinks, alternate storage and compression]

[Gionatan Danti]

Il 30-11-2015 23:23 Timo Sirainen ha scritto:

On 30 Nov 2015, at 17:48, Gionatan Danti  wrote:


Hi Timo,
glad to know it is in your TODO list ;)


It's been for many years.


Any rough ETA on that?


Right now it doesn't seem likely to be developed anytime soon.



Thank you anyway :)

It let me wonder if I am the only one caring about hardlinks...
Maybe I overstate the space savings?

Thanks.

--
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.da...@assyoma.it - i...@assyoma.it
GPG public key ID: FF5F32A8

Re: Questions about hardlinks, alternate storage and compression]

[Gionatan Danti]

Hi Timo,
glad to know it is in your TODO list ;)

Any rough ETA on that?
Thanks.

On 30/11/2015 14:23, Timo Sirainen wrote:

On 30 Nov 2015, at 10:21, Gionatan Danti  wrote:


So, let me do a straigth question: is someone using dovecot/LMTP with 
hardlinking? To me, this seems a _very_ important feature, and I wonder if I am 
doing something wrong or if the feature (hardlink+sieve) simply does not exists.


Hardlink+Sieve has never worked. The fix is a bit complicated. Here's my TODO 
entry about it:

  - remove mail_deliver_session after all, do all the stuff transparently
by hooking into mailbox_copy().
  - use this hook also to do the mail deduplication: 1) sort all destination
users, 2) create mail_user only once for each user, 3) remember in
src_mail the previously copied mail, 4) use that for mailbox_copy()ing
to following recipients
  - make sure this removes duplicate dbox mails when sieve saves mail to
multiple mailboxes



--
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.da...@assyoma.it - i...@assyoma.it
GPG public key ID: FF5F32A8

Re: Questions about hardlinks, alternate storage and compression]

[Timo Sirainen]

On 30 Nov 2015, at 17:48, Gionatan Danti  wrote:
> 
> Hi Timo,
> glad to know it is in your TODO list ;)

It's been for many years.

> Any rough ETA on that?

Right now it doesn't seem likely to be developed anytime soon.

> Thanks.
> 
> On 30/11/2015 14:23, Timo Sirainen wrote:
>> On 30 Nov 2015, at 10:21, Gionatan Danti  wrote:
>>> 
>>> So, let me do a straigth question: is someone using dovecot/LMTP with 
>>> hardlinking? To me, this seems a _very_ important feature, and I wonder if 
>>> I am doing something wrong or if the feature (hardlink+sieve) simply does 
>>> not exists.
>> 
>> Hardlink+Sieve has never worked. The fix is a bit complicated. Here's my 
>> TODO entry about it:
>> 
>>  - remove mail_deliver_session after all, do all the stuff transparently
>>by hooking into mailbox_copy().
>>  - use this hook also to do the mail deduplication: 1) sort all 
>> destination
>>users, 2) create mail_user only once for each user, 3) remember in
>>src_mail the previously copied mail, 4) use that for mailbox_copy()ing
>>to following recipients
>>  - make sure this removes duplicate dbox mails when sieve saves mail to
>>multiple mailboxes
>> 
> 
> -- 
> Danti Gionatan
> Supporto Tecnico
> Assyoma S.r.l. - www.assyoma.it
> email: g.da...@assyoma.it - i...@assyoma.it
> GPG public key ID: FF5F32A8

Re: Questions about hardlinks, alternate storage and compression]

[Gionatan Danti]

Hi all,
I have some updates on the hard-link discussion.

First, let me explain that I installed a test machine with CentOS 7.1 
and dovecot/pigeonhole version 2.2.10-4 and the results where identical 
on what I had on CentOS 6.7 and dovecot 2.0.9-19


The bottom line is that hardlinking works only when no, or at most only 
one, RCPT have sieve filtering. For example:

- if no RCPT has sieve filters, hardlink is OK
- if only one RCPT has sieve filters, hardlink is OK
- if two or more RCPTs have sieve filters, hardlink is NOT working.

Some time ago I reported that, when disabling the sieve_before and 
sieve_after options, hardlinks worked; however, this really was a side 
effect of what written above.


So, let me do a straigth question: is someone using dovecot/LMTP with 
hardlinking? To me, this seems a _very_ important feature, and I wonder 
if I am doing something wrong or if the feature (hardlink+sieve) simply 
does not exists.


Thanks.

On 27/11/2015 09:44, Gionatan Danti wrote:

centos 7.x is latest, so if you need help upgrade make a bug report to
centos, not waste your time here since you choiced a precompiled
problem, reporting here 7s helpfull if you use dovecot compiled with
the latest version here, this migth be a unknown bug you then have
found, but not if you use precompiled problems, then report to the
maintainers of such software


Hi,
does it means that when you use before_sieve and after_sieve in your
dovecot configuration the hard-linking part works?

I don't need help to upgrade to CentOS 7.x, but before going this route
I would understand if this is how dovecot/pigenhole are expected to work
(don't use hardlinks) or if I misconfigured something.

RedHat / CentOS are a very common base OS, so I wonder if I am the only
one hitting this problem or no.
Thanks.



--
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.da...@assyoma.it - i...@assyoma.it
GPG public key ID: FF5F32A8

Re: Questions about hardlinks, alternate storage and compression]

[Timo Sirainen]

On 30 Nov 2015, at 10:21, Gionatan Danti  wrote:
> 
> So, let me do a straigth question: is someone using dovecot/LMTP with 
> hardlinking? To me, this seems a _very_ important feature, and I wonder if I 
> am doing something wrong or if the feature (hardlink+sieve) simply does not 
> exists.

Hardlink+Sieve has never worked. The fix is a bit complicated. Here's my TODO 
entry about it:

 - remove mail_deliver_session after all, do all the stuff transparently
   by hooking into mailbox_copy().
 - use this hook also to do the mail deduplication: 1) sort all destination
   users, 2) create mail_user only once for each user, 3) remember in
   src_mail the previously copied mail, 4) use that for mailbox_copy()ing
   to following recipients
 - make sure this removes duplicate dbox mails when sieve saves mail to
   multiple mailboxes

Re: [g.da...@assyoma.it: [Dovecot] Re: Questions about hardlinks, alternate storage and compression]

[Gionatan Danti]

centos 7.x is latest, so if you need help upgrade make a bug report to
centos, not waste your time here since you choiced a precompiled
problem, reporting here 7s helpfull if you use dovecot compiled with
the latest version here, this migth be a unknown bug you then have
found, but not if you use precompiled problems, then report to the
maintainers of such software


Hi,
does it means that when you use before_sieve and after_sieve in your 
dovecot configuration the hard-linking part works?


I don't need help to upgrade to CentOS 7.x, but before going this route 
I would understand if this is how dovecot/pigenhole are expected to work 
(don't use hardlinks) or if I misconfigured something.


RedHat / CentOS are a very common base OS, so I wonder if I am the only 
one hitting this problem or no.

Thanks.

--
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.da...@assyoma.it - i...@assyoma.it
GPG public key ID: FF5F32A8

Re: Questions about hardlinks, alternate storage and compression

[Gionatan Danti]

Il 14-07-2015 14:44 Gionatan Danti ha scritto:

On 14/07/15 12:26, Steffen Kaiser wrote:



You asked about "newer dovecot versions", v2.2 does so.



Fair enough :)

So, with v2.2+ the hardlink approach is irremediably gone, at least
with LMTP (and without relying to SiS)?


Dear list,
sorry if I resume this (relatively) old post, but I would like to know 
if someone has some good suggestions/ideas.


A quick recap...
System spec:
- CentOS 6.6 x64
- dovecot-2.0.9-8.el6_6.4.x86_64 RPM package/version
- sdbox mail store

I have two problems:
1) hardlinks are not created when the same email is sent to multiple 
rcpt, nor when sending to the very same user (a non-linked copy is kept 
both in Inbox and Sent)
2) I would like to have an alternate storage and to enable compression 
on it only, leaving the main location without compression. It is 
possible?


Relative to the hard-link problem, I found that not using before_sieve 
and after_sieve solves my problem, so that hardlinks are created 
correctly. However, this is far from ideal because I lose global sieve 
rules (global/defaul rules are only applied when no customized rules 
exist).


Any idea to how to solve these two problem?
Thanks.

--
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.da...@assyoma.it - i...@assyoma.it
GPG public key ID: FF5F32A8

Re: [g.da...@assyoma.it: [Dovecot] Re: Questions about hardlinks, alternate storage and compression]

[Gionatan Danti]

Il 26-11-2015 15:15 John R. Dennison ha scritto:

You are strongly encouraged to update that CentOS system.  Current is
6.7 (released some 3 months ago) and dovecot-2.0.9-19.


Ouch! I copied outdated information from my old post.
My current system _is_ CentOS 6.7 with dovecot 
dovecot-2.0.9-19.el6_7.2.x86_64


Sorry for the confusion. Still, the problems remain


If you find you need a more modern dovecot version you will find that
dovecot-2.2.19-1 is in the GhettoForge 3rd-party repo, packaged and
maintained by a respected CentOS community member.  Information
on this repo can be found by following the links off of:

http://wiki.centos.org/AdditionalResources/Repositories



The point is that I _don't know_ if more recent Dovecot version have the 
same problem or not. Moreover, as this is a production machine, I would 
like to leave a key package as Dovecot as near as possible to the 
shipping O.S.


Thanks.

--
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.da...@assyoma.it - i...@assyoma.it
GPG public key ID: FF5F32A8

Re: [g.da...@assyoma.it: [Dovecot] Re: Questions about hardlinks, alternate storage and compression]

[Benny Pedersen]

On November 26, 2015 3:28:22 PM Gionatan Danti  wrote:


Il 26-11-2015 15:15 John R. Dennison ha scritto:

You are strongly encouraged to update that CentOS system.  Current is
6.7 (released some 3 months ago) and dovecot-2.0.9-19.


Ouch! I copied outdated information from my old post.
My current system _is_ CentOS 6.7 with dovecot


centos 7.x is latest, so if you need help upgrade make a bug report to 
centos, not waste your time here since you choiced a precompiled problem, 
reporting here 7s helpfull if you use dovecot compiled with the latest 
version here, this migth be a unknown bug you then have found, but not if 
you use precompiled problems, then report to the maintainers of such software

Re: Questions about hardlinks, alternate storage and compression

[Steffen Kaiser]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Mon, 13 Jul 2015, Gionatan Danti wrote:

On the other hand, private (per-user) sieve file works without interfering 
with hardlinks. In a similar manner, disabling sieve also permits dovecot to 
create multiple hardlinks for a single message.


Does someone know if newer dovecot versions change anything in this regard?


LMTP adds Delivered-To header, so all delivered messages are unique and 
you cannot hardlink messages regardless of Sieve.


If Dovecot LDA adds headers, too, I do not know.

- -- 
Steffen Kaiser

-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iQEVAwUBVaSpX3z1H7kL/d9rAQKuPwf/e4GddZvm/qj9sfAnVgV3H5iC62fnS6Ny
/TPaXcuLcN5Tx9slhLTwIx8/GRROUVwLVqKYjaXwQciV2yytBu5vkC0+lowIZGq9
kJAAKPp4h3Ia6SDGhI8E5Im9VGGSpbXyLKR+V3rf1G/sOyvJTITliVe4ckf76xrI
c1LGYumW0BGZeNZAAA0lYHZGrgy5meCrL20CMupmahoHsOFw5cA3HhJ/dEBRPlOJ
y886BScRh7dWJXyS+PUzPFlbFOeULKvh6fVwCK7b4+aFkfjLedDLew5TThWiblK5
c5+rx0pAh8xVdXGZyQXzPjUl22KbQmGfzv78XWlN2WksCnMVaFXe2g==
=3iPP
-END PGP SIGNATURE-

Re: Questions about hardlinks, alternate storage and compression

[Gionatan Danti]

On 14/07/15 08:17, Steffen Kaiser wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Mon, 13 Jul 2015, Gionatan Danti wrote:


On the other hand, private (per-user) sieve file works without
interfering with hardlinks. In a similar manner, disabling sieve also
permits dovecot to create multiple hardlinks for a single message.

Does someone know if newer dovecot versions change anything in this
regard?


LMTP adds Delivered-To header, so all delivered messages are unique and
you cannot hardlink messages regardless of Sieve.

If Dovecot LDA adds headers, too, I do not know.



Mmm... I'm using LMTP, but I can't find the Delivered-To header.
Below you can see an example of successfully hard-linked email [1]

I am missing something?

[1]
Return-Path: g.da...@assyoma.it
Received: from mail.gruppocrimi.it
by mail.gruppocrimi.it (Dovecot) with LMTP id 
VFA8Fj3OmlUStwAA6qgwnw

; Mon, 06 Jul 2015 20:51:41 +0200
Received: from localhost (localhost.localdomain [127.0.0.1])
by mail.gruppocrimi.it (Postfix) with ESMTP id 22AB4A1A85;
Mon,  6 Jul 2015 20:51:41 +0200 (CEST)
X-Virus-Scanned: amavisd-new at gruppocrimi.it
Received: from mail.gruppocrimi.it ([127.0.0.1])
by localhost (mail.gruppocrimi.it [127.0.0.1]) (amavisd-new, 
port 10024)

with LMTP id NC3YcizeDFPO; Mon,  6 Jul 2015 20:51:40 +0200 (CEST)
Received: from mr003msb.fastweb.it (mr003msb.fastweb.it [85.18.95.87])
by mail.gruppocrimi.it (Postfix) with ESMTP id 4380DA1A7C;
Mon,  6 Jul 2015 20:51:40 +0200 (CEST)
Received: from ceres.assyoma.it (93.63.55.57) by mr003msb.fastweb.it 
(8.5.140.03)

id 55501C9F0432631D; Mon, 6 Jul 2015 20:51:40 +0200
Received: by ceres.assyoma.it (Postfix, from userid 48)
id B7B912643B4; Mon,  6 Jul 2015 20:51:39 +0200 (CEST)
To: gionatan.da...@gruppocrimi.it, g.da...@gruppocrimi.it
Subject: test invio
X-PHP-Originating-Script: 0:rcube.php
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit
Date: Mon, 06 Jul 2015 20:51:39 +0200
From: Gionatan Danti g.da...@assyoma.it
Organization: Assyoma s.r.l.
Message-ID: ef3149d82ef4350c1c9b6da9c1f03...@assyoma.it
X-Sender: g.da...@assyoma.it
User-Agent: Roundcube Webmail/1.0.5


--
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.da...@assyoma.it - i...@assyoma.it
GPG public key ID: FF5F32A8

Re: Questions about hardlinks, alternate storage and compression

[Steffen Kaiser]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Tue, 14 Jul 2015, Gionatan Danti wrote:

On 14/07/15 08:17, Steffen Kaiser wrote:

On Mon, 13 Jul 2015, Gionatan Danti wrote:


On the other hand, private (per-user) sieve file works without
interfering with hardlinks. In a similar manner, disabling sieve also
permits dovecot to create multiple hardlinks for a single message.

Does someone know if newer dovecot versions change anything in this
regard?


LMTP adds Delivered-To header, so all delivered messages are unique and
you cannot hardlink messages regardless of Sieve.

If Dovecot LDA adds headers, too, I do not know.



Mmm... I'm using LMTP, but I can't find the Delivered-To header.
Below you can see an example of successfully hard-linked email [1]


You asked about newer dovecot versions, v2.2 does so.

- -- 
Steffen Kaiser

-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iQEVAwUBVaTjzXz1H7kL/d9rAQK1oAf/fdUuBY8kseVEFa5kyXG01cyUjc3RfBNl
o0EYm+e2hvoz5B4N96pkbmilYjaCtgUw/qlMnGkzFbmJDwrqOiAhxOG71Aewjvbx
q42cXHtw7CsOCr6y+eshNUfU3T20f7wgvyJDqLAOwg/pSP3CjU9m93D2zCqUgDXO
MHuDV1zEEljlrxXmtdG8GI5YlwkBqvWXQuPbXr7PhoQ4HTKhvKHWurGvVkfBlg6k
cpuy40mSWY3ZXwNDcnHP0o82EezGAdgzDE/EoV4fV0JDvANbTjpwwqE4gMW+wOM+
lUJnMyawkVuvfbB85K/tkK+a0lIVnZOwdUy0RaUcJFeZHXdRsixvIg==
=HzBs
-END PGP SIGNATURE-

Re: Questions about hardlinks, alternate storage and compression

[Gionatan Danti]

On 14/07/15 12:26, Steffen Kaiser wrote:



You asked about newer dovecot versions, v2.2 does so.



Fair enough :)

So, with v2.2+ the hardlink approach is irremediably gone, at least with 
LMTP (and without relying to SiS)?


--
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.da...@assyoma.it - i...@assyoma.it
GPG public key ID: FF5F32A8

Re: Questions about hardlinks, alternate storage and compression

[Felix Zielcke]

Am Dienstag, den 14.07.2015, 12:26 +0200 schrieb Steffen Kaiser:
 On Tue, 14 Jul 2015, Gionatan Danti wrote:
  On 14/07/15 08:17, Steffen Kaiser wrote:
   On Mon, 13 Jul 2015, Gionatan Danti wrote:
   
On the other hand, private (per-user) sieve file works without
interfering with hardlinks. In a similar manner, disabling 
sieve also
permits dovecot to create multiple hardlinks for a single 
message.

Does someone know if newer dovecot versions change anything in 
this
regard?
   
   LMTP adds Delivered-To header, so all delivered messages are 
   unique and
   you cannot hardlink messages regardless of Sieve.
   
   If Dovecot LDA adds headers, too, I do not know.
   
  
  Mmm... I'm using LMTP, but I can't find the Delivered-To header.
  Below you can see an example of successfully hard-linked email [1]
 
 You asked about newer dovecot versions, v2.2 does so.
 

I just updated my Dovecot 2.2.13 to the current 2.2.18
This config option was added in the meanwhile:

# Which recipient address to use for Delivered-To: header and Received:
# header. The default is final, which is the same as the one given to
# RCPT TO command. original uses the address given in RCPT TO's ORCPT
# parameter, none uses nothing. Note that none is currently always
used
# when a mail has multiple recipients.
#lmtp_hdr_delivery_address = final

Doestn't that mean if you set it to none that no Delived-To: header
gets added then?

Re: Questions about hardlinks, alternate storage and compression

[Gionatan Danti]

Hi Javier,
thanks for your reply.

I already checked SIS and, while interesting, is not what I want, because:
1) it can be difficult to restore a single message/attachment from a backup
2) only the attachments, and not the entire messages, are deduped.

Message-based hardlinks really exists for a reason. The good news is 
that I found _why_ they are not working: it depends from how dovecot and 
its sieve plugin (pigenhole) interact. Basically, if I define anything 
for the before_sieve and after_sieve variables, dovecot stops creating 
hardlinks for multiple copies of the same message.


On the other hand, private (per-user) sieve file works without 
interfering with hardlinks. In a similar manner, disabling sieve also 
permits dovecot to create multiple hardlinks for a single message.


Does someone know if newer dovecot versions change anything in this regard?

Thank you all.

On 13/07/15 11:10, Javier Miguel Rodríguez wrote:

Search about single instance storage dovecot. This is what you need.

Regards

Javier

On 27/06/2015 18:18, Gionatan Danti wrote:

Hi all,
I have some questions about hardlinks, alternate storage and
compression. I already scanned the list for related information and I
have an idea of how things works, but I would like to have a definite
answer.

System spec:
- CentOS 6.6 x64
- dovecot-2.0.9-8.el6_6.4.x86_64 RPM package/version
- sdbox mail store

1) About hardlinks: when sending the same message to two different
recipients, I see that the two u.x files are created as two different
files. Diffing them, I see that the only difference is a single char
(see [1] for an example). My questions are:
  a) it is possible to tell dovecot to create a single file + a single
hardlink (linkref=2)? As other IMAP servers support that features (eg:
Cyrus, CommunigatePro, etc) I am wondering if I missed something in
configuring dovecot...
  b) If it is not possible, can I run a script that compare the
various files and substitute equal ones (minus the changing line) with
hardlinks, or it will confuse dovecot? As a side note, why the
changing line ever exists?

2) About alternate storage and compression: actually, I use a single
mail_location without compression. I would like to have an alternate
storage and to enable compression on it only, leaving the main
location without compression. I if understand it correctly, it
_should_ be done using a command similar to doveadm -Dv -o
plugin/zlib_save=gz altmove -uu testuser sentbefore 8d. I'm right
thinking that it should work? I will really end with a primary
uncompressed mail store and an alternate, zlib-compressed one?

Thank you all and sorry if I did some naive questions.

[1]
63c63
 G2fd0811c64be8e553d97eaa8309f
---

G2ed0811c64be8e553d97eaa8309f








--
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.da...@assyoma.it - i...@assyoma.it
GPG public key ID: FF5F32A8

Re: Questions about hardlinks, alternate storage and compression

[Gionatan Danti]

Sorry list,
anyone with a good idea / suggestion?

Thanks.

Il 29-06-2015 11:20 Gionatan Danti ha scritto:

Hi all,
any ideas?

Especially point n.1 (no hardlink when sending the same email to
multiple addresses) confuse me a bit. Searching in old messages I even
stumbled on some users stating that, using Dovecot LMTP server, they
achieved what I want (one messagge, multiple hardlinks), but I am
_already_ using LMTP with no avail...

Regards.

On 27/06/15 18:18, Gionatan Danti wrote:

Hi all,
I have some questions about hardlinks, alternate storage and
compression. I already scanned the list for related information and I
have an idea of how things works, but I would like to have a definite
answer.

System spec:
- CentOS 6.6 x64
- dovecot-2.0.9-8.el6_6.4.x86_64 RPM package/version
- sdbox mail store

1) About hardlinks: when sending the same message to two different
recipients, I see that the two u.x files are created as two different
files. Diffing them, I see that the only difference is a single char
(see [1] for an example). My questions are:
   a) it is possible to tell dovecot to create a single file + a 
single

hardlink (linkref=2)? As other IMAP servers support that features (eg:
Cyrus, CommunigatePro, etc) I am wondering if I missed something in
configuring dovecot...
   b) If it is not possible, can I run a script that compare the 
various

files and substitute equal ones (minus the changing line) with
hardlinks, or it will confuse dovecot? As a side note, why the 
changing

line ever exists?

2) About alternate storage and compression: actually, I use a single
mail_location without compression. I would like to have an alternate
storage and to enable compression on it only, leaving the main 
location

without compression. I if understand it correctly, it _should_ be done
using a command similar to doveadm -Dv -o plugin/zlib_save=gz 
altmove

-uu testuser sentbefore 8d. I'm right thinking that it should work? I
will really end with a primary uncompressed mail store and an 
alternate,

zlib-compressed one?

Thank you all and sorry if I did some naive questions.

[1]
63c63
 G2fd0811c64be8e553d97eaa8309f
---

G2ed0811c64be8e553d97eaa8309f






--
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.da...@assyoma.it - i...@assyoma.it
GPG public key ID: FF5F32A8

Re: Questions about hardlinks, alternate storage and compression

[Gionatan Danti]

Hi all,
any ideas?

Especially point n.1 (no hardlink when sending the same email to 
multiple addresses) confuse me a bit. Searching in old messages I even 
stumbled on some users stating that, using Dovecot LMTP server, they 
achieved what I want (one messagge, multiple hardlinks), but I am 
_already_ using LMTP with no avail...


Regards.

On 27/06/15 18:18, Gionatan Danti wrote:

Hi all,
I have some questions about hardlinks, alternate storage and
compression. I already scanned the list for related information and I
have an idea of how things works, but I would like to have a definite
answer.

System spec:
- CentOS 6.6 x64
- dovecot-2.0.9-8.el6_6.4.x86_64 RPM package/version
- sdbox mail store

1) About hardlinks: when sending the same message to two different
recipients, I see that the two u.x files are created as two different
files. Diffing them, I see that the only difference is a single char
(see [1] for an example). My questions are:
   a) it is possible to tell dovecot to create a single file + a single
hardlink (linkref=2)? As other IMAP servers support that features (eg:
Cyrus, CommunigatePro, etc) I am wondering if I missed something in
configuring dovecot...
   b) If it is not possible, can I run a script that compare the various
files and substitute equal ones (minus the changing line) with
hardlinks, or it will confuse dovecot? As a side note, why the changing
line ever exists?

2) About alternate storage and compression: actually, I use a single
mail_location without compression. I would like to have an alternate
storage and to enable compression on it only, leaving the main location
without compression. I if understand it correctly, it _should_ be done
using a command similar to doveadm -Dv -o plugin/zlib_save=gz altmove
-uu testuser sentbefore 8d. I'm right thinking that it should work? I
will really end with a primary uncompressed mail store and an alternate,
zlib-compressed one?

Thank you all and sorry if I did some naive questions.

[1]
63c63
 G2fd0811c64be8e553d97eaa8309f
---

G2ed0811c64be8e553d97eaa8309f






--
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.da...@assyoma.it - i...@assyoma.it
GPG public key ID: FF5F32A8

Questions about hardlinks, alternate storage and compression

[Gionatan Danti]

Hi all,
I have some questions about hardlinks, alternate storage and 
compression. I already scanned the list for related information and I 
have an idea of how things works, but I would like to have a definite 
answer.


System spec:
- CentOS 6.6 x64
- dovecot-2.0.9-8.el6_6.4.x86_64 RPM package/version
- sdbox mail store

1) About hardlinks: when sending the same message to two different 
recipients, I see that the two u.x files are created as two different 
files. Diffing them, I see that the only difference is a single char 
(see [1] for an example). My questions are:
  a) it is possible to tell dovecot to create a single file + a single 
hardlink (linkref=2)? As other IMAP servers support that features (eg: 
Cyrus, CommunigatePro, etc) I am wondering if I missed something in 
configuring dovecot...
  b) If it is not possible, can I run a script that compare the various 
files and substitute equal ones (minus the changing line) with 
hardlinks, or it will confuse dovecot? As a side note, why the changing 
line ever exists?


2) About alternate storage and compression: actually, I use a single 
mail_location without compression. I would like to have an alternate 
storage and to enable compression on it only, leaving the main location 
without compression. I if understand it correctly, it _should_ be done 
using a command similar to doveadm -Dv -o plugin/zlib_save=gz altmove 
-uu testuser sentbefore 8d. I'm right thinking that it should work? I 
will really end with a primary uncompressed mail store and an alternate, 
zlib-compressed one?


Thank you all and sorry if I did some naive questions.

[1]
63c63
 G2fd0811c64be8e553d97eaa8309f
---

G2ed0811c64be8e553d97eaa8309f




--
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.da...@assyoma.it - i...@assyoma.it
GPG public key ID: FF5F32A8