Re: sieve and utf-8 MIME/base64 content

2018-02-19 Thread Robert Stepanek
On Mon, Feb 19, 2018, at 14:33, Eugene M. Zheganin wrote:
> But my users kind of demand more and more :), now they want not only the 
> header/body search, but also sorting out the letters to the folders that 
> contain the localized symbols. So, it's working fine when a folder is 
> named 'Junk e-mail', but when it's named 'Спам', this stops working. Of 
> course, the workaround is to determine on-disk name of the folder, this 
> can be done by ls'ing the folder name on the disk, but only few users 
> can ssh to server, so the rest have a choice of either using latin names 
> for folders or to ask their system engineer. So I wanted to ask - is 
> there any more elegant solution ?

The folder name either needs to be in UTF-8 or modified UTF-7 encoding (aka 
IMAP UTF-7). If you want to use UTF-8, you'll need to enable the 
sieve_utf8fileinto option in imapd.conf (see 
https://www.cyrusimap.org/imap/reference/manpages/configs/imapd.conf.html).

Cheers,
Robert

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: sieve and utf-8 MIME/base64 content

2018-02-19 Thread Eugene M. Zheganin

Hi,

On 16.02.2018 15:37, Robert Stepanek wrote:

Thanks for making this more clear to me :)

On Fri, Feb 16, 2018, at 11:07, Vladislav Kurz wrote:

The problem is that headers with non-ascii chars are encoded in form
like this:
Subject: =?UTF-8?B?UmU6I.gVG9v?=
And also the body is sometimes completely in base64, even though it is
just plaintext or HTML in UTF-8. Depends on sender's mail client.

AFAIK the Sieve implementation in Cyrus 2.5 fully implements RFC 5228, 
including the string comparison requirements of section 2.7 [1]. That is, it 
implements the ascii-casemap and octet collations. It decodes MIME headers to 
UTF-8 before matching (e.g. see [2] and [3]). The RFC 5173 Sieve body extension 
is also supported [4].

Eugene, does that work for you?

Yes, thanks, I experimented and found that it's fully working, thanks a 
lot (I'm truly impressed).


But my users kind of demand more and more :), now they want not only the 
header/body search, but also sorting out the letters to the folders that 
contain the localized symbols. So, it's working fine when a folder is 
named 'Junk e-mail', but when it's named 'Спам', this stops working. Of 
course, the workaround is to determine on-disk name of the folder, this 
can be done by ls'ing the folder name on the disk, but only few users 
can ssh to server, so the rest have a choice of either using latin names 
for folders or to ask their system engineer. So I wanted to ask - is 
there any more elegant solution ?



Eugene.


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: sieve and utf-8 MIME/base64 content

2018-02-16 Thread Robert Stepanek
Thanks for making this more clear to me :)

On Fri, Feb 16, 2018, at 11:07, Vladislav Kurz wrote:
> The problem is that headers with non-ascii chars are encoded in form
> like this:
> Subject: =?UTF-8?B?UmU6I.gVG9v?=
> And also the body is sometimes completely in base64, even though it is
> just plaintext or HTML in UTF-8. Depends on sender's mail client.

AFAIK the Sieve implementation in Cyrus 2.5 fully implements RFC 5228, 
including the string comparison requirements of section 2.7 [1]. That is, it 
implements the ascii-casemap and octet collations. It decodes MIME headers to 
UTF-8 before matching (e.g. see [2] and [3]). The RFC 5173 Sieve body extension 
is also supported [4].

Eugene, does that work for you?

Cheers,
Robert

[1] https://tools.ietf.org/html/rfc5228#section-2.7
[2] 
https://github.com/cyrusimap/cyrus-imapd/blob/cyrus-imapd-2.5/sieve/script.c#L261
[3] 
https://github.com/cyrusimap/cyrus-imapd/blob/cyrus-imapd-2.5/sieve/bc_eval.c#L809
[4] https://cyrusimap.org/imap/rfc-support.html

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus


Re: sieve and utf-8 MIME/base64 content

2018-02-16 Thread Vladislav Kurz
On 02/16/18 10:25, Robert Stepanek wrote:
> On Fri, Feb 16, 2018, at 10:17, Eugene M. Zheganin wrote:
> 
>> I'm using sieve with cyrus to sort incoming mail, and it works perfectly 
>> with latin symbols. But what if I need to sort out the mail that has all 
>> sorts utf-8 sumbols in it ? Like MIME-encoded headers and base64 
>> -encoded body ?
> 
> The main developer for Sieve support (and 2.5) in Cyrus IMAP might not be 
> able to respond the next days. That being said, I'm not sure I understand 
> what use case you are trying to accomplish?

I'm not sure if it works either, but just to clarify - I think what
Eugene wants, is something like this:

if header :contains "from" "Štěpánek" { ... do whatever ... }

The problem is that headers with non-ascii chars are encoded in form
like this:

Subject: =?UTF-8?B?UmU6I.gVG9v?=

And also the body is sometimes completely in base64, even though it is
just plaintext or HTML in UTF-8. Depends on sender's mail client.

So the question is whether Cyrus decodes this stuff before checking and
applying the sieve rules.

-- 
Best Regards
Vladislav Kurz


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: sieve and utf-8 MIME/base64 content

2018-02-16 Thread Robert Stepanek
On Fri, Feb 16, 2018, at 10:17, Eugene M. Zheganin wrote:

> I'm using sieve with cyrus to sort incoming mail, and it works perfectly 
> with latin symbols. But what if I need to sort out the mail that has all 
> sorts utf-8 sumbols in it ? Like MIME-encoded headers and base64 
> -encoded body ?

The main developer for Sieve support (and 2.5) in Cyrus IMAP might not be able 
to respond the next days. That being said, I'm not sure I understand what use 
case you are trying to accomplish?

Cheers,
Robert

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus