Re: Cyrus IMAP 3.2.2 (Re: cyrus imap 2.5.5 build in Linux: problem with pcre)

[ellie timoney]

Hi Sergey,

Quoting out of order, because it's a bit easier to explain that way:

> And there is another problem that is not obvious. -lpcreposix is needed
> in perl/imap/Makefile.PL and in perl/sieve/managesieve/Makefile.PL I seems.

This bit sounds a lot like https://github.com/cyrusimap/cyrus-imapd/issues/2629

There's an experimental patch from me in that issue, which seems to have worked 
around the issue for Fedora package maintainers.  But I can't reproduce the 
problem on Debian, so I can't confirm myself that it fixes things correctly.  
More feedback would be great.

> I returned to this question. 3.2.2 still does not find 
> /usr/include/pcre/pcre.h
> 
> $ pkg-config --cflags libpcre
> -I/usr/include/pcre
> 
> Build is successful with CFLAGS="-I/usr/include/pcre" ./configure ...

Cyrus's configure script does not currently use pkg-config to locate libpcre.  
Instead it uses bespoke detection built from AC_CHECK_HEADER, which I guess 
doesn't know to look in /usr/include/pcre.  There might be a generic way for a 
sysadmin to add this path to the paths that AC_CHECK_HEADER checks, but I don't 
know it offhand.  Otherwise, specifying it in CFLAGS like you have done seems 
like the right thing to do.

The issue linked above also touches on the possibility of changing the libpcre 
detection to use pkg-config instead of bespoke detection, in which case this 
wouldn't be necessary, but it has its own complications.

> Except perl/imap: CFLAGS is not used by Makefile.PL.

Aaand this sounds like its own whole separate bug!  I'm not sure offhand what 
the right way to pass this down will be, but it clearly needs doing.  New issue 
here: https://github.com/cyrusimap/cyrus-imapd/issues/3087

Cheers,

ellie

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Cyrus IMAP 3.2.2 (Re: cyrus imap 2.5.5 build in Linux: problem with pcre)

[Sergey]

On Friday 16 October 2015, Sergey wrote:

> I wanted to build Cyrus-IMAP with libpcre but it did not success.
> System libpcre-devel package install headers to /usr/include/pcre.
 
I returned to this question. 3.2.2 still does not find /usr/include/pcre/pcre.h

$ pkg-config --cflags libpcre
-I/usr/include/pcre

Build is successful with CFLAGS="-I/usr/include/pcre" ./configure ...
Except perl/imap: CFLAGS is not used by Makefile.PL.

And there is another problem that is not obvious. -lpcreposix is needed
in perl/imap/Makefile.PL and in perl/sieve/managesieve/Makefile.PL I seems.

-- 
Regards,
Sergey

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: another problem with conversations db

[Michael Menge]

Replying to myself,

I still see

IOERROR: conversations_audit on store

and

IOERROR: conversations_audit on load



Quoting Michael Menge :


Quoting Bron Gondwana :



ctl_conversationsdb -b should update the cid. BUT - if you're  
running old mailboxes which have a format which doesn't support  
saving the CID, that would for sure break things!




I did run "reconstruct -V max" during the upgrade. So all mailboxes  
should have been updated but

checking with

mbexamine user/* | grep "  Minor Version: 12"

I found some mailboxes that where not updated correctly. I didn't  
see any errors during the
initial upgrade, but I don't have the logs anymore so i don't know  
what happened.

Any idea what could lead to "reconstruct -V max" failing for some mailboxes?

I did rerun "reconstruct -r -V max user/LoginID" for those users,  
but I had two accounts where
the reconstruct command didn't update the index for the INBOX of  
that account. Rerunning it

again did update it.

The rebuild of the conversation_db worked fine after the Index was updated..
Do this did resolve the endless loop in ctl_conversationsdb. I would suggest
to add a check for the index version in ctl_conversationsdb

I will monitor my log files if the IOERRORS will reappear, or if the  
wrong index version

was the root of that problem too.

@Bron Thanks for the help





M.MengeTel.: (49) 7071/29-70316
Universität Tübingen   Fax.: (49) 7071/29-5912
Zentrum für Datenverarbeitung  mail:  
michael.me...@zdv.uni-tuebingen.de

Wächterstraße 76
72074 Tübingen


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: another problem with conversations db

[Michael Menge]

Quoting Bron Gondwana :



ctl_conversationsdb -b should update the cid. BUT - if you're  
running old mailboxes which have a format which doesn't support  
saving the CID, that would for sure break things!




I did run "reconstruct -V max" during the upgrade. So all mailboxes  
should have been updated but

checking with

mbexamine user/* | grep "  Minor Version: 12"

I found some mailboxes that where not updated correctly. I didn't see  
any errors during the
initial upgrade, but I don't have the logs anymore so i don't know  
what happened.

Any idea what could lead to "reconstruct -V max" failing for some mailboxes?

I did rerun "reconstruct -r -V max user/LoginID" for those users, but  
I had two accounts where
the reconstruct command didn't update the index for the INBOX of that  
account. Rerunning it

again did update it.

The rebuild of the conversation_db worked fine after the Index was updated..
Do this did resolve the endless loop in ctl_conversationsdb. I would suggest
to add a check for the index version in ctl_conversationsdb

I will monitor my log files if the IOERRORS will reappear, or if the  
wrong index version

was the root of that problem too.

@Bron Thanks for the help




M.MengeTel.: (49) 7071/29-70316
Universität Tübingen   Fax.: (49) 7071/29-5912
Zentrum für Datenverarbeitung  mail:  
michael.me...@zdv.uni-tuebingen.de

Wächterstraße 76
72074 Tübingen


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: another problem with conversations db

[Bron Gondwana]

On Wed, Feb 13, 2019, at 00:39, Michael Menge wrote:
> Hi Bron,
> 
> sorry, i had to rearrange some quotes to put them my answers in a more 
> meaningful order.
> 
> 
> Quoting Bron Gondwana :
> 
> >> >> The file was already at 
> >> /srv/cyrus-hdd-be/archive/ssd-part/L/user//2185.
> 
> I was able to fix these problems with reconstruct, and the didn't 
> reappear till now.
> Also there where other accounts which had IOERRORS regarding the 
> conversation db,
> with no cyr_expire archive errors, so i believe that these problems 
> are not related.
> 
> I tried rebuilding the conversation db for the accounts with errors, 
> but some other
> accounts will show up with errors some time later. I counldn't find a 
> some thing in
> common jet.

OK. That's hard to disagnore from remotely :(

> 
> >> >> > Anyway, I don't think that would break anything.
> >> >> >
> >> >> > metapartition-ssd: /srv/cyrus-ssd-be/meta/ssd-part
> >> >> > metapartition_files: header index cache expunge squat annotations
> >> >> > lock dav archivecache
> >> >> >
> >> >> > Ooh, I haven't tested having cache and archivecache on the same
> >> >> > location. That's really interesting. Again, I'd be in favour of
> >> >> > separation here, give them different paths. That might be tricky
> >> >> > with ssd though, the way this is laid out. I assume you have some
> >> >> > kind of symlink farm going on?
> >> >> >
> >> >>
> >> >> I didn't know that there could be a problem with cache and archivecache.
> >> >> At the time we decided on the configuration for cyrus 3.0 I looked at 
> >> >> the
> >> >> imapd.conf man page and for metapartition_files decided that I want all
> >> >> meta files on the ssd storage. There was no indication in the man page
> >> >> that there could be a problem.
> >> >
> >> > Fair. I'd have to test that to see if it works correctly. I would
> >> > hope so, but I haven't tested that configuration. This is the
> >> > downside with having lots of different ways to do things!
> >> >
> >> >> How do I separate location of archivecache from the other
> >> >> metapartition path?
> >> >> And fix the cache and archivecache files?
> >> >
> >> > This I don't know a good answer for. I will test if having the same
> >> > path for cache and archivecache could fail. I THINK that I made the
> >> > code safe for it, but I'm not sure that it's been tested.
> >> >
> >> >> No there is no sysmlink farm. We have mounted different iSCSI volumes to
> >> >> /srv/cyrus-ssd-be, /srv/cyrus-hdd-be and /srv/cyrus-be
> >> >
> >> > Right. That makes sense.
> 
> Did you have time to look into the cache/archivecache situation jet?

Yes, I've looked at the code!

in mailbox_archive():

 /* got a new cache record to write */
 if (differentcache)
 {
 dirtycache = 1;
 copyrecord.cache_offset = 0;
 if (mailbox_append_cache(mailbox, ))
 continue;
 }

And the code for differentcache is:

 char *spoolcache = xstrdup(mailbox_meta_fname(mailbox, META_CACHE));
 char *archivecache = xstrdup(mailbox_meta_fname(mailbox, META_ARCHIVECACHE));
 int differentcache = strcmp(spoolcache, archivecache);

So it looks like the answer is cache/archivecache is fine. It is not your 
problem.

> 
> >> > Right! I do wonder if there are some bugs in 3.0.x which are fixed
> >> > on master around delivery to archive partition. We definitely had
> >> > bugs on master, but I thought they were newly introduced on master
> >> > as well, which is why the fixes weren't backported. But if you're
> >> > having files be in the wrong location, maybe there are bugs on 3.0.x
> >> > as well.
> 
> Are all fixes from master backported to 3.0?

Unfortunately it's hard to tell, because many of the fixes on master are fixes 
to bugs that were only introduced on master, and some bugs on 3.0 we just say 
"the fix is so invasive that it's basically just backporting 90% of master, 
which is pointless for a stable release".

> Is the new Commit "I will try your new commits regarding CID" related to the
> "IOERROR: conversations_audit on load:" and "IOERROR: 
> conversations_audit on store"?

Shouldn't be. It just means we store the G keys regardless of whether the 
record has a CID.

> I will try your new commits in the next days on my test s

Re: another problem with conversations db

[Michael Menge]

Hi Bron,

sorry, i had to rearrange some quotes to put them my answers in a more  
meaningful order.



Quoting Bron Gondwana :


On Mon, Feb 4, 2019, at 22:00, Michael Menge wrote:


Quoting Bron Gondwana :

> On Mon, Feb 4, 2019, at 20:21, Michael Menge wrote:
>>



>> Feb 4 01:10:55 mailserv03 be/cyr_expire[7626]: IOERROR: opening
>> /srv/cyrus-be/ssd-part/L/user//2185.: No such file or directory
>> Feb 4 01:10:55 mailserv03 be/cyr_expire[7626]: IOERROR: opening
>> /srv/cyrus-be/ssd-part/L/user//2185.: No such file or directory
>> Feb 4 01:10:55 mailserv03 be/cyr_expire[7626]: IOERROR archive
>> user. 2185 failed to copyfile
>> (/srv/cyrus-be/ssd-part/L/user//2185. =>
>> /srv/cyrus-hdd-be/archive/ssd-part/L/user//2185.): Unknown code
>>  255
>
>
> Ouch. Yeah, that could have been caused by a bug in delivery, and
> would definitely cause conversations DB corruption if the index file
> was updated but the conversations DB wasn't or vice versa.
>
>> The file was already at  
/srv/cyrus-hdd-be/archive/ssd-part/L/user//2185.


I was able to fix these problems with reconstruct, and the didn't  
reappear till now.
Also there where other accounts which had IOERRORS regarding the  
conversation db,
with no cyr_expire archive errors, so i believe that these problems  
are not related.


I tried rebuilding the conversation db for the accounts with errors,  
but some other
accounts will show up with errors some time later. I counldn't find a  
some thing in

common jet.



>> > Anyway, I don't think that would break anything.
>> >
>> > metapartition-ssd: /srv/cyrus-ssd-be/meta/ssd-part
>> > metapartition_files: header index cache expunge squat annotations
>> > lock dav archivecache
>> >
>> > Ooh, I haven't tested having cache and archivecache on the same
>> > location. That's really interesting. Again, I'd be in favour of
>> > separation here, give them different paths. That might be tricky
>> > with ssd though, the way this is laid out. I assume you have some
>> > kind of symlink farm going on?
>> >
>>
>> I didn't know that there could be a problem with cache and archivecache.
>> At the time we decided on the configuration for cyrus 3.0 I looked at the
>> imapd.conf man page and for metapartition_files decided that I want all
>> meta files on the ssd storage. There was no indication in the man page
>> that there could be a problem.
>
> Fair. I'd have to test that to see if it works correctly. I would
> hope so, but I haven't tested that configuration. This is the
> downside with having lots of different ways to do things!
>
>> How do I separate location of archivecache from the other
>> metapartition path?
>> And fix the cache and archivecache files?
>
> This I don't know a good answer for. I will test if having the same
> path for cache and archivecache could fail. I THINK that I made the
> code safe for it, but I'm not sure that it's been tested.
>
>> No there is no sysmlink farm. We have mounted different iSCSI volumes to
>> /srv/cyrus-ssd-be, /srv/cyrus-hdd-be and /srv/cyrus-be
>
> Right. That makes sense.


Did you have time to look into the cache/archivecache situation jet?



> Right! I do wonder if there are some bugs in 3.0.x which are fixed
> on master around delivery to archive partition. We definitely had
> bugs on master, but I thought they were newly introduced on master
> as well, which is why the fixes weren't backported. But if you're
> having files be in the wrong location, maybe there are bugs on 3.0.x
> as well.


Are all fixes from master backported to 3.0?

Is the new Commit "I will try your new commits regarding CID" related to the
"IOERROR: conversations_audit on load:" and "IOERROR:  
conversations_audit on store"?


I will try your new commits in the next days on my test servers to sea  
if the fix

the endless loop in ctl_conversationsdb I have seen for some accounts.

Quoting myself (Re: prblems rebuilding conversations db) Jan 24, 2019


The program loops in build_cid_cb (imap/ctl_conversationsdb.c:189)

For the problematic mailbox that I tested, for every message
record->cid was NULLCONVERSATION, so mailbox_cacherecord,
message_update_conversations and mailbox_rewrite_index_record
where called, each returned 0.

After iterating trough all messages in the mailbox count was > 0, and r==0,
so the while condition (!r && count) was true for the next run.
At this point record->cid was still NULLCONVERSATION for every message,
which I guess should not be the case.


Michael


M.MengeTel.: (49

Re: another problem with conversations db

[Bron Gondwana]

On Mon, Feb 4, 2019, at 22:00, Michael Menge wrote:
> 
> Quoting Bron Gondwana :
> 
> > On Mon, Feb 4, 2019, at 20:21, Michael Menge wrote:
> >> Hi,
> >>
> >> Quoting Bron Gondwana :
> >>
> >> > Hi Michael,
> >> >
> >> > Sorry about the delay in looking at this - I was mad crazy busy
> >> > getting ready to go overseas. At Fosdem now, about to give a talk
> >> > about JMAP!
> >> >
> >> > OK, let's start with the things that give me a little bit of hives...
> >> >
> >> > configdirectory: /srv/cyrus-be
> >> > partition-default: /srv/cyrus-be
> >> > partition-ssd: /srv/cyrus-be/ssd-part
> >> >
> >> > Ouch. There's a couple of things I wouldn't do there - having the
> >> > partition be the same as the config directory, and having a separate
> >> > partition be a subdirectory of a partition. They're both asking for
> >> > trouble. I would probably lay my system out like:
> >> >
> >> > configdirectory: /srv/cyrus-be/conf
> >> > partition-default: /srv/cyrus-be/default-part
> >> > partition-ssd: /srv/cyrus-be/ssd-part
> >> >
> >>
> >> partition-default isn't used any more. To use the metapartition we moved
> >> all accounts form the default partition to the ssd partition which is the
> >> the new defaultpartition ("defaultpartition: ssd")
> >
> > Right - that makes sense.
> >
> >> > And then each tree would only have one type of thing in it.
> >> >
> >> > Anyway, I don't think that would break anything.
> >> >
> >> > metapartition-ssd: /srv/cyrus-ssd-be/meta/ssd-part
> >> > metapartition_files: header index cache expunge squat annotations
> >> > lock dav archivecache
> >> >
> >> > Ooh, I haven't tested having cache and archivecache on the same
> >> > location. That's really interesting. Again, I'd be in favour of
> >> > separation here, give them different paths. That might be tricky
> >> > with ssd though, the way this is laid out. I assume you have some
> >> > kind of symlink farm going on?
> >> >
> >>
> >> I didn't know that there could be a problem with cache and archivecache.
> >> At the time we decided on the configuration for cyrus 3.0 I looked at the
> >> imapd.conf man page and for metapartition_files decided that I want all
> >> meta files on the ssd storage. There was no indication in the man page
> >> that there could be a problem.
> >
> > Fair. I'd have to test that to see if it works correctly. I would 
> > hope so, but I haven't tested that configuration. This is the 
> > downside with having lots of different ways to do things!
> >
> >> How do I separate location of archivecache from the other 
> >> metapartition path?
> >> And fix the cache and archivecache files?
> >
> > This I don't know a good answer for. I will test if having the same 
> > path for cache and archivecache could fail. I THINK that I made the 
> > code safe for it, but I'm not sure that it's been tested.
> >
> >> No there is no sysmlink farm. We have mounted different iSCSI volumes to
> >> /srv/cyrus-ssd-be, /srv/cyrus-hdd-be and /srv/cyrus-be
> >
> > Right. That makes sense.
> >
> >> > Otherwise it all looks OK. Are you getting other IOERRORs in your
> >> > log files which could show things aborting? It really looks like
> >> > your conversations DB is getting out of sync due to other failures.
> >>
> >> I found a few instances of 3 related errors.
> >>
> >> Feb 4 01:10:55 mailserv03 be/cyr_expire[7626]: IOERROR: opening
> >> /srv/cyrus-be/ssd-part/L/user//2185.: No such file or directory
> >> Feb 4 01:10:55 mailserv03 be/cyr_expire[7626]: IOERROR: opening
> >> /srv/cyrus-be/ssd-part/L/user//2185.: No such file or directory
> >> Feb 4 01:10:55 mailserv03 be/cyr_expire[7626]: IOERROR archive
> >> user. 2185 failed to copyfile
> >> (/srv/cyrus-be/ssd-part/L/user//2185. =>
> >> /srv/cyrus-hdd-be/archive/ssd-part/L/user//2185.): Unknown code
> >>  255
> >
> >
> > Ouch. Yeah, that could have been caused by a bug in delivery, and 
> > would definitely cause conversations DB corruption if the index file 
> > was updated but the conversations DB wasn't or vice versa.
> >
> >> The file was already at 
> >> /srv

Re: another problem with conversations db

[Michael Menge]

Quoting Bron Gondwana :


On Mon, Feb 4, 2019, at 20:21, Michael Menge wrote:

Hi,

Quoting Bron Gondwana :

> Hi Michael,
>
> Sorry about the delay in looking at this - I was mad crazy busy
> getting ready to go overseas. At Fosdem now, about to give a talk
> about JMAP!
>
> OK, let's start with the things that give me a little bit of hives...
>
> configdirectory: /srv/cyrus-be
> partition-default: /srv/cyrus-be
> partition-ssd: /srv/cyrus-be/ssd-part
>
> Ouch. There's a couple of things I wouldn't do there - having the
> partition be the same as the config directory, and having a separate
> partition be a subdirectory of a partition. They're both asking for
> trouble. I would probably lay my system out like:
>
> configdirectory: /srv/cyrus-be/conf
> partition-default: /srv/cyrus-be/default-part
> partition-ssd: /srv/cyrus-be/ssd-part
>

partition-default isn't used any more. To use the metapartition we moved
all accounts form the default partition to the ssd partition which is the
the new defaultpartition ("defaultpartition: ssd")


Right - that makes sense.


> And then each tree would only have one type of thing in it.
>
> Anyway, I don't think that would break anything.
>
> metapartition-ssd: /srv/cyrus-ssd-be/meta/ssd-part
> metapartition_files: header index cache expunge squat annotations
> lock dav archivecache
>
> Ooh, I haven't tested having cache and archivecache on the same
> location. That's really interesting. Again, I'd be in favour of
> separation here, give them different paths. That might be tricky
> with ssd though, the way this is laid out. I assume you have some
> kind of symlink farm going on?
>

I didn't know that there could be a problem with cache and archivecache.
At the time we decided on the configuration for cyrus 3.0 I looked at the
imapd.conf man page and for metapartition_files decided that I want all
meta files on the ssd storage. There was no indication in the man page
that there could be a problem.


Fair. I'd have to test that to see if it works correctly. I would  
hope so, but I haven't tested that configuration. This is the  
downside with having lots of different ways to do things!


How do I separate location of archivecache from the other  
metapartition path?

And fix the cache and archivecache files?


This I don't know a good answer for. I will test if having the same  
path for cache and archivecache could fail. I THINK that I made the  
code safe for it, but I'm not sure that it's been tested.



No there is no sysmlink farm. We have mounted different iSCSI volumes to
/srv/cyrus-ssd-be, /srv/cyrus-hdd-be and /srv/cyrus-be


Right. That makes sense.


> Otherwise it all looks OK. Are you getting other IOERRORs in your
> log files which could show things aborting? It really looks like
> your conversations DB is getting out of sync due to other failures.

I found a few instances of 3 related errors.

Feb 4 01:10:55 mailserv03 be/cyr_expire[7626]: IOERROR: opening
/srv/cyrus-be/ssd-part/L/user//2185.: No such file or directory
Feb 4 01:10:55 mailserv03 be/cyr_expire[7626]: IOERROR: opening
/srv/cyrus-be/ssd-part/L/user//2185.: No such file or directory
Feb 4 01:10:55 mailserv03 be/cyr_expire[7626]: IOERROR archive
user. 2185 failed to copyfile
(/srv/cyrus-be/ssd-part/L/user//2185. =>
/srv/cyrus-hdd-be/archive/ssd-part/L/user//2185.): Unknown code
 255



Ouch. Yeah, that could have been caused by a bug in delivery, and  
would definitely cause conversations DB corruption if the index file  
was updated but the conversations DB wasn't or vice versa.



The file was already at /srv/cyrus-hdd-be/archive/ssd-part/L/user//2185.


Right! I do wonder if there are some bugs in 3.0.x which are fixed  
on master around delivery to archive partition. We definitely had  
bugs on master, but I thought they were newly introduced on master  
as well, which is why the fixes weren't backported. But if you're  
having files be in the wrong location, maybe there are bugs on 3.0.x  
as well.


Do you have the syslog lines at the time that email was delivered?


I dont' have the log, for that message, but I will search for a
more recent example.


From the mail headers i know that it was not dilivered to the archive  
partition

but moved by cyr_expire. The conversation db was not used at that time.

PS.: the timesamp of the file is not the internal date but the time
the mail was moved to the archive partition. I was wondering about the reason.

Michael



M.MengeTel.: (49) 7071/29-70316
Universität Tübingen   Fax.: (49) 7071/29-5912
Zentrum für Datenverarbeitung  mail:  
michael.me...@zdv.uni-tuebingen.de

Wächterstraße 76
72074 Tübingen


Cyrus Home Page: http://www.cyrusimap.org/
List Arch

Re: another problem with conversations db

[Bron Gondwana]

On Mon, Feb 4, 2019, at 20:21, Michael Menge wrote:
> Hi,
> 
> Quoting Bron Gondwana :
> 
> > Hi Michael,
> >
> > Sorry about the delay in looking at this - I was mad crazy busy 
> > getting ready to go overseas. At Fosdem now, about to give a talk 
> > about JMAP!
> >
> > OK, let's start with the things that give me a little bit of hives...
> >
> > configdirectory: /srv/cyrus-be
> > partition-default: /srv/cyrus-be
> > partition-ssd: /srv/cyrus-be/ssd-part
> >
> > Ouch. There's a couple of things I wouldn't do there - having the 
> > partition be the same as the config directory, and having a separate 
> > partition be a subdirectory of a partition. They're both asking for 
> > trouble. I would probably lay my system out like:
> >
> > configdirectory: /srv/cyrus-be/conf
> > partition-default: /srv/cyrus-be/default-part
> > partition-ssd: /srv/cyrus-be/ssd-part
> >
> 
> partition-default isn't used any more. To use the metapartition we moved
> all accounts form the default partition to the ssd partition which is the
> the new defaultpartition ("defaultpartition: ssd")

Right - that makes sense.

> > And then each tree would only have one type of thing in it.
> >
> > Anyway, I don't think that would break anything.
> >
> > metapartition-ssd: /srv/cyrus-ssd-be/meta/ssd-part
> > metapartition_files: header index cache expunge squat annotations 
> > lock dav archivecache
> >
> > Ooh, I haven't tested having cache and archivecache on the same 
> > location. That's really interesting. Again, I'd be in favour of 
> > separation here, give them different paths. That might be tricky 
> > with ssd though, the way this is laid out. I assume you have some 
> > kind of symlink farm going on?
> >
> 
> I didn't know that there could be a problem with cache and archivecache.
> At the time we decided on the configuration for cyrus 3.0 I looked at the
> imapd.conf man page and for metapartition_files decided that I want all
> meta files on the ssd storage. There was no indication in the man page
> that there could be a problem.

Fair. I'd have to test that to see if it works correctly. I would hope so, but 
I haven't tested that configuration. This is the downside with having lots of 
different ways to do things!

> How do I separate location of archivecache from the other metapartition path?
> And fix the cache and archivecache files?

This I don't know a good answer for. I will test if having the same path for 
cache and archivecache could fail. I THINK that I made the code safe for it, 
but I'm not sure that it's been tested.

> No there is no sysmlink farm. We have mounted different iSCSI volumes to
> /srv/cyrus-ssd-be, /srv/cyrus-hdd-be and /srv/cyrus-be

Right. That makes sense.

> > Otherwise it all looks OK. Are you getting other IOERRORs in your 
> > log files which could show things aborting? It really looks like 
> > your conversations DB is getting out of sync due to other failures.
> 
> I found a few instances of 3 related errors.
> 
> Feb 4 01:10:55 mailserv03 be/cyr_expire[7626]: IOERROR: opening 
> /srv/cyrus-be/ssd-part/L/user//2185.: No such file or directory
> Feb 4 01:10:55 mailserv03 be/cyr_expire[7626]: IOERROR: opening 
> /srv/cyrus-be/ssd-part/L/user//2185.: No such file or directory
> Feb 4 01:10:55 mailserv03 be/cyr_expire[7626]: IOERROR archive 
> user. 2185 failed to copyfile 
> (/srv/cyrus-be/ssd-part/L/user//2185. => 
> /srv/cyrus-hdd-be/archive/ssd-part/L/user//2185.): Unknown code 
>  255


Ouch. Yeah, that could have been caused by a bug in delivery, and would 
definitely cause conversations DB corruption if the index file was updated but 
the conversations DB wasn't or vice versa.

> The file was already at /srv/cyrus-hdd-be/archive/ssd-part/L/user//2185.

Right! I do wonder if there are some bugs in 3.0.x which are fixed on master 
around delivery to archive partition. We definitely had bugs on master, but I 
thought they were newly introduced on master as well, which is why the fixes 
weren't backported. But if you're having files be in the wrong location, maybe 
there are bugs on 3.0.x as well.

Do you have the syslog lines at the time that email was delivered?

Bron.
-- 
 Bron Gondwana
 br...@fastmail.fm


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: another problem with conversations db

[Michael Menge]

Hi,

Quoting Bron Gondwana :


Hi Michael,

Sorry about the delay in looking at this - I was mad crazy busy  
getting ready to go overseas. At Fosdem now, about to give a talk  
about JMAP!


OK, let's start with the things that give me a little bit of hives...

configdirectory: /srv/cyrus-be
partition-default: /srv/cyrus-be
partition-ssd: /srv/cyrus-be/ssd-part

Ouch. There's a couple of things I wouldn't do there - having the  
partition be the same as the config directory, and having a separate  
partition be a subdirectory of a partition. They're both asking for  
trouble. I would probably lay my system out like:


configdirectory: /srv/cyrus-be/conf
partition-default: /srv/cyrus-be/default-part
partition-ssd: /srv/cyrus-be/ssd-part



partition-default isn't used any more. To use the metapartition we moved
all accounts form the default partition to the ssd partition which is the
the new defaultpartition ("defaultpartition: ssd")


And then each tree would only have one type of thing in it.

Anyway, I don't think that would break anything.

metapartition-ssd: /srv/cyrus-ssd-be/meta/ssd-part
metapartition_files: header index cache expunge squat annotations  
lock dav archivecache


Ooh, I haven't tested having cache and archivecache on the same  
location. That's really interesting. Again, I'd be in favour of  
separation here, give them different paths. That might be tricky  
with ssd though, the way this is laid out. I assume you have some  
kind of symlink farm going on?




I didn't know that there could be a problem with cache and archivecache.
At the time we decided on the configuration for cyrus 3.0 I looked at the
imapd.conf man page and for metapartition_files decided that I want all
meta files on the ssd storage. There was no indication in the man page
that there could be a problem.

How do I separate location of archivecache from the other metapartition path?
And fix the cache and archivecache files?

No there is no sysmlink farm.  We have mounted different iSCSI volumes to
/srv/cyrus-ssd-be, /srv/cyrus-hdd-be and /srv/cyrus-be


Otherwise it all looks OK. Are you getting other IOERRORs in your  
log files which could show things aborting? It really looks like  
your conversations DB is getting out of sync due to other failures.


I found a few instances of 3 related errors.

Feb  4 01:10:55 mailserv03 be/cyr_expire[7626]: IOERROR: opening  
/srv/cyrus-be/ssd-part/L/user//2185.: No such file or directory
Feb  4 01:10:55 mailserv03 be/cyr_expire[7626]: IOERROR: opening  
/srv/cyrus-be/ssd-part/L/user//2185.: No such file or directory
Feb  4 01:10:55 mailserv03 be/cyr_expire[7626]: IOERROR archive  
user. 2185 failed to copyfile  
(/srv/cyrus-be/ssd-part/L/user//2185. =>  
/srv/cyrus-hdd-be/archive/ssd-part/L/user//2185.): Unknown code  
 255


The file was already at /srv/cyrus-hdd-be/archive/ssd-part/L/user//2185.

stat 2185.
  File: '2185.'
  Size: 424542  Blocks: 832IO Block: 4096   regular file
Device: 860h/2144d  Inode: 4677438462  Links: 1
Access: (0600/-rw---)  Uid: (   96/   cyrus)   Gid: (   12/mail)
Context: system_u:object_r:unlabeled_t:s0
Access: 2018-11-27 01:12:46.585864239 +0100
Modify: 2018-11-27 01:12:46.585864239 +0100
Change: 2018-11-27 01:12:46.585864239 +0100
 Birth: -




Cheers,

Bron.



On Thu, Jan 31, 2019, at 19:36, Michael Menge wrote:


Quoting Bron Gondwana :

> On Fri, Jan 25, 2019, at 20:00, Michael Menge wrote:
>> Hi Bron
>>
>>
>> Quoting Bron Gondwana :
>>
>> > Sorry I haven't been following along with this earlier. Can you post
>> > your imapd.conf and cyrus.conf as well as let her know if you run
>> > anything which directly messes with files on disk.
>>
>> Thanks for looking into it. I have attached the backend and replica
>> configs. There should be nothing messing with the files on disk
>> directly. Some times we need to restore mails from file based
>> backup (bacula) followed by a reconstruct but this was not the
>> case this time.
>
> Do you always rebuild the conversations db after doing the
> reconstruct? That will be necessary now. We switched to doing all
> our restores using IMAP append a while back so we're never fiddling
> the file system under Cyrus.
>
>> >
>> > Also, what operating system is this on and what Cyrus version?
>> >
>>
>> We are running a cyrus 3.0.8 compiled with the following Options
>> (./configure --enable-murder --enable-http --enable-calalarmd
>> --enable-replication --enable-backup --enable-idled
>> --enable-autocreate CFLAGS="-fPIC -g")
>> in a murder configuration on a RHEL 7.5 System. As filesystem we use xfs.
>
> They should be fine. I'll have a read of the config when I'm at a
> real computer.
>

did you find anything in the config that would ex

Re: another problem with conversations db

[Bron Gondwana]

Hi Michael,

Sorry about the delay in looking at this - I was mad crazy busy getting ready 
to go overseas. At Fosdem now, about to give a talk about JMAP!

OK, let's start with the things that give me a little bit of hives...

configdirectory: /srv/cyrus-be
partition-default: /srv/cyrus-be
partition-ssd: /srv/cyrus-be/ssd-part

Ouch. There's a couple of things I wouldn't do there - having the partition be 
the same as the config directory, and having a separate partition be a 
subdirectory of a partition. They're both asking for trouble. I would probably 
lay my system out like:

configdirectory: /srv/cyrus-be/conf
partition-default: /srv/cyrus-be/default-part
partition-ssd: /srv/cyrus-be/ssd-part

And then each tree would only have one type of thing in it.

Anyway, I don't think that would break anything.

metapartition-ssd: /srv/cyrus-ssd-be/meta/ssd-part
metapartition_files: header index cache expunge squat annotations lock dav 
archivecache

Ooh, I haven't tested having cache and archivecache on the same location. 
That's really interesting. Again, I'd be in favour of separation here, give 
them different paths. That might be tricky with ssd though, the way this is 
laid out. I assume you have some kind of symlink farm going on?

Otherwise it all looks OK. Are you getting other IOERRORs in your log files 
which could show things aborting? It really looks like your conversations DB is 
getting out of sync due to other failures.

Cheers,

Bron.



On Thu, Jan 31, 2019, at 19:36, Michael Menge wrote:
> 
> Quoting Bron Gondwana :
> 
> > On Fri, Jan 25, 2019, at 20:00, Michael Menge wrote:
> >> Hi Bron
> >>
> >>
> >> Quoting Bron Gondwana :
> >>
> >> > Sorry I haven't been following along with this earlier. Can you post
> >> > your imapd.conf and cyrus.conf as well as let her know if you run
> >> > anything which directly messes with files on disk.
> >>
> >> Thanks for looking into it. I have attached the backend and replica
> >> configs. There should be nothing messing with the files on disk
> >> directly. Some times we need to restore mails from file based
> >> backup (bacula) followed by a reconstruct but this was not the
> >> case this time.
> >
> > Do you always rebuild the conversations db after doing the 
> > reconstruct? That will be necessary now. We switched to doing all 
> > our restores using IMAP append a while back so we're never fiddling 
> > the file system under Cyrus.
> >
> >> >
> >> > Also, what operating system is this on and what Cyrus version?
> >> >
> >>
> >> We are running a cyrus 3.0.8 compiled with the following Options
> >> (./configure --enable-murder --enable-http --enable-calalarmd
> >> --enable-replication --enable-backup --enable-idled
> >> --enable-autocreate CFLAGS="-fPIC -g")
> >> in a murder configuration on a RHEL 7.5 System. As filesystem we use xfs.
> >
> > They should be fine. I'll have a read of the config when I'm at a 
> > real computer.
> >
> 
> did you find anything in the config that would explain the problems?
> 
> >>
> >> > Bron.
> >> >
> >> > On Fri, Jan 25, 2019, at 04:08, Michael Menge wrote:
> >> >> Hi,
> >> >>
> >> >> I have discovered an other problem with the conversations db:
> >> >>
> >> >> Thousends of lines with "IOERROR: conversations_audit on load:" and
> >> >> "IOERROR: conversations_audit on store:"
> >> >> A look at the source code shows that these errors are logged after
> >> >> "_sanity_check_counts" is called.
> >> >> The log level LOG_ERR and the prefix IOERROR indicate that I have a
> >> >> serious problem. Do I?
> >> >>
> >> >> This problem occurred for accounts where the rebuild of the
> >> >> conversations db was successful.
> >> >>
> >> >> I don't want to rebuild the conversations db every few days.
> >> >>
> >> >> Any help is appreciated.
> >> >>
> >> >> Kind regards
> >> >>
> >> >>
> >> >> Michael Menge
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >> 
> >> 
> >> >> M.Menge Tel.: (49) 7071/29-70316
> >> >> Universität Tübingen Fax.: (49) 7071/29-5912
> >> >> Zentrum für Datenverarbeitung mail:
> &

Re: another problem with conversations db

[Michael Menge]

Quoting Bron Gondwana :


On Fri, Jan 25, 2019, at 20:00, Michael Menge wrote:

Hi Bron


Quoting Bron Gondwana :

> Sorry I haven't been following along with this earlier. Can you post
> your imapd.conf and cyrus.conf as well as let her know if you run
> anything which directly messes with files on disk.

Thanks for looking into it. I have attached the backend and replica
configs. There should be nothing messing with the files on disk
directly. Some times we need to restore mails from file based
backup (bacula) followed by a reconstruct but this was not the
case this time.


Do you always rebuild the conversations db after doing the  
reconstruct? That will be necessary now. We switched to doing all  
our restores using IMAP append a while back so we're never fiddling  
the file system under Cyrus.



>
> Also, what operating system is this on and what Cyrus version?
>

We are running a cyrus 3.0.8 compiled with the following Options
(./configure --enable-murder --enable-http --enable-calalarmd
--enable-replication --enable-backup --enable-idled
--enable-autocreate CFLAGS="-fPIC -g")
in a murder configuration on a RHEL 7.5 System. As filesystem we use xfs.


They should be fine. I'll have a read of the config when I'm at a  
real computer.




did you find anything in the config that would explain the problems?



> Bron.
>
> On Fri, Jan 25, 2019, at 04:08, Michael Menge wrote:
>> Hi,
>>
>> I have discovered an other problem with the conversations db:
>>
>> Thousends of lines with "IOERROR: conversations_audit on load:" and
>> "IOERROR: conversations_audit on store:"
>> A look at the source code shows that these errors are logged after
>> "_sanity_check_counts" is called.
>> The log level LOG_ERR and the prefix IOERROR indicate that I have a
>> serious problem. Do I?
>>
>> This problem occurred for accounts where the rebuild of the
>> conversations db was successful.
>>
>> I don't want to rebuild the conversations db every few days.
>>
>> Any help is appreciated.
>>
>> Kind regards
>>
>>
>> Michael Menge
>>
>>
>>
>>
>>
>>  


>> M.Menge Tel.: (49) 7071/29-70316
>> Universität Tübingen Fax.: (49) 7071/29-5912
>> Zentrum für Datenverarbeitung mail:
>> michael.me...@zdv.uni-tuebingen.de
>> Wächterstraße 76
>> 72074 Tübingen
>>
>> 
>> Cyrus Home Page: http://www.cyrusimap.org/
>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>> To Unsubscribe:
>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
>
> --
> Bron Gondwana
> br...@fastmail.fm




M.Menge Tel.: (49) 7071/29-70316
Universität Tübingen Fax.: (49) 7071/29-5912
Zentrum für Datenverarbeitung mail:
michael.me...@zdv.uni-tuebingen.de
Wächterstraße 76
72074 Tübingen


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

*Attachments:*
 * imapd_be.conf
 * cyrus_be.conf
 * cyrus_re.conf
 * imapd_re.conf


--
 Bron Gondwana
 br...@fastmail.fm





M.MengeTel.: (49) 7071/29-70316
Universität Tübingen   Fax.: (49) 7071/29-5912
Zentrum für Datenverarbeitung  mail:  
michael.me...@zdv.uni-tuebingen.de

Wächterstraße 76
72074 Tübingen


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: another problem with conversations db

[Bron Gondwana]

Yes, updating reconstruct is on our roadmap. The biggest problem is that there 
are multiple file types across mailboxes and conversations and the changes 
can't be atomic, so if reconstruct discovers something partially done, it can't 
tell if the conversations were correct.

The longer term grand plan is to reduce the number of db types and make 
conversations updates able to be atomic.

Cheers,

Bron.

On Fri, Jan 25, 2019, at 22:30, Michael Menge wrote:
> Hi,
> 
> 
> Quoting Bron Gondwana :
> 
> > On Fri, Jan 25, 2019, at 20:00, Michael Menge wrote:
> >> Hi Bron
> >>
> >>
> >> Quoting Bron Gondwana :
> >>
> >> > Sorry I haven't been following along with this earlier. Can you post
> >> > your imapd.conf and cyrus.conf as well as let her know if you run
> >> > anything which directly messes with files on disk.
> >>
> >> Thanks for looking into it. I have attached the backend and replica
> >> configs. There should be nothing messing with the files on disk
> >> directly. Some times we need to restore mails from file based
> >> backup (bacula) followed by a reconstruct but this was not the
> >> case this time.
> >
> > Do you always rebuild the conversations db after doing the 
> > reconstruct? That will be necessary now. We switched to doing all 
> > our restores using IMAP append a while back so we're never fiddling 
> > the file system under Cyrus.
> >
> 
> We didn't have to restore any mails from backup since we enabled 
> conversation db
> a few weeks ago. It is good to know that the rebuild is necessary, but 
> shouldn't
> reconstruct also update the conversation db if it re-appends the message?
> At least a hint should be put in the reconstruct man page, like the 
> one for "quota -f"
> 
> 
> >> >
> >> > Also, what operating system is this on and what Cyrus version?
> >> >
> >>
> >> We are running a cyrus 3.0.8 compiled with the following Options
> >> (./configure --enable-murder --enable-http --enable-calalarmd
> >> --enable-replication --enable-backup --enable-idled
> >> --enable-autocreate CFLAGS="-fPIC -g")
> >> in a murder configuration on a RHEL 7.5 System. As filesystem we use xfs.
> >
> > They should be fine. I'll have a read of the config when I'm at a 
> > real computer.
> >
> >>
> >> > Bron.
> >> >
> >> > On Fri, Jan 25, 2019, at 04:08, Michael Menge wrote:
> >> >> Hi,
> >> >>
> >> >> I have discovered an other problem with the conversations db:
> >> >>
> >> >> Thousends of lines with "IOERROR: conversations_audit on load:" and
> >> >> "IOERROR: conversations_audit on store:"
> >> >> A look at the source code shows that these errors are logged after
> >> >> "_sanity_check_counts" is called.
> >> >> The log level LOG_ERR and the prefix IOERROR indicate that I have a
> >> >> serious problem. Do I?
> >> >>
> >> >> This problem occurred for accounts where the rebuild of the
> >> >> conversations db was successful.
> >> >>
> >> >> I don't want to rebuild the conversations db every few days.
> >> >>
> >> >> Any help is appreciated.
> >> >>
> >> >> Kind regards
> >> >>
> >> >>
> >> >> Michael Menge
> 
> 
> 
> 
> M.Menge Tel.: (49) 7071/29-70316
> Universität Tübingen Fax.: (49) 7071/29-5912
> Zentrum für Datenverarbeitung mail: 
> michael.me...@zdv.uni-tuebingen.de
> Wächterstraße 76
> 72074 Tübingen
> 
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

-- 
 Bron Gondwana
 br...@fastmail.fm

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: another problem with conversations db

[Michael Menge]

Hi,


Quoting Bron Gondwana :


On Fri, Jan 25, 2019, at 20:00, Michael Menge wrote:

Hi Bron


Quoting Bron Gondwana :

> Sorry I haven't been following along with this earlier. Can you post
> your imapd.conf and cyrus.conf as well as let her know if you run
> anything which directly messes with files on disk.

Thanks for looking into it. I have attached the backend and replica
configs. There should be nothing messing with the files on disk
directly. Some times we need to restore mails from file based
backup (bacula) followed by a reconstruct but this was not the
case this time.


Do you always rebuild the conversations db after doing the  
reconstruct? That will be necessary now. We switched to doing all  
our restores using IMAP append a while back so we're never fiddling  
the file system under Cyrus.




We didn't have to restore any mails from backup since we enabled  
conversation db
a few weeks ago. It is good to know that the rebuild is necessary, but  
shouldn't

reconstruct also update the conversation db if it re-appends the message?
At least a hint should be put in the reconstruct man page, like the  
one for "quota -f"




>
> Also, what operating system is this on and what Cyrus version?
>

We are running a cyrus 3.0.8 compiled with the following Options
(./configure --enable-murder --enable-http --enable-calalarmd
--enable-replication --enable-backup --enable-idled
--enable-autocreate CFLAGS="-fPIC -g")
in a murder configuration on a RHEL 7.5 System. As filesystem we use xfs.


They should be fine. I'll have a read of the config when I'm at a  
real computer.




> Bron.
>
> On Fri, Jan 25, 2019, at 04:08, Michael Menge wrote:
>> Hi,
>>
>> I have discovered an other problem with the conversations db:
>>
>> Thousends of lines with "IOERROR: conversations_audit on load:" and
>> "IOERROR: conversations_audit on store:"
>> A look at the source code shows that these errors are logged after
>> "_sanity_check_counts" is called.
>> The log level LOG_ERR and the prefix IOERROR indicate that I have a
>> serious problem. Do I?
>>
>> This problem occurred for accounts where the rebuild of the
>> conversations db was successful.
>>
>> I don't want to rebuild the conversations db every few days.
>>
>> Any help is appreciated.
>>
>> Kind regards
>>
>>
>> Michael Menge





M.MengeTel.: (49) 7071/29-70316
Universität Tübingen   Fax.: (49) 7071/29-5912
Zentrum für Datenverarbeitung  mail:  
michael.me...@zdv.uni-tuebingen.de

Wächterstraße 76
72074 Tübingen


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: another problem with conversations db

[Bron Gondwana]

On Fri, Jan 25, 2019, at 20:00, Michael Menge wrote:
> Hi Bron
> 
> 
> Quoting Bron Gondwana :
> 
> > Sorry I haven't been following along with this earlier. Can you post 
> > your imapd.conf and cyrus.conf as well as let her know if you run 
> > anything which directly messes with files on disk.
> 
> Thanks for looking into it. I have attached the backend and replica
> configs. There should be nothing messing with the files on disk
> directly. Some times we need to restore mails from file based
> backup (bacula) followed by a reconstruct but this was not the
> case this time.

Do you always rebuild the conversations db after doing the reconstruct? That 
will be necessary now. We switched to doing all our restores using IMAP append 
a while back so we're never fiddling the file system under Cyrus.

> >
> > Also, what operating system is this on and what Cyrus version?
> >
> 
> We are running a cyrus 3.0.8 compiled with the following Options 
> (./configure --enable-murder --enable-http --enable-calalarmd 
> --enable-replication --enable-backup --enable-idled 
> --enable-autocreate CFLAGS="-fPIC -g")
> in a murder configuration on a RHEL 7.5 System. As filesystem we use xfs.

They should be fine. I'll have a read of the config when I'm at a real computer.

> 
> > Bron.
> >
> > On Fri, Jan 25, 2019, at 04:08, Michael Menge wrote:
> >> Hi,
> >>
> >> I have discovered an other problem with the conversations db:
> >>
> >> Thousends of lines with "IOERROR: conversations_audit on load:" and
> >> "IOERROR: conversations_audit on store:"
> >> A look at the source code shows that these errors are logged after
> >> "_sanity_check_counts" is called.
> >> The log level LOG_ERR and the prefix IOERROR indicate that I have a
> >> serious problem. Do I?
> >>
> >> This problem occurred for accounts where the rebuild of the
> >> conversations db was successful.
> >>
> >> I don't want to rebuild the conversations db every few days.
> >>
> >> Any help is appreciated.
> >>
> >> Kind regards
> >>
> >>
> >> Michael Menge
> >>
> >>
> >>
> >>
> >>
> >> 
> >> M.Menge Tel.: (49) 7071/29-70316
> >> Universität Tübingen Fax.: (49) 7071/29-5912
> >> Zentrum für Datenverarbeitung mail:
> >> michael.me...@zdv.uni-tuebingen.de
> >> Wächterstraße 76
> >> 72074 Tübingen
> >>
> >> 
> >> Cyrus Home Page: http://www.cyrusimap.org/
> >> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> >> To Unsubscribe:
> >> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
> >
> > --
> > Bron Gondwana
> > br...@fastmail.fm
> 
> 
> 
> 
> M.Menge Tel.: (49) 7071/29-70316
> Universität Tübingen Fax.: (49) 7071/29-5912
> Zentrum für Datenverarbeitung mail: 
> michael.me...@zdv.uni-tuebingen.de
> Wächterstraße 76
> 72074 Tübingen
> 
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
> 
> *Attachments:*
>  * imapd_be.conf
>  * cyrus_be.conf
>  * cyrus_re.conf
>  * imapd_re.conf

-- 
 Bron Gondwana
 br...@fastmail.fm

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: another problem with conversations db

[Michael Menge]

Hi Bron


Quoting Bron Gondwana :

Sorry I haven't been following along with this earlier. Can you post  
your imapd.conf and cyrus.conf as well as let her know if you run  
anything which directly messes with files on disk.


Thanks for looking into it. I have attached the backend and replica
configs. There should be nothing messing with the files on disk
directly. Some times we need to restore mails from file based
backup (bacula) followed by a reconstruct but this was not the
case this time.



Also, what operating system is this on and what Cyrus version?



We are running a cyrus 3.0.8 compiled with the following Options  
(./configure --enable-murder --enable-http --enable-calalarmd  
--enable-replication --enable-backup --enable-idled  
--enable-autocreate CFLAGS="-fPIC -g")

in a murder configuration on a RHEL 7.5 System. As filesystem we use xfs.



Bron.

On Fri, Jan 25, 2019, at 04:08, Michael Menge wrote:

Hi,

I have discovered an other problem with the conversations db:

Thousends of lines with "IOERROR: conversations_audit on load:" and
"IOERROR: conversations_audit on store:"
A look at the source code shows that these errors are logged after
"_sanity_check_counts" is called.
The log level LOG_ERR and the prefix IOERROR indicate that I have a
serious problem. Do I?

This problem occurred for accounts where the rebuild of the
conversations db was successful.

I don't want to rebuild the conversations db every few days.

Any help is appreciated.

Kind regards


 Michael Menge






M.Menge Tel.: (49) 7071/29-70316
Universität Tübingen Fax.: (49) 7071/29-5912
Zentrum für Datenverarbeitung mail:
michael.me...@zdv.uni-tuebingen.de
Wächterstraße 76
72074 Tübingen


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus


--
 Bron Gondwana
 br...@fastmail.fm





M.MengeTel.: (49) 7071/29-70316
Universität Tübingen   Fax.: (49) 7071/29-5912
Zentrum für Datenverarbeitung  mail:  
michael.me...@zdv.uni-tuebingen.de

Wächterstraße 76
72074 Tübingen
# Configuration for Backend/Failover Instance
# Template MD5SUM: b5b04095d552bb1cbc2b178f7edfd1e2  conf/imapd_master.template
servername: ma01.mail.localhost
configdirectory: /srv/cyrus-be
partition-default: /srv/cyrus-be
partition-ssd: /srv/cyrus-be/ssd-part
metapartition-ssd: /srv/cyrus-ssd-be/meta/ssd-part
metapartition_files: header index cache expunge squat annotations lock dav 
archivecache
archivepartition-ssd: /srv/cyrus-hdd-be/archive/ssd-part
archive_enabled: 1
archive_days: 31
archive_maxsize: 10240

proc_path: /srv/tmpfs/proc-be
mboxname_lockpath: /srv/tmpfs/lock-be
defaultpartition: ssd
admins:  

mupdate_server: mupdate.mail.localhost 
mupdate_port: 3905
# mupdate_authname verwenden nicht mupdate_username
# sonst wird login als root/cyrus versucht
#mupdate_username: 
mupdate_authname: 
mupdate_password: 
proxy_authname: 
proxy_password: 
proxyservers: 

allowusermoves: 1
allowallsubscribe: 1

sync_host: sl01.mail.localhost 
sync_authname: 
sync_password: 
sync_port: 2005
guid_mode: sha1
sync_log: 1
sync_shutdown_file: /srv/cyrus-be/sync/shutdown

sievedir: /srv/cyrus-be/sieve
sieve_extensions: fileinto reject vacation imapflags notify include envelope 
body relational regex subaddress copy
sieve_maxscriptsize: 150

sasl_pwcheck_method: saslauthd
sasl_mech_list: plain login

allowanonymouslogin: no
reject8bit: no
quotawarn: 90
timeout: 45
poptimeout: 10
dracinterval: 0
drachost: localhost
lmtp_over_quota_perm_failure: 1
altnamespace: 1
#flushseenstate: 1
unixhierarchysep: 1
hashimapspool: 1
fulldirhash: 1
duplicatesuppression: 0
expunge_mode: delayed
delete_mode: delayed
deletedprefix: DELETED
anysievefolder: 1
#annotation_db: skiplist
#duplicate_db: skiplist
#mboxlist_db: skiplist
#ptscache_db: skiplist
quota_db: quotalegacy
#seenstate_db: skiplist
subscriptions_db: flat
xlist-sent: Mail.sent
xlist-trash: Mail.trash
xlist-drafts: Mail.drafts
xlist-junk: Mail.v-spam
xlist-spam: Mail.v-spam
specialusealways: 1
syslog_prefix: be 

# https://bugzilla.cyrusimap.org/show_bug.cgi?id=3850
# Vermutlich gefixed
#suppress_capabilities: LIST-EXTENDED


tls_server_cert: /etc/pki/cyrus-imapd/cyrus-imapd.pem
tls_server_key: /etc/pki/cyrus-imapd/cyrus-imapd.pem
tls_client_ca_file: /etc/pki/tls/certs/ca-bundle.crt
tls_ciphers: 
EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA:EECDH:EDH+AESGCM:EDH:+3DES:ECDH+AESGCM:ECDH+AES:ECDH:AES:HIGH:MEDIUM:!RC4:!CAMELLIA:!SEED:!aNULL:!MD5:!eNULL:!LOW:!EXP:!DSS:!PSK:!SRP:!3DES:!IDEA
tls_prefer_server_ciphers: 1

# Änderungen 27.06.2018
reverseacls: 1
search_engine: squat
delete_unsubscribe: 1

another problem with conversations db

[Michael Menge]

Hi,

I have discovered an other problem with the conversations db:

Thousends of lines with "IOERROR: conversations_audit on load:" and  
"IOERROR: conversations_audit on store:"
A look at the source code shows that these errors are logged after  
"_sanity_check_counts" is called.
The log level LOG_ERR and the prefix IOERROR indicate that I have a  
serious problem. Do I?


This problem occurred for accounts where the rebuild of the  
conversations db was successful.


I don't want to rebuild the  conversations db every few days.

Any help is appreciated.

Kind regards


   Michael Menge






M.MengeTel.: (49) 7071/29-70316
Universität Tübingen   Fax.: (49) 7071/29-5912
Zentrum für Datenverarbeitung  mail:  
michael.me...@zdv.uni-tuebingen.de

Wächterstraße 76
72074 Tübingen


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Big problem with replication

[Albert Shih]

Le 16/01/2019 à 17:10:30+0100, Egoitz Aurrekoetxea a écrit
> Good afternoon,
> 
> 
> I would try doing it user by user (with -u). This way you would have all 
> synced
> except the problematic mailbox.

Hi, thanks for the help. 

I got some progress in my problem :

>   [root@imap-mirror-p /bals/DELETED]# /usr/local/cyrus/sbin/sync_client -S
> slave_3 -A -v
>   MAILBOXES DELETED.DIO.5AEAD6F9
>   Error from do_user(DELETED.DIO.5AEAD6F9): bailing out!
>   [root@imap-mirror-p /bals/DELETED]#
> 
> and the DIO folder don't event exist
> 
>   [root@imap-mirror-p /bals/DELETED]# ls DIO
>   ls: DIO: No such file or directory
>   [root@imap-mirror-p /bals/DELETED]#
> 

For some strange reason the I was unable to destroy the mailbox either
(with cyradm), so I copy some junk mailbox on the filesystem, and run
reconstruct and finally I'm was able to destroy those mailbox.

But that's not really solve my problem because now when I run the
sync_client he crash at the beginning with a shared mailbox. It stop with 


[root@imap-mirror-p /usr/home/jas-adm]# /usr/local/cyrus/sbin/sync_client -S 
imap-mirror-m-tmp -A -v
MAILBOXES shared.*
MAILBOX shared.*
Error from do_user(shared.*): bailing out!
[root@imap-mirror-p /usr/home/jas-adm]# 

I've no idea if it's normal or not. I don't think so, because the first
level (Master -- replica --> slave_1) work well event with thoses
shared_mailbox.

Any help would be very welcome.

Regards.
--
Albert SHIH
Heure local/Local time:
Wed Jan 16 18:11:53 CET 2019

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Big problem with replication

[Egoitz Aurrekoetxea]

Good afternoon, 

I would try doing it user by user (with -u). This way you would have all
synced except the problematic mailbox. 

Cheers!

---

EGOITZ AURREKOETXEA 
Departamento de sistemas 
944 209 470
Parque Tecnológico. Edificio 103
48170 Zamudio (Bizkaia) 
ego...@sarenet.es 
www.sarenet.es [1] 
Antes de imprimir este correo electrónico piense si es necesario
hacerlo. 

El 16-01-2019 16:15, Albert Shih escribió:

> Hi everyone.
> 
> I've got some big issue with replication.
> 
> I've 
> 
> master --- replica ---> slave_1 --- replica ---> slave_2
> 
> The replication between master and slave_1 work nice. 
> 
> Between slave_1 and slave_2 I've got some issue (log to big after network
> failure and work nagios_supervision). 
> 
> So now I'm trying to build a new slave_3 to replace slave_2. And I'm unable
> to launch sync_client. Each time I try to manually launch I got 
> 
> [root@imap-mirror-p /bals/DELETED]# /usr/local/cyrus/sbin/sync_client -S 
> slave_3 -A -v
> MAILBOXES DELETED.DIO.5AEAD6F9
> Error from do_user(DELETED.DIO.5AEAD6F9): bailing out!
> [root@imap-mirror-p /bals/DELETED]# 
> 
> and the DIO folder don't event exist
> 
> [root@imap-mirror-p /bals/DELETED]# ls DIO
> ls: DIO: No such file or directory
> [root@imap-mirror-p /bals/DELETED]# 
> 
> Any help would be *very* welcome ;-) 
> 
> Regards.
> 
> --
> Albert SHIH
> DIO bâtiment 15
> Observatoire de Paris
> Heure local/Local time:
> Wed Jan 16 16:08:47 CET 2019
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
 

Links:
--
[1] http://www.sarenet.es
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Big problem with replication

[Albert Shih]

Hi everyone.

I've got some big issue with replication.

I've 
  
master --- replica ---> slave_1 --- replica ---> slave_2

The replication between master and slave_1 work nice. 

Between slave_1 and slave_2 I've got some issue (log to big after network
failure and work nagios_supervision). 

So now I'm trying to build a new slave_3 to replace slave_2. And I'm unable
to launch sync_client. Each time I try to manually launch I got 

  [root@imap-mirror-p /bals/DELETED]# /usr/local/cyrus/sbin/sync_client -S 
slave_3 -A -v
  MAILBOXES DELETED.DIO.5AEAD6F9
  Error from do_user(DELETED.DIO.5AEAD6F9): bailing out!
  [root@imap-mirror-p /bals/DELETED]# 

and the DIO folder don't event exist

  [root@imap-mirror-p /bals/DELETED]# ls DIO
  ls: DIO: No such file or directory
  [root@imap-mirror-p /bals/DELETED]# 


Any help would be *very* welcome ;-) 

Regards.

--
Albert SHIH
DIO bâtiment 15
Observatoire de Paris
Heure local/Local time:
Wed Jan 16 16:08:47 CET 2019

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Problem with sync

[Albert Shih]

Le 08/10/2018 à 10:36:10+0200, Albert Shih a écrit
Hi everyone,

>
I still got my problem :

> I got two level of synchro:
>
> master --- sync --> imap-mirror-1 --- sync --> imap-mirror-2
>
> The first level work fine, the second level (imap-mirror-1 --> imap-mirror-2)
> crash sometime ago.
>
> Now I try to restart the sync, and currently I'm not sure it's working, in
> the imap-mirror-1 I got
>
>   [root@imap-mirror-1 /var/imap/sync/log]# ls -l
>   total 133519
>   -rw---  1 cyrus  cyrus   16080624 Oct  8 10:27 log
>   -rw---  1 cyrus  cyrus  256631941 Oct  3 16:39 log-run
>   [root@imap-mirror-1 /var/imap/sync/log]# top
>
> When I restart the imap daemon on both mirror, I see a sync_client on the
> imap-mirror-1 and imapd on the imap-mirror-2 at 100% during sometime ( 4
> days), now I still got the
>
> /var/imap/sync/log/log-run
>
> file and the
>
> /var/imap/sync/log/log
>
> who still growing up, but very few activity on those mirror.

But it seem the sync working, but it's very slow and stop very often. So
how can I speedup me sync, knowning at this speed I will never end the sync
because the

/var/imap/sync/log/log

still growing up. So if it wait the end of

/var/imap/sync/log/log-run

I don't see how this going to happen.

So is they are any way to synchronise what's in the

/var/imap/sync/log/log-run

manually without waiting the imap daemon to launch the sync.

Other question, at this time (during the sync of log-run) can I restart the
imap daemon without breaking everything ?

Regards.


--
Albert SHIH
DIO bâtiment 15
Observatoire de Paris
xmpp: j...@obspm.fr
Heure local/Local time:
Mon Oct 15 13:12:05 CEST 2018

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Problem with sync

[Albert Shih]

Hi everyone,


I got two level of synchro:

master --- sync --> imap-mirror-1 --- sync --> imap-mirror-2

The first level work fine, the second level (imap-mirror-1 --> imap-mirror-2)
crash sometime ago.

Now I try to restart the sync, and currently I'm not sure it's working, in
the imap-mirror-1 I got

  [root@imap-mirror-1 /var/imap/sync/log]# ls -l
  total 133519
  -rw---  1 cyrus  cyrus   16080624 Oct  8 10:27 log
  -rw---  1 cyrus  cyrus  256631941 Oct  3 16:39 log-run
  [root@imap-mirror-1 /var/imap/sync/log]# top

When I restart the imap daemon on both mirror, I see a sync_client on the
imap-mirror-1 and imapd on the imap-mirror-2 at 100% during sometime ( 4
days), now I still got the

/var/imap/sync/log/log-run

file and the

/var/imap/sync/log/log

who still growing up, but very few activity on those mirror.

What should I do after long period of missing sync ? How can I re-sync
everything ?

Regards


--
Albert SHIH
DIO bâtiment 15
Observatoire de Paris
xmpp: j...@obspm.fr
Heure local/Local time:
Mon Oct 8 10:30:46 CEST 2018

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: cyradm problem

[Paul van der Vlis]

Op 15-09-18 om 08:25 schreef bluntroller via Info-cyrus:
> Day,
> I totally dislike it but I need help here.
> I have postfix installed, up and running as a MTA.
> I have saslauthd installed up and running and an authentication server.
> I use the auxprop-sasldb2 alternative as a user/password database (and
> thought this were the easiest way to get it all up before turning to the
> mysql option, automating procedures, php-scripting etc)
> I can do remote-logins into my server via sasl authentication.
> I can do remote-logings into my (imaps) server with the aid of TLS
> Certificates only.
> I do not use the POP3 protocol at all.
> I do not use unsecured connections at all.
> Everything goes over TLS/sasl authentication/authorization.
> 
> However...
> If it comes to testsaslauthd, imtest or cyradm I can't connect to
> localhost.localdomain (via SSH) on my remote server or get a '*can't
> connect to server*' (cyradm) reply.

Not sure what you mean with "with ssh". What I do is log into the
machine with ssh, and then:
cyradm -u cyrus localhost
testsaslauthd -u paul -p xx -f /var/spool/postfix/var/run/saslauthd/mux

> I'm pretty sure it's a simple configuration problem or misunderstanding
> of the stack at all but I am stuck finding the needle in the haystack.
> It's probably a SSH problem but I am not sure.
> Inside SSH I use a certificate-based authentication too with root-logins
> not allowed ('without password')
> 
> Any help is very appreciated.

Hope it helps!

With regards,
Paul




-- 
Paul van der Vlis Linux systeembeheer Groningen
https://www.vandervlis.nl/


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: cyrus imap 3.0.8 build in Linux: problem with unit test, test "badservice" failed

[Sergey]

On Monday 17 September 2018, ellie timoney wrote:

> Does this patch help?
> https://github.com/cyrusimap/cyrus-imapd/commit/1cd8e90305093970d5d5953138a2ef8a723ea96f
>  

No, the error remains.

-- 
Regards,
Sergey

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: cyrus imap 3.0.8 build in Linux: problem with unit test, test "badservice" failed

[ellie timoney]

Does this patch help? 
https://github.com/cyrusimap/cyrus-imapd/commit/1cd8e90305093970d5d5953138a2ef8a723ea96f

On Mon, Sep 17, 2018, at 1:21 PM, ellie timoney wrote:
> Interesting, that sounds like the backend suite has some state that's 
> not being cleaned up/reinitialised properly between tests!  Thanks for 
> the report :)
> 
> I've raised this as https://github.com/cyrusimap/cyrus-imapd/issues/2526
> 
> On Sat, Sep 15, 2018, at 9:27 PM, Sergey wrote:
> > On Monday 19 October 2015, Sergey wrote:
> > 
> > > I found messages with similag error (threads.c:342: krb5int_key_register 
> > > ...)
> > > with krb5-1.13 (I use 1.13.1) in Google. I attempted to rollback to krb5 
> > > 1.12
> > > and test was passed. That is probably it is the problem of krb5.
> > 
> > The problem remains with Cyrus-IMAP 3.0.8 and krb5-1.16.1:
> > 
> > Suite: backend
> >   Test: badhost ...passed
> >   Test: badservice ...lt-unit: threads.c:353: krb5int_key_register: 
> > Assertion `destructors_set[keynum] == 0' failed.
> > 
> > Ivan A. Melnikov found an interesting nuance (sorry, comments on Russian):
> > https://bugzilla.altlinux.org/show_bug.cgi?id=31381#c9
> > 
> > In short: test suite is not passed in all, but it passed by one by one.
> > 
> > It works:
> > 
> > for test in `./unit -l | grep backend` ; do ./unit -v "$test"; done
> > 
> > It isn't:
> > 
> > ./unit -v "backend"
> > 
> > -- 
> > Regards,
> > Sergey
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: cyrus imap 3.0.8 build in Linux: problem with unit test, test "badservice" failed

[ellie timoney]

Interesting, that sounds like the backend suite has some state that's not being 
cleaned up/reinitialised properly between tests!  Thanks for the report :)

I've raised this as https://github.com/cyrusimap/cyrus-imapd/issues/2526

On Sat, Sep 15, 2018, at 9:27 PM, Sergey wrote:
> On Monday 19 October 2015, Sergey wrote:
> 
> > I found messages with similag error (threads.c:342: krb5int_key_register 
> > ...)
> > with krb5-1.13 (I use 1.13.1) in Google. I attempted to rollback to krb5 
> > 1.12
> > and test was passed. That is probably it is the problem of krb5.
> 
> The problem remains with Cyrus-IMAP 3.0.8 and krb5-1.16.1:
> 
> Suite: backend
>   Test: badhost ...passed
>   Test: badservice ...lt-unit: threads.c:353: krb5int_key_register: 
> Assertion `destructors_set[keynum] == 0' failed.
> 
> Ivan A. Melnikov found an interesting nuance (sorry, comments on Russian):
> https://bugzilla.altlinux.org/show_bug.cgi?id=31381#c9
> 
> In short: test suite is not passed in all, but it passed by one by one.
> 
> It works:
> 
> for test in `./unit -l | grep backend` ; do ./unit -v "$test"; done
> 
> It isn't:
> 
> ./unit -v "backend"
> 
> -- 
> Regards,
> Sergey

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: cyrus imap 3.0.8 build in Linux: problem with unit test, test "badservice" failed

[Sergey]

On Monday 19 October 2015, Sergey wrote:

> I found messages with similag error (threads.c:342: krb5int_key_register ...)
> with krb5-1.13 (I use 1.13.1) in Google. I attempted to rollback to krb5 1.12
> and test was passed. That is probably it is the problem of krb5.

The problem remains with Cyrus-IMAP 3.0.8 and krb5-1.16.1:

Suite: backend
  Test: badhost ...passed
  Test: badservice ...lt-unit: threads.c:353: krb5int_key_register: Assertion 
`destructors_set[keynum] == 0' failed.

Ivan A. Melnikov found an interesting nuance (sorry, comments on Russian):
https://bugzilla.altlinux.org/show_bug.cgi?id=31381#c9

In short: test suite is not passed in all, but it passed by one by one.

It works:

for test in `./unit -l | grep backend` ; do ./unit -v "$test"; done

It isn't:

./unit -v "backend"

-- 
Regards,
Sergey

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

cyradm problem

[bluntroller via Info-cyrus]

Day,
I totally dislike it but I need help here.
I have postfix installed, up and running as a MTA.
I have saslauthd installed up and running and an authentication server.
I use the auxprop-sasldb2 alternative as a user/password database (and thought 
this were the easiest way to get it all up before turning to the mysql option, 
automating procedures, php-scripting etc)
I can do remote-logins into my server via sasl authentication.
I can do remote-logings into my (imaps) server with the aid of TLS Certificates 
only.
I do not use the POP3 protocol at all.
I do not use unsecured connections at all.
Everything goes over TLS/sasl authentication/authorization.

However...
If it comes to testsaslauthd, imtest or cyradm I can't connect to 
localhost.localdomain (via SSH) on my remote server or get a 'can't connect to 
server' (cyradm) reply.
I'm pretty sure it's a simple configuration problem or misunderstanding of the 
stack at all but I am stuck finding the needle in the haystack.
It's probably a SSH problem but I am not sure.
Inside SSH I use a certificate-based authentication too with root-logins not 
allowed ('without password')

Any help is very appreciated.

Greets

Gee
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Cyrus 2.4 and vacation problem (smtputf8)

[Josef Karliak]

  Good morning,
  I upgraded servers to new Postfix versions, we had some issue with 
bounced vacation with czech characters. But postfix has smtputf8 support 
enabled :

smtputf8_enable = yes
smtputf8_autodetect_classes = sendmail, verify

  and the issue with emails (vacation) persists.
  After some checking I've found that thereis not postfix that bounce 
the message - it is cyrus'es LMTP, listenning on TCP/24:
Sep 13 08:08:12 postman11 postfix/cleanup[2916]: 59F2A280DB7: 
message-id=
Sep 13 08:08:12 postman11 postfix/qmgr[2885]: 59F2A280DB7: from=<>, 
size=658, nrcpt=1 (queue active)
Sep 13 08:08:12 postman11 postfix/lmtp[2918]: 55497280DB5: 
to=, orig_to=, 
relay=postman.domain.cz[1.2.3.101]:24, delay=0.04, delays=0/0/0/0.04, 
dsn=2.1.5, status=sent (250 2.1.5 Ok 
SESSIONID=)

Sep 13 08:08:12 postman11 postfix/qmgr[2885]: 55497280DB5: removed
Sep 13 08:08:12 postman11 postfix/lmtp[2918]: 59F2A280DB7: 
to=, orig_to=, 
relay=postman.domain.cz[1.2.3.101]:24, delay=0.11, delays=0/0/0.1/0, 
dsn=5.6.7, status=bounced (SMTPUTF8 is required, but was not offered by 
host postman.domain.cz[1.2.3.101])

Sep 13 08:08:12 postman11 postfix/qmgr[2885]: 59F2A280DB7: removed

  So do I need to set something in cyrus config ? What did I missed ?
  Thanks and best regards
  J.Karliak


--
Ma domena pouziva zabezpeceni a kontrolu SPF (www.openspf.org) a
DomainKeys/DKIM (s ADSP) a implementaci DMARC. Pokud mate problemy s
dorucenim emailu, zacnete pouzivat metody overeni puvody emailu
zminene vyse. Dekuji.
My domain use SPF (www.openspf.org) and DomainKeys/DKIM (with ADSP)
policy and implementation of the DMARC. If you've problem with sending
emails to me, start using email origin methods mentioned above. Thank
you.

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: reconstruct problem

[James B. Byrne via Info-cyrus]

On Tue, May 1, 2018 14:44, Sebastian Hagedorn wrote:
> reconstruct only works when the cyrus files exist. You can just touch
> them prior to running reconstruct.
>
>> Am 01.05.2018 um 20:18 schrieb Dr. Harry Knitter
>> <ha...@knitter-edv-beratung.de>:
>>
>> Hello
>>
>> reconstruct doesn't process submailboxes even when using the -r
>> option. No cyrus.* files are created in subfolders.
>> What can I do?
>>
>> Thanks


I have the same problem with Cyrus 3.0.5 reconstruct running on a
FreeBSd-11.1 host.  Even with the -f option it still does not recurse.

sudo -u cyrus /usr/local/cyrus/sbin/reconstruct -r -f user.testuser

ll
/var/spool/imap/t/user/testuser_hll/delivery/imports/cyrus.index-rw---
 1 cyrus  cyrus  96 Apr 27 09:13
/var/spool/imap/t/user/testuser_hll/delivery/imports/cyrus.index

rm /var/spool/imap/t/user/testuser_hll/delivery/imports/cyrus.index
remove
/var/spool/imap/t/user/testuser_hll/delivery/imports/cyrus.index? y

sudo -u cyrus /usr/local/cyrus/sbin/reconstruct -r -f user.testuser_hll

ll /var/spool/imap/t/user/testuser_hll/delivery/imports/cyrus.index
ls: /var/spool/imap/t/user/testuser_hll/delivery/imports/cyrus.index:
No such file or directory

sudo -u cyrus /usr/local/cyrus/sbin/reconstruct -r -f
user.testuser_hll/delivery/imports/cyrus.index

ll /var/spool/imap/t/user/testuser_hll/delivery/imports/cyrus.index
ls: /var/spool/imap/t/user/testuser_hll/delivery/imports/cyrus.index:
No such file or directory


Clearly either I am not using reconstruct correctly or it is not
working as it did formerly.  I expected that the missing index file
would be recreated by reconstruct and it was not.  In fact on other
mailboxes in the same maildir tree I can see absolutely no evidence
that reconstruct does anything at all.


-- 


***  e-Mail is NOT a SECURE channel  ***
Do NOT transmit sensitive data via e-Mail
 Do NOT open attachments nor follow links sent by e-Mail

James B. Byrnemailto:byrn...@harte-lyne.ca
Harte & Lyne Limited  http://www.harte-lyne.ca
9 Brockley Drive  vox: +1 905 561 1241
Hamilton, Ontario fax: +1 905 561 0757
Canada  L8E 3C3


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: reconstruct problem

[Sebastian Hagedorn]

reconstruct only works when the cyrus files exist. You can just touch them 
prior to running reconstruct.

> Am 01.05.2018 um 20:18 schrieb Dr. Harry Knitter 
> :
> 
> Hello
> 
> reconstruct doesn't process submailboxes even when using the -r option.
> No cyrus.* files are created in subfolders.
> What can I do?
> 
> Thanks
> 
> Harry
> using cyrus 2.4
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus


smime.p7s
Description: S/MIME cryptographic signature

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

reconstruct problem

[Dr. Harry Knitter]

Hello

reconstruct doesn't process submailboxes even when using the -r option.
No cyrus.* files are created in subfolders.
What can I do?

Thanks

Harry
 using cyrus 2.4


signature.asc
Description: This is a digitally signed message part.

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Problem after upgrading debian wheezy to jessie

[Dr. Harry Knitter]

Solution:
1. cyrus was not enabled via systemctl
2. Some symbolic links were not set:
ln -s /usr/sbin/cyrus /usr/sbin/ctl_cyrusdb
ln -s /usr/sbin/cyrus /usr/sbin/cyr_expire
ln -s /usr/sbin/cyrus /usr/sbin/tls_prune

WTF caused this?

Greetings

Harry



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Problem after upgrading debian wheezy to jessie

[Dr. Harry Knitter]

Am Samstag, 28. April 2018, 14:51:07 CEST schrieb Dan White:
> On 04/28/18 20:43 +0200, Dr. Harry Knitter wrote:
> >after upgrading debian wheezy to jessie a socket has gone:
> >/var/run/cyrus/socket/lmtp
> >
> >How to get out of this problem?
> 
> The lmtp unix domain socket is started by master via its /etc/cyrus.conf
> config file, commonly in an entry called 'lmtpunix', which will specificy
> the location for the socket. Check your syslog for errors, such as a
> permissions problem with the path.

Cyrus does not start at all. I found root@gateway:~# /etc/init.d/cyrus-imapd 
start
[ ok ] Starting cyrus-imapd (via systemctl): cyrus-imapd.service.
root@gateway:~# /etc/init.d/cyrus-imapd status
* cyrus-imapd.service - Cyrus IMAP/POP3 daemons
   Loaded: loaded (/lib/systemd/system/cyrus-imapd.service; disabled)
   Active: active (running) since Sun 2018-04-29 09:00:56 CEST; 3s ago
  Process: 30341 ExecStartPre=/usr/sbin/cyrus init-helper start (code=exited, 
status=0/SUCCESS)
 Main PID: 30365 (cyrmaster)
   CGroup: /system.slice/cyrus-imapd.service
   |-30365 /usr/sbin/cyrmaster -l 32 -C /etc/imapd.conf -M /etc/
cyrus.conf
   `-30389 notifyd

Apr 29 09:00:56 gateway cyrus/master[30387]: about to exec /usr/sbin/tls_prune
Apr 29 09:00:56 gateway cyrus/master[30365]: process 30387 exited, status 71
Apr 29 09:00:56 gateway cyrus/master[30365]: unable to setsocketopt(IP_TOS): 
Operation not supported
Apr 29 09:00:56 gateway cyrus/master[30365]: unable to setsocketopt(IP_TOS): 
Operation not supported
Apr 29 09:00:56 gateway cyrus/master[30365]: ready for work
Apr 29 09:00:56 gateway cyrus/master[30388]: about to exec /usr/sbin/
ctl_cyrusdb
Apr 29 09:00:56 gateway cyrus/master[30389]: about to exec /usr/lib/cyrus/bin/
notifyd
Apr 29 09:00:56 gateway cyrus/master[30388]: can't exec /usr/sbin/ctl_cyrusdb 
on schedule: No such file or directory
Apr 29 09:00:56 gateway cyrus/master[30365]: process 30388 exited, status 71
Apr 29 09:00:56 gateway cyrus/notify[30389]: executed
root@gateway:~# tail /var/log/mail.err
Apr 29 08:40:40 gateway cyrus/deliver[28762]: connect(/var/run/cyrus/socket/
lmtp) failed: No such file or directory
Apr 29 08:40:40 gateway cyrus/deliver[28765]: connect(/var/run/cyrus/socket/
lmtp) failed: No such file or directory
Apr 29 09:00:40 gateway cyrus/deliver[30319]: connect(/var/run/cyrus/socket/
lmtp) failed: No such file or directory
Apr 29 09:00:56 gateway cyrus/master[30385]: can't exec /usr/sbin/ctl_cyrusdb 
for startup: No such file or directory
Apr 29 09:00:56 gateway cyrus/master[30365]: process 30385 exited, status 71
Apr 29 09:00:56 gateway cyrus/master[30386]: can't exec /usr/sbin/cyr_expire 
for startup: No such file or directory
Apr 29 09:00:56 gateway cyrus/master[30365]: process 30386 exited, status 71
Apr 29 09:00:56 gateway cyrus/master[30387]: can't exec /usr/sbin/tls_prune 
for startup: No such file or directory
Apr 29 09:00:56 gateway cyrus/master[30365]: process 30387 exited, status 71
Apr 29 09:00:56 gateway cyrus/master[30388]: can't exec /usr/sbin/ctl_cyrusdb 
on schedule: No such file or directory

Harry




signature.asc
Description: This is a digitally signed message part.

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Problem after upgrading debian wheezy to jessie

[Dr. Harry Knitter]

Am Samstag, 28. April 2018, 14:51:07 CEST schrieb Dan White:
> On 04/28/18 20:43 +0200, Dr. Harry Knitter wrote:
> >after upgrading debian wheezy to jessie a socket has gone:
> >
> >
> >How to get out of this problem?
> 
> The lmtp unix domain socket is started by master via its /etc/cyrus.conf
> config file, commonly in an entry called 'lmtpunix', which will specificy
> the location for the socket. Check your syslog for errors, such as a
> permissions problem with the path.


Thanks for your answer.
The socket /var/run/cyrus/socket/lmtp does not exist at all anymore. I can't 
see any permission problem.

Harry

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Problem after upgrading debian wheezy to jessie

[Dan White]

On 04/28/18 20:43 +0200, Dr. Harry Knitter wrote:

after upgrading debian wheezy to jessie a socket has gone:
/var/run/cyrus/socket/lmtp

How to get out of this problem?


The lmtp unix domain socket is started by master via its /etc/cyrus.conf
config file, commonly in an entry called 'lmtpunix', which will specificy
the location for the socket. Check your syslog for errors, such as a
permissions problem with the path.

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Problem after upgrading debian wheezy to jessie

[Dr. Harry Knitter]

Hello folks,

after upgrading debian wheezy to jessie a socket has gone:
/var/run/cyrus/socket/lmtp

How to get out of this problem?

Thanks

Harry


signature.asc
Description: This is a digitally signed message part.

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

rsync problem backing up cyrus-imap emails

[Nikos Gatsis - Qbit]

Title: Untitled Document

  
  
Hello list.

I send the following some years ago and still facing same problem again:

I use rsync to backup cyrus mail dirs using the following command:

rsync -vaR --delete --log-file=/var/log/rsync /var/lib/imap /var/spool/imap/ /mnt/backup

The destination is a WD external network drive in unknown format, probably ntfs.
The problem I have is that rsync change email names from, lets say 102. to 2RB3UX~6.

The external disk cant change from ntfs to other format.

Does someone had the same problem?
Thank you in advance.
-- 
  
  
  
  Γατσής
  Νίκος - Gatsis Nikos
Web developer
tel.: 2108256721 - 2108256722
fax: 2108256712
email: ngat...@qbit.gr
http://www.qbit.gr 
  




smime.p7s
Description: S/MIME Cryptographic Signature

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Problem after update (deliver not working anymore)

[Dr. Peer-Joachim Koch]

Hi,

I updated our cyrus server (SuSE SLES 12) to the latest release. First tests

did not show any problem. But now I noticed the following:


Oct 24 11:48:44 mail postfix/local[3438]: 0F6604A6F0: 
to=<x...@bgc-jena.mpg.de>, relay=local, delay=0.03, delays=0.01/0/0/0.01, 
dsn=4.3.0, status=deferred (temporary failure. Command output: couldn't 
connect to lmtpd: Permission denied 421 4.3.0 deliver: couldn't connect 
to lmtpd_



We are using for some shared mailfolders /etc/aliases with

NAME:  ""|/usr/bin/formail -I \"From \" |/usr/lib/cyrus/bin/deliver -e 
-a cyrus -m NAME"


It worked without any problems before.

Normal mail (to users) are working perfectly ...

I tried to change the permission of /var/lib/imap/socket/lmp without any 
success.


Any idea ?


--
Bye,
Peer


Max-Planck-Institut für Biogeochemie
Dr. Peer-Joachim Koch
Hans-Knöll Str.10Telefon: ++49 3641 57-6705
D-07745 Jena Telefax: ++49 3641 57-7705




smime.p7s
Description: S/MIME Cryptographic Signature

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: odd problem with cyradm

[Per olof Ljungmark]

On 2017-08-21 08:57, Egoitz Aurrekoetxea wrote:
> Have you copied from another machine or similar the quota database??
> 
> 
> You should never do that

No. The mailstore was transferred with imapsync many months ago.

The test enviroment behaves identical, ie. the MUA'a and bin/quota with
and without -f all report correct values for quota.

Only thing that does not work is cyradm for reading qoutas, "set quota"
works.

I think I'm giving up on this one for now because it is not a problem
really production-wise.

Thanks,

> 
> El 21/8/17 a las 8:42, Per olof Ljungmark escribió:
>> On 2017-08-21 08:11, Egoitz Aurrekoetxea wrote:
>>> Good morning,
>>>
>>>
>>> What happens if you launch the cyradm from a remote machine?. For
>>> instance in a FreeBSD with another Perl version?. Does it work?.
>>>
>>
>> No, tried that and results are the same. Current line of thought is
>> that something is not right with the quota database, I am building a
>> testing setup now to verify.
>>
>>>
>>> El 19/8/17 a las 13:18, Per olof Ljungmark escribió:
>>>> Hi all,
>>>>
>>>> Wonder if someone can offer help.
>>>>
>>>> Host is FreeBSD 11.0-STABLE #0 r316644M and cyrus 2.5.11 in a jail.
>>>> If I run cyradm as user cyrus (admin) and issue the lq command, usually
>>>> there is a proper response at first, but subsequent commands fail. This
>>>> could very well be a FreeBSD problem but I thought I'll ask here first.
>>>>
>>>> 1st:
>>>>
>>>> read(0,"lq user/\n",8192)  = 15 (0xf)
>>>> write(3,"8 GETQUOTA user/\r\n",24) = 24 (0x18)
>>>> select(4,{ 3 },{ },0x0,0x0)  = 1 (0x1)
>>>> read(3,"* QUOTA user/ (STORAGE 888619 1000)\r\n8 OK
>>>> Completed\r\n",4096) = 63 (0x3f)
>>>> write(1," STORAGE 888619/1000",24)   = 24 (0x18)
>>>> write(1," (8.88619%)",11)= 11 (0xb)
>>>> write(1,"\n",1)  = 1 (0x1)
>>>> write(1,"192.168.64.12> ",15)= 15 (0xf)
>>>>
>>>> and following
>>>>
>>>> read(0,"lq user/\n",8192)  = 15 (0xf)
>>>> write(3,"10 GETQUOTA user/\r\n",25)= 25 (0x19)
>>>> select(4,{ 3 },{ },0x0,0x0)  = 1 (0x1)
>>>> read(3,"* QUOTA user/ (STORAGE 888619 1000)\r\n10 OK
>>>> Completed\r\n",4096) = 64 (0x40)
>>>> write(1,"192.168.64.12> ",15)= 15 (0xf)
>>>>
>>>> As one can see, cyradm does not write out the info, just reads it.
>>>>
>>>> There are no quota problems AFAICS, bin/quota and -f all works as
>>>> expected and mail agents sees proper quota info, so I am inclined to
>>>> think there is something fishy with cyradm or possibly something I
>>>> cannot see with the quotas.
>>>>
>>>> Same with both quotas.db twoskip and quotalegacy. Other cyradm commands
>>>> works fine.
>>>>
>>>> Thanks!
>>>>
>>>> //per
>>>> 
>>>> Cyrus Home Page: http://www.cyrusimap.org/
>>>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>>>> To Unsubscribe:
>>>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
>>>
>>> -- 
>>>
>>>
>>> sarenet
>>> *Egoitz Aurrekoetxea*
>>> Departamento de sistemas
>>> 944 209 470
>>> Parque Tecnológico. Edificio 103
>>> 48170 Zamudio (Bizkaia)
>>> ego...@sarenet.es <mailto:ego...@sarenet.es>
>>> www.sarenet.es <https://www.sarenet.es>
>>>
>>> Antes de imprimir este correo electrónico piense si es necesario
>>> hacerlo.
>>>
>>>
>>> 
>>> Cyrus Home Page: http://www.cyrusimap.org/
>>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>>> To Unsubscribe:
>>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
>>>
>> 
>> Cyrus Home Page: http://www.cyrusimap.org/
>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>> To Unsubscribe:
>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
> 
> -- 
> 
> 
> sarenet
> *Egoitz Aurrekoetxea*
> Departamento de sistemas
> 944 209 470
> Parque Tecnológico. Edificio 103
> 48170 Zamudio (Bizkaia)
> ego...@sarenet.es <mailto:ego...@sarenet.es>
> www.sarenet.es <https://www.sarenet.es>
> 
> Antes de imprimir este correo electrónico piense si es necesario hacerlo.
> 
> 
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
> 

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: odd problem with cyradm

[Egoitz Aurrekoetxea]

Have you copied from another machine or similar the quota database??


You should never do that


Best regards,


El 21/8/17 a las 8:42, Per olof Ljungmark escribió:

On 2017-08-21 08:11, Egoitz Aurrekoetxea wrote:

Good morning,


What happens if you launch the cyradm from a remote machine?. For
instance in a FreeBSD with another Perl version?. Does it work?.



No, tried that and results are the same. Current line of thought is 
that something is not right with the quota database, I am building a 
testing setup now to verify.




El 19/8/17 a las 13:18, Per olof Ljungmark escribió:

Hi all,

Wonder if someone can offer help.

Host is FreeBSD 11.0-STABLE #0 r316644M and cyrus 2.5.11 in a jail.
If I run cyradm as user cyrus (admin) and issue the lq command, usually
there is a proper response at first, but subsequent commands fail. This
could very well be a FreeBSD problem but I thought I'll ask here first.

1st:

read(0,"lq user/\n",8192)  = 15 (0xf)
write(3,"8 GETQUOTA user/\r\n",24) = 24 (0x18)
select(4,{ 3 },{ },0x0,0x0)  = 1 (0x1)
read(3,"* QUOTA user/ (STORAGE 888619 1000)\r\n8 OK
Completed\r\n",4096) = 63 (0x3f)
write(1," STORAGE 888619/1000",24)   = 24 (0x18)
write(1," (8.88619%)",11)= 11 (0xb)
write(1,"\n",1)  = 1 (0x1)
write(1,"192.168.64.12> ",15)= 15 (0xf)

and following

read(0,"lq user/\n",8192)  = 15 (0xf)
write(3,"10 GETQUOTA user/\r\n",25)= 25 (0x19)
select(4,{ 3 },{ },0x0,0x0)  = 1 (0x1)
read(3,"* QUOTA user/ (STORAGE 888619 1000)\r\n10 OK
Completed\r\n",4096) = 64 (0x40)
write(1,"192.168.64.12> ",15)= 15 (0xf)

As one can see, cyradm does not write out the info, just reads it.

There are no quota problems AFAICS, bin/quota and -f all works as
expected and mail agents sees proper quota info, so I am inclined to
think there is something fishy with cyradm or possibly something I
cannot see with the quotas.

Same with both quotas.db twoskip and quotalegacy. Other cyradm commands
works fine.

Thanks!

//per

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus


--


sarenet
*Egoitz Aurrekoetxea*
Departamento de sistemas
944 209 470
Parque Tecnológico. Edificio 103
48170 Zamudio (Bizkaia)
ego...@sarenet.es <mailto:ego...@sarenet.es>
www.sarenet.es <https://www.sarenet.es>

Antes de imprimir este correo electrónico piense si es necesario 
hacerlo.




Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus


--


sarenet
*Egoitz Aurrekoetxea*
Departamento de sistemas
944 209 470
Parque Tecnológico. Edificio 103
48170 Zamudio (Bizkaia)
ego...@sarenet.es <mailto:ego...@sarenet.es>
www.sarenet.es <https://www.sarenet.es>

Antes de imprimir este correo electrónico piense si es necesario hacerlo.

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: odd problem with cyradm

[Per olof Ljungmark]

On 2017-08-21 08:11, Egoitz Aurrekoetxea wrote:

Good morning,


What happens if you launch the cyradm from a remote machine?. For
instance in a FreeBSD with another Perl version?. Does it work?.



No, tried that and results are the same. Current line of thought is that 
something is not right with the quota database, I am building a testing 
setup now to verify.




El 19/8/17 a las 13:18, Per olof Ljungmark escribió:

Hi all,

Wonder if someone can offer help.

Host is FreeBSD 11.0-STABLE #0 r316644M and cyrus 2.5.11 in a jail.
If I run cyradm as user cyrus (admin) and issue the lq command, usually
there is a proper response at first, but subsequent commands fail. This
could very well be a FreeBSD problem but I thought I'll ask here first.

1st:

read(0,"lq user/\n",8192)  = 15 (0xf)
write(3,"8 GETQUOTA user/\r\n",24) = 24 (0x18)
select(4,{ 3 },{ },0x0,0x0)  = 1 (0x1)
read(3,"* QUOTA user/ (STORAGE 888619 1000)\r\n8 OK
Completed\r\n",4096) = 63 (0x3f)
write(1," STORAGE 888619/1000",24)   = 24 (0x18)
write(1," (8.88619%)",11)= 11 (0xb)
write(1,"\n",1)  = 1 (0x1)
write(1,"192.168.64.12> ",15)= 15 (0xf)

and following

read(0,"lq user/\n",8192)  = 15 (0xf)
write(3,"10 GETQUOTA user/\r\n",25)= 25 (0x19)
select(4,{ 3 },{ },0x0,0x0)  = 1 (0x1)
read(3,"* QUOTA user/ (STORAGE 888619 1000)\r\n10 OK
Completed\r\n",4096) = 64 (0x40)
write(1,"192.168.64.12> ",15)= 15 (0xf)

As one can see, cyradm does not write out the info, just reads it.

There are no quota problems AFAICS, bin/quota and -f all works as
expected and mail agents sees proper quota info, so I am inclined to
think there is something fishy with cyradm or possibly something I
cannot see with the quotas.

Same with both quotas.db twoskip and quotalegacy. Other cyradm commands
works fine.

Thanks!

//per

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus


--


sarenet
*Egoitz Aurrekoetxea*
Departamento de sistemas
944 209 470
Parque Tecnológico. Edificio 103
48170 Zamudio (Bizkaia)
ego...@sarenet.es <mailto:ego...@sarenet.es>
www.sarenet.es <https://www.sarenet.es>

Antes de imprimir este correo electrónico piense si es necesario hacerlo.



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

odd problem with cyradm

[Per olof Ljungmark]

Hi all,

Wonder if someone can offer help.

Host is FreeBSD 11.0-STABLE #0 r316644M and cyrus 2.5.11 in a jail.
If I run cyradm as user cyrus (admin) and issue the lq command, usually
there is a proper response at first, but subsequent commands fail. This
could very well be a FreeBSD problem but I thought I'll ask here first.

1st:

read(0,"lq user/\n",8192)  = 15 (0xf)
write(3,"8 GETQUOTA user/\r\n",24) = 24 (0x18)
select(4,{ 3 },{ },0x0,0x0)  = 1 (0x1)
read(3,"* QUOTA user/ (STORAGE 888619 1000)\r\n8 OK
Completed\r\n",4096) = 63 (0x3f)
write(1," STORAGE 888619/1000",24)   = 24 (0x18)
write(1," (8.88619%)",11)= 11 (0xb)
write(1,"\n",1)  = 1 (0x1)
write(1,"192.168.64.12> ",15)= 15 (0xf)

and following

read(0,"lq user/\n",8192)  = 15 (0xf)
write(3,"10 GETQUOTA user/\r\n",25)= 25 (0x19)
select(4,{ 3 },{ },0x0,0x0)  = 1 (0x1)
read(3,"* QUOTA user/ (STORAGE 888619 1000)\r\n10 OK
Completed\r\n",4096) = 64 (0x40)
write(1,"192.168.64.12> ",15)= 15 (0xf)

As one can see, cyradm does not write out the info, just reads it.

There are no quota problems AFAICS, bin/quota and -f all works as
expected and mail agents sees proper quota info, so I am inclined to
think there is something fishy with cyradm or possibly something I
cannot see with the quotas.

Same with both quotas.db twoskip and quotalegacy. Other cyradm commands
works fine.

Thanks!

//per

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Asterisk problem in Sieve Script

[Walter H. via Info-cyrus]

Hello,

with the, there is not any match ...

'.*' would be a regular expression matching anything,
but sieve filter language doesn't support regular expressions, or does it?

I did a workaround on the other side, I did a reverse DNS lookup, and 
most times
there is a DNS name instead of an IP address, and with this I didn't 
find any sample that

doesn't work ...

Greetings,
Walter

On 17.05.2017 14:41, Eric W. Bates wrote:

Try:

elsif header :matches "subject" "[proxy] File-URL (.*) detected"

On 5/16/2017 9:53 AM, Walter H. via Info-cyrus wrote:

Hello,

I've got the following in the sieve script

elsif header :matches "subject" "[proxy] File-URL (*) detected"
{
 fileinto "INBOX._Info.ftpFileURLs";
}
else
{
 fileinto "INBOX._Info";
}

when the subject is e.g.
[proxy] File-URL (Media-PC) detected
then the if is true, this correct

when the subject is e.g.
[proxy] File-URL (2001:dead:beef:123::1234:5678) detected
then the if is also true, this is correct

when there is an IPv4 address e.g.
[proxy] File-URL (192.168.0.122) detected
this matches also,

but why
doesn't the following match:

[proxy] File-URL (2001:dead:beef:123::12:3456:7890) detected
[proxy] File-URL (2001:dead:beef:123:0:12:3456:7890) detected

Thanks,
Walter






smime.p7s
Description: S/MIME Cryptographic Signature

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Asterisk problem in Sieve Script

[Eric W. Bates]

Try:

elsif header :matches "subject" "[proxy] File-URL (.*) detected"

On 5/16/2017 9:53 AM, Walter H. via Info-cyrus wrote:
> Hello,
> 
> I've got the following in the sieve script
> 
> elsif header :matches "subject" "[proxy] File-URL (*) detected"
> {
> fileinto "INBOX._Info.ftpFileURLs";
> }
> else
> {
> fileinto "INBOX._Info";
> }
> 
> when the subject is e.g.
> [proxy] File-URL (Media-PC) detected
> then the if is true, this correct
> 
> when the subject is e.g.
> [proxy] File-URL (2001:dead:beef:123::1234:5678) detected
> then the if is also true, this is correct
> 
> when there is an IPv4 address e.g.
> [proxy] File-URL (192.168.0.122) detected
> this matches also,
> 
> but why
> doesn't the following match:
> 
> [proxy] File-URL (2001:dead:beef:123::12:3456:7890) detected
> [proxy] File-URL (2001:dead:beef:123:0:12:3456:7890) detected
> 
> Thanks,
> Walter
> 
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
> 

-- 
Clark 159a, MS 46
508/289-3112



smime.p7s
Description: S/MIME Cryptographic Signature

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Asterisk problem in Sieve Script

[Walter H. via Info-cyrus]

Hello,

I've got the following in the sieve script

elsif header :matches "subject" "[proxy] File-URL (*) detected"
{
fileinto "INBOX._Info.ftpFileURLs";
}
else
{
fileinto "INBOX._Info";
}

when the subject is e.g.
[proxy] File-URL (Media-PC) detected
then the if is true, this correct

when the subject is e.g.
[proxy] File-URL (2001:dead:beef:123::1234:5678) detected
then the if is also true, this is correct

when there is an IPv4 address e.g.
[proxy] File-URL (192.168.0.122) detected
this matches also,

but why
doesn't the following match:

[proxy] File-URL (2001:dead:beef:123::12:3456:7890) detected
[proxy] File-URL (2001:dead:beef:123:0:12:3456:7890) detected

Thanks,
Walter


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Problem moving mailboxes

[Jiří Pavlovský]

Hello,


I need to move some mailboxes to a new partition.
I'm following procedure given here
https://lists.andrew.cmu.edu/pipermail/info-cyrus/2006-October/024096.html

But when I issue the renamemailbox command i cyradm I get the following
error:
"renamemailbox: Server(s) unavailable to complete operation"

Any idea how to fix this?


Thank you,

-- 
Jiří Pavlovský


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: strange sieve problem

[Walter H. via Info-cyrus]

Hello,

Oh, didn't change waldinet.local to my.local

I found out why I had problems, the 2nd worked, the first not ...

in
/var/lib/imap/sieve

there I had 2 folders:
global and w/walter
and both contained a sieve script, the one under w/walter was older and
didn't handle the 2nd mail, and I guess that the other in global didn't
come to work ...

I did the following

service cyrus-imapd stop
rm -rf w/walter
mv global w/walter
service cyrus-imapd start

and this solved it, but I don't know why there were 2 sieve scripts - one
global and for user walter (me)

when I update my sieve script I do the following:
sieveshell --authname=cyrus --user=walter localhost
and there  put filter.script
quite strange how this happened;

Greetings,
Walter


On Tue, April 4, 2017 12:15, Patrick Boutilier wrote:
> Is it the second email sample that does not work? If so the To: on that
> one is walter@waldinet.local and I don't see any rules for
> walter@waldinet.local. Also, where does the mail that doesn't work end up?
>
>
> On 04/04/2017 03:00 AM, Walter H. via Info-cyrus wrote:
>> Hello,
>>
>> I've found a Sieve Tester, where everything works as I expect
>>
>> https://www.fastmail.com/cgi-bin/sievetest.pl
>>
>> but Cyrus Sieve doesn't
>>
>> here the Sieve-Script
>>
>> 
>> # Sieve filter
>>
>> require ["fileinto", "relational"];
>>
>> if not exists ["from"]
>> {
>> discard;
>> }
>> elsif allof (address :all :is "from" "sq...@proxy.my.local",
>> address :all :is "to" "walter@my.local")
>> {
>> if header :matches "subject" "[proxy] Video-URL (*) detected"
>> {
>> fileinto "INBOX._Info.hbbtvVideoURLs";
>> }
>> elsif header :matches "subject" "[proxy] File-URL (*) detected"
>> {
>> fileinto "INBOX._Info.ftpFileURLs";
>> }
>> else
>> {
>> fileinto "INBOX._Info";
>> }
>> }
>> elsif allof (address :all :is "from" "cla...@mail.my.local",
>> address :all :is "to" "walter@my.local")
>> {
>> if header :matches "subject" "[mail] Virus detected in E-mail"
>> {
>> fileinto "INBOX._Alert";
>> }
>> }
>> elsif header :matches "list-id" "*"
>> {
>> fileinto "INBOX._MailLists._CENTOS";
>> }
>> elsif header :is "precedence" "bulk"
>> {
>> fileinto "INBOX.Trash";
>> }
>> else
>> {
>> keep;
>> }
>> 
>>
>> and this is the Mail
>>
>> 
>> Return-Path: 
>> Received: from storage.mail ([unix socket])
>>  by storage.mail (Cyrus v2.3.16-Fedora-RPM-2.3.16-13.el6_6) with
>> LMTPA;
>>  Mon, 03 Apr 2017 21:27:35 +0200
>> X-Sieve: CMU Sieve 2.3
>> Received: from proxy.host by storage.mail (Postfix) with ESMTP id
>> 19B2C79235
>> Received: by proxy.host (Postfix, userid 23) id EB81D2B0BE
>> Date: Mon, 03 Apr 2017 21:27:34 +0200
>> To: walter@my.local
>> Subject: [proxy] File-URL (PC) detected
>> User-Agent: Heirloom mailx 12.4 7/29/08
>> MIME-Version: 1.0
>> Content-Type: text/plain; charset=us-ascii
>> Content-Transfer-Encoding: 7bit
>> Message-Id: <20170403192734.eb81d2b...@proxy.my.local>
>> From: sq...@proxy.my.local (Squid)
>>
>> The following information came from the Squid proxy virtual machine.
>>
>> --[ Data submitted
>> ]---
>>
>> File-URL: ftp://ftp.adobe.com/lbtest.txt
>>
>> 
>>
>> this Mail is sorted correct by the sieve script
>>
>> 
>> Return-Path: 
>> Received: from storage.mail ([unix socket])
>> by storage.mail (Cyrus v2.3.16-Fedora-RPM-2.3.16-13.el6_6) with LMTPA;
>> Sun, 05 Feb 2017 19:14:15 +0100
>> X-Sieve: CMU Sieve 2.3
>> Received: from filter.mail by storage.mail (Postfix) with ESMTP id
>> 5634078BA8
>> Received: by filter.mail (Postfix) id 48F198E9
>> Delivered-To: r...@filter.mail
>> Received: from filter.mail [local] by filter.mail (Postfix) with ESMTP
>> id
>> 35E838E8
>> Received: by filter.mail (Postfix, userid 496) id 2A20D8E9
>> From: ClamAV 
>> To: walter@waldinet.local
>> Subject: [mail] Virus detected in E-mail
>> Message-Id: <20170205181415.2a20d...@mail.my.local>
>> Date: Sun, 5 Feb 2017 19:14:15 +0100 (CET)
>> X-AV-Scanned: ClamAV using ClamSMTP (filter.mail)
>>
>> The following information came from the Mail filter virtual machine.
>>
>> --[ Data submitted
>> ]---
>>
>> Virus name: Heuristics.Phishing.Email.SpoofedDomain
>> Sender:
>> rte+ne-null-b1cb1a01203481e6zubgcse...@sellernotifications.amazon.com
>>
>> Quarantined to: /var/lib/clamd.clamsmtp/virus.XeKpYL
>>
>> --[ E-Mail header
>> ]
>>
>> ...
>>
>> 
>>
>> can someone give me a hint, what is wrong,
>>
>> Thanks,
>> Walter
>>
>> 
>> Cyrus Home Page: http://www.cyrusimap.org/
>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>> To Unsubscribe:
>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: strange sieve problem

[Patrick Boutilier]

Is it the second email sample that does not work? If so the To: on that 
one is walter@waldinet.local and I don't see any rules for 
walter@waldinet.local. Also, where does the mail that doesn't work end up?



On 04/04/2017 03:00 AM, Walter H. via Info-cyrus wrote:

Hello,

I've found a Sieve Tester, where everything works as I expect

https://www.fastmail.com/cgi-bin/sievetest.pl

but Cyrus Sieve doesn't

here the Sieve-Script


# Sieve filter

require ["fileinto", "relational"];

if not exists ["from"]
{
discard;
}
elsif allof (address :all :is "from" "sq...@proxy.my.local",
address :all :is "to" "walter@my.local")
{
if header :matches "subject" "[proxy] Video-URL (*) detected"
{
fileinto "INBOX._Info.hbbtvVideoURLs";
}
elsif header :matches "subject" "[proxy] File-URL (*) detected"
{
fileinto "INBOX._Info.ftpFileURLs";
}
else
{
fileinto "INBOX._Info";
}
}
elsif allof (address :all :is "from" "cla...@mail.my.local",
address :all :is "to" "walter@my.local")
{
if header :matches "subject" "[mail] Virus detected in E-mail"
{
fileinto "INBOX._Alert";
}
}
elsif header :matches "list-id" "*"
{
fileinto "INBOX._MailLists._CENTOS";
}
elsif header :is "precedence" "bulk"
{
fileinto "INBOX.Trash";
}
else
{
keep;
}


and this is the Mail


Return-Path: 
Received: from storage.mail ([unix socket])
 by storage.mail (Cyrus v2.3.16-Fedora-RPM-2.3.16-13.el6_6) with LMTPA;
 Mon, 03 Apr 2017 21:27:35 +0200
X-Sieve: CMU Sieve 2.3
Received: from proxy.host by storage.mail (Postfix) with ESMTP id 19B2C79235
Received: by proxy.host (Postfix, userid 23) id EB81D2B0BE
Date: Mon, 03 Apr 2017 21:27:34 +0200
To: walter@my.local
Subject: [proxy] File-URL (PC) detected
User-Agent: Heirloom mailx 12.4 7/29/08
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-Id: <20170403192734.eb81d2b...@proxy.my.local>
From: sq...@proxy.my.local (Squid)

The following information came from the Squid proxy virtual machine.

--[ Data submitted ]---

File-URL: ftp://ftp.adobe.com/lbtest.txt



this Mail is sorted correct by the sieve script


Return-Path: 
Received: from storage.mail ([unix socket])
by storage.mail (Cyrus v2.3.16-Fedora-RPM-2.3.16-13.el6_6) with LMTPA;
Sun, 05 Feb 2017 19:14:15 +0100
X-Sieve: CMU Sieve 2.3
Received: from filter.mail by storage.mail (Postfix) with ESMTP id 5634078BA8
Received: by filter.mail (Postfix) id 48F198E9
Delivered-To: r...@filter.mail
Received: from filter.mail [local] by filter.mail (Postfix) with ESMTP id
35E838E8
Received: by filter.mail (Postfix, userid 496) id 2A20D8E9
From: ClamAV 
To: walter@waldinet.local
Subject: [mail] Virus detected in E-mail
Message-Id: <20170205181415.2a20d...@mail.my.local>
Date: Sun, 5 Feb 2017 19:14:15 +0100 (CET)
X-AV-Scanned: ClamAV using ClamSMTP (filter.mail)

The following information came from the Mail filter virtual machine.

--[ Data submitted ]---

Virus name: Heuristics.Phishing.Email.SpoofedDomain
Sender: rte+ne-null-b1cb1a01203481e6zubgcse...@sellernotifications.amazon.com

Quarantined to: /var/lib/clamd.clamsmtp/virus.XeKpYL

--[ E-Mail header ]

...



can someone give me a hint, what is wrong,

Thanks,
Walter


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus



<>
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: strange sieve problem

[Walter H. via Info-cyrus]

On Tue, April 4, 2017 08:42, ellie timoney wrote:
>> Received: from storage.mail ([unix socket])
>>  by storage.mail (Cyrus v2.3.16-Fedora-RPM-2.3.16-13.el6_6) with
>>  LMTPA;
>>  Mon, 03 Apr 2017 21:27:35 +0200
>> X-Sieve: CMU Sieve 2.3
>
> Wild guess, is your script using sieve features that are not available
> in 2.3.16?
can't imagine, because when you look at the samples below, you see, that
one works and the other not, but why?
and there is used the same feature inside the script for both ...

> 2.3.16 was released in 2009.
its the release that comes with CentOS 6

Thanks,
Walter



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: strange sieve problem

[ellie timoney]

> Received: from storage.mail ([unix socket])
>  by storage.mail (Cyrus v2.3.16-Fedora-RPM-2.3.16-13.el6_6) with
>  LMTPA;
>  Mon, 03 Apr 2017 21:27:35 +0200
> X-Sieve: CMU Sieve 2.3

Wild guess, is your script using sieve features that are not available
in 2.3.16?  2.3.16 was released in 2009.

On Tue, Apr 4, 2017, at 04:00 PM, Walter H. via Info-cyrus wrote:
> Hello,
> 
> I've found a Sieve Tester, where everything works as I expect
> 
> https://www.fastmail.com/cgi-bin/sievetest.pl
> 
> but Cyrus Sieve doesn't
> 
> here the Sieve-Script
> 
> 
> # Sieve filter
> 
> require ["fileinto", "relational"];
> 
> if not exists ["from"]
> {
> discard;
> }
> elsif allof (address :all :is "from" "sq...@proxy.my.local",
> address :all :is "to" "walter@my.local")
> {
> if header :matches "subject" "[proxy] Video-URL (*) detected"
> {
> fileinto "INBOX._Info.hbbtvVideoURLs";
> }
> elsif header :matches "subject" "[proxy] File-URL (*) detected"
> {
> fileinto "INBOX._Info.ftpFileURLs";
> }
> else
> {
> fileinto "INBOX._Info";
> }
> }
> elsif allof (address :all :is "from" "cla...@mail.my.local",
> address :all :is "to" "walter@my.local")
> {
> if header :matches "subject" "[mail] Virus detected in E-mail"
> {
> fileinto "INBOX._Alert";
> }
> }
> elsif header :matches "list-id" "*"
> {
> fileinto "INBOX._MailLists._CENTOS";
> }
> elsif header :is "precedence" "bulk"
> {
> fileinto "INBOX.Trash";
> }
> else
> {
> keep;
> }
> 
> 
> and this is the Mail
> 
> 
> Return-Path: 
> Received: from storage.mail ([unix socket])
>  by storage.mail (Cyrus v2.3.16-Fedora-RPM-2.3.16-13.el6_6) with
>  LMTPA;
>  Mon, 03 Apr 2017 21:27:35 +0200
> X-Sieve: CMU Sieve 2.3
> Received: from proxy.host by storage.mail (Postfix) with ESMTP id
> 19B2C79235
> Received: by proxy.host (Postfix, userid 23) id EB81D2B0BE
> Date: Mon, 03 Apr 2017 21:27:34 +0200
> To: walter@my.local
> Subject: [proxy] File-URL (PC) detected
> User-Agent: Heirloom mailx 12.4 7/29/08
> MIME-Version: 1.0
> Content-Type: text/plain; charset=us-ascii
> Content-Transfer-Encoding: 7bit
> Message-Id: <20170403192734.eb81d2b...@proxy.my.local>
> From: sq...@proxy.my.local (Squid)
> 
> The following information came from the Squid proxy virtual machine.
> 
> --[ Data submitted
> ]---
> 
> File-URL: ftp://ftp.adobe.com/lbtest.txt
> 
> 
> 
> this Mail is sorted correct by the sieve script
> 
> 
> Return-Path: 
> Received: from storage.mail ([unix socket])
> by storage.mail (Cyrus v2.3.16-Fedora-RPM-2.3.16-13.el6_6) with LMTPA;
> Sun, 05 Feb 2017 19:14:15 +0100
> X-Sieve: CMU Sieve 2.3
> Received: from filter.mail by storage.mail (Postfix) with ESMTP id
> 5634078BA8
> Received: by filter.mail (Postfix) id 48F198E9
> Delivered-To: r...@filter.mail
> Received: from filter.mail [local] by filter.mail (Postfix) with ESMTP id
> 35E838E8
> Received: by filter.mail (Postfix, userid 496) id 2A20D8E9
> From: ClamAV 
> To: walter@waldinet.local
> Subject: [mail] Virus detected in E-mail
> Message-Id: <20170205181415.2a20d...@mail.my.local>
> Date: Sun, 5 Feb 2017 19:14:15 +0100 (CET)
> X-AV-Scanned: ClamAV using ClamSMTP (filter.mail)
> 
> The following information came from the Mail filter virtual machine.
> 
> --[ Data submitted
> ]---
> 
> Virus name: Heuristics.Phishing.Email.SpoofedDomain
> Sender:
> rte+ne-null-b1cb1a01203481e6zubgcse...@sellernotifications.amazon.com
> 
> Quarantined to: /var/lib/clamd.clamsmtp/virus.XeKpYL
> 
> --[ E-Mail header
> ]
> 
> ...
> 
> 
> 
> can someone give me a hint, what is wrong,
> 
> Thanks,
> Walter
> 
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

strange sieve problem

[Walter H. via Info-cyrus]

Hello,

I've found a Sieve Tester, where everything works as I expect

https://www.fastmail.com/cgi-bin/sievetest.pl

but Cyrus Sieve doesn't

here the Sieve-Script


# Sieve filter

require ["fileinto", "relational"];

if not exists ["from"]
{
discard;
}
elsif allof (address :all :is "from" "sq...@proxy.my.local",
address :all :is "to" "walter@my.local")
{
if header :matches "subject" "[proxy] Video-URL (*) detected"
{
fileinto "INBOX._Info.hbbtvVideoURLs";
}
elsif header :matches "subject" "[proxy] File-URL (*) detected"
{
fileinto "INBOX._Info.ftpFileURLs";
}
else
{
fileinto "INBOX._Info";
}
}
elsif allof (address :all :is "from" "cla...@mail.my.local",
address :all :is "to" "walter@my.local")
{
if header :matches "subject" "[mail] Virus detected in E-mail"
{
fileinto "INBOX._Alert";
}
}
elsif header :matches "list-id" "*"
{
fileinto "INBOX._MailLists._CENTOS";
}
elsif header :is "precedence" "bulk"
{
fileinto "INBOX.Trash";
}
else
{
keep;
}


and this is the Mail


Return-Path: 
Received: from storage.mail ([unix socket])
 by storage.mail (Cyrus v2.3.16-Fedora-RPM-2.3.16-13.el6_6) with LMTPA;
 Mon, 03 Apr 2017 21:27:35 +0200
X-Sieve: CMU Sieve 2.3
Received: from proxy.host by storage.mail (Postfix) with ESMTP id 19B2C79235
Received: by proxy.host (Postfix, userid 23) id EB81D2B0BE
Date: Mon, 03 Apr 2017 21:27:34 +0200
To: walter@my.local
Subject: [proxy] File-URL (PC) detected
User-Agent: Heirloom mailx 12.4 7/29/08
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-Id: <20170403192734.eb81d2b...@proxy.my.local>
From: sq...@proxy.my.local (Squid)

The following information came from the Squid proxy virtual machine.

--[ Data submitted ]---

File-URL: ftp://ftp.adobe.com/lbtest.txt



this Mail is sorted correct by the sieve script


Return-Path: 
Received: from storage.mail ([unix socket])
by storage.mail (Cyrus v2.3.16-Fedora-RPM-2.3.16-13.el6_6) with LMTPA;
Sun, 05 Feb 2017 19:14:15 +0100
X-Sieve: CMU Sieve 2.3
Received: from filter.mail by storage.mail (Postfix) with ESMTP id 5634078BA8
Received: by filter.mail (Postfix) id 48F198E9
Delivered-To: r...@filter.mail
Received: from filter.mail [local] by filter.mail (Postfix) with ESMTP id
35E838E8
Received: by filter.mail (Postfix, userid 496) id 2A20D8E9
From: ClamAV 
To: walter@waldinet.local
Subject: [mail] Virus detected in E-mail
Message-Id: <20170205181415.2a20d...@mail.my.local>
Date: Sun, 5 Feb 2017 19:14:15 +0100 (CET)
X-AV-Scanned: ClamAV using ClamSMTP (filter.mail)

The following information came from the Mail filter virtual machine.

--[ Data submitted ]---

Virus name: Heuristics.Phishing.Email.SpoofedDomain
Sender: rte+ne-null-b1cb1a01203481e6zubgcse...@sellernotifications.amazon.com

Quarantined to: /var/lib/clamd.clamsmtp/virus.XeKpYL

--[ E-Mail header ]

...



can someone give me a hint, what is wrong,

Thanks,
Walter


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: mailboxes.db locking problem after updating from 2.4 to 2.5.9

[Wolfgang Breyha via Info-cyrus]

On 18/11/16 01:07, Bron Gondwana via Info-cyrus wrote:
> On Fri, 18 Nov 2016, at 10:51, Wolfgang Breyha via Info-cyrus wrote:
>> I already filed a bug https://github.com/cyrusimap/cyrus-imapd/issues/43 
>> but no response so far. I directly asked Bron, but no response as well.
> 
> Sorry, I really don't have a clue.  2.5 does have a different mailboxes.db
> format, so it's a bit more CPU intensive.  The real massive win for CPU
> usage is going to come with reverse ACLs:

Thanks for the response in the first place! I'm sorry to push this topic
continuously because I appreciate all your hard work (and Ellies as well) on
cyrus very much!

We had a quite hard time keeping our infrastructure operational after doing
the final step upgrading our frontends to 2.5. Seeing others having the same
stress should be quite a warning for people thinking about an upgrade.

Since Deniss has these troubles with skiplist as well on 2.5 it looks like the
size
triggers it. twoskip seems to perform even worse if mailboxes.db reaches a
certain size,
or it's only the size penalty which hits twoskip earlier.
150-200 MB seems critical for locking issues.

> https://blog.fastmail.com/2015/12/05/reverse-acls-making-imap-list-fast/

I read about them some time ago in the 3.0.0-beta changelog. Hopefully we're
on a stable cyrus release supporting it before reaching the critical size for
skiplist;-)

> But to get there, we need to solve reverse ACLs for groups.  I did ask about 
> it here:
> 
> https://lists.andrew.cmu.edu/pipermail/info-cyrus/2015-November/038628.html

Sorry, we don't use groups.

> But then didn't follow up to add group reverse ACL support in Cyrus, so 
> reverse ACLs are broken if you're using groups.

So, 2.5 wont get them anyway I guess;-)

Greetings, Wolfgang
-- 
Wolfgang Breyha  | http://www.blafasel.at/
Vienna University Computer Center | Austria


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: mailboxes.db locking problem after updating from 2.4 to 2.5.9

[Bron Gondwana via Info-cyrus]

On Fri, 18 Nov 2016, at 10:51, Wolfgang Breyha via Info-cyrus wrote:
> On 17/11/16 14:00, Deniss via Info-cyrus wrote:
> > Any ideas or suggestion for investigation ?
> 
> I already filed a bug
> https://github.com/cyrusimap/cyrus-imapd/issues/43
> but no response so far. I directly asked Bron, but no response as well.

Sorry, I really don't have a clue.  2.5 does have a different mailboxes.db 
format, so it's a bit more CPU intensive.  The real massive win for CPU usage 
is going to come with reverse ACLs:

https://blog.fastmail.com/2015/12/05/reverse-acls-making-imap-list-fast/

But to get there, we need to solve reverse ACLs for groups.  I did ask about it 
here:

https://lists.andrew.cmu.edu/pipermail/info-cyrus/2015-November/038628.html

But then didn't follow up to add group reverse ACL support in Cyrus, so reverse 
ACLs are broken if you're using groups.

Bron.

-- 
  Bron Gondwana
  br...@fastmail.fm

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: mailboxes.db locking problem after updating from 2.4 to 2.5.9

[Wolfgang Breyha via Info-cyrus]

On 17/11/16 14:00, Deniss via Info-cyrus wrote:
> Hello,
> 
> I trying to migrate one big cyrus imap server from 2.4 to 2.5.9.
> 
> I updated binaries, fix db backend in imapd.conf and converted
> mailboxes.db with ctl_mboxlist -d & -u to twoskip.
> 
> cyrus ran fine until morning when a count of simultanious sessions
> started to rise.

Sounds familiar;-)

> I converted mailboxes.db back to skiplist and got a bunch (10-20 per
> sec) of following logs:

I did the same. twoskip was and is useless on my frontends. With skiplist my
loads dropped at least down to 2-5. After doubling our vCPUs it runs quite well.

Do you use roundcube by any chance? If yes update to 2.5.10 and do not use
$config['imap_disabled_caps'] = array('LIST-EXTENDED');

Otherwise roundcube sends two 'LIST "" "*"' on each login and triggers two
full table scans on your mailboxes.db

> 180M /var/imap/mailboxes.db.skiplist.24
> 215M /var/imap/mailboxes.db.skiplist.25
> 262M /var/imap/mailboxes.db.twoskip.25

Wow, larger then ours. We've ~140MB on skiplist.25.

> Any ideas or suggestion for investigation ?

I already filed a bug
https://github.com/cyrusimap/cyrus-imapd/issues/43
but no response so far. I directly asked Bron, but no response as well.

Greetings, Wolfgang
-- 
Wolfgang Breyha  | http://www.blafasel.at/
Vienna University Computer Center | Austria


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: mailboxes.db locking problem after updating from 2.4 to 2.5.9

[Shawn Bakhtiar via Info-cyrus]

Did you run reconstruct like the upgrade documentation recommends?

When I did my upgrade, from 2.4.x to 2.5 that's about all I had to do. I don't 
recall having to run ctl_mboxlist -du

https://www.cyrusimap.org/docs/cyrus-imapd/2.5.0/install-upgrade.php

http://www.cyrusimap.org/~vanmeeuwen/imap/release-notes/2.5.0.html


On Nov 17, 2016, at 5:00 AM, Deniss via Info-cyrus 
> wrote:

Hello,

I trying to migrate one big cyrus imap server from 2.4 to 2.5.9.

I updated binaries, fix db backend in imapd.conf and converted
mailboxes.db with ctl_mboxlist -d & -u to twoskip.

cyrus ran fine until morning when a count of simultanious sessions
started to rise.

then imapd processes become locked on mailboxes.db when logs show
something like
Nov 17 08:04:21 srv1 imap[11019]: cmdtimer: 
'y...@mail.xxx' 'list'
'' '46.741322' '0.00' '46.741322'
Nov 17 08:04:21 srv1 imap[11019]: buf: A1 LIST "" "*"

I converted mailboxes.db back to skiplist and got a bunch (10-20 per
sec) of following logs:

Nov 17 09:59:09 srv1 imap[5352]: skiplist: longlock
/var/imap/mailboxes.db for 177.7 seconds
Nov 17 09:59:09 srv1 imap[6760]: skiplist: longlock
/var/imap/mailboxes.db for 169.7 seconds
Nov 17 09:59:09 srv1 imap[6764]: skiplist: longlock
/var/imap/mailboxes.db for 171.7 seconds
Nov 17 09:59:09 srv1 imap[5230]: skiplist: longlock
/var/imap/mailboxes.db for 171.8 seconds
Nov 17 09:59:09 srv1 imap[5251]: skiplist: longlock
/var/imap/mailboxes.db for 203.2 seconds
Nov 17 09:59:09 srv1 imap[5221]: skiplist: longlock
/var/imap/mailboxes.db for 168.9 seconds


then I rolled back to cyrus 2.4 (converting mailboxes.db back to old
format) and the server started to work just fine again.

I run 8core 16gb box with mailboxes.db on SSD disk, noswap,
Linux 4.4.8-hardened glibc-2.20 x86_64-pc-linux-gnu-4.8.4 on Gentoo

mailboxes.db is ~200Mb depending of format
135M /var/imap/mailboxes.plain
180M /var/imap/mailboxes.db.skiplist.24
215M /var/imap/mailboxes.db.skiplist.25
262M /var/imap/mailboxes.db.twoskip.25

Any ideas or suggestion for investigation ?

Best,
Deniss

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

mailboxes.db locking problem after updating from 2.4 to 2.5.9

[Deniss via Info-cyrus]

Hello,

I trying to migrate one big cyrus imap server from 2.4 to 2.5.9.

I updated binaries, fix db backend in imapd.conf and converted
mailboxes.db with ctl_mboxlist -d & -u to twoskip.

cyrus ran fine until morning when a count of simultanious sessions
started to rise.

then imapd processes become locked on mailboxes.db when logs show
something like
Nov 17 08:04:21 srv1 imap[11019]: cmdtimer: 'y...@mail.xxx' 'list'
'' '46.741322' '0.00' '46.741322'
Nov 17 08:04:21 srv1 imap[11019]: buf: A1 LIST "" "*"

I converted mailboxes.db back to skiplist and got a bunch (10-20 per
sec) of following logs:

Nov 17 09:59:09 srv1 imap[5352]: skiplist: longlock
/var/imap/mailboxes.db for 177.7 seconds
Nov 17 09:59:09 srv1 imap[6760]: skiplist: longlock
/var/imap/mailboxes.db for 169.7 seconds
Nov 17 09:59:09 srv1 imap[6764]: skiplist: longlock
/var/imap/mailboxes.db for 171.7 seconds
Nov 17 09:59:09 srv1 imap[5230]: skiplist: longlock
/var/imap/mailboxes.db for 171.8 seconds
Nov 17 09:59:09 srv1 imap[5251]: skiplist: longlock
/var/imap/mailboxes.db for 203.2 seconds
Nov 17 09:59:09 srv1 imap[5221]: skiplist: longlock
/var/imap/mailboxes.db for 168.9 seconds


then I rolled back to cyrus 2.4 (converting mailboxes.db back to old
format) and the server started to work just fine again.

I run 8core 16gb box with mailboxes.db on SSD disk, noswap,
Linux 4.4.8-hardened glibc-2.20 x86_64-pc-linux-gnu-4.8.4 on Gentoo

mailboxes.db is ~200Mb depending of format
135M /var/imap/mailboxes.plain
180M /var/imap/mailboxes.db.skiplist.24
215M /var/imap/mailboxes.db.skiplist.25
262M /var/imap/mailboxes.db.twoskip.25

Any ideas or suggestion for investigation ?

Best,
Deniss

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Performance problem

[Kenneth Marshall via Info-cyrus]

On Fri, Aug 05, 2016 at 11:11:33AM -0400, Eric W. Bates via Info-cyrus wrote:
> We are migrating our cyrus from a venerable computer to a new one.
> 
> We've been gradually moving individuals over to the new machine and are
> quite surprised at the poor performance characteristics we're seeing on
> the new machine.
> 
> The new machine:
>  2 @ 6 core Xeon E5-2640
>  256 Gb memory
>  zfs built on 18 SAS drives with SSDs for zfs cache and log
>  Freebsd 10.2-RELEASE-p7
>  Cyrus imapd 2.5.7
> 
> There are approximately 1000 copies of imapd running at any given moment.
> 
> The load average hovers around 5.
> 
> Can anyone suggest what we might look at to assess why imapd seems to be
> such heavy lifting?
> 
> -- 
> Clark 159a, MS 46
> 508/289-3112
> 

Hi Eric,

How many users do you have? It is typical to have one or more imapd
processes per user. Look at the I/O stats for your filesystems to
see if you have a bottleneck somewhere. Also, what type of SSDs are
you using for your ZFS log devices? We found we needed RAM-style
SSDs to maximize our I/O performance under ZFS.

Regards,
Ken

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Performance problem

[Eric W. Bates via Info-cyrus]

We are migrating our cyrus from a venerable computer to a new one.

We've been gradually moving individuals over to the new machine and are
quite surprised at the poor performance characteristics we're seeing on
the new machine.

The new machine:
 2 @ 6 core Xeon E5-2640
 256 Gb memory
 zfs built on 18 SAS drives with SSDs for zfs cache and log
 Freebsd 10.2-RELEASE-p7
 Cyrus imapd 2.5.7

There are approximately 1000 copies of imapd running at any given moment.

The load average hovers around 5.

Can anyone suggest what we might look at to assess why imapd seems to be
such heavy lifting?

-- 
Clark 159a, MS 46
508/289-3112



smime.p7s
Description: S/MIME Cryptographic Signature

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Problem moving between folders during 2.3.16 -> 2.5.9 upgrade

[ellie timoney via Info-cyrus]

On Mon, Jul 25, 2016, at 10:53 PM, Kenneth Marshall wrote:
> On Mon, Jul 25, 2016 at 03:53:43PM +1000, ellie timoney wrote:
> > I don't have a 2.3.x environment handy to try to reproduce this from the
> > outside in, so I'm starting at the error code and working outwards...
> > 
> > > > <1469384361 > > > >1469384362>a0080 NO Mailbox format corruption detected
> > 
> > "Mailbox format corruption detected" corresponds with
> > IMAP_MAILBOX_CHECKSUM
> > 
> > This error code is exclusively returned by functions in imap/mailbox.c,
> > for index/record crc mismatches.  Given that the mailbox is selectable
> > and such, I'm supposing that the whole-file/whole-record crc checks are
> > okay...
> > 
> > It looks like 2.3.16 had MAILBOX_MINOR_VERSION 10, and in particular
> > doesn't have the record->cache_crc field, which many of the
> > IMAP_MAILBOX_CHECKSUM checks are for.  When reading mailboxes < 12 in
> > newer Cyrus versions, this field is initialised to zero.
> > 
> > Some of the places that want to do a crc check against record->cache_crc
> > first make sure record->cache_crc is non-zero before doing so: 
> > cache_append_record(), mailbox_cacherecord().
> > 
> > But some don't, or at least don't appear to: mailbox_append_cache()
> > initially uses cache_append_record() (good), but then reads it straight
> > back in to ensure freshness -- and then having done so, appears to do an
> > unconditional check against record->cache_crc (dodgy?)
> > 
> > Actually that seems like all the spots this check happens -- so the fix
> > might be as simple as the attached patch?
> > 
> > Kenneth, are you able to try this out?
> > 
> > Cheers,
> > 
> > ellie
> > 
> 
> Hi Ellie,
> 
> That fixes the problem. Thank you so much for the quick response.
> 
> Regards,
> Ken

That's great to hear, thanks for trying it out.  This patch will be
included in 2.5.10 when it comes out.

Cheers,

ellie

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Problem moving between folders during 2.3.16 -> 2.5.9 upgrade

[Kenneth Marshall via Info-cyrus]

On Mon, Jul 25, 2016 at 03:53:43PM +1000, ellie timoney wrote:
> I don't have a 2.3.x environment handy to try to reproduce this from the
> outside in, so I'm starting at the error code and working outwards...
> 
> > > <1469384361 > > >1469384362>a0080 NO Mailbox format corruption detected
> 
> "Mailbox format corruption detected" corresponds with
> IMAP_MAILBOX_CHECKSUM
> 
> This error code is exclusively returned by functions in imap/mailbox.c,
> for index/record crc mismatches.  Given that the mailbox is selectable
> and such, I'm supposing that the whole-file/whole-record crc checks are
> okay...
> 
> It looks like 2.3.16 had MAILBOX_MINOR_VERSION 10, and in particular
> doesn't have the record->cache_crc field, which many of the
> IMAP_MAILBOX_CHECKSUM checks are for.  When reading mailboxes < 12 in
> newer Cyrus versions, this field is initialised to zero.
> 
> Some of the places that want to do a crc check against record->cache_crc
> first make sure record->cache_crc is non-zero before doing so: 
> cache_append_record(), mailbox_cacherecord().
> 
> But some don't, or at least don't appear to: mailbox_append_cache()
> initially uses cache_append_record() (good), but then reads it straight
> back in to ensure freshness -- and then having done so, appears to do an
> unconditional check against record->cache_crc (dodgy?)
> 
> Actually that seems like all the spots this check happens -- so the fix
> might be as simple as the attached patch?
> 
> Kenneth, are you able to try this out?
> 
> Cheers,
> 
> ellie
> 

Hi Ellie,

That fixes the problem. Thank you so much for the quick response.

Regards,
Ken

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Problem moving between folders during 2.3.16 -> 2.5.9 upgrade

[Kenneth Marshall via Info-cyrus]

On Mon, Jul 25, 2016 at 03:53:43PM +1000, ellie timoney wrote:
> I don't have a 2.3.x environment handy to try to reproduce this from the
> outside in, so I'm starting at the error code and working outwards...
> 
> > > <1469384361 > > >1469384362>a0080 NO Mailbox format corruption detected
> 
> "Mailbox format corruption detected" corresponds with
> IMAP_MAILBOX_CHECKSUM
> 
> This error code is exclusively returned by functions in imap/mailbox.c,
> for index/record crc mismatches.  Given that the mailbox is selectable
> and such, I'm supposing that the whole-file/whole-record crc checks are
> okay...
> 
> It looks like 2.3.16 had MAILBOX_MINOR_VERSION 10, and in particular
> doesn't have the record->cache_crc field, which many of the
> IMAP_MAILBOX_CHECKSUM checks are for.  When reading mailboxes < 12 in
> newer Cyrus versions, this field is initialised to zero.
> 
> Some of the places that want to do a crc check against record->cache_crc
> first make sure record->cache_crc is non-zero before doing so: 
> cache_append_record(), mailbox_cacherecord().
> 
> But some don't, or at least don't appear to: mailbox_append_cache()
> initially uses cache_append_record() (good), but then reads it straight
> back in to ensure freshness -- and then having done so, appears to do an
> unconditional check against record->cache_crc (dodgy?)
> 
> Actually that seems like all the spots this check happens -- so the fix
> might be as simple as the attached patch?
> 
> Kenneth, are you able to try this out?
> 
> Cheers,
> 
> ellie
> 

Hi Ellie,

I will try it this morning.

Regards,
Ken

> On Mon, Jul 25, 2016, at 02:02 PM, Bron Gondwana wrote:
> > Thanks for reporting this.  Ellie, if you get a chance can you look at
> > it?  I'm in the middle of the the FastMail security rollout right now, so
> > I can't do anything today.
> > 
> > Bron.
> > 
> > On Mon, Jul 25, 2016, at 11:45, Kenneth Marshall wrote:
> > > Hi,
> > > 
> > > I accidentally hijacked another thread. So here is a new message. I am
> > > testing a Cyrus IMAP 2.3.16 to 2.5.9 upgrade and I am having problems
> > > moving messages between folders before the 'reconstruct -V max' has been
> > > run on the mailboxes. Here is the error from the IMAP client:
> > > 
> > > imap_copy_messages [a0080 NO Mailbox format corruption detected]?
> > > 
> > > and the details from the IMAP logs:
> > > 
> > > <1469384341 > > >1469384341>a0077 OK Completed
> > > <1469384354 > > >1469384355>* LIST (\Noinferiors \HasNoChildren) "/" INBOX
> > > * LIST (\HasChildren) "/" DSPAM_notspam
> > > * LIST (\HasChildren) "/" DSPAM_spam
> > > * LIST (\HasNoChildren) "/" Drafts
> > > * LIST (\HasChildren) "/" Mail
> > > * LIST (\HasNoChildren) "/" Sent
> > > * LIST (\HasNoChildren) "/" Spam
> > > * LIST (\HasNoChildren) "/" Trash
> > > a0078 OK Completed (0.240 secs 356 calls)
> > > <1469384361 > > >1469384361>* STATUS Drafts (UIDVALIDITY 1345619846)
> > > a0079 OK Completed
> > > <1469384361 > > >1469384362>a0080 NO Mailbox format corruption detected
> > > <1469384370 > > >1469384370>* LIST (\Noinferiors \HasNoChildren) "/" INBOX
> > > * LIST (\HasChildren) "/" DSPAM_notspam
> > > * LIST (\HasChildren) "/" DSPAM_spam
> > > * LIST (\HasNoChildren) "/" Drafts
> > > * LIST (\HasChildren) "/" Mail
> > > * LIST (\HasNoChildren) "/" Sent
> > > * LIST (\HasNoChildren) "/" Spam
> > > * LIST (\HasNoChildren) "/" Trash
> > > a0081 OK Completed (0.230 secs 356 calls)
> > > <1469384372 > > a0083 MYRIGHTS "Drafts"
> > > a0084 SELECT "Drafts"
> > > >1469384372>a0082 OK Completed
> > > >1469384372>* MYRIGHTS Drafts lrswipkxtecda
> > > a0083 OK Completed
> > > >1469384372>* 0 EXISTS
> > > * 0 RECENT
> > > * FLAGS (\Answered \Flagged \Draft \Deleted \Seen $mdnsent Old)
> > > * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen $mdnsent 
> > > Old \*)] Ok
> > > * OK [UIDVALIDITY 1345619846] Ok
> > > * OK [UIDNEXT 6] Ok
> > > * OK [HIGHESTMODSEQ 1] Ok
> > > * OK [URLMECH INTERNAL] Ok
> > > * OK [ANNOTATIONS 65536] Ok
> > > a0084 OK [READ-WRITE] Completed
> > > >1469384355>* LIST (\Noinferiors \HasNoChildren) "/" INBOX
> > > * LIST (\HasChildren) "/" DSPAM_notspam
> > > * LIST (\HasChildren) "/" DSPAM_spam
> > > * LIST (\HasNoChildren) "/" Drafts
> > > * LIST (\HasChildren) "/" Mail
> > > * LIST (\HasNoChildren) "/" Sent
> > > * LIST (\HasNoChildren) "/" Spam
> > > * LIST (\HasNoChildren) "/" Trash
> > > a0078 OK Completed (0.240 secs 356 calls)
> > > <1469384361 > > >1469384361>* STATUS Drafts (UIDVALIDITY 1345619846)
> > > a0079 OK Completed
> > > <1469384361 > > >1469384362>a0080 NO Mailbox format corruption detected
> > > <1469384370 > > >1469384370>* LIST (\Noinferiors \HasNoChildren) "/" INBOX
> > > * LIST (\HasChildren) "/" DSPAM_notspam
> > > * LIST (\HasChildren) "/" DSPAM_spam
> > > * LIST (\HasNoChildren) "/" Drafts
> > > * LIST (\HasChildren) "/" Mail
> > > * LIST (\HasNoChildren) "/" Sent
> > > * LIST (\HasNoChildren) "/" Spam
> > > * LIST (\HasNoChildren) "/" Trash
> > > a0081 OK Completed 

Re: Problem moving between folders during 2.3.16 -> 2.5.9 upgrade

[ellie timoney via Info-cyrus]

I don't have a 2.3.x environment handy to try to reproduce this from the
outside in, so I'm starting at the error code and working outwards...

> > <1469384361 > >1469384362>a0080 NO Mailbox format corruption detected

"Mailbox format corruption detected" corresponds with
IMAP_MAILBOX_CHECKSUM

This error code is exclusively returned by functions in imap/mailbox.c,
for index/record crc mismatches.  Given that the mailbox is selectable
and such, I'm supposing that the whole-file/whole-record crc checks are
okay...

It looks like 2.3.16 had MAILBOX_MINOR_VERSION 10, and in particular
doesn't have the record->cache_crc field, which many of the
IMAP_MAILBOX_CHECKSUM checks are for.  When reading mailboxes < 12 in
newer Cyrus versions, this field is initialised to zero.

Some of the places that want to do a crc check against record->cache_crc
first make sure record->cache_crc is non-zero before doing so: 
cache_append_record(), mailbox_cacherecord().

But some don't, or at least don't appear to: mailbox_append_cache()
initially uses cache_append_record() (good), but then reads it straight
back in to ensure freshness -- and then having done so, appears to do an
unconditional check against record->cache_crc (dodgy?)

Actually that seems like all the spots this check happens -- so the fix
might be as simple as the attached patch?

Kenneth, are you able to try this out?

Cheers,

ellie

On Mon, Jul 25, 2016, at 02:02 PM, Bron Gondwana wrote:
> Thanks for reporting this.  Ellie, if you get a chance can you look at
> it?  I'm in the middle of the the FastMail security rollout right now, so
> I can't do anything today.
> 
> Bron.
> 
> On Mon, Jul 25, 2016, at 11:45, Kenneth Marshall wrote:
> > Hi,
> > 
> > I accidentally hijacked another thread. So here is a new message. I am
> > testing a Cyrus IMAP 2.3.16 to 2.5.9 upgrade and I am having problems
> > moving messages between folders before the 'reconstruct -V max' has been
> > run on the mailboxes. Here is the error from the IMAP client:
> > 
> > imap_copy_messages [a0080 NO Mailbox format corruption detected]?
> > 
> > and the details from the IMAP logs:
> > 
> > <1469384341 > >1469384341>a0077 OK Completed
> > <1469384354 > >1469384355>* LIST (\Noinferiors \HasNoChildren) "/" INBOX
> > * LIST (\HasChildren) "/" DSPAM_notspam
> > * LIST (\HasChildren) "/" DSPAM_spam
> > * LIST (\HasNoChildren) "/" Drafts
> > * LIST (\HasChildren) "/" Mail
> > * LIST (\HasNoChildren) "/" Sent
> > * LIST (\HasNoChildren) "/" Spam
> > * LIST (\HasNoChildren) "/" Trash
> > a0078 OK Completed (0.240 secs 356 calls)
> > <1469384361 > >1469384361>* STATUS Drafts (UIDVALIDITY 1345619846)
> > a0079 OK Completed
> > <1469384361 > >1469384362>a0080 NO Mailbox format corruption detected
> > <1469384370 > >1469384370>* LIST (\Noinferiors \HasNoChildren) "/" INBOX
> > * LIST (\HasChildren) "/" DSPAM_notspam
> > * LIST (\HasChildren) "/" DSPAM_spam
> > * LIST (\HasNoChildren) "/" Drafts
> > * LIST (\HasChildren) "/" Mail
> > * LIST (\HasNoChildren) "/" Sent
> > * LIST (\HasNoChildren) "/" Spam
> > * LIST (\HasNoChildren) "/" Trash
> > a0081 OK Completed (0.230 secs 356 calls)
> > <1469384372 > a0083 MYRIGHTS "Drafts"
> > a0084 SELECT "Drafts"
> > >1469384372>a0082 OK Completed
> > >1469384372>* MYRIGHTS Drafts lrswipkxtecda
> > a0083 OK Completed
> > >1469384372>* 0 EXISTS
> > * 0 RECENT
> > * FLAGS (\Answered \Flagged \Draft \Deleted \Seen $mdnsent Old)
> > * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen $mdnsent Old 
> > \*)] Ok
> > * OK [UIDVALIDITY 1345619846] Ok
> > * OK [UIDNEXT 6] Ok
> > * OK [HIGHESTMODSEQ 1] Ok
> > * OK [URLMECH INTERNAL] Ok
> > * OK [ANNOTATIONS 65536] Ok
> > a0084 OK [READ-WRITE] Completed
> > >1469384355>* LIST (\Noinferiors \HasNoChildren) "/" INBOX
> > * LIST (\HasChildren) "/" DSPAM_notspam
> > * LIST (\HasChildren) "/" DSPAM_spam
> > * LIST (\HasNoChildren) "/" Drafts
> > * LIST (\HasChildren) "/" Mail
> > * LIST (\HasNoChildren) "/" Sent
> > * LIST (\HasNoChildren) "/" Spam
> > * LIST (\HasNoChildren) "/" Trash
> > a0078 OK Completed (0.240 secs 356 calls)
> > <1469384361 > >1469384361>* STATUS Drafts (UIDVALIDITY 1345619846)
> > a0079 OK Completed
> > <1469384361 > >1469384362>a0080 NO Mailbox format corruption detected
> > <1469384370 > >1469384370>* LIST (\Noinferiors \HasNoChildren) "/" INBOX
> > * LIST (\HasChildren) "/" DSPAM_notspam
> > * LIST (\HasChildren) "/" DSPAM_spam
> > * LIST (\HasNoChildren) "/" Drafts
> > * LIST (\HasChildren) "/" Mail
> > * LIST (\HasNoChildren) "/" Sent
> > * LIST (\HasNoChildren) "/" Spam
> > * LIST (\HasNoChildren) "/" Trash
> > a0081 OK Completed (0.230 secs 356 calls)
> > <1469384372 > a0083 MYRIGHTS "Drafts"
> > a0084 SELECT "Drafts"
> > >1469384372>a0082 OK Completed
> > >1469384372>* MYRIGHTS Drafts lrswipkxtecda
> > a0083 OK Completed
> > >1469384372>* 0 EXISTS
> > * 0 RECENT
> > * FLAGS (\Answered \Flagged \Draft \Deleted \Seen $mdnsent Old)
> > * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted 

Re: Problem moving between folders during 2.3.16 -> 2.5.9 upgrade

[Bron Gondwana via Info-cyrus]

Thanks for reporting this.  Ellie, if you get a chance can you look at it?  I'm 
in the middle of the the FastMail security rollout right now, so I can't do 
anything today.

Bron.

On Mon, Jul 25, 2016, at 11:45, Kenneth Marshall wrote:
> Hi,
> 
> I accidentally hijacked another thread. So here is a new message. I am
> testing a Cyrus IMAP 2.3.16 to 2.5.9 upgrade and I am having problems
> moving messages between folders before the 'reconstruct -V max' has been
> run on the mailboxes. Here is the error from the IMAP client:
> 
> imap_copy_messages [a0080 NO Mailbox format corruption detected]?
> 
> and the details from the IMAP logs:
> 
> <1469384341 >1469384341>a0077 OK Completed
> <1469384354 >1469384355>* LIST (\Noinferiors \HasNoChildren) "/" INBOX
> * LIST (\HasChildren) "/" DSPAM_notspam
> * LIST (\HasChildren) "/" DSPAM_spam
> * LIST (\HasNoChildren) "/" Drafts
> * LIST (\HasChildren) "/" Mail
> * LIST (\HasNoChildren) "/" Sent
> * LIST (\HasNoChildren) "/" Spam
> * LIST (\HasNoChildren) "/" Trash
> a0078 OK Completed (0.240 secs 356 calls)
> <1469384361 >1469384361>* STATUS Drafts (UIDVALIDITY 1345619846)
> a0079 OK Completed
> <1469384361 >1469384362>a0080 NO Mailbox format corruption detected
> <1469384370 >1469384370>* LIST (\Noinferiors \HasNoChildren) "/" INBOX
> * LIST (\HasChildren) "/" DSPAM_notspam
> * LIST (\HasChildren) "/" DSPAM_spam
> * LIST (\HasNoChildren) "/" Drafts
> * LIST (\HasChildren) "/" Mail
> * LIST (\HasNoChildren) "/" Sent
> * LIST (\HasNoChildren) "/" Spam
> * LIST (\HasNoChildren) "/" Trash
> a0081 OK Completed (0.230 secs 356 calls)
> <1469384372 a0083 MYRIGHTS "Drafts"
> a0084 SELECT "Drafts"
> >1469384372>a0082 OK Completed
> >1469384372>* MYRIGHTS Drafts lrswipkxtecda
> a0083 OK Completed
> >1469384372>* 0 EXISTS
> * 0 RECENT
> * FLAGS (\Answered \Flagged \Draft \Deleted \Seen $mdnsent Old)
> * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen $mdnsent Old 
> \*)] Ok
> * OK [UIDVALIDITY 1345619846] Ok
> * OK [UIDNEXT 6] Ok
> * OK [HIGHESTMODSEQ 1] Ok
> * OK [URLMECH INTERNAL] Ok
> * OK [ANNOTATIONS 65536] Ok
> a0084 OK [READ-WRITE] Completed
> >1469384355>* LIST (\Noinferiors \HasNoChildren) "/" INBOX
> * LIST (\HasChildren) "/" DSPAM_notspam
> * LIST (\HasChildren) "/" DSPAM_spam
> * LIST (\HasNoChildren) "/" Drafts
> * LIST (\HasChildren) "/" Mail
> * LIST (\HasNoChildren) "/" Sent
> * LIST (\HasNoChildren) "/" Spam
> * LIST (\HasNoChildren) "/" Trash
> a0078 OK Completed (0.240 secs 356 calls)
> <1469384361 >1469384361>* STATUS Drafts (UIDVALIDITY 1345619846)
> a0079 OK Completed
> <1469384361 >1469384362>a0080 NO Mailbox format corruption detected
> <1469384370 >1469384370>* LIST (\Noinferiors \HasNoChildren) "/" INBOX
> * LIST (\HasChildren) "/" DSPAM_notspam
> * LIST (\HasChildren) "/" DSPAM_spam
> * LIST (\HasNoChildren) "/" Drafts
> * LIST (\HasChildren) "/" Mail
> * LIST (\HasNoChildren) "/" Sent
> * LIST (\HasNoChildren) "/" Spam
> * LIST (\HasNoChildren) "/" Trash
> a0081 OK Completed (0.230 secs 356 calls)
> <1469384372 a0083 MYRIGHTS "Drafts"
> a0084 SELECT "Drafts"
> >1469384372>a0082 OK Completed
> >1469384372>* MYRIGHTS Drafts lrswipkxtecda
> a0083 OK Completed
> >1469384372>* 0 EXISTS
> * 0 RECENT
> * FLAGS (\Answered \Flagged \Draft \Deleted \Seen $mdnsent Old)
> * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen $mdnsent Old 
> \*)] Ok
> * OK [UIDVALIDITY 1345619846] Ok
> * OK [UIDNEXT 6] Ok
> * OK [HIGHESTMODSEQ 1] Ok
> * OK [URLMECH INTERNAL] Ok
> * OK [ANNOTATIONS 65536] Ok
> a0084 OK [READ-WRITE] Completed
> <1469384392 >1469384392>a0085 OK Completed
> <1469384412 >1469384412>a0086 OK Completed
> 
> Is this a known issue with the upgrade process? Messages can be delivered to
> the folders via a sieve script, but both message moves and copies fail. If 
> there
> is any fix for this issue, it would be greatly appreciated. I have been 
> waiting
> since the release of 2.4.x for a version that would handle the I/O storm 
> caused
> by automatically upgrading the mailbox format on first access, and 2.5.x does
> manage that successfully. Thank you for any assistance.
> 
> Regards,
> Ken
> 
> On Fri, Jul 22, 2016 at 02:27:01PM -0500, Kenneth Marshall wrote:
> > On Sat, Jul 16, 2016 at 12:04:26AM +1000, Bron Gondwana via Info-cyrus 
> > wrote:
> > > On Fri, Jul 15, 2016, at 23:52, Bron Gondwana via Info-cyrus wrote:
> > > > On Fri, Jul 15, 2016, at 23:46, Andy Dorman via Info-cyrus wrote:
> > > > > So if the issue apparently lies with twoskip, can we keep our dbs 
> > > > > using 
> > > > > skiplist and do the 2.4 -> 2.5 upgrade?  Is it possible -h could 
> > > > > revert 
> > > > > back to skiplist?
> > > > 
> > > > You can convert a db back just by changing the setting in imapd.conf 
> > > > and restarting - it will convert itself.
> > > > 
> > > > mboxlist_db: skiplist
> > > > 
> > > > > If it could help we could test upgrading to 2.5.8 on our dev server 
> > > > > while leaving our database(s) as skiplist.
> > > > 
> > > > 

Problem moving between folders during 2.3.16 -> 2.5.9 upgrade

[Kenneth Marshall via Info-cyrus]

Hi,

I accidentally hijacked another thread. So here is a new message. I am
testing a Cyrus IMAP 2.3.16 to 2.5.9 upgrade and I am having problems
moving messages between folders before the 'reconstruct -V max' has been
run on the mailboxes. Here is the error from the IMAP client:

imap_copy_messages [a0080 NO Mailbox format corruption detected]?

and the details from the IMAP logs:

<14693843411469384341>a0077 OK Completed
<14693843541469384355>* LIST (\Noinferiors \HasNoChildren) "/" INBOX
* LIST (\HasChildren) "/" DSPAM_notspam
* LIST (\HasChildren) "/" DSPAM_spam
* LIST (\HasNoChildren) "/" Drafts
* LIST (\HasChildren) "/" Mail
* LIST (\HasNoChildren) "/" Sent
* LIST (\HasNoChildren) "/" Spam
* LIST (\HasNoChildren) "/" Trash
a0078 OK Completed (0.240 secs 356 calls)
<14693843611469384361>* STATUS Drafts (UIDVALIDITY 1345619846)
a0079 OK Completed
<14693843611469384362>a0080 NO Mailbox format corruption detected
<14693843701469384370>* LIST (\Noinferiors \HasNoChildren) "/" INBOX
* LIST (\HasChildren) "/" DSPAM_notspam
* LIST (\HasChildren) "/" DSPAM_spam
* LIST (\HasNoChildren) "/" Drafts
* LIST (\HasChildren) "/" Mail
* LIST (\HasNoChildren) "/" Sent
* LIST (\HasNoChildren) "/" Spam
* LIST (\HasNoChildren) "/" Trash
a0081 OK Completed (0.230 secs 356 calls)
<14693843721469384372>a0082 OK Completed
>1469384372>* MYRIGHTS Drafts lrswipkxtecda
a0083 OK Completed
>1469384372>* 0 EXISTS
* 0 RECENT
* FLAGS (\Answered \Flagged \Draft \Deleted \Seen $mdnsent Old)
* OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen $mdnsent Old 
\*)] Ok
* OK [UIDVALIDITY 1345619846] Ok
* OK [UIDNEXT 6] Ok
* OK [HIGHESTMODSEQ 1] Ok
* OK [URLMECH INTERNAL] Ok
* OK [ANNOTATIONS 65536] Ok
a0084 OK [READ-WRITE] Completed
>1469384355>* LIST (\Noinferiors \HasNoChildren) "/" INBOX
* LIST (\HasChildren) "/" DSPAM_notspam
* LIST (\HasChildren) "/" DSPAM_spam
* LIST (\HasNoChildren) "/" Drafts
* LIST (\HasChildren) "/" Mail
* LIST (\HasNoChildren) "/" Sent
* LIST (\HasNoChildren) "/" Spam
* LIST (\HasNoChildren) "/" Trash
a0078 OK Completed (0.240 secs 356 calls)
<14693843611469384361>* STATUS Drafts (UIDVALIDITY 1345619846)
a0079 OK Completed
<14693843611469384362>a0080 NO Mailbox format corruption detected
<14693843701469384370>* LIST (\Noinferiors \HasNoChildren) "/" INBOX
* LIST (\HasChildren) "/" DSPAM_notspam
* LIST (\HasChildren) "/" DSPAM_spam
* LIST (\HasNoChildren) "/" Drafts
* LIST (\HasChildren) "/" Mail
* LIST (\HasNoChildren) "/" Sent
* LIST (\HasNoChildren) "/" Spam
* LIST (\HasNoChildren) "/" Trash
a0081 OK Completed (0.230 secs 356 calls)
<14693843721469384372>a0082 OK Completed
>1469384372>* MYRIGHTS Drafts lrswipkxtecda
a0083 OK Completed
>1469384372>* 0 EXISTS
* 0 RECENT
* FLAGS (\Answered \Flagged \Draft \Deleted \Seen $mdnsent Old)
* OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen $mdnsent Old 
\*)] Ok
* OK [UIDVALIDITY 1345619846] Ok
* OK [UIDNEXT 6] Ok
* OK [HIGHESTMODSEQ 1] Ok
* OK [URLMECH INTERNAL] Ok
* OK [ANNOTATIONS 65536] Ok
a0084 OK [READ-WRITE] Completed
<14693843921469384392>a0085 OK Completed
<14693844121469384412>a0086 OK Completed

Is this a known issue with the upgrade process? Messages can be delivered to
the folders via a sieve script, but both message moves and copies fail. If there
is any fix for this issue, it would be greatly appreciated. I have been waiting
since the release of 2.4.x for a version that would handle the I/O storm caused
by automatically upgrading the mailbox format on first access, and 2.5.x does
manage that successfully. Thank you for any assistance.

Regards,
Ken

On Fri, Jul 22, 2016 at 02:27:01PM -0500, Kenneth Marshall wrote:
> On Sat, Jul 16, 2016 at 12:04:26AM +1000, Bron Gondwana via Info-cyrus wrote:
> > On Fri, Jul 15, 2016, at 23:52, Bron Gondwana via Info-cyrus wrote:
> > > On Fri, Jul 15, 2016, at 23:46, Andy Dorman via Info-cyrus wrote:
> > > > So if the issue apparently lies with twoskip, can we keep our dbs using 
> > > > skiplist and do the 2.4 -> 2.5 upgrade?  Is it possible -h could revert 
> > > > back to skiplist?
> > > 
> > > You can convert a db back just by changing the setting in imapd.conf and 
> > > restarting - it will convert itself.
> > > 
> > > mboxlist_db: skiplist
> > > 
> > > > If it could help we could test upgrading to 2.5.8 on our dev server 
> > > > while leaving our database(s) as skiplist.
> > > 
> > > Twoskip should be fixed in 2.5.9.  I've worked out what went wrong, and 
> > > am working on patches now :)
> > 
> > Patches have passed all the tests - they're on master and 2.5.  I'm pushing 
> > to FastMail's testing stores and then going to bed :)
> > 
> > -- 
> >   Bron Gondwana
> >   br...@fastmail.fm
> > 
> 
> Hi Bron and Ellie,
> 
> Thank you for the new release. I am currently testing my upgrade process and 
> while
> I can access my mailboxes after the upgrade to 2.5.x but before the 
> 'reconstruct -V max',
> I cannot move messages between mailboxes. When I try I get the following 
> error:
> 
> 

Problem setting up cyrus imapd server

[Juergen Wolf via Info-cyrus]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Hi,

 I am trying to setup a cyrus on CentOS 7 systems with a
 cyrus murder backend. Unfortunately I fail quite early while setting
 up the cyrus. 

 The Problem ist that a cyrus murder setup with an cyrus-imapd newer
 than 2.4.17 seems to have either problems with the imap proxy, or I am
 simply doing it completely wrong.  The Version 2.4.17 works with the
 config, version 2.5.8 or 3.0beta does not work.

 To track down the error, I have the cyrus running with a very
 minimalistic configuration. Even this configuration shows the same
 error. After creating a mailbox on the backend, all information is
 propagated to the frontend and the mupdate server. But the information
 in the frontend is garbled and the the proxyd process even crashes
 while accessing the mailbox.

 So new server, one mailbox on one fronted. Ctl_mboxlist shows the
 following:

 at backend:
/usr/lib/cyrus-imapd/ctl_mboxlist -d
user.wolf   0 default wolf  lrswipkxtecdan

 at the mupdate server:
/usr/lib/cyrus-imapd/ctl_mboxlist -d
user.wolf   1 imailbackend00!default wolf   lrswipkxtecdan

 at the frontend:
/usr/lib/cyrus-imapd/ctl_mboxlist -d
user.wolf   1 imailbackend00!defaultH�H��f� wolf lrswipkxtecdan

 The proxyd process at the fronted crashes while executing the info
 user.wolf command via cyradmin as cyrus-admin:

Jun 20 12:04:31 ilserv15 imap[73259]: Fatal error: Internal error:  assertion 
failed: imap/annotate.c: 1615: state->mailbox
Jun 20 12:04:31 ilserv15 master[73250]: process type:SERVICE name:imap 
path:/usr/lib/cyrus-imapd/proxyd age:1239.927s pid:73259 exited, status 75

 The wanted setup is one frontend server, one mupdate server and two
 backend servers with the following configs:

 frontend-config:
#
# Servername
#
servername: ilserv15
#
# Storage Part
#
configdirectory:   /var/lib/imap
partition-default: /var/spool/imap
sievedir:  /var/lib/imap/sieve
#
# Admins
#
admins:   cyrus-admin cyrus-murder
# 
# Executables 
#
sendmail: /usr/sbin/sendmail
#
# User Authentication
#
# General
allowplaintext: yes
# Sasl
sasl_pwcheck_method: auxprop saslauthd
sasl_mech_list:  PLAIN LOGIN
#
# Security / Encryption
tls_client_ca_dir:  /etc/pki/tls/certs/
tls_server_cert:/etc/pki/cyrus-imapd/cyrus-imapd.pem
tls_server_key: /etc/pki/cyrus-imapd/cyrus-imapd.pem
#
# Murder Setup
#
mupdate_server: ilserv18.idmt.fraunhofer.de
mupdate_username:   cyrus-murder
mupdate_authname:   cyrus-murder
mupdate_password:   
mupdate_mechs:  plain
proxy_authname: cyrus-murder
proxy_password: 
proxyservers:   cyrus-murder

START {
recover cmd="ctl_cyrusdb -r"
idled   cmd="idled"
mupdatepush cmd="ctl_mboxlist -m"
}
SERVICES {
imapcmd="proxyd" listen="imap" prefork=5
imaps   cmd="proxyd -s" listen="imaps" prefork=1
pop3cmd="pop3d" listen="pop3" prefork=3
pop3s   cmd="pop3d -s" listen="pop3s" prefork=1
sieve   cmd="timsieved" listen="sieve" prefork=0
lmtpunixcmd="lmtpd" listen="/var/lib/imap/socket/lmtp" prefork=1 
mupdate  cmd="mupdate" listen="3905" prefork=1 
notify  cmd="notifyd" listen="/var/lib/imap/socket/notify" proto="udp" 
prefork=1 
}
EVENTS {
checkpoint  cmd="ctl_cyrusdb -c" period=30
duplicateprune cmd="cyr_expire -E 3" at=0400
deleteprune cmd="cyr_expire -E 4 -D 69" at=0430
expungeprune cmd="cyr_expire -E 4 -X 69" at=0445
tlsprunecmd="tls_prune" at=0400
}

 backend-config:
#
# Servername
#
servername: imailbackend00
#
# Storage Part
#
configdirectory:   /var/lib/imap
partition-default: /cyrus_pool
sievedir:  /var/lib/imap/sieve
#
# Admins
#
admins:   cyrus-admin cyrus-murder
# 
# Executables 
#
sendmail: /usr/sbin/sendmail
#
# User Authentication
#
# General
allowplaintext: yes
# Sasl
sasl_pwcheck_method: auxprop saslauthd
sasl_mech_list:  PLAIN LOGIN
#
# Security / Encryption
tls_client_ca_dir:  /etc/pki/tls/certs/
tls_server_cert:/etc/pki/cyrus-imapd/cyrus-imapd.pem
tls_server_key: /etc/pki/cyrus-imapd/cyrus-imapd.pem
#
# Murder Setup
#
mupdate_server: ilserv18.idmt.fraunhofer.de
mupdate_username:   cyrus-murder
mupdate_authname:   cyrus-murder
mupdate_password:   
mupdate_mechs:  plain
proxy_authname: cyrus-murder
proxy_password: 
proxyservers:   cyrus-murder

START {
recover cmd="ctl_cyrusdb -r"
idled   cmd="idled"
mupdatepush cmd="ctl_mboxlist -m"
}
SERVICES {
imapcmd="imapd" lis

Re: 2.4.18, problem with reconstruct

[Andrew Morgan via Info-cyrus]

On Fri, 5 Feb 2016, Sergey via Info-cyrus wrote:


Hello.

I attempted to reconstruct some damaged mailboxes with empty
folders, but it does not work. I use this command:

su -l cyrus -s /bin/bash -c "/usr/lib/cyrus/reconstruct -f -r user/user@domain"

Mail directory contains "Trash" subdirectory without any files (manualy
created from backup). Reconstruct works if I put any of files cyrus.* to
this subdirectory. At the same time there was the opposite problem:
I can not delete existing directory, reconstruct restores it.

Is this is a bug or require any other settings to run reconstruct ?


I usually use these steps to add a new folder using reconstruct:

  touch cyrus.header
  chown cyrus:mail cyrus.header
  reconstruct -f -r user.

So, I think the behavior you are seeing is expected.  Create an empty 
cyrus.header file, with the correct ownership, before running reconstruct.


Andy

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

2.4.18, problem with reconstruct

[Sergey via Info-cyrus]

Hello.

I attempted to reconstruct some damaged mailboxes with empty
folders, but it does not work. I use this command:

su -l cyrus -s /bin/bash -c "/usr/lib/cyrus/reconstruct -f -r user/user@domain"

Mail directory contains "Trash" subdirectory without any files (manualy
created from backup). Reconstruct works if I put any of files cyrus.* to
this subdirectory. At the same time there was the opposite problem:
I can not delete existing directory, reconstruct restores it.

Is this is a bug or require any other settings to run reconstruct ?

-- 
Regards,
Sergey

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: 2.4.18, problem with reconstruct

[Sergey via Info-cyrus]

On Friday 05 February 2016, Andrew Morgan wrote:

> > Is this is a bug or require any other settings to run reconstruct ?
> 
> I usually use these steps to add a new folder using reconstruct:
> 
>touch cyrus.header
>chown cyrus:mail cyrus.header
>reconstruct -f -r user.
> 
> So, I think the behavior you are seeing is expected.  Create an empty 
> cyrus.header file, with the correct ownership, before running reconstruct.

Good when boxes are few and known. I don't know exactly how many boxes
I need to fix. Ok, I wrote script. It may be useful to someone.

=
BASE_DIR=/var/spool/imap/domain/M/

[ -d $BASE_DIR ] || exit

find $BASE_DIR -type d | \
while read DIR; do
[ "$DIR" = "$BASE_DIR" ] && continue
if [ -d $DIR ]; then
FILES=`find $DIR -type f | wc -l`
if [ $FILES -eq 0 ]; then
touch $DIR/cyrus.header
chown cyrus:cyrus $DIR/cyrus.header
echo $DIR/cyrus.header
fi
fi
done
==

-- 
Regards,
Sergey

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Seen problem after xfer command

[Miguel Mucio Santos Moreira via Info-cyrus]

Hi everybody,

SERPRO (It's an IT services corporation of Brazilian Federal Government) has 
discovered what was going on about my xfer problem, they had the same situation 
in their environment.

We've suppressed the Server Information turning the serverinfo parameter off 
and this was causing the problem. We thought better to suppress it because of 
security reasons, turning it on (default value), cyrus put a lot of environment 
information into email header.
Actually the serverinformation parameter only causes problem if it turned off 
in backend servers (Murder Environment).

During the mailbox transfer the source server does a downgrade of index to 
version 6 and the destination server does an upgrade from version 6 to 12:
Jan 25 16:35:48 srcserver cyrus/imap[4608]: user.X.Templates downgrading 
index to version 6 for XFER
Jan 25 16:35:55 dstserver cyrus/imap[325]: Index upgrade: user.x.Templates 
(6 -> 12)

When the serverinfo parameter is turned on, the above information doesn't 
appear in log file.

I'd like to thank Gustavo from SERPRO for sharing the information, so if 
someone else has the same problem put the serverinfo parameter on and have the 
problem solved.

We'd prefer to suppress this information but mailbox transfer is more important 
than security reasons for now.


See you

-- 
Miguel Mucio Santos Moreira
 Analista - LPIC 1 Linux Professional Institute Certified
 GRE - Gerência de Redes
 (31)3339-1401
 PRODEMGE - Companhia de Tecnologia da Informação do Estado de Minas Gerais 

Aviso:
 Esta mensagem é destinada exclusivamente para a(s) pessoa(s) a quem é 
dirigida, podendo conter informação confidencial e legalmente protegida.
 Se você não for destinatário dela, desde já fica notificado de 
abster-se a divulgar, copiar, distribuir, examinar ou, de qualquer 
forma, utilizar a informação contida nesta mensagem, por ser ilegal. 
Caso você tenha recebido por engano, pedimos que responda essa mensagem 
informando o acontecido.



Em 16/07/2015 16:21:49, Miguel Mucio Santos Moreira escreveu:
> Hi everybody,
> 
> I've had a weird trouble in my cyrus murder environment.
> After I move a mailbox between the backend servers, for each non empty folder 
> which has all email as read, it put the most recent email as unread.
> It seems a seen problem but I wasn't able to find out the reason.
> Have anyone already had this problem? Is it a known bug?
> My backend environment is:
> Debian 7.0
> Cyrus Version: 2.4.16-4
> 
> I've checked the changes on this link > 
> https://cyrusimap.org/docs/cyrus-imapd/2.4.18/changes.php>  looking for this 
> problem.
> 
> Thanks in advance
> 
> Sincerely
> 
> -- 
> Miguel Mucio Santos Moreira
>  Analista - LPIC 1 Linux Professional Institute Certified
>  GRE - Gerência de Redes
>  (31)3339-1401
>  PRODEMGE - Companhia de Tecnologia da Informação do Estado de Minas Gerais>  
> 
> Aviso:
 Esta mensagem é destinada exclusivamente para a(s) pessoa(s) a quem é 
dirigida, podendo conter informação confidencial e legalmente protegida.
 Se você não for destinatário dela, desde já fica notificado de 
abster-se a divulgar, copiar, distribuir, examinar ou, de qualquer 
forma, utilizar a informação contida nesta mensagem, por ser ilegal. 
Caso você tenha recebido por engano, pedimos que responda essa mensagem 
informando o acontecido.
> 
> 
Cyrus Home Page: > http://www.cyrusimap.org/> 
List Archives/Info: > http://lists.andrew.cmu.edu/pipermail/info-cyrus/> 
To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Replication problem do_folders(): failed to rename

[Marcus Schopen via Info-cyrus]

Am Montag, den 14.12.2015, 07:31 -0400 schrieb Patrick Boutilier via
Info-cyrus:
> On 12/14/2015 06:25 AM, Marcus Schopen via Info-cyrus wrote:
> > Am Freitag, den 11.12.2015, 19:10 +0100 schrieb Marcus Schopen via
> > Info-cyrus:
> >> Hi,
> >>
> >> I have a problem with a single mailbox. The user's Outlook crashed and
> >> since then the sync_client is running wild on this user account and
> >> produces high load on the master. I stopped sync_client on master side
> >> for the moment.
> >>
> >> When I try to sync the user by hand
> >>
> >> /bin/su - cyrus -c "/usr/lib/cyrus/bin/sync_client -S replicaserver -v
> >> -u testuser
> >>
> >> I do get the following error.
> >>
> >> Dec 11 17:54:48 master cyrus/sync_client[22727]: RENAME received NO
> >> response: Operation is not supported on mailbox
> >> Dec 11 17:54:48 master cyrus/sync_client[22727]: do_folders(): failed to
> >> rename: user.elsa-secgen -> user.testuser.Archives.Gesch
> >> 2014-15.25-Jahr-Feier
> >> Dec 11 17:54:48 master cyrus/sync_client[22727]: IOERROR: The remote
> >> Server(s) denied the operation
> >> Dec 11 17:54:48 master cyrus/sync_client[22727]: Error in
> >> do_user(testuser): bailing out!
> >>
> >> Comparing master and slave on filesystem I do see the subfolder
> >> "25-Jahr-Feier" in "user.testuser.Archives.Gesch 2014-15.",
> >> but only on master but not on slave side. And why does sync_client want
> >> to rename and where does it get this order from?
> >>
> >> I can login into the users' mailbox on master side and new message are
> >> shown in the INBOX.
> >>
> >> How can I fix it?
> >>
> >> Should I try a "reconstruct -r user.testuser" on master and slave or
> >> just on slave? (do I have to shutdown cyrus for a reconstruct -r on a
> >> user box?)
> >>
> >> Or can I delete the complete mailbox on slave side start an "sync_client
> >> -S replicaserver -v -u testuser"?
> >>
> >> Thanks for helping
> >> Marcus
> >
> > I did a reconstruct on the replica whichs runs through on the 12 GB
> > mailbox ot the user within a second (too fast?).
> >
> > /bin/su - cyrus -c "/usr/lib/cyrus/bin/reconstruct -r user.testuser"
> >
> > A following sync ended up with the same error:
> >
> > /bin/su - cyrus -c "/usr/lib/cyrus/bin/sync_client -S replicaserver -v
> > -u testuser
> >
> > Any ideas?
> 
> No, but this seems weird. Was this user ever renamed?
> 
> user.testuser -> user.testuser.Archives.Gesch

No, sorry, forgot to hide the real user name ;)

Ciao!


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Replication problem do_folders(): failed to rename

[Michael Menge via Info-cyrus]

Hi,


Quoting Patrick Boutilier via Info-cyrus <info-cyrus@lists.andrew.cmu.edu>:


On 12/14/2015 06:25 AM, Marcus Schopen via Info-cyrus wrote:

Am Freitag, den 11.12.2015, 19:10 +0100 schrieb Marcus Schopen via
Info-cyrus:

Hi,

I have a problem with a single mailbox. The user's Outlook crashed and
since then the sync_client is running wild on this user account and
produces high load on the master. I stopped sync_client on master side
for the moment.

When I try to sync the user by hand

/bin/su - cyrus -c "/usr/lib/cyrus/bin/sync_client -S replicaserver -v
-u testuser

I do get the following error.

Dec 11 17:54:48 master cyrus/sync_client[22727]: RENAME received NO
response: Operation is not supported on mailbox
Dec 11 17:54:48 master cyrus/sync_client[22727]: do_folders(): failed to
rename: user.elsa-secgen -> user.testuser.Archives.Gesch
2014-15.25-Jahr-Feier
Dec 11 17:54:48 master cyrus/sync_client[22727]: IOERROR: The remote
Server(s) denied the operation
Dec 11 17:54:48 master cyrus/sync_client[22727]: Error in
do_user(testuser): bailing out!

Comparing master and slave on filesystem I do see the subfolder
"25-Jahr-Feier" in "user.testuser.Archives.Gesch 2014-15.",
but only on master but not on slave side. And why does sync_client want
to rename and where does it get this order from?

I can login into the users' mailbox on master side and new message are
shown in the INBOX.

How can I fix it?

Should I try a "reconstruct -r user.testuser" on master and slave or
just on slave? (do I have to shutdown cyrus for a reconstruct -r on a
user box?)

Or can I delete the complete mailbox on slave side start an "sync_client
-S replicaserver -v -u testuser"?

Thanks for helping
Marcus


I did a reconstruct on the replica whichs runs through on the 12 GB
mailbox ot the user within a second (too fast?).

/bin/su - cyrus -c "/usr/lib/cyrus/bin/reconstruct -r user.testuser"

A following sync ended up with the same error:

/bin/su - cyrus -c "/usr/lib/cyrus/bin/sync_client -S replicaserver -v
-u testuser

Any ideas?


No, but this seems weird. Was this user ever renamed?

user.elsa-secgen -> user.testuser.Archives.Gesch



Cyrus uses the folder "Unique ID" for syncronisation. If this "Unique  
ID" is NOT unique

it will confuse syncronisation.





Ciao
Marcus



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus






M.MengeTel.: (49) 7071/29-70316
Universität Tübingen   Fax.: (49) 7071/29-5912
Zentrum für Datenverarbeitung  mail:  
michael.me...@zdv.uni-tuebingen.de

Wächterstraße 76
72074 Tübingen


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Replication problem do_folders(): failed to rename

[Marcus Schopen via Info-cyrus]

Hi,

Am Montag, den 14.12.2015, 12:53 +0100 schrieb Michael Menge via
Info-cyrus:
> Hi,
> 
> 
> Quoting Patrick Boutilier via Info-cyrus <info-cyrus@lists.andrew.cmu.edu>:
> 
> > On 12/14/2015 06:25 AM, Marcus Schopen via Info-cyrus wrote:
> >> Am Freitag, den 11.12.2015, 19:10 +0100 schrieb Marcus Schopen via
> >> Info-cyrus:
> >>> Hi,
> >>>
> >>> I have a problem with a single mailbox. The user's Outlook crashed and
> >>> since then the sync_client is running wild on this user account and
> >>> produces high load on the master. I stopped sync_client on master side
> >>> for the moment.
> >>>
> >>> When I try to sync the user by hand
> >>>
> >>> /bin/su - cyrus -c "/usr/lib/cyrus/bin/sync_client -S replicaserver -v
> >>> -u testuser
> >>>
> >>> I do get the following error.
> >>>
> >>> Dec 11 17:54:48 master cyrus/sync_client[22727]: RENAME received NO
> >>> response: Operation is not supported on mailbox
> >>> Dec 11 17:54:48 master cyrus/sync_client[22727]: do_folders(): failed to
> >>> rename: user.elsa-secgen -> user.testuser.Archives.Gesch
> >>> 2014-15.25-Jahr-Feier
> >>> Dec 11 17:54:48 master cyrus/sync_client[22727]: IOERROR: The remote
> >>> Server(s) denied the operation
> >>> Dec 11 17:54:48 master cyrus/sync_client[22727]: Error in
> >>> do_user(testuser): bailing out!
> >>>
> >>> Comparing master and slave on filesystem I do see the subfolder
> >>> "25-Jahr-Feier" in "user.testuser.Archives.Gesch 2014-15.",
> >>> but only on master but not on slave side. And why does sync_client want
> >>> to rename and where does it get this order from?
> >>>
> >>> I can login into the users' mailbox on master side and new message are
> >>> shown in the INBOX.
> >>>
> >>> How can I fix it?
> >>>
> >>> Should I try a "reconstruct -r user.testuser" on master and slave or
> >>> just on slave? (do I have to shutdown cyrus for a reconstruct -r on a
> >>> user box?)
> >>>
> >>> Or can I delete the complete mailbox on slave side start an "sync_client
> >>> -S replicaserver -v -u testuser"?
> >>>
> >>> Thanks for helping
> >>> Marcus
> >>
> >> I did a reconstruct on the replica whichs runs through on the 12 GB
> >> mailbox ot the user within a second (too fast?).
> >>
> >> /bin/su - cyrus -c "/usr/lib/cyrus/bin/reconstruct -r user.testuser"
> >>
> >> A following sync ended up with the same error:
> >>
> >> /bin/su - cyrus -c "/usr/lib/cyrus/bin/sync_client -S replicaserver -v
> >> -u testuser
> >>
> >> Any ideas?
> >
> > No, but this seems weird. Was this user ever renamed?
> >
> > user.testuser -> user.testuser.Archives.Gesch
> >
> 
> Cyrus uses the folder "Unique ID" for syncronisation. If this "Unique  
> ID" is NOT unique
> it will confuse syncronisation.


I connected with an imap client to the mailbox and removed all folders
on the master to which sync_client was bailing out. After that
sync_client was runnig through without problems. Strange ...

Ciao!





Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Replication problem do_folders(): failed to rename

[Patrick Boutilier via Info-cyrus]

On 12/14/2015 06:25 AM, Marcus Schopen via Info-cyrus wrote:

Am Freitag, den 11.12.2015, 19:10 +0100 schrieb Marcus Schopen via
Info-cyrus:

Hi,

I have a problem with a single mailbox. The user's Outlook crashed and
since then the sync_client is running wild on this user account and
produces high load on the master. I stopped sync_client on master side
for the moment.

When I try to sync the user by hand

/bin/su - cyrus -c "/usr/lib/cyrus/bin/sync_client -S replicaserver -v
-u testuser

I do get the following error.

Dec 11 17:54:48 master cyrus/sync_client[22727]: RENAME received NO
response: Operation is not supported on mailbox
Dec 11 17:54:48 master cyrus/sync_client[22727]: do_folders(): failed to
rename: user.elsa-secgen -> user.testuser.Archives.Gesch
2014-15.25-Jahr-Feier
Dec 11 17:54:48 master cyrus/sync_client[22727]: IOERROR: The remote
Server(s) denied the operation
Dec 11 17:54:48 master cyrus/sync_client[22727]: Error in
do_user(testuser): bailing out!

Comparing master and slave on filesystem I do see the subfolder
"25-Jahr-Feier" in "user.testuser.Archives.Gesch 2014-15.",
but only on master but not on slave side. And why does sync_client want
to rename and where does it get this order from?

I can login into the users' mailbox on master side and new message are
shown in the INBOX.

How can I fix it?

Should I try a "reconstruct -r user.testuser" on master and slave or
just on slave? (do I have to shutdown cyrus for a reconstruct -r on a
user box?)

Or can I delete the complete mailbox on slave side start an "sync_client
-S replicaserver -v -u testuser"?

Thanks for helping
Marcus


I did a reconstruct on the replica whichs runs through on the 12 GB
mailbox ot the user within a second (too fast?).

/bin/su - cyrus -c "/usr/lib/cyrus/bin/reconstruct -r user.testuser"

A following sync ended up with the same error:

/bin/su - cyrus -c "/usr/lib/cyrus/bin/sync_client -S replicaserver -v
-u testuser

Any ideas?


No, but this seems weird. Was this user ever renamed?

user.elsa-secgen -> user.testuser.Archives.Gesch




Ciao
Marcus



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus



<>
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Replication problem do_folders(): failed to rename

[Marcus Schopen via Info-cyrus]

Am Freitag, den 11.12.2015, 19:10 +0100 schrieb Marcus Schopen via
Info-cyrus:
> Hi,
> 
> I have a problem with a single mailbox. The user's Outlook crashed and
> since then the sync_client is running wild on this user account and
> produces high load on the master. I stopped sync_client on master side
> for the moment.
> 
> When I try to sync the user by hand 
> 
> /bin/su - cyrus -c "/usr/lib/cyrus/bin/sync_client -S replicaserver -v
> -u testuser
> 
> I do get the following error.
> 
> Dec 11 17:54:48 master cyrus/sync_client[22727]: RENAME received NO
> response: Operation is not supported on mailbox
> Dec 11 17:54:48 master cyrus/sync_client[22727]: do_folders(): failed to
> rename: user.elsa-secgen -> user.testuser.Archives.Gesch
> 2014-15.25-Jahr-Feier 
> Dec 11 17:54:48 master cyrus/sync_client[22727]: IOERROR: The remote
> Server(s) denied the operation
> Dec 11 17:54:48 master cyrus/sync_client[22727]: Error in
> do_user(testuser): bailing out!
> 
> Comparing master and slave on filesystem I do see the subfolder
> "25-Jahr-Feier" in "user.testuser.Archives.Gesch 2014-15.",
> but only on master but not on slave side. And why does sync_client want
> to rename and where does it get this order from?
> 
> I can login into the users' mailbox on master side and new message are
> shown in the INBOX.
> 
> How can I fix it? 
> 
> Should I try a "reconstruct -r user.testuser" on master and slave or
> just on slave? (do I have to shutdown cyrus for a reconstruct -r on a
> user box?)
> 
> Or can I delete the complete mailbox on slave side start an "sync_client
> -S replicaserver -v -u testuser"?
> 
> Thanks for helping
> Marcus

I did a reconstruct on the replica whichs runs through on the 12 GB
mailbox ot the user within a second (too fast?).

/bin/su - cyrus -c "/usr/lib/cyrus/bin/reconstruct -r user.testuser"

A following sync ended up with the same error:

/bin/su - cyrus -c "/usr/lib/cyrus/bin/sync_client -S replicaserver -v
-u testuser

Any ideas?

Ciao
Marcus



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Replication problem do_folders(): failed to rename

[Marcus Schopen via Info-cyrus]

Hi,

forgot the cyrus version: 2.4.12 on Ubuntu 12.04 LTS

Ciao!



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Replication problem do_folders(): failed to rename

[Marcus Schopen via Info-cyrus]

Hi,

I have a problem with a single mailbox. The user's Outlook crashed and
since then the sync_client is running wild on this user account and
produces high load on the master. I stopped sync_client on master side
for the moment.

When I try to sync the user by hand 

/bin/su - cyrus -c "/usr/lib/cyrus/bin/sync_client -S replicaserver -v
-u testuser

I do get the following error.

Dec 11 17:54:48 master cyrus/sync_client[22727]: RENAME received NO
response: Operation is not supported on mailbox
Dec 11 17:54:48 master cyrus/sync_client[22727]: do_folders(): failed to
rename: user.elsa-secgen -> user.testuser.Archives.Gesch
2014-15.25-Jahr-Feier 
Dec 11 17:54:48 master cyrus/sync_client[22727]: IOERROR: The remote
Server(s) denied the operation
Dec 11 17:54:48 master cyrus/sync_client[22727]: Error in
do_user(testuser): bailing out!

Comparing master and slave on filesystem I do see the subfolder
"25-Jahr-Feier" in "user.testuser.Archives.Gesch 2014-15.",
but only on master but not on slave side. And why does sync_client want
to rename and where does it get this order from?

I can login into the users' mailbox on master side and new message are
shown in the INBOX.

How can I fix it? 

Should I try a "reconstruct -r user.testuser" on master and slave or
just on slave? (do I have to shutdown cyrus for a reconstruct -r on a
user box?)

Or can I delete the complete mailbox on slave side start an "sync_client
-S replicaserver -v -u testuser"?

Thanks for helping
Marcus



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

2.5.x startup problem with mupadatepush enabled

[Khalid Mehmood via Info-cyrus]

Cyrus-imapd-2.5.x wouldn't start with "mupatepush -m" enabled in cyrus.conf on 
RHEL-7 after a system reboot. After disabling mupdatepush in cyrus.conf the 
service starts just fine, and subsequent restarts with mupdatepush enabled also 
work fine. Running a kerberized murder setup based on 2.4.x for more than a 
decade, we are planning to migrate our mail system to 2.5.x. Frontends and 
Muder itself don't have this problem, the cyrus service starts and works 
properly after system reboots. Is anyone else facing such issues?

Backend cyrus.conf:

START {
auth  cmd="/usr/bin/kinit -k -t /etc/xxx.d/krb5.keytab xxx"
recover   cmd="ctl_cyrusdb -r"
#   syncclientcmd="/usr/lib/cyrus-imapd/sync_client -r" prefork=1
mupdatepush   cmd="ctl_mboxlist -m"
idled cmd="idled"
}
SERVICES {
imap  cmd="imapd" listen="imap" prefork=10 maxchild=650
imaps cmd="imapd -s" listen="imaps" prefork=10 maxchild=650
lmtp  cmd="lmtpd" listen="lmtp" prefork=5 maxchild=10
lmtpunix  cmd="lmtpd" listen="/var/lib/imap/socket/lmtp" prefork=5 
maxchild=10
sieve cmd="timsieved" listen="sieve" prefork=1 maxchild=10
}
EVENTS {
checkpointcmd="ctl_cyrusdb -c" period=5
delprune  cmd="cyr_expire -E 1 -D 7 -X 7 -a" at=2300
deleteprune   cmd="cyr_expire -E 1 -D 7 -X 7 -a" at=2300
expungeprune  cmd="cyr_expire -E 1 -D 7 -X 7 -a" at=2300
tlsprune  cmd="tls_prune" period=1440
squatter  cmd="/usr/lib/cyrus-imapd/squatter -r *" at=0200
#   squatter  cmd="/usr/lib/cyrus-imapd/squatter -r *@*" at=0400
reauthcmd="/usr/bin/kinit -k -t /etc/xxx.d/krb5.keytab xxx" period="360"
}

maillog:
Nov  2 10:14:02 vcas ctl_cyrusdb[4502]: skiplist: clean shutdown file missing, 
updating recovery stamp
Nov  2 10:14:02 vcas ctl_cyrusdb[4502]: recovering cyrus databases
Nov  2 10:14:02 vcas ctl_cyrusdb[4502]: done recovering cyrus databases
Nov  2 10:14:02 vcas ctl_mboxlist[4504]: extra arguments recieved, aborting 
connection
Nov  2 10:14:02 vcas master[4500]: process type:START name:mupdatepush 
path:/usr/lib/cyrus-imapd/ctl_mboxlist age:0.000s pid:4504 exited, status 1
Nov  2 10:14:02 vcas master[4500]: can't run startup
Nov  2 10:14:02 vcas master[4500]: exiting
.

Thanks

Khalid

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: cyrus imap 2.5.5 build in Linux: problem with pcre

[Sergey]

On Monday 19 October 2015, ellie timoney wrote:

> Do you have pkg-config installed on your system? 

$ pkg-config --version
0.25

> Does your pcre library support utf8?

I think yes: configure call in pcre.spec contains "--enable-pcre8 
--enable-pcre16 --enable-utf" options. 

> Do you have a line in your configure output like
> "checking for utf8 enabled pcre..."?

No. I have not the sting "checking for utf8" in output of configure.
I look into "configure" and found what $ac_cv_header_pcreposix_h="".

If I to set ac_cv_header_pcreposix_h="yes" then package is configured
with pcre

checking pcre/pcreposix.h usability... yes
checking pcre/pcreposix.h presence... yes
checking for pcre/pcreposix.h... yes
checking for utf8 enabled pcre... yes

External dependencies:
   ldap:   yes
   openssl:yes
   pcre:   yes

-- 
Regards, Sergey

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: cyrus imap 2.5.5 build in Linux: problem with unit test, test "badservice" failed

[Sergey]

On Monday 19 October 2015, ellie timoney wrote:

> >   Test: badservice ...lt-unit: threads.c:342: krb5int_key_register:
> >   Assertion `destructors_set[keynum] == 0' failed.
> > /bin/sh: line 10:   791 Aborted $vg ./unit $f
> 
> This looks like it's related to krb5 somehow, which I know nothing
> about.  Can someone who uses or has used Cyrus with Kerberos chime in
> please?

I found messages with similag error (threads.c:342: krb5int_key_register ...)
with krb5-1.13 (I use 1.13.1) in Google. I attempted to rollback to krb5 1.12
and test was passed. That is probably it is the problem of krb5.

-- 
Regards, Sergey

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: cyrus imap 2.5.5 build in Linux: problem with pcre

[ellie timoney]

Hi Sergey,

Do you have pkg-config installed on your system?  If not, does
installing it help?

Does your pcre library support utf8?  Cyrus depends on a utf8-enabled
pcre library.  If your pcre library does not appear to support utf-8,
then the configure script will find the headers but will decide to not
use the library.  Do you have a line in your configure output like
"checking for utf8 enabled pcre..."?  What does it say?

Cheers,

ellie

On Sat, Oct 17, 2015, at 05:49 AM, Sergey wrote:
> Hello.
> 
> I built and run version 2.5.5 but I have some questions about
> build process. I will ask this in some different messages.
> 
> I wanted to build Cyrus-IMAP with libpcre but it did not success.
> System libpcre-devel package install headers to /usr/include/pcre.
> I tried to replace pcreposix.h in configure.ac to pcre/pcreposix.h
> but it didn't yield results. "configure" found it:
> 
> checking pcre/pcreposix.h usability... yes
> checking pcre/pcreposix.h presence... yes
> checking for pcre/pcreposix.h... yes
> 
> but pcre missed in final configuration:
> 
> External dependencies:
>ldap:   yes
>openssl:yes
>pcre:   no
> 
> I'm not very familiar with autotools. What can I missed ?
> btw: "configure" contains "--with-bdb-incdir" option.
> I think what "--with-pcre-incdir" will be good.
> 
> -- 
> Regards,
> Sergey
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: cyrus imap 2.5.5 build in Linux: problem with unit test, test "badservice" failed

[ellie timoney]

>   Test: badservice ...lt-unit: threads.c:342: krb5int_key_register:
>   Assertion `destructors_set[keynum] == 0' failed.
> /bin/sh: line 10:   791 Aborted $vg ./unit $f

This looks like it's related to krb5 somehow, which I know nothing
about.  Can someone who uses or has used Cyrus with Kerberos chime in
please?

> Is this a bug of Cyrus IMAP or Is this a bug of my build environment ?

Hard to say at this point.  It would help if you would attach your
config.log, and include the commands you run (not just their output), in
your emails.

Cheers,

ellie

On Sat, Oct 17, 2015, at 06:33 AM, Sergey wrote:
> Hello.
> 
> I found what Cyrus IMAP contains unit tests now.
> I try it but test failed:
> 
>   Test: setentryatt ...passed
>   Test: clearentryatt ...passed
> Suite: backend
>   Test: badhost ...passed
>   Test: badservice ...lt-unit: threads.c:342: krb5int_key_register:
>   Assertion `destructors_set[keynum] == 0' failed.
> /bin/sh: line 10:   791 Aborted $vg ./unit $f
> make[3]: *** [check-local] Error 134
> lt-unit: threads.c:342: krb5int_key_register: Assertion
> `destructors_set[keynum] == 0' failed.
> 
> Is this a bug of Cyrus IMAP or Is this a bug of my build environment ?
> 
> -- 
> Regards,
> Sergey
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

cyrus imap 2.5.5 build in Linux: problem with pcre

[Sergey]

Hello.

I built and run version 2.5.5 but I have some questions about
build process. I will ask this in some different messages.

I wanted to build Cyrus-IMAP with libpcre but it did not success.
System libpcre-devel package install headers to /usr/include/pcre.
I tried to replace pcreposix.h in configure.ac to pcre/pcreposix.h
but it didn't yield results. "configure" found it:

checking pcre/pcreposix.h usability... yes
checking pcre/pcreposix.h presence... yes
checking for pcre/pcreposix.h... yes

but pcre missed in final configuration:

External dependencies:
   ldap:   yes
   openssl:yes
   pcre:   no

I'm not very familiar with autotools. What can I missed ?
btw: "configure" contains "--with-bdb-incdir" option.
I think what "--with-pcre-incdir" will be good.

-- 
Regards,
Sergey

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

cyrus imap 2.5.5 build in Linux: problem with unit test, test "badservice" failed

[Sergey]

Hello.

I found what Cyrus IMAP contains unit tests now.
I try it but test failed:

  Test: setentryatt ...passed
  Test: clearentryatt ...passed
Suite: backend
  Test: badhost ...passed
  Test: badservice ...lt-unit: threads.c:342: krb5int_key_register: Assertion 
`destructors_set[keynum] == 0' failed.
/bin/sh: line 10:   791 Aborted $vg ./unit $f
make[3]: *** [check-local] Error 134
lt-unit: threads.c:342: krb5int_key_register: Assertion 
`destructors_set[keynum] == 0' failed.

Is this a bug of Cyrus IMAP or Is this a bug of my build environment ?

-- 
Regards,
Sergey

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Seen problem after transfering a mailbox

[Miguel Mucio Santos Moreira]

Hello everybody,

I have a weird trouble in my cyrus murder environment.
After
 I move a mailbox between backend servers, for each non empty folder 
which has all email as read, it put the most recent email as unread.
It seems a seen problem but I wasn't able to find out the reason that it 
happens after a transfer of a mailbox.
Could anyone help me?


My backend/frontend environment is:
Debian 7.0
Cyrus Version: 2.4.16-4

Cheers

-- 
Miguel Mucio Santos Moreira


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Sieve, a problem with if/elsif/else

[Sergey]

Hello.

I have a strange problem. I use a filter

if address :contains ["To"] ["bounceerror@"] {
   ifaddress :contains ["From"] ["MAILER-DAEMON@srv1.domain"] {
  fileinto "INBOX/BounceError/srv1";
   }
   elsif address :contains ["From"] ["MAILER-DAEMON@srv2.domain"] {
  fileinto "INBOX/BounceError/srv2";
   }
   elsif address :contains ["From"] ["MAILER-DAEMON@srv3.domain"] {
  fileinto "INBOX/BounceError/srv3";
   }
#   elsif address :contains ["From"] ["MAILER-DAEMON@srv4.domain"] {
#  fileinto "INBOX/BounceError/srv4";
#   }
   else {
  fileinto "INBOX/BounceError";
   }
}

Messages from srv4 remain in the inbox instead of move to
BounceError. Messages from srv4 are moved to INBOX/BounceError/srv4
if uncomment lines about srv4.

Why "else fileinto" doesn't work ? Discovered in 2.4.17, reproduced
in 2.4.18.

-- 
Regards, Sergey

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Sieve, a problem with if/elsif/else

[Eric Luyten]

On Mon, August 31, 2015 9:22 am, Sergey wrote:
> Hello.
>
>
> I have a strange problem. I use a filter
>
>
> if address :contains ["To"] ["bounceerror@"] { ifaddress :contains
> ["From"] ["MAILER-DAEMON@srv1.domain"] {
> fileinto "INBOX/BounceError/srv1"; }
> elsif address :contains ["From"] ["MAILER-DAEMON@srv2.domain"] { fileinto
> "INBOX/BounceError/srv2";
> }
> elsif address :contains ["From"] ["MAILER-DAEMON@srv3.domain"] { fileinto
> "INBOX/BounceError/srv3";
> }
> #   elsif address :contains ["From"] ["MAILER-DAEMON@srv4.domain"] {
> #  fileinto "INBOX/BounceError/srv4";
> #   }
> else { fileinto "INBOX/BounceError"; }
> }
>
>
> Messages from srv4 remain in the inbox instead of move to
> BounceError. Messages from srv4 are moved to INBOX/BounceError/srv4
> if uncomment lines about srv4.
>
> Why "else fileinto" doesn't work ? Discovered in 2.4.17, reproduced
> in 2.4.18.


Sergey,


Did you define "INBOX/BounceError" as a mailbox ?
This is not implied by the fact that it is a hierarchy.
Some IMAP clients may create it in the process of creating
sub-mailboxes, but certainly not all.


Regards,
Eric Luyten, Computing Centre VUB/ULB.



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Sieve, a problem with if/elsif/else

[Sergey]

On Monday 31 August 2015, Eric Luyten wrote:

> > else { fileinto "INBOX/BounceError"; }
> > }

> > Why "else fileinto" doesn't work ? Discovered in 2.4.17,
> > reproduced in 2.4.18.

> Did you define "INBOX/BounceError" as a mailbox ?
> This is not implied by the fact that it is a hierarchy.
> Some IMAP clients may create it in the process of creating
> sub-mailboxes, but certainly not all.

It seems you are right. Apparently, "BounceError" has an unusual
permissions. I can create subfolders but I can't move messages to
"BounceError". I understand behavior of the sieve now, thanks.

-- 
Regards, Sergey

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Seen problem after xfer command

[Miguel Mucio Santos Moreira]

Hi everybody,

I've had a weird trouble in my cyrus murder environment.
After I move a mailbox between the backend servers, for each non empty folder 
which has all email as read, it put the most recent email as unread.
It seems a seen problem but I wasn't able to find out the reason.
Have anyone already had this problem? Is it a known bug?
My backend environment is:
Debian 7.0
Cyrus Version: 2.4.16-4

I've checked the changes on this link 
https://cyrusimap.org/docs/cyrus-imapd/2.4.18/changes.php looking for this 
problem.

Thanks in advance

Sincerely

-- 
Miguel Mucio Santos Moreira
 Analista - LPIC 1 Linux Professional Institute Certified
 GRE - Gerência de Redes
 (31)3339-1401
 PRODEMGE - Companhia de Tecnologia da Informação do Estado de Minas Gerais 

Aviso:
 Esta mensagem é destinada exclusivamente para a(s) pessoa(s) a quem é 
dirigida, podendo conter informação confidencial e legalmente protegida.
 Se você não for destinatário dela, desde já fica notificado de 
abster-se a divulgar, copiar, distribuir, examinar ou, de qualquer 
forma, utilizar a informação contida nesta mensagem, por ser ilegal. 
Caso você tenha recebido por engano, pedimos que responda essa mensagem 
informando o acontecido.



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Murder frontend problem

[Major Csaba]

On 06/05/2015 07:42 PM, Andrew Morgan wrote:
 On Fri, 5 Jun 2015, Major Csaba wrote:

 There is one more small question: why the proxied LMTP needs to have 
 admins permission on the backend? I thought the proxyservers 
 setting is for this, but LMTP doesn't work without adding my proxy 
 user in the admins...

 Play around with lmtp_admins in imapd.conf.  Our mail relays connect 
 to our frontends over lmtp and auth as cyr_lmtp.  That authentication is
I missed the separated admins, but good idea.

Thanks for the hints guys.

Regards,
Csaba

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Murder frontend problem

[Major Csaba]

Hi,

I'm rying to build a cyrus server with murder frontend/backend.  The 
topology is quiet simple at the moment: there is one frontend + mupdate 
master, and there is one backend server.

(Cyrus version: 2.4.17)

Almost everything is working fine:

 * The backend populates the folder changes to the mupdate master server
 * The frontend (which is the same as the mupdate master) can see the
   mailboxes on the backend
 * the user can log in to the frontend with imap and can reach the
   mailboxes and mails

The only thing which is not working is to create new subfolders on the 
frontend. If I create the subfolder on the backend, then it's visible on 
the frontend correctly.


The frontend server reports this when I try to create a subfolder:
*cyrus/imap[10130]: IOERROR: Mailbox name too long 
(domain.com!user.johndoe)*


This is reported directly by the frontend, as I can not see any 
communications to the backend. So, I think this is a config issue on the 
frontend, but I can not see where.


See the configs below.
Do you have any hint how to go on or debug this situation?

Thanks in advace,
Csaba

*The frontend cyrus.conf:*
START {
recovercmd=/usr/sbin/cyrus ctl_cyrusdb -r
delprunecmd=/usr/sbin/cyrus expire -E 3
tlsprunecmd=/usr/sbin/cyrus tls_prune
}

SERVICES {
sievecmd=timsieved listen=localhost:sieve prefork=0 
maxchild=100
notifycmd=notifyd listen=/var/run/cyrus/socket/notify 
proto=udp prefork=1

mupdate   cmd=mupdate -m listen=3905 prefork=1
imapcmd=proxyd listen=imap prefork=0 maxchild=100
imapscmd=proxyd -s listen=imaps prefork=0 maxchild=100
pop3cmd=pop3proxyd listen=pop3 prefork=0 maxchild=50
pop3scmd=pop3proxyd -s listen=pop3s prefork=0 maxchild=50
lmtpcmd=lmtpproxyd listen=/var/run/cyrus/socket/lmtp 
prefork=1 maxchild=20

}

EVENTS {
checkpointcmd=/usr/sbin/cyrus ctl_cyrusdb -c period=30
delprunecmd=/usr/sbin/cyrus expire -E 3 at=0401
tlsprunecmd=/usr/sbin/cyrus tls_prune at=0401
}

*The frontend imap.conf:*
configdirectory: /var/lib/cyrus
proc_path: /run/cyrus/proc
mboxname_lockpath: /run/cyrus/lock
defaultpartition: default
partition-default: /var/spool/cyrus/mail
partition-news: /var/spool/cyrus/news
newsspool: /var/spool/news
altnamespace: no
unixhierarchysep: yes
reject8bit: no
munge8bit: no
lmtp_downcase_rcpt: yes
admins: cyrus2 mupdate
allowanonymouslogin: no
popminpoll: 1
autocreatequota: 0
umask: 077
sieveusehomedir: false
sievedir: /var/spool/sieve
httpmodules: caldav carddav
hashimapspool: true
allowplaintext: yes
sasl_mech_list: PLAIN
virtdomains: userid
sasl_pwcheck_method: saslauthd
sasl_auto_transition: no
proxy_authname: murderproxy
proxy_password: password
tls_cert_file: /etc/ssl/certs/ssl.crt
tls_key_file: /etc/ssl/private/ssl.key
tls_ca_path: /etc/ssl/certs
tls_session_timeout: 1440
tls_cipher_list: TLSv1+HIGH:!aNULL:@STRENGTH
lmtpsocket: /var/run/cyrus/socket/lmtp
idlesocket: /var/run/cyrus/socket/idle
notifysocket: /var/run/cyrus/socket/notify
syslog_prefix: cyrus

*Backend cyrus.conf:*
START {
recovercmd=/usr/sbin/cyrus ctl_cyrusdb -r
mupdatepush   cmd=/usr/sbin/cyrus ctl_mboxlist -m
delprunecmd=/usr/sbin/cyrus expire -E 3
tlsprunecmd=/usr/sbin/cyrus tls_prune
}

SERVICES {
imapcmd=imapd -U 30 listen=imap prefork=0 maxchild=100
imapscmd=imapd -s -U 30 listen=imaps prefork=0 maxchild=100
pop3cmd=pop3d -U 30 listen=pop3 prefork=0 maxchild=50
pop3scmd=pop3d -s -U 30 listen=pop3s prefork=0 maxchild=50
lmtpunixcmd=lmtpd listen=/var/run/cyrus/socket/lmtp 
prefork=0 maxchild=20
sievecmd=timsieved listen=localhost:sieve prefork=0 
maxchild=100
notifycmd=notifyd listen=/var/run/cyrus/socket/notify 
proto=udp prefork=1

}

EVENTS {
checkpointcmd=/usr/sbin/cyrus ctl_cyrusdb -c period=30
delprunecmd=/usr/sbin/cyrus expire -E 3 at=0401
tlsprunecmd=/usr/sbin/cyrus tls_prune at=0401
}

*Backend imapd.conf:*
configdirectory: /var/lib/cyrus
proc_path: /run/cyrus/proc
mboxname_lockpath: /run/cyrus/lock
defaultpartition: common
partition-common: /var/spool/cyrus/mail
partition-news: /var/spool/cyrus/news
newsspool: /var/spool/news
duplicatesuppression: 0
altnamespace: no
unixhierarchysep: yes
reject8bit: no
munge8bit: no
lmtp_downcase_rcpt: yes
admins: cyrus2
proxyservers: murderproxy
allowanonymouslogin: no
popminpoll: 0
autocreatequota: 20971520
umask: 077
sieveusehomedir: false
sievedir: /var/spool/sieve
httpmodules: caldav carddav
mailnotifier: zephyr
sievenotifier: zephyr
hashimapspool: true
allowplaintext: yes
sasl_mech_list: PLAIN
virtdomains: userid
sasl_pwcheck_method: saslauthd
sasl_auto_transition: no
tls_cert_file: /etc/ssl/certs/ssl.crt
tls_key_file: /etc/ssl/private/ssl.key
tls_ca_path: /etc/ssl/certs
tls_session_timeout: 1440
tls_cipher_list: TLSv1+HIGH:!aNULL:@STRENGTH
mupdate_server: mx1

Re: Murder frontend problem

[Major Csaba]

Hi,

Thanks for the quick answer.
I managed to get further as I realized I missed a small piece from the 
documentation. My fronted server and master update server is on the same 
machine and I didn't configure the mupdate_* parameter. But as I can 
see, the proxy still has to speak to mupdate when I would like to create 
a new mailbox and the auth info is necessary even if they are on the 
same host.


So, it seems to be a misundersanding of the documentation which is not 
so verbose :)
I added the mupdate_* parameters (pointing to the host itself) and it is 
working fine now.


There is one more small question: why the proxied LMTP needs to have 
admins permission on the backend? I thought the proxyservers setting 
is for this, but LMTP doesn't work without adding my proxy user in the 
admins...


Best regards,
Csaba

This error is thrown in a few places where the code is attempting to 
verify

the validity of the partition/mailbox. I suspect this error would be more
accurate if it said 'invalid partition'.

Do you have the ability to debug and gather a backtrace at the moment 
this

error is thrown? It's thrown on lines 2726, 2740, 2753, and 2773 within
mailbox.c (for version 2.4.17).

See:

http://members.sange.fi/~atehwa/vc/packaging/cyrus-imapd/debian/README.Debian.debug 



This is reported directly by the frontend, as I can not see any 
communications to the backend. So, I think this is a config issue on 
the frontend, but I can not see where.


This smells like a bug. Although the following options may affect the
problem:

altnamespace
unixhierarchysep
defaultdomain
defaultserver
hashimapspool
improved_mboxlist_sort
proxyd_disable_mailbox_referrals
sharedprefix
virtdomains

I don't recommend changing any of these on a production system however.


See the configs below.
Do you have any hint how to go on or debug this situation?




*The frontend cyrus.conf:*
START {
   recovercmd=/usr/sbin/cyrus ctl_cyrusdb -r
   delprunecmd=/usr/sbin/cyrus expire -E 3
   tlsprunecmd=/usr/sbin/cyrus tls_prune
}

SERVICES {
   sievecmd=timsieved listen=localhost:sieve prefork=0 
maxchild=100
   notifycmd=notifyd listen=/var/run/cyrus/socket/notify 
proto=udp prefork=1

   mupdate   cmd=mupdate -m listen=3905 prefork=1
   imapcmd=proxyd listen=imap prefork=0 maxchild=100
   imapscmd=proxyd -s listen=imaps prefork=0 maxchild=100
   pop3cmd=pop3proxyd listen=pop3 prefork=0 maxchild=50
   pop3scmd=pop3proxyd -s listen=pop3s prefork=0 maxchild=50
   lmtpcmd=lmtpproxyd listen=/var/run/cyrus/socket/lmtp 
prefork=1 maxchild=20

}

EVENTS {
   checkpointcmd=/usr/sbin/cyrus ctl_cyrusdb -c period=30
   delprunecmd=/usr/sbin/cyrus expire -E 3 at=0401
   tlsprunecmd=/usr/sbin/cyrus tls_prune at=0401
}

*The frontend imap.conf:*
configdirectory: /var/lib/cyrus
proc_path: /run/cyrus/proc
mboxname_lockpath: /run/cyrus/lock
defaultpartition: default
partition-default: /var/spool/cyrus/mail
partition-news: /var/spool/cyrus/news
newsspool: /var/spool/news
altnamespace: no
unixhierarchysep: yes
reject8bit: no
munge8bit: no
lmtp_downcase_rcpt: yes
admins: cyrus2 mupdate
allowanonymouslogin: no
popminpoll: 1
autocreatequota: 0
umask: 077
sieveusehomedir: false
sievedir: /var/spool/sieve
httpmodules: caldav carddav
hashimapspool: true
allowplaintext: yes
sasl_mech_list: PLAIN
virtdomains: userid
sasl_pwcheck_method: saslauthd
sasl_auto_transition: no
proxy_authname: murderproxy
proxy_password: password
tls_cert_file: /etc/ssl/certs/ssl.crt
tls_key_file: /etc/ssl/private/ssl.key
tls_ca_path: /etc/ssl/certs
tls_session_timeout: 1440
tls_cipher_list: TLSv1+HIGH:!aNULL:@STRENGTH
lmtpsocket: /var/run/cyrus/socket/lmtp
idlesocket: /var/run/cyrus/socket/idle
notifysocket: /var/run/cyrus/socket/notify
syslog_prefix: cyrus

*Backend cyrus.conf:*
START {
   recovercmd=/usr/sbin/cyrus ctl_cyrusdb -r
   mupdatepush   cmd=/usr/sbin/cyrus ctl_mboxlist -m
   delprunecmd=/usr/sbin/cyrus expire -E 3
   tlsprunecmd=/usr/sbin/cyrus tls_prune
}

SERVICES {
   imapcmd=imapd -U 30 listen=imap prefork=0 maxchild=100
   imapscmd=imapd -s -U 30 listen=imaps prefork=0 
maxchild=100

   pop3cmd=pop3d -U 30 listen=pop3 prefork=0 maxchild=50
   pop3scmd=pop3d -s -U 30 listen=pop3s prefork=0 
maxchild=50
   lmtpunixcmd=lmtpd listen=/var/run/cyrus/socket/lmtp 
prefork=0 maxchild=20
   sievecmd=timsieved listen=localhost:sieve prefork=0 
maxchild=100
   notifycmd=notifyd listen=/var/run/cyrus/socket/notify 
proto=udp prefork=1

}

EVENTS {
   checkpointcmd=/usr/sbin/cyrus ctl_cyrusdb -c period=30
   delprunecmd=/usr/sbin/cyrus expire -E 3 at=0401
   tlsprunecmd=/usr/sbin/cyrus tls_prune at=0401
}

*Backend imapd.conf:*
configdirectory: /var/lib/cyrus
proc_path: /run/cyrus/proc
mboxname_lockpath: /run/cyrus/lock
defaultpartition: common
partition-common: /var/spool/cyrus/mail
partition-news: /var/spool

Re: Murder frontend problem

[Andrew Morgan]

On Fri, 5 Jun 2015, Major Csaba wrote:

 There is one more small question: why the proxied LMTP needs to have admins 
 permission on the backend? I thought the proxyservers setting is for this, 
 but LMTP doesn't work without adding my proxy user in the admins...

Play around with lmtp_admins in imapd.conf.  Our mail relays connect to 
our frontends over lmtp and auth as cyr_lmtp.  That authentication is 
proxied to the backends for delivery.

Here is our admin-related config on the backends:

admins: cyrus cyr_proxy
lmtp_admins: cyr_lmtp cyr_proxy
# Only set proxyservers on Standard Murder BACKENDS
proxyservers: cyr_proxy

and on our frontends:

admins: cyrus
lmtp_admins: cyr_lmtp
proxy_authname: cyr_proxy
proxy_password: redacted


Andy

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Murder frontend problem

[Dan White]

On 06/05/15 16:44 +0200, Major Csaba wrote:
Hi,

Thanks for the quick answer.
I managed to get further as I realized I missed a small piece from the 
documentation. My fronted server and master update server is on the 
same machine and I didn't configure the mupdate_* parameter. But as 
I can see, the proxy still has to speak to mupdate when I would like 
to create a new mailbox and the auth info is necessary even if they 
are on the same host.

So, it seems to be a misundersanding of the documentation which is not 
so verbose :)
I added the mupdate_* parameters (pointing to the host itself) and it 
is working fine now.

There is one more small question: why the proxied LMTP needs to have 
admins permission on the backend? I thought the proxyservers setting 
is for this, but LMTP doesn't work without adding my proxy user in the 
admins...

On your backend, you should set 'lmtp_admins: murderproxy', rather than
specifying it as an admin, which limits its security impact.

With imap, the frontend proxy 'authenticates' as the user connecting to the
front end, which gains the permissions of the connecting user (on the
backend). E.g. you should see log entries on your backend with a successful
imap select which appears to be authenticating as the end user (e.g.
jsm...@domain.com).

lmtp may not proxy authenticate at all. If it does, you could specify
*that* user (e.g., the 'mail' account on your frontend) in the backend's
lmtp_admin, but I'm not sure that gains you much security wise.

Referencing syslog on the backend is the best way to flesh this out.

-- 
Dan White

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: single instance message store problem.

[John McMonagle]

On Saturday 30 May 2015 9:34:44 AM Michael Neumann wrote:
 Am 30.05.2015 um 13:43 schrieb John McMonagle:
  
  Michael
  Thanks that works.
  
  Now maps in /etc/aliases are not working.
  I have aliases in ldap that are working.
  
  Read http://www.postfix.org/LOCAL_RECIPIENT_README.html
  Sounds like need local_recipient_maps.
  It already existed and has hash:/etc/aliases.
  
  These are may maps:
  #maps
  alias_maps = hash:/etc/aliases
  alias_database = hash:/etc/aliases
  virtual_alias_maps = hash:/etc/postfix/virtual,
  ldap:/etc/postfix/ldapdistlist.cf,
  ldap:/etc/postfix/ldapvirtual.cf,
  ldap:/etc/postfix/ldap-aliases.cf
  #relocated_maps = hash:/etc/postfix/relocated
  transport_maps = ldap:/etc/postfix/ldaptransport.cf
  #virtual_mailbox_maps = $virtual_alias_maps
  local_recipient_maps = $virtual_alias_maps, $alias_maps,
  hash:/etc/postfix/local-accounts
  
  Just add hash:/etc/aliases to virtual_alias_maps ?
 
 Yes, we have it there too. Please consider that only mail address
 aliases will work, a pipe to a command, that is normally possible in
 /etc/aliases, wont work anymore (because you redefined the local_transport).
 
 Best regards
 
Thanks

Adding hash:/etc/aliases to virtual_alias_maps worked :-)
Do not have any code in aliases.


John

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: single instance message store problem.

[John McMonagle]

On Friday 29 May 2015 3:30:29 PM Michael Neumann wrote:
 Am 29.05.2015 um 22:19 schrieb Ken Murchison:
  
  Also, are you sure that your MTA isn't breaking up the recipients into
  separate LMTP transactions?  lmtpd can/will only do single instance
  store for recipients that are part of the same transaction (MAIL FROM,
  RCPT TO, DATA sequence).  If the recipients are sent one at a time with
  duplicate DATA commands, there are treated as distinct messages.
 
 That is the problem, postfix default local transport does only one at a
 time, that breaks single instance storage. We had to redefine it to:
 local_transport = lmtp:unix:/lmtp/lmtp
 Please read about the consequences in postconf manual.
 
 Hope that helps
 
Michael

Thanks that works.

Now maps in /etc/aliases are not working.
I have aliases in ldap that are working.

Read http://www.postfix.org/LOCAL_RECIPIENT_README.html
Sounds like need local_recipient_maps.
It already existed and has hash:/etc/aliases.

These are may maps:
#maps
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
virtual_alias_maps =  hash:/etc/postfix/virtual,
ldap:/etc/postfix/ldapdistlist.cf,
ldap:/etc/postfix/ldapvirtual.cf,
ldap:/etc/postfix/ldap-aliases.cf
#relocated_maps = hash:/etc/postfix/relocated
transport_maps = ldap:/etc/postfix/ldaptransport.cf
#virtual_mailbox_maps = $virtual_alias_maps
local_recipient_maps = $virtual_alias_maps, $alias_maps, 
hash:/etc/postfix/local-accounts

Just add hash:/etc/aliases to virtual_alias_maps ?

John


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: single instance message store problem.

[Michael Neumann]

Am 30.05.2015 um 13:43 schrieb John McMonagle:
 
 Michael
 Thanks that works.
 
 Now maps in /etc/aliases are not working.
 I have aliases in ldap that are working.
 
 Read http://www.postfix.org/LOCAL_RECIPIENT_README.html
 Sounds like need local_recipient_maps.
 It already existed and has hash:/etc/aliases.
 
 These are may maps:
 #maps
 alias_maps = hash:/etc/aliases
 alias_database = hash:/etc/aliases
 virtual_alias_maps = hash:/etc/postfix/virtual,
 ldap:/etc/postfix/ldapdistlist.cf,
 ldap:/etc/postfix/ldapvirtual.cf,
 ldap:/etc/postfix/ldap-aliases.cf
 #relocated_maps = hash:/etc/postfix/relocated
 transport_maps = ldap:/etc/postfix/ldaptransport.cf
 #virtual_mailbox_maps = $virtual_alias_maps
 local_recipient_maps = $virtual_alias_maps, $alias_maps,
 hash:/etc/postfix/local-accounts
 
 Just add hash:/etc/aliases to virtual_alias_maps ?

Yes, we have it there too. Please consider that only mail address
aliases will work, a pipe to a command, that is normally possible in
/etc/aliases, wont work anymore (because you redefined the local_transport).

Best regards
-- 
Michael Neumann

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: single instance message store problem.

[Nic Bernstein]

On 05/29/2015 03:30 PM, Michael Neumann wrote:

Am 29.05.2015 um 22:19 schrieb Ken Murchison:

Also, are you sure that your MTA isn't breaking up the recipients into
separate LMTP transactions?  lmtpd can/will only do single instance
store for recipients that are part of the same transaction (MAIL FROM,
RCPT TO, DATA sequence).  If the recipients are sent one at a time with
duplicate DATA commands, there are treated as distinct messages.

That is the problem, postfix default local transport does only one at a
time, that breaks single instance storage. We had to redefine it to:
local_transport = lmtp:unix:/lmtp/lmtp
Please read about the consequences in postconf manual.

Hope that helps


Michael is right on the money here.  I hadn't noticed that you had 
defined mailbox_transport rather than local_transport.


This is our typical postfix configuration in support of singleinstancestore:

   local_recipient_maps = ldap:/etc/postfix/local_recipient_map.ldap,
ldap:/etc/postfix/alias_map.ldap
   local_transport = lmtp:unix:/mailstore/lib/imap/socket/lmtp
   local_destination_recipient_limit = 300
   local_destination_concurrency_limit = 5

The warnings Michael was referring to are covered in 
http://www.postfix.org/LOCAL_RECIPIENT_README.html


Specifically, if one overrides the local_transport, it's important that 
one also properly qualifies all local_recipients (quoting):


   Problem: you don't use the default Postfix local(8)
   http://www.postfix.org/local.8.html delivery agent for domains
   matching $mydestination
   http://www.postfix.org/postconf.5.html#mydestination,
   $inet_interfaces
   http://www.postfix.org/postconf.5.html#inet_interfaces, or
   $proxy_interfaces
   http://www.postfix.org/postconf.5.html#proxy_interfaces. For
   example, you redefined the local_transport
   http://www.postfix.org/postconf.5.html#local_transport setting in
   main.cf http://www.postfix.org/postconf.5.html.

   Solution: your local_recipient_maps
   http://www.postfix.org/postconf.5.html#local_recipient_maps
   setting needs to specify a database that lists all the known user
   names or addresses for that delivery agent.

In our case we most often use LDAP for this, as shown above, but 
whatever DB is best for you, etc.


Cheers,
-nic

--
Nic Bernstein n...@onlight.com
Onlight llc.  www.onlight.com
219 N. Milwaukee St., Ste. 2A v. 414.272.4477
Milwaukee, Wisconsin  53202   f. 414.290.0335


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: single instance message store problem.

[John McMonagle]

On Friday, May 29, 2015 04:19:42 PM Ken Murchison wrote:
 On 05/29/2015 03:19 PM, John McMonagle wrote:
  On Friday 29 May 2015 2:08:58 PM you wrote:
   On 05/29/2015 01:48 PM, John McMonagle wrote:
Having trouble getting single instance message store to work.



Running debian jessie with cyrus imap 2.4.17+caldav~beta10-18 postfix

2.11.3-1.



Am following instructions in:



https://cyrusimap.org/docs/cyrus-imapd/2.4.17/install-configure.php



master.cf has:



lmtp unix - - n - - lmtp



main.cf has:



mailbox_transport = lmtp:unix:/var/run/cyrus/socket/lmtp



local_destination_recipient_limit = 200



local_destination_concurrency_limit = 5



imap.conf has



singleinstancestore: 1



lmtpsocket: /var/run/cyrus/socket/lmtp



Delivery works but each is an independent file.



Any suggestions?



John
   
   John,
   
   At the risk of sounding insulting, have you checked the link count (via
   
   'ls -l') on these duplicate copies? For example, here I have part of my
   
   Inbox:
   
   
   
   -rw--- 1 cyrus cyrus 5821 2013-07-09 12:26 17201.
   
   -rw--- 6 cyrus cyrus 36816 2013-07-09 14:22 17202.
   
   -rw--- 1 cyrus cyrus 17305 2013-07-09 14:43 17203.
   
   -rw--- 3 cyrus cyrus 44952 2013-07-09 14:53 17204.
   
   -rw--- 3 cyrus cyrus 44942 2013-07-09 14:56 17205.
   
   -rw--- 1 cyrus cyrus 1395 2013-07-09 15:00 17206.
   
   
   
   The 2nd, 4th  5th messages have multiple hard-links (second column),
   
   which is how singleinstancestore does its thing.
   
   
   
   Cheers,
   
   -nic
  
  No problem
  
  Yes I checked.
  
  -rw--- 1 cyrus mail 3919 May 29 10:35 145026.
  
  -rw--- 1 cyrus mail 87189663 May 29 12:34 cyrus.squat
  
  drwx-- 2 cyrus mail 204800 May 29 13:15 Sent
  
  -rw--- 1 cyrus mail 2116 May 29 13:15 145027.
  
  The 2 mails should have had links.
 
 Are the Message-IDs of both 145026 and 27 the same?  I doubt it given
 the 1k+ diff in size and the almost 3hr diff in delivery time.
 
 Also, are you sure that your MTA isn't breaking up the recipients into
 separate LMTP transactions?  lmtpd can/will only do single instance
 store for recipients that are part of the same transaction (MAIL FROM,
 RCPT TO, DATA sequence).  If the recipients are sent one at a time with
 duplicate DATA commands, there are treated as distinct messages.
 
 Finally, lmtpd can't do single instancestore for mailboxes on different
 filesystems, unless newer fs allow this now.

Did not mean they were the same message, just that they were mails sent to 
more than one person that should have been linked.

It's not in production yet so need to create the emails.
I set up sub domain, mx record etc for testing.

Log has one email to jo...@nmail.advocap.org, mi...@nmail.advocap.org and 
jam...@nmail.advocap.org
All I removed was a lot of imaps entries.

John





mail.log.gz
Description: application/gzip

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: single instance message store problem.

[Nic Bernstein]

On 05/29/2015 02:19 PM, John McMonagle wrote:


On Friday 29 May 2015 2:08:58 PM you wrote:

 On 05/29/2015 01:48 PM, John McMonagle wrote:

 

  Having trouble getting single instance message store to work.

 

  Running debian jessie with cyrus imap 2.4.17+caldav~beta10-18 postfix

  2.11.3-1.

 

  Am following instructions in:

 

  https://cyrusimap.org/docs/cyrus-imapd/2.4.17/install-configure.php

 

  master.cf has:

 

  lmtp unix - - n - - lmtp

 

  main.cf has:

 

  mailbox_transport = lmtp:unix:/var/run/cyrus/socket/lmtp

 

  local_destination_recipient_limit = 200

 

  local_destination_concurrency_limit = 5

 

  imap.conf has

 

  singleinstancestore: 1

 

  lmtpsocket: /var/run/cyrus/socket/lmtp

 

  Delivery works but each is an independent file.

 

  Any suggestions?

 

  John

 

 



 John,

 At the risk of sounding insulting, have you checked the link count (via

 'ls -l') on these duplicate copies? For example, here I have part of my

 Inbox:



 -rw--- 1 cyrus cyrus 5821 2013-07-09 12:26 17201.

 -rw--- 6 cyrus cyrus 36816 2013-07-09 14:22 17202.

 -rw--- 1 cyrus cyrus 17305 2013-07-09 14:43 17203.

 -rw--- 3 cyrus cyrus 44952 2013-07-09 14:53 17204.

 -rw--- 3 cyrus cyrus 44942 2013-07-09 14:56 17205.

 -rw--- 1 cyrus cyrus 1395 2013-07-09 15:00 17206.



 The 2nd, 4th  5th messages have multiple hard-links (second column),

 which is how singleinstancestore does its thing.



 Cheers,

 -nic





No problem

Yes I checked.

-rw--- 1 cyrus mail 3919 May 29 10:35 145026.

-rw--- 1 cyrus mail 87189663 May 29 12:34 cyrus.squat

drwx-- 2 cyrus mail 204800 May 29 13:15 Sent

-rw--- 1 cyrus mail 2116 May 29 13:15 145027.

The 2 mails should have had links.

john



John,
Okay, I had hoped you'd already checked that.

How about this; have you got the duplicatesuppression disabled?  If you 
disable duplicatesuppression, you'll also lose singleinstancestore, as 
it relies upon the duplicate.db to perform its work.


Cheers,
-nic

--
Nic Bernstein n...@onlight.com
Onlight, Inc. www.onlight.com
1442 N Farwell Ave., Suite 600v. 414.272.4477
Milwaukee, Wisconsin  53202

attachment: nic.vcf
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus