Re: LMTP said: 550-Mailbox unknown or you do not have authorization to see it

[Ezsra McDonald]

Sebastian,
Thank you for the response.

I have never heard of this tool but it looks interesting. I will give it a
try.

Will let you all know if I find anything.

-Ez


On Thu, Oct 15, 2020 at 9:28 AM Sebastian Hagedorn 
wrote:

>
> Am 15.10.20 um 15:49 schrieb Ezsra McDonald:
> > I wonder if there is a way to test LMTP manually to verify LMTP can see
> > the imap accounts? I have not done much with LMTP because it always
> > worked for us in the past.
>
> My favorite tool for mail delivery testing is swaks. You can test LMTP
> this way:
>
> swaks --to YOUR-TEST-USER --socket /var/lib/imap/socket/lmtp --protocol
> LMTP
>
> --
> .:.Sebastian Hagedorn - Weyertal 121 (Gebäude 133), Zimmer 2.02.:.
>  .:.Regionales Rechenzentrum (RRZK).:.
>.:.Universität zu Köln / Cologne University - ✆ +49-221-470-89578.:.
>
>

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: LMTP said: 550-Mailbox unknown or you do not have authorization to see it

[Sebastian Hagedorn]

Am 15.10.20 um 15:49 schrieb Ezsra McDonald:
I wonder if there is a way to test LMTP manually to verify LMTP can see 
the imap accounts? I have not done much with LMTP because it always 
worked for us in the past.


My favorite tool for mail delivery testing is swaks. You can test LMTP 
this way:


swaks --to YOUR-TEST-USER --socket /var/lib/imap/socket/lmtp --protocol LMTP

--
   .:.Sebastian Hagedorn - Weyertal 121 (Gebäude 133), Zimmer 2.02.:.
.:.Regionales Rechenzentrum (RRZK).:.
  .:.Universität zu Köln / Cologne University - ✆ +49-221-470-89578.:.



smime.p7s
Description: S/MIME Cryptographic Signature

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: LMTP said: 550-Mailbox unknown or you do not have authorization to see it

[Ezsra McDonald]

Albert,
Thank you for your response.

LDAP is only used for the Postfix/Imap servers. We do not configure Pam  to
use LDAP. We are using saslauthd.

I wonder if there is a way to test LMTP manually to verify LMTP can see the
imap accounts? I have not done much with LMTP because it always worked for
us in the past.

ldapsearch, testsaslauthd and imtest all tested successfully.

I deleted and recreated my test user's imap account
cm user.testuser
sam user.testuser testuser write
sq user.testuser 100

-Ez

On Wed, Oct 14, 2020 at 4:15 PM Albert Shih  wrote:

> Le 14/10/2020 à 14:30:31-0500, Ezsra McDonald a écrit
> > I am building a new mail server to replace an older EL6 server. The new
> server
> > is Centos 8. I keep getting this response when trying to deliver email
> to a
> > local account stored in LDAP.
> >
> > host mail.example.org[/var/lib/imap/socket/lmtp] said:
> > 550-Mailbox unknown.  Either there is no mailbox associated with this
> > 550-name or you do not have authorization to see it.
> > 550 5.1.1 User unknown (in reply to RCPT TO command))
> >
> > I have tried replacing the new configs with my old working configs from
> the EL6
> > server but they get the same result.
> >
> > a postmap -q against the LDAP table config returns the appropriate
> information.
> > I am wondering if the key is the 'or you do not have authorization to
> see it`
> > part of the message. What exactly does LMTP need to authorize the
> delivery?
> >
> > Enabling verbose logging on LMTP and LDAP did not give any clues.
>
> If you run
>
>   getent passwd
>
> what you got ?
>
> Personnaly I don't run the lmtp against ldap, to risky IMHO, if you got any
> problem with the connection betwen your postfix/cyrus server and the ldap
> server your are going to loose email.
>
> So for me I'm using a script who dump the ldap inside the /etc/passwd, so
> the all account are local.
>
> Regards
>
> --
> Albert SHIH
> Observatoire de Paris
> xmpp: j...@obspm.fr
> Heure local/Local time:
> Wed Oct 14 11:13:14 PM CEST 2020
>

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: LMTP said: 550-Mailbox unknown or you do not have authorization to see it

[Albert Shih]

Le 14/10/2020 à 14:30:31-0500, Ezsra McDonald a écrit
> I am building a new mail server to replace an older EL6 server. The new server
> is Centos 8. I keep getting this response when trying to deliver email to a
> local account stored in LDAP.
>
> host mail.example.org[/var/lib/imap/socket/lmtp] said:
> 550-Mailbox unknown.  Either there is no mailbox associated with this
> 550-name or you do not have authorization to see it.
> 550 5.1.1 User unknown (in reply to RCPT TO command))
>
> I have tried replacing the new configs with my old working configs from the 
> EL6
> server but they get the same result.
>
> a postmap -q against the LDAP table config returns the appropriate 
> information.
> I am wondering if the key is the 'or you do not have authorization to see it`
> part of the message. What exactly does LMTP need to authorize the delivery?
>
> Enabling verbose logging on LMTP and LDAP did not give any clues.

If you run

  getent passwd

what you got ?

Personnaly I don't run the lmtp against ldap, to risky IMHO, if you got any
problem with the connection betwen your postfix/cyrus server and the ldap
server your are going to loose email.

So for me I'm using a script who dump the ldap inside the /etc/passwd, so
the all account are local.

Regards

--
Albert SHIH
Observatoire de Paris
xmpp: j...@obspm.fr
Heure local/Local time:
Wed Oct 14 11:13:14 PM CEST 2020

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Mailshare conversations

[Bron Gondwana]

Sorry for the delay.  Yes, this does seem like the right thing to do.  The 
conversation root being the user is a convenience for a ton of things to the 
point that we're basically using "fake users" for storing related things for a 
business.

But doing the same thing for mailshare would be quite easy I think.  There are 
only a few places where we convert mailbox name to conversations_db path.

Bron.

On Tue, Sep 22, 2020, at 01:00, Thomas Fricker wrote:

> Hello,
> 
> I noticed that Cyrus does not keep track of conversations in mailshare 
> folders. 
> There is a code comment addressing this issue in conversations.c:
> 
> /* only users have conversations.  Later we may introduce the idea of
> * "conversationroot" in the same way we have quotaroot, but for now
> * it's hard-coded as the user */
> 
> Is it still imaginable adding this feature in a future Cyrus version?
> Can you elaborate on a reasonable approach of implementing this on the 
> current codebase?
> My goal is to have 1 conversation.db per mailshare subtree (not per single 
> folder).
> 
> Any ideas?
> 
> Thanks,
> --
> Thomas Fricker

>  

>  

> BlueMind LogoLinkedin 
>   Facebook 
>   Twitter 
>   
> 


> *Thomas Fricker * 
> DEVELOPPEUR 
> +33 (0)5 81 91 55 60
> www.bluemind.net / Blog 

>  [*WEBINAR*] 

> *Migrer sa messagerie vers BlueMind depuis Exchange*

> *24/09/20 - 11h*

> *INSCRIPTION* 
> 

>  

> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
> 
> *Attachments:*
>  * x-disclaimer1569892563-0.png
>  * x-disclaimer1569892563-1.png
>  * x-disclaimer1569892563-2.png
>  * x-disclaimer1569892563-3.png
>  * x-disclaimer1569892563-4.png

--
  Bron Gondwana, CEO, Fastmail Pty Ltd
  br...@fastmailteam.com


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: sync_client: IOERROR: fetching subscriptions

[Jean Charles Delépine]

Jean Charles Delépine  écrivait (wrote) :

> 2020-10-06T10:51:45.070636+02:00 cyrus-3.0.8  cyrus/imap[2714758]: IOERROR: 
> fetching subscriptions for user1

user1.sub didn't have correct tab line termination. Certainly my fault
sometime in the past.

Jean Chales Delépine

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: sync_clients bails out - failed to order folders correctly

[ellie timoney]

Hi Konrad,

> do_folders: failed to order folders correctly

Do you have "improved_mboxlist_sort" enabled in your imapd.conf?  My guess is 
you don't.

> every time on a specific mailbox of a specific user.

It sounds like this user has two mailboxes where one is an exact substring of 
another, and the difference starts with a  character earlier than your 
hierarchy separator in ascii order (space and hyphen are the usual culprits).  
e.g. Something like this can cause a variety of problems:

user.ellie.foo
user.ellie.foo-etc
user.ellie.foo.bar

Note the "-" sorts earlier than the hierarchy separator, and so even though 
"bar" is a child of "foo" and should be processed right after "foo", "foo-etc" 
sorts earlier than it and gets in the way...

I guess if this only started happening in the last days, then one of these 
folders must have been created recently.

If this is the cause, the quick workaround is to find the offending folder pair 
and ask the user to rename one of them.  They could replace the space or hyphen 
with an underscore ("_") and this will be fine.

The proper fix is to turn on "improved_mboxlist_sort", but note that the 
documentation says this SHOULD NOT be turned on on a live system, and provides 
instructions for converting the mailboxes.db and subscription databases with 
the service down.  I believe the most up to date documentation for this process 
is here:
https://www.cyrusimap.org/imap/developer/thoughts/improved_mboxlist_sort.html

Hope this helps,

ellie

On Wed, 30 Sep 2020, at 11:03 PM, Konrad Mauz wrote:
> Hello list,
> 
> I had a working replication between to cyrus 3.0 servers running.
> 
> The sync_client is started once a day.
> 
> In the last days the sync_client bails out with 
> 
> do_folders: failed to order folders correctly
> 
> every time on a  specific mailbox of a specific user.
> 
> I can't see that there is a problem with the mailbox ( cyr_expire and 
> squatter are running correctly ).
> 
> How can I get the sync_client work again?
> 
> thanks in advance and kind regards,
> 
> Konrad
> -- 
> Konrad Mauz - Rechenzentrum HTWG Konstanz
> Tel: +497531206472 - Fax: +497531206153
> E-Mail: km...@htwg-konstanz.de
> Web: http://www.htwg-konstanz.de/rz.html
> 
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
>

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Replication : IMAP_PROTOCOL_ERROR Protocol error

[Jean Charles Delépine]

Jean Charles Delépine  écrivait (wrote) :

> Jean Charles Delépine  écrivait (wrote) :
> 
> > Hello,
> > 
> > I'm on the way to migrate one quite big murder config with Cyrus IMAP
> > 3.0.8-Debian-3.0.8-6+deb10u4
> > to Cyrus IMAP 3.2.3-Debian-3.2.3-1~bpo10+1.
> > 
> > My plan is to replicate 3.0.8's backends on 3.2.3 ones. This plan has work
> > before for 2.5
> > to 3.0. migration.
> > 
> > Il can replicate empty mailboxes. So the conf (attached) seems ok.
> > But if the mailbox isn't empty here's the result (completes traces 
> > attached) :
> 
> The conf might not be _that_ ok.
> 
> If the option conversation is set to 1 and I create new mailboxes, ond
> send mails to this mailboxes, no sync of these mailboxes is possible.
> 
> If I remove the option 'conversation: 1', any new populated mailboxes can 
> be sync.
> 
> So, the problem seems to be in this option.
> 
> But even after removing "conversation: 1" and zeroed conversation indexes
> (ctl_conversationsdb -z), the old "bad" mailboxes can't be synced. Even
> removing their .conversations and .counters files doesn't help.
> 
> Can I, and how can I,  get rid of those conversation indexes in order to have
> my mailboxes being "as if there never been conversation" ?

After removing "conversations: 1" option AND restarted the serveur,
ctl_conversationsdb -z did correctly remove conversations indexes and
the replication finally works fine.

I did have to revert xapian usage (which needs conversations) and switch
back to squat indexes.

Jean Charles Delépine

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Replication : IMAP_PROTOCOL_ERROR Protocol error

[Jean Charles Delépine]

ellie timoney  écrivait (wrote) :

> On Thu, 24 Sep 2020, at 1:44 AM, Jean Charles Delépine wrote:
> > Is this a known problem corrected after 3.0.9 ?
> 
> Off the top of my head I no longer remember, but the current release in the 
> 3.0 series is 3.0.14.  I'd suggest, if you haven't already, that you look in 
> the release notes from 3.0.8-3.0.14 and see if anything looks relevant.  
> They're here: 
> https://www.cyrusimap.org/imap/download/release-notes/3.0/index.html

I didn't find anything relevant.

> We don't generally recommend in-place upgrades between series (so, upgrading 
> in-place from 3.0.8 to 3.2.3 is not something we'd recommend).  
> Within-series, an in-place upgrade ought to be safe -- but please check the 
> release notes carefully for extra steps/considerations you may need to make, 
> depending on your deployment.  I think you'll probably want to upgrade your 
> 3.0 systems in place as far forward as you can (while staying 3.0), and then 
> use the replication strategy to upgrade to 3.2 after that.

I just do that. My test server is now using 3.0.14 (self build debian package) :

>1601118406>APPLY MAILBOX %(UNIQUEID m8lfz12835tr5y3dfucebk95 MBOXNAME 
>user.testes2 SYNC_CRC 2393559122 SYNC_CRC_ANNOT 4164967045 LAST_UID 2 
>HIGHESTMODSEQ 4 RECENTUID 0 RECENTTIME 0 LAST_APPENDDATE 1601117744 
>POP3_LAST_LOGIN 0 POP3_SHOW_AFTER 0 XCONVMODSEQ 4 UIDVALIDITY 1601117689 
>PARTITION default ACL "testes2lrswipkxtecdan  " OPTIONS P RECORD 
>(%(UID 1 MODSEQ 4 LAST_UPDATED 1601118406 FLAGS (\Expunged) INTERNALDATE 
>1601117744 SIZE 2890 GUID 6f160d7026f4adbaebb2d0941c6398272a8692da ANNOTATIONS 
>(%(ENTRY /vendor/cmu/cyrus-imapd/thrid USERID NIL VALUE fee5d4912d9da6a8))) 
>%(UID 2 MODSEQ 3 LAST_UPDATED 1601118406 FLAGS () INTERNALDATE 1601117744 SIZE 
>2890 GUID 6f160d7026f4adbaebb2d0941c6398272a8692da ANNOTATIONS (%(ENTRY 
>/vendor/cmu/cyrus-imapd/thrid USERID NIL VALUE fee5d4912d9da6a8)
<16011184061601118406>EXIT
<1601118406http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Replication : IMAP_PROTOCOL_ERROR Protocol error

[ellie timoney]

On Thu, 24 Sep 2020, at 1:44 AM, Jean Charles Delépine wrote:
> Is this a known problem corrected after 3.0.9 ?

Off the top of my head I no longer remember, but the current release in the 3.0 
series is 3.0.14.  I'd suggest, if you haven't already, that you look in the 
release notes from 3.0.8-3.0.14 and see if anything looks relevant.  They're 
here: https://www.cyrusimap.org/imap/download/release-notes/3.0/index.html

We don't generally recommend in-place upgrades between series (so, upgrading 
in-place from 3.0.8 to 3.2.3 is not something we'd recommend).  Within-series, 
an in-place upgrade ought to be safe -- but please check the release notes 
carefully for extra steps/considerations you may need to make, depending on 
your deployment.  I think you'll probably want to upgrade your 3.0 systems in 
place as far forward as you can (while staying 3.0), and then use the 
replication strategy to upgrade to 3.2 after that.
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Replication : IMAP_PROTOCOL_ERROR Protocol error

[Jean Charles Delépine]

Jean Charles Delépine  écrivait (wrote) :

> Hello,
> 
> I'm on the way to migrate one quite big murder config with Cyrus IMAP
> 3.0.8-Debian-3.0.8-6+deb10u4
> to Cyrus IMAP 3.2.3-Debian-3.2.3-1~bpo10+1.
> 
> My plan is to replicate 3.0.8's backends on 3.2.3 ones. This plan has work
> before for 2.5
> to 3.0. migration.
> 
> Il can replicate empty mailboxes. So the conf (attached) seems ok.
> But if the mailbox isn't empty here's the result (completes traces attached) :

The conf might not be _that_ ok.

If the option conversation is set to 1 and I create new mailboxes, ond
send mails to this mailboxes, no sync of these mailboxes is possible.

If I remove the option 'conversation: 1', any new populated mailboxes can 
be sync.

So, the problem seems to be in this option.

But even after removing "conversation: 1" and zeroed conversation indexes
(ctl_conversationsdb -z), the old "bad" mailboxes can't be synced. Even
removing their .conversations and .counters files doesn't help.

Can I, and how can I,  get rid of those conversation indexes in order to have
my mailboxes being "as if there never been conversation" ?

Sincerly,
  Jean charles Delépine

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Sync failing. Possible db corruption

[Michael Sofka]

Problem resolved, or resolving.  This version of Cyrus may be old, but 
it's been rock solid and compared to previous versions I've had no 
problems with sync_client dying. It turns out the behavior I was seeing 
is normal recover behavior.  There are still errors I have not seen 
before, but they appear to be from turning on verbose logging, and the 
synchronization being out of step from the previous errors.  As people 
check mailboxes, it is catching up.


Mike

On 9/15/20 4:20 PM, Michael Sofka wrote:
I have confirmed that running sync_client on individual mailboxes 
works, as does running sync on the accumulated log files.  But rolling 
replication is not working.   After sync_client is restarted there is 
a burst of sync activity on the accumulated log, then nothing. The new 
log file continues to accumulate, sync_client continues to run, but is 
not processing anything so far as I can determine.




--
--
Michael D. Sofka   sof...@rpi.edu
ITI Software Architect,   Email, TeX, Epistemology
Rensselaer Polytechnic Institute, Troy, NY.  http://www.rpi.edu/~sofkam/


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: How to find mailbox name from id

[Ismaël Tanguy]

On 14/09/2020 11:43, Ismaël Tanguy has written:

Hello,

cyrus-imapd-3.0.7-16.el8.src.rpm installed on Centos 8.2

An user fails to receive an email from  a mailing list of 1000 
subscribers.
 From SMTP server, I found the uid of the mail and look for it in the 
IMAP store logs.


While the delivery is OK for the most part, I found 5 mistakes :

# grep 280aa530-f9a0-c1fd-b100-51201aa6b2fc /var/log/maillog | grep 
-v Delivered


dupelim: eliminated duplicate message to i21mdycbfokfu8yq6jul3hvt id 
<280aa530-f9a0-c1fd-b100-51201aa6b2fc>

[...]

Hello,
 good question!


I answer myself.
With this command :

$ cyr_dbtool /var/lib/imap/mailboxes.db twoskip show | grep 
i21mdycbfokfu8yq6jul3hvt


But I found two mailboxes :

user.e22009540  %(A %(e22009540 lrswipkxtecdan) I 
i21mdycbfokfu8yq6jul3hvt P splitmeta V 1599471223 F 1 M 1599471222)
user.e22009563  %(A %(e22009563 lrswipkxtecdan) I 
i21mdycbfokfu8yq6jul3hvt P splitmeta V 1599471224 F 1 M 1599471223)


Is it the expected behavior?

I thought Unique ID was unique for one mailbox..




btw... on Cyrus 2.4 these logs told me the mailbox name too. On Cyrus 
3.x this info disappears. Ouch :(


If you delete the mailbox the syslog shows the mailbox name and the 
uniqueid together, only dupelim lacks of this info.


Cheers
Marco

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Sync failing. Possible db corruption

[Michael Sofka]

I have confirmed that running sync_client on individual mailboxes works, 
as does running sync on the accumulated log files.  But rolling 
replication is not working.   After sync_client is restarted there is a 
burst of sync activity on the accumulated log, then nothing.  The new 
log file continues to accumulate, sync_client continues to run, but is 
not processing anything so far as I can determine.


Sep 15 15:59:17 imap-be6 cyrus/sync_client[22370]: SYNCNOTICE: replica 
uid:7451 modseq:19935 last_updated:1600023598 internaldate:1600023598 
flags:( \Seen)
Sep 15 15:59:17 imap-be6 cyrus/sync_client[22370]: inefficient 
replication (1487 > 1484) user.jins4.Sent
Sep 15 15:59:18 imap-be6 cyrus/sync_client[22370]: inefficient 
replication (182427 > 182409) user.newelj
Sep 15 15:59:18 imap-be6 cyrus/sync_client[22370]: SYNCERROR: only 
exists on replica user.newelj 121180 
(a27c1dffed0326fb930b072fd85f04a8c2cbdf24)
Sep 15 15:59:23 imap-be6 cyrus/sync_client[22370]: inefficient 
replication (5598 > 5571) user.berryk3
Sep 15 15:59:23 imap-be6 cyrus/sync_client[22370]: SYNCNOTICE: record 
mismatch with replica: user.berryk3 more recent on master
Sep 15 15:59:23 imap-be6 cyrus/sync_client[22370]: SYNCNOTICE: master 
uid:2614 modseq:5636 last_updated:1600191010 internaldate:1599245383 
flags:(\Seen)


From the previous day's log I see:

Sep 15 06:25:11 imap-be6 cyrus/sync_client[29665]: MAILBOX received NO 
response: System I/O error
Sep 15 06:25:11 imap-be6 cyrus/sync_client[29665]: do_folders(): update 
failed: user.chauhs2 'The remote Server(s) denied the operation'
Sep 15 06:25:11 imap-be6 cyrus/sync_client[29665]: seen_db: user swankd 
opened /var/lib/cyrus/user/s/swankd.seen
Sep 15 06:25:11 imap-be6 cyrus/sync_client[29665]: seen_db: user edena2 
opened /var/lib/cyrus/user/e/edena2.seen
Sep 15 06:25:11 imap-be6 cyrus/sync_client[29665]: seen_db: user searsm2 
opened /var/lib/cyrus/user/s/searsm2.seen
Sep 15 06:25:11 imap-be6 cyrus/sync_client[29665]: USER received NO 
response: System I/O error
Sep 15 06:25:11 imap-be6 cyrus/sync_client[29665]: Error in do_sync(): 
bailing out! The remote Server(s) denied the operation
Sep 15 06:25:11 imap-be6 cyrus/sync_client[29665]: Processing sync log 
file /var/lib/cyrus/sync/log-29665 failed: The remote Server(s) denied 
the operation
Sep 15 06:25:11 imap-be6 cyrus/sync_client[29665]: Reprocessing sync log 
file /var/lib/cyrus/sync/log-29665



I have not seen a repeat of the DBERROR, and have since restarted cyrus 
services.  So I'm not sure a ctl_cyrusdb -r is needed, or even it it 
would fix things.




--
--
Michael D. sofkasof...@rpi.edu
ITI Software Architect,   Email, TeX, Epistemology
Rensselaer Polytechnic Institute, Troy, NY.http://www.rpi.edu/~sofkam/


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Sync failing. Possible db corruption

[Michael Sofka]

Okay, these errors may be admin error. In running the old logs I 
specified -u, instead of -m.


I am still concerned about the DBERROR, but those did not appear after I 
restarted cyrus.  And rolling replication is still not working.


Mike

On 9/15/20 10:49 AM, Michael Sofka wrote:


Sep 15 10:21:18 imap-be6 cyrus/sync_client[19816]: Inbox missing on 
master for MAILBOX user.travi

t
Sep 15 10:21:18 imap-be6 cyrus/sync_client[19816]: Inbox missing on 
master for MAILBOX user.sutto

e
Sep 15 10:21:18 imap-be6 cyrus/sync_client[19816]: Inbox missing on 
master for MAILBOX user.maran

v


On the replication server I have:

Sep 15 10:34:40 imap-be5 cyrus/syncserver[31550]: cannot unlink 
/var/lib/cyrus/user/m/MAILBOX use

r.linr7.mboxkey: No such file or directory
Sep 15 10:34:40 imap-be5 cyrus/syncserver[31550]: cannot unlink 
/var/lib/cyrus/user/m/MAILBOX use

r.kjells.mboxkey: No such file or directory
Sep 15 10:34:40 imap-be5 cyrus/syncserver[31550]: cannot unlink 
/var/lib/cyrus/user/m/MAILBOX use

r.ngang.mboxkey: No such file or directory

When I try to process the sync/log- files.


Any pointers or help appreciated.  IMAP is functioning as far as I can 
tell, and only the replication process is failing.


Thank you,

MIke



--
--
Michael D. Sofka   sof...@rpi.edu
ITI Software Architect,   Email, TeX, Epistemology
Rensselaer Polytechnic Institute, Troy, NY.  http://www.rpi.edu/~sofkam/


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: How to find mailbox name from id

[Marco]

On 14/09/2020 11:43, Ismaël Tanguy has written:

Hello,

cyrus-imapd-3.0.7-16.el8.src.rpm installed on Centos 8.2

An user fails to receive an email from  a mailing list of 1000 subscribers.
 From SMTP server, I found the uid of the mail and look for it in the 
IMAP store logs.


While the delivery is OK for the most part, I found 5 mistakes :

# grep 280aa530-f9a0-c1fd-b100-51201aa6b2fc /var/log/maillog | grep -v 
Delivered


dupelim: eliminated duplicate message to i21mdycbfokfu8yq6jul3hvt id 
<280aa530-f9a0-c1fd-b100-51201aa6b2fc>

[...]

Hello,
 good question!

btw... on Cyrus 2.4 these logs told me the mailbox name too. On Cyrus 
3.x this info disappears. Ouch :(


If you delete the mailbox the syslog shows the mailbox name and the 
uniqueid together, only dupelim lacks of this info.


Cheers
Marco

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

RE: OutLook support

[Deborah Pickett]

Hi Andrea,

> What I found really annoying is "non-read" receipts.

Ouch, I did not know about this!  I will file it away in case it ever bites me.

> > Rarely, Outlook will decide that a folder is local-only
> > [...]
> Does it at least shows the folder is local?
> Then I could train the users to stop using it and call me.

Haha, no, no one notices until (if you're lucky) someone else who sees
the same folder in a shared account notices they can't see It, or (if you're 
not)
the computer housing these unexpected local folders dies, and then
you discover that the messages have been absent for months.



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: OutLook support

[Andrea Venturoli]

On 2020-09-05 03:11, Deborah Pickett wrote:

On 2020-09-04 18.30, Andrea Venturoli wrote:

What's the status of interoperability today?
Will OL 2013 work reliably with CyrusIMAP 3.0? 3.2?
What about newer versions of OL? 


Hi Andrea,


Hello.



I can offer anecdata of interoperability between Cyrus 3.0.x and Outlook 
2016 from ten months of experience.


Thank you very much for sharing.



Like Outlook in general, it works fine, until it doesn't. With ~40 users 
with mailboxes up to 15 GB in size, I've had to help users start with 
fresh Outlook profiles about half a dozen times when their Outlook data 
file got corrupted.


Nothing new here: when I said pre-2013 version worked, what I meant is a 
~50 users installation required 2-3 reconfigurations per month.

I can live with that.
Of course using ThunderBird would mean max. 2-3 per *year* or even less, 
but it's up to the customer to choose.





Some specific observations:

Like any IMAP account in Outlook, you lose the ability to tag messages 
with colour.  All you can do is flag or unflag a message.


Fine, I guess.



Outlook does not handle the \Deleted flag well, and it has a habit of 
showing messages which are \Deleted as if they are still present 
(especially in Drafts). I recommend configuring all clients sharing an 
account with Outlook to purge folders early and often.


True, I remember that.
Showing deleted messages with an overstroke and forcing the user to 
"delete twice" was puzzling most of my users.




Outlook respects special-use folders about as well as other IMAP 
clients, which is to say, not super-well.  If you try to change which 
folder is the Trash folder using cyradm, Outlook may not notice the 
change.


Fine as long as I can set this up for a new account.
Then I can always delete/recreate.



64-bit Outlook is needed if both (a) your data file is > 2 GB, and (b) 
you want search indexing to work.  With 32-bit Outlook, search will 
silently miss messages.


:-O
Thanks for pointing this out!



Outlook is inefficient at IMAP synchronization.  If you have large 
mailboxes, it can spend a lot of its time synchronizing subscribed 
folders.  There are ways of winnowing the list of folders that Outlook 
tries to sync during Send And Receive, and this helps a bit.


Now I remember this too...




Outlook by default wants to send read receipts.


What I found really annoying is "non-read" receipts.
In case a user loses view of a shared folders, OL will send a "non-read" 
receipt for any unread message, potentially queueing thousands of 
messages in a few seconds!
This could be turned off in older versions; on post-2010 the option is 
either not there or hidden.

>:-<



To sync Outlook with Cyrus contacts, calendars and tasks, the free 
Outlook CalDav Synchronizer add-in works well (but sync rules are 
tedious to set up, and I haven't found a way to deploy preconfigured 
sync rules to users).  Outlook tries to disable the add-in because it 
slows down startup, but you can prevent this with a registry setting.


Thanks again.
I don't think I'll need this soon, but it's useful to know.



Rarely, Outlook will decide that a folder is local-only, and any 
messages moved into that folder will stop syncing with Cyrus.  The only 
fix I have found is to create a new Outlook profile (and then hunt for 
lost messages to drag back under synchronized folders).


Yes, I remember this too :-<
Does it at least shows the folder is local?
Then I could train the users to stop using it and call me.



These corruptions seem to occur more often in "Other Users" and "Shared 
Folders", but this might just be because said folders are huge (> 4 GB) 
in my company.


I hope I don't need to use this, at least for now.



Of course, it's unlikely that any of these irritations will ever get 
fixed by Microsoft.


That's sure.



In the long term, I am considering migrating my 
users from Outlook to Thunderbird or webmail.


All the users of my servers are using TB (along with mobile devices and 
occasionally RoundCube).
Now I'm evaulating installing a new instance at a customer who is 
already using OutLook (so to overcome the limitations of their ISP, 
which only offers POP3).

I'll suggest they move to ThunderBird but:
a) habits are hard to lose;
b) some use a management software who might require OL :-(



 bye & Thanks
av.

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: OutLook support

[Deborah Pickett]

On 2020-09-04 18.30, Andrea Venturoli wrote:

What's the status of interoperability today?
Will OL 2013 work reliably with CyrusIMAP 3.0? 3.2?
What about newer versions of OL? 


Hi Andrea,

I can offer anecdata of interoperability between Cyrus 3.0.x and Outlook 
2016 from ten months of experience.


Like Outlook in general, it works fine, until it doesn't. With ~40 users 
with mailboxes up to 15 GB in size, I've had to help users start with 
fresh Outlook profiles about half a dozen times when their Outlook data 
file got corrupted.


Some specific observations:

Like any IMAP account in Outlook, you lose the ability to tag messages 
with colour.  All you can do is flag or unflag a message.


Outlook does not handle the \Deleted flag well, and it has a habit of 
showing messages which are \Deleted as if they are still present 
(especially in Drafts). I recommend configuring all clients sharing an 
account with Outlook to purge folders early and often.


Outlook respects special-use folders about as well as other IMAP 
clients, which is to say, not super-well.  If you try to change which 
folder is the Trash folder using cyradm, Outlook may not notice the 
change.  That said, it's not as bad as Windows 10 mail, which flat out 
refuses to respect special-use folder tags at all.


64-bit Outlook is needed if both (a) your data file is > 2 GB, and (b) 
you want search indexing to work.  With 32-bit Outlook, search will 
silently miss messages.


Outlook is inefficient at IMAP synchronization.  If you have large 
mailboxes, it can spend a lot of its time synchronizing subscribed 
folders.  There are ways of winnowing the list of folders that Outlook 
tries to sync during Send And Receive, and this helps a bit.


Outlook by default wants to send read receipts.  These invisible 
messages can silently clog up the outbox, preventing any messages from 
being sent.  It is a very good idea to disable sending read receipts 
entirely; I have set a Group Policy object to do this. I've had to use a 
free program called MFCMAPI to manually delete queued read receipts that 
accidentally got through.


To sync Outlook with Cyrus contacts, calendars and tasks, the free 
Outlook CalDav Synchronizer add-in works well (but sync rules are 
tedious to set up, and I haven't found a way to deploy preconfigured 
sync rules to users).  Outlook tries to disable the add-in because it 
slows down startup, but you can prevent this with a registry setting.


Rarely, Outlook will decide that a folder is local-only, and any 
messages moved into that folder will stop syncing with Cyrus.  The only 
fix I have found is to create a new Outlook profile (and then hunt for 
lost messages to drag back under synchronized folders).


Renaming a folder, or moving it somewhere else, in Outlook usually 
works.  Rarely, it'll make the folder local-only.  A slightly more 
robust way of renaming a folder is to make a new folder with the new 
name, and move the contents of the old folder over to the new folder.  I 
have seen this fail too.


These corruptions seem to occur more often in "Other Users" and "Shared 
Folders", but this might just be because said folders are huge (> 4 GB) 
in my company.


Of course, it's unlikely that any of these irritations will ever get 
fixed by Microsoft.  In the long term, I am considering migrating my 
users from Outlook to Thunderbird or webmail.


Hope these data points are useful.

Deborah


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: IOERROR: conversations_audit on store

[ellie timoney]

Hi Frederik,

>From the source, it looks like this error is reported when the internal counts 
>in the conversations data are out of sync.

I don't have any insight as to what might cause this, but I think you should be 
able to run "ctl_conversationsdb" with the "-R" argument for the affected 
users, to recalculate their counts.

Cheers,

ellie

On Mon, 24 Aug 2020, at 6:22 PM, Frederik Himpe via Info-cyrus wrote:
> I am using Cyrus 3.2.2 on Debian Buster 10 (package from buster-
> backports).
> 
> I am having lots of errors similar to these in the logs:
> 
> cyrus/imaps[24052]: IOERROR: conversations_audit on store: 
> /var/lib/cyrus/user/e/example.conversations Bff62cd0852db22e1 0 
> (1452110 1 0 0 () ((1 1452109 1 1 0)) () foobar. 0 () 1452109)
> 
> The same problem happend with Cyrus 3.0.x and seems to happen often for
> specific users, usually with big mailboxes (one of them 40GB+). When
> upgrading to 3.2.2, I ran
> 
> reconstruct -V max
> ctl_conversationsdb -b -r
> quota -f
> dav_reconstruct -a
> 
> as per the instructions given in the documentation.
> 
> 
> /var/lib/cyrus and /var/spool/cyrus are on ext4, Linux kernel at this
> moment is 5.4.48 (also happend with different kernels).
> 
> imapd.conf:
> configdirectory: /var/lib/cyrus
> proc_path: /run/cyrus/proc
> mboxname_lockpath: /run/cyrus/lock
> defaultpartition: default
> partition-default: /var/spool/cyrus/mail
> partition-news: /var/spool/cyrus/news
> altnamespace: yes
> unixhierarchysep: yes
> lmtp_downcase_rcpt: yes
> admins: cyradm
> allowanonymouslogin: no
> popminpoll: 1
> autocreate_quota: 500
> umask: 077
> sieveusehomedir: false
> sievedir: /var/spool/sieve
> httpmodules: caldav carddav
> hashimapspool: true
> allowplaintext: yes
> sasl_mech_list: PLAIN
> sasl_saslauthd_path: /var/spool/postfix/var/run/saslauthd/mux
> sasl_minimum_layer: 128
> sasl_pwcheck_method: saslauthd
> sasl_auto_transition: no
> tls_server_cert: /etc/letsencrypt/live/ai.vub.ac.be/fullchain.pem
> tls_server_key:  /etc/letsencrypt/live/ai.vub.ac.be/privkey.pem
> tls_client_ca_dir: /etc/ssl/certs
> tls_session_timeout: 1440
> tls_ciphers: TLSv1.2:+TLSv1:+HIGH:!aNULL:@STRENGTH
> tls_versions: tls1_0 tls1_1 tls1_2
> lmtpsocket: /run/cyrus/socket/lmtp
> idlesocket: /run/cyrus/socket/idle
> notifysocket: /run/cyrus/socket/notify
> syslog_prefix: cyrus
> delete_mode: delayed
> expunge_mode: delayed
> sync_log: on
> sync_log_channels: squatter
> conversations: 1
> 
> cyrus.conf:
> START {
>   recover cmd="/usr/sbin/cyrus ctl_cyrusdb -r"
>   
>   idled   cmd="idled"
> }
> SERVICES {
>   imapcmd="imapd -U 30" listen="imap" prefork=0 maxchild=100
>   imaps   cmd="imapd -s -U 30" listen="imaps" prefork=0 
> maxchild=100
>   pop3cmd="pop3d -U 30" listen="pop3" prefork=0 maxchild=50
>   pop3s   cmd="pop3d -s -U 30" listen="pop3s" prefork=0 
> maxchild=50
>   httpcmd="httpd -U 30 -p 256" listen="localhost:8008" 
> prefork=0 
> maxchild=100
>   lmtpunixcmd="lmtpd" 
> listen="/var/spool/postfix/var/run/cyrus/socket/lmtp" prefork=0 
> maxchild=20
>   sieve   cmd="timsieved" listen="sieve" prefork=0 maxchild=100
>   notify  cmd="notifyd" listen="/run/cyrus/socket/notify" 
> proto="udp" 
> prefork=1
> }
> EVENTS {
>   checkpoint  cmd="/usr/sbin/cyrus ctl_cyrusdb -c" period=30
>   delprunecmd="/usr/sbin/cyrus expire -E 3" at=0401
>   tlsprunecmd="/usr/sbin/cyrus tls_prune" at=0415
>   deleteprune cmd="/usr/sbin/cyrus expire -E 4 -D 28" at=0430
>   expungeprunecmd="/usr/sbin/cyrus expire -E 4 -X 28" at=0445
>   squatter1   cmd="/usr/bin/nice -n 19 /usr/sbin/cyrus squatter -i" 
> period=120
>   squattera   cmd="/usr/bin/nice -n 19 /usr/sbin/cyrus squatter -i" 
> at=0517
> }
> 
> Any idea what is going wrong?
> 
> -- 
> Frederik Himpe 
> 
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
>

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cryus 3.2.2: Remote does not support ANNOTATEMORE

[Rainer Ruprechtsberger]

On 8/12/20 11:59 AM, Marco wrote:
[...]
> I suggest to use the Cyrus::IMAP::Admin version provided by the Cyrus
> IMAP server. So, when you upgrade Cyrus IMAP, upgrade the Perl admin
> utility too and use that.

Thanks a lot. My problem came from installing cyrus from the debian
backports repository and I indeed forgot to pull the cyrus-admin package
from there as well.

/r

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cryus 3.2.2: Remote does not support ANNOTATEMORE

[Marco]

Il 12/08/2020 11:16, Rainer Ruprechtsberger ha scritto:

Hi,
not sure if it is only after the upgrade to 3.2.2 since the features are
not that much in use. But I did set 'expire' and 'sharedseen' before.

Now I get 'Remote does not support ANNOTATEMORE' using 'mboxconfig ..
expire' or 'sharedseen' or even 'info'.

[...]
Hello Rainer,

  if I well remember, from 3.2 version Cyrus IMAP does not support the 
ANNOTATEMORE experimental extension by default no more. It provides now 
the support for METADATA only.


Sharedseen, comment, expire, etc are all annotations.

I have a lot of problems using Cyrus::IMAP::Admin between different 
Cyrus IMAP server versions.


I suggest to use the Cyrus::IMAP::Admin version provided by the Cyrus 
IMAP server. So, when you upgrade Cyrus IMAP, upgrade the Perl admin 
utility too and use that.



Alternatively, you can try to configure your Cyrus IMAP 3.2 with

annotation_enable_legacy_commands: 1

but I never verified this chance. Upgrade the client to use METADATA 
could be the better and definitive choice.


Cheers
Marco

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Restore message date from "Date:" field

[Nic Bernstein]

Gionatan,
I believe the tool you're looking for is 'mbtool'  From the man page:

   DESCRIPTION
   mbtool  is  a  tool  for performing various actions on the indexes 
of a
   list of mailboxes. The only actions currently supported are  -t,  
which
   will  normalize the internaldate time stamp of each record in the 
index
   to GMT, and -r which will create a new unique ID for each mailbox.
   ...
   -t Normalize  internaldate on all index records of all listed 
mail‐
  boxes to match the Date: header if theyâre off by  more  than 
 a
  day,  which  can  be used to fix up a mailbox which has been 
re‐
  stored from backup and lost its internaldate information.
   ...
   EXAMPLES*mbtool -t*  user.jsmith

   Normalize |internaldate| on all index records in /user.jsmith/.

   Working  on  user.jsmith...
   0001:  Tue,  08  Jul  2014  16:45:18  -0500  =>  Mon,  07  Jul  2014  
20:44:18  +
   0002:  Tue  Jul  08  16:45:13  CDT  2013  =>  Fri,  30  Aug  2013  
19:46:03  +
   <...>

http://www.cyrusimap.org/imap/reference/manpages/systemcommands/mbtool.html?highlight=mbtool

Cheers,
    -nic

On 7/31/20 6:39 AM, Gionatan Danti wrote:

Hi all,
I just noticed the dates of some old emails are wrongly displayed on 
roundcube webmail.


In short, the list view shows the filesystem date of the affected 
messages (ie: mtime of u.1 file), rather than what is found in the 
"Date:" header field


These were emails migrated from an old system, but I vaguely remember 
I had some issue at the time which I solved with some combination of 
rsync+imapsync.


Can "reconstruct" be used to repopulate the index file with the 
correct date from "Date:" field? If not, what I can do to solve the 
issue? I already tried "reconstruct -u user@domain -x -f -r -G", but 
with no avail.


Thanks.



--
Nic Bernstein   n...@nicbernstein.com
https://www.nicbernstein.com
https://www.linkedin.com/in/nic-b-26577a178/


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: backupd and sync_client Syntax error

[Marco]

Ouch, sorry, ignore this.

I suppose to have found the problem.

My goal is to sync from Cyrus IMAP 2.4 to 3.2, and to 3.2 to a backupd host.
I see it works just starting by hand the sync_client in Cyrus IMAP 2.4, 
meanwhile in the middle host the rolling mode is sufficient with 
sync_log_chain.


But I had a bad Cyrus IMAP 2.4 with some mailboxes which had references 
in mailboxes.db only, and not in file system and metadata.


Solving above error, then sync_client from 3.2 IMAP host to backupd host 
works.


I don't know why I don't see errors in sync_client from 2.4 to 3.2 already.

Cheers
Marco


Il 22/07/2020 11:51, Marco ha scritto:

Hello,

  I would ask an help with a new replication error in backup channel.
I have this never-ending loop error:

2020-07-22T11:35:16.212126+02:00 tst-msg03 cyrus/sync_client[22465]: 
MAILBOXES received NO response: IMAP_PROTOCOL_BAD_PARAMETERS Bad parameters
2020-07-22T11:35:16.212185+02:00 tst-msg03 cyrus/sync_client[22465]: 
Error in do_sync(): bailing out! Syntax error in parameters
2020-07-22T11:35:16.212228+02:00 tst-msg03 cyrus/sync_client[22465]: 
Processing sync log file /var/lib/imap/sync/bck/log-run failed: Syntax 
error in parameters
2020-07-22T11:35:16.216962+02:00 tst-msg03 cyrus/sync_client[22465]: 
Reprocessing sync log file /var/lib/imap/sync/bck/log-run



The "log-run" file ends with:

APPEND "example.com!user.gianni^ferromagnetic.  Cartella Molto Spaziosa"
MAILBOX "example.com!user.gianni^ferromagnetic.  Cartella Molto Spaziosa"
MAILBOX "example.com!user.gianni^ferromagnetic.  Cartella Molto Spaziosa"
APPEND "example.com!user.gianni^ferromagnetic.  Cartella Molto 
Spaziosa.Sottocartella"
MAILBOX "example.com!user.gianni^ferromagnetic.  Cartella Molto 
Spaziosa.Sottocartella"
MAILBOX "example.com!user.gianni^ferromagnetic.  Cartella Molto 
Spaziosa.Sottocartella"

APPEND example.com!user.gianni^ferromagnetic.10
MAILBOX example.com!user.gianni^ferromagnetic.10
MAILBOX example.com!user.gianni^ferromagnetic.10
APPEND example.com!user.gianni^ferromagnetic.Abbcc
MAILBOX example.com!user.gianni^ferromagnetic.Abbcc
MAILBOX example.com!user.gianni^ferromagnetic.Abbcc
APPEND example.com!user.gianni^ferromagnetic.Abc
MAILBOX example.com!user.gianni^ferromagnetic.Abc
MAILBOX example.com!user.gianni^ferromagnetic.Abc
MAILBOX "example.com!user.gianni^ferromagnetic.Cartella 1"
MAILBOX "example.com!user.gianni^ferromagnetic.Cartella 1"
APPEND "example.com!user.gianni^ferromagnetic.Cartella 1.Cestino"
MAILBOX "example.com!user.gianni^ferromagnetic.Cartella 1.Cestino"
MAILBOX "example.com!user.gianni^ferromagnetic.Cartella 1.Cestino"
APPEND "example.com!user.gianni^ferromagnetic.Cartella 1.Cestino.Test"
MAILBOX "example.com!user.gianni^ferromagnetic.Cartella 1.Cestino.Test"
MAILBOX "example.com!user.gianni^ferromagnetic.Cartella 1.Cestino.Test"
MAILBOX "example.com!user.gianni^ferromagnetic.Cartella 1.Spam"
MAILBOX "example.com!user.gianni^ferromagnetic.Cartella 1.Spam"
APPEND "example.com!user.gianni^ferromagnetic.Cartella 1.ma,cia"
MAILBOX "example.com!user.gianni^ferromagnetic.Cartella 1.ma,cia"
MAILBOX "example.com!user.gianni^ferromagnetic.Cartella 1.ma,cia"
MAILBOX "example.com!user.gianni^ferromagnetic.Cartella 1.ma,cia.Archivio"
MAILBOX "example.com!user.gianni^ferromagnetic.Cartella 1.ma,cia.Archivio"
APPEND "example.com!user.gianni^ferromagnetic.Cartella Spaziosa"
MAILBOX "example.com!user.gianni^ferromagnetic.Cartella Spaziosa"
MAILBOX "example.com!user.gianni^ferromagnetic.Cartella Spaziosa"

The log-run has the same last-modified date of the first syslog error 
message of syntax error.



Other strange folder names I created are

MAILBOX example.com!user.baraccone.prova:prova:
MAILBOX example.com!user.baraccone.prova:prova:
MAILBOX example.com!user.baraccone.prova^1
MAILBOX example.com!user.baraccone.prova^1


I suspect that some folder name breaks the backup replication...

Thank you very much

Cheers
Marco

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus




Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: sync_client Sieve from 2.4.20 to 3.2.2 issue

[Marco]

For Sieve the new path "C" 
is reported by mbpath, but sync_client replicates the sieve scripts 
elsewhere.


Sorry, I would mean "For Sieve the new path with "H" is reported by 
mbpath, but sync_client replicates the scripts elsewhere (in the path 
with "S").


Many many thanks for every help

Cheers
Marco

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: assertion failed: imap/mboxevent.c: 743: filled_params(type, event)

[Matthew Schumacher]

Bug filed:

https://github.com/cyrusimap/cyrus-imapd/issues/3115

On 7/17/20 1:52 AM, Matthew Schumacher wrote:

HI Ellie,

I agree that it's probably a bug.  I'll open a github issue.

I'll report back with the issue number.

Thanks,
Matt

On 7/16/20 5:47 PM, ellie timoney wrote:

Hi,

I've seen something like this before, and my gut feel is that this is 
going to turn out to be a bug in Cyrus.


I think what's happening is that, somewhere in Cyrus, an event is 
being generated with a type that's supposed to contain a 
serverAddress field, but the serverAddress field is not being 
initialised.


Before a generated event actually gets sent out to the notifier, we 
validate that all the required parameters have been filled 
("filled_params()"), and the fatal assertion is telling us that this 
one has not been, even though it should have been.


Would you like to open a GitHub issue at 
https://github.com/cyrusimap/cyrus-imapd/issues ?  If you don't, I 
will.  But if I do it, then you won't get automatic notifications 
about updates, so if you can, it's better if you do it.  Feel free to 
just paste your previous email as the issue text. :)


Cheers,

ellie

On Thu, Jul 16, 2020, at 11:23 AM, Matthew Schumacher wrote:

I'm trying to use external notifications on 3.2.2 but it doesn't work.
If I define

event_notifier: external
notify_external: /usr/cyrus/bin/cyrus_notify
event_groups: access
event_extra_params: clientAddress timestamp service

Then the imapd thread dies with this assertion:

Jul 15 18:01:54 snow imaps[28108]: Cannot notify event Login: missing
parameters: serverAddress clientAddress
Jul 15 18:01:54 snow imaps[28108]: Fatal error: Internal error:
assertion failed: imap/mboxevent.c: 743: filled_params(type, event)

If I remove event_groups and event_extra_params then notify never calls
my external script and notify breaks.

If I define "event_groups: access" and omit event_extra_params then I'm
back to:

Jul 15 18:14:26 snow imaps[28934]: Cannot notify event Login: missing
parameters: serverAddress
Jul 15 18:14:26 snow imaps[28934]: Fatal error: Internal error:
assertion failed: imap/mboxevent.c: 743: filled_params(type, event)

Anyone know where this serverAddress is coming from and how to fix it?

schu

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: assertion failed: imap/mboxevent.c: 743: filled_params(type, event)

[Matthew Schumacher]

HI Ellie,

I agree that it's probably a bug.  I'll open a github issue.

I'll report back with the issue number.

Thanks,
Matt

On 7/16/20 5:47 PM, ellie timoney wrote:

Hi,

I've seen something like this before, and my gut feel is that this is going to 
turn out to be a bug in Cyrus.

I think what's happening is that, somewhere in Cyrus, an event is being 
generated with a type that's supposed to contain a serverAddress field, but the 
serverAddress field is not being initialised.

Before a generated event actually gets sent out to the notifier, we validate that all the 
required parameters have been filled ("filled_params()"), and the fatal 
assertion is telling us that this one has not been, even though it should have been.

Would you like to open a GitHub issue at 
https://github.com/cyrusimap/cyrus-imapd/issues ?  If you don't, I will.  But 
if I do it, then you won't get automatic notifications about updates, so if you 
can, it's better if you do it.  Feel free to just paste your previous email as 
the issue text. :)

Cheers,

ellie

On Thu, Jul 16, 2020, at 11:23 AM, Matthew Schumacher wrote:

I'm trying to use external notifications on 3.2.2 but it doesn't work.
If I define

event_notifier: external
notify_external: /usr/cyrus/bin/cyrus_notify
event_groups: access
event_extra_params: clientAddress timestamp service

Then the imapd thread dies with this assertion:

Jul 15 18:01:54 snow imaps[28108]: Cannot notify event Login: missing
parameters: serverAddress clientAddress
Jul 15 18:01:54 snow imaps[28108]: Fatal error: Internal error:
assertion failed: imap/mboxevent.c: 743: filled_params(type, event)

If I remove event_groups and event_extra_params then notify never calls
my external script and notify breaks.

If I define "event_groups: access" and omit event_extra_params then I'm
back to:

Jul 15 18:14:26 snow imaps[28934]: Cannot notify event Login: missing
parameters: serverAddress
Jul 15 18:14:26 snow imaps[28934]: Fatal error: Internal error:
assertion failed: imap/mboxevent.c: 743: filled_params(type, event)

Anyone know where this serverAddress is coming from and how to fix it?

schu

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: assertion failed: imap/mboxevent.c: 743: filled_params(type, event)

[ellie timoney]

Hi,

I've seen something like this before, and my gut feel is that this is going to 
turn out to be a bug in Cyrus.

I think what's happening is that, somewhere in Cyrus, an event is being 
generated with a type that's supposed to contain a serverAddress field, but the 
serverAddress field is not being initialised.

Before a generated event actually gets sent out to the notifier, we validate 
that all the required parameters have been filled ("filled_params()"), and the 
fatal assertion is telling us that this one has not been, even though it should 
have been.

Would you like to open a GitHub issue at 
https://github.com/cyrusimap/cyrus-imapd/issues ?  If you don't, I will.  But 
if I do it, then you won't get automatic notifications about updates, so if you 
can, it's better if you do it.  Feel free to just paste your previous email as 
the issue text. :)

Cheers,

ellie

On Thu, Jul 16, 2020, at 11:23 AM, Matthew Schumacher wrote:
> I'm trying to use external notifications on 3.2.2 but it doesn't work.   
> If I define
> 
> event_notifier: external
> notify_external: /usr/cyrus/bin/cyrus_notify
> event_groups: access
> event_extra_params: clientAddress timestamp service
> 
> Then the imapd thread dies with this assertion:
> 
> Jul 15 18:01:54 snow imaps[28108]: Cannot notify event Login: missing 
> parameters: serverAddress clientAddress
> Jul 15 18:01:54 snow imaps[28108]: Fatal error: Internal error: 
> assertion failed: imap/mboxevent.c: 743: filled_params(type, event)
> 
> If I remove event_groups and event_extra_params then notify never calls 
> my external script and notify breaks.
> 
> If I define "event_groups: access" and omit event_extra_params then I'm 
> back to:
> 
> Jul 15 18:14:26 snow imaps[28934]: Cannot notify event Login: missing 
> parameters: serverAddress
> Jul 15 18:14:26 snow imaps[28934]: Fatal error: Internal error: 
> assertion failed: imap/mboxevent.c: 743: filled_params(type, event)
> 
> Anyone know where this serverAddress is coming from and how to fix it?
> 
> schu
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: www.cyrusimap.org is down

[ellie timoney]

CMU report that it's been fixed, and it does indeed seem to be working again 
for me now.  The usual caveats about DNS propagation probably apply.

I don't know why the entry disappeared, and if the cause was something 
automated I suppose it might disappear again... I guess we'll keep an eye on it 
and see what unfolds.

Cheers,

ellie

On Wed, Jul 15, 2020, at 10:39 AM, ellie timoney wrote:
> It's hosted on GitHub Pages.  There's supposed to be a DNS CNAME entry at 
> "www.cyrusimap.org" pointing to "cyrusimap.github.io", but it seems to have 
> disappeared. /sigh
> 
> Without that DNS entry, even going directly to "cyrusimap.github.io" isn't 
> working, because GitHub Pages wants to redirect to our custom domain... which 
> is currently broken.
> 
> Thanks for the alert, we'll try and get hold of someone at CMU to get this 
> fixed (or maybe they're watching the list and see this thread).
> 
> Cheers,
> 
> ellie
> 
> On Wed, Jul 15, 2020, at 12:52 AM, Sebastian Hagedorn wrote:
>> Hi,
>> 
>> currently the hostname www.cyrusimap.org cannot be resolved.
>> cyrusimap.org still works, but it appears to point to Github, from where
>> there is a redirect to www.cyrusimap.org
>> -- 
>>.:.Sebastian Hagedorn - Weyertal 121 (Gebäude 133), Zimmer 2.02.:.
>> .:.Regionales Rechenzentrum (RRZK).:.
>>   .:.Universität zu Köln / Cologne University - ✆ +49-221-470-89578.:.
>> 
>> 
>> 
>> Cyrus Home Page: http://www.cyrusimap.org/
>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>> To Unsubscribe:
>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
>> 
>> *Attachments:*
>>  * smime.p7s
> 
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: www.cyrusimap.org is down

[ellie timoney]

It's hosted on GitHub Pages.  There's supposed to be a DNS CNAME entry at 
"www.cyrusimap.org" pointing to "cyrusimap.github.io", but it seems to have 
disappeared. /sigh

Without that DNS entry, even going directly to "cyrusimap.github.io" isn't 
working, because GitHub Pages wants to redirect to our custom domain... which 
is currently broken.

Thanks for the alert, we'll try and get hold of someone at CMU to get this 
fixed (or maybe they're watching the list and see this thread).

Cheers,

ellie

On Wed, Jul 15, 2020, at 12:52 AM, Sebastian Hagedorn wrote:
> Hi,
> 
> currently the hostname www.cyrusimap.org cannot be resolved.
> cyrusimap.org still works, but it appears to point to Github, from where
> there is a redirect to www.cyrusimap.org
> -- 
>.:.Sebastian Hagedorn - Weyertal 121 (Gebäude 133), Zimmer 2.02.:.
> .:.Regionales Rechenzentrum (RRZK).:.
>   .:.Universität zu Köln / Cologne University - ✆ +49-221-470-89578.:.
> 
> 
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
> 
> *Attachments:*
>  * smime.p7s

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: lmtp Trying to unput wrong character

[Sebastian Hagedorn]

Am 09.07.20 um 11:47 Uhr schrieb Stephan:
> Hello,
> 
> I am having trouble with a mail stuck in the queue, it can't get
> delivered because lmtpd has some problem, it says:
> 
> lmtp: FATAL: Trying to unput wrong character
> 
> I'm not sure what to do about this. I found the error message in
> prot_ungetc() which is sitting in lib/prot.c but don't know what this
> actually does. If I interpret this correctly, it gets called in
> pushmsg() in lmtpengine.c and the comment says that there is a carriage
> return that should be put back, which apparently doesn't work.
> 
> Any hints ?

If you check the list archives you will find that this happens
occasionally. IMO your best bet is to delete the mail from the queue.

Cheers,
Sebastian
<>

smime.p7s
Description: S/MIME Cryptographic Signature

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: backupd and sync_client IOERROR

[Marco]

Hi Ellie,

On 08/07/2020 06:23, ellie timoney has written:

Oh that's very curious.  It suggests that something about the mailbox contents 
is causing it to fail under -A but not under -u.  I was hoping it would just be 
the dot in the name, cause something like that should be fairly easy to 
reproduce and debug.  But it's sounding like it'll be more difficult!

It might be worth running "mbexamine" over some of these mailboxes -- both some 
that work correctly, and the ones that are misbehaving -- and see if any interesting 
patterns emerge?  The man page is 
here:https://www.cyrusimap.org/imap/reference/manpages/systemcommands/mbexamine.html


The mailbox which fails is quite large (over 32000 msgs and over 1GB). 
mbexamine produces a very large output for all mails. I can see that it 
doesn't fail: it terminates without errors and with 0 exit status.


Even switches -c or -q terminates without errors. All quotas checks are 
correct.



Does it fail in the same way replicating to a normal replica, instead of a 
backup server?

At this moment I don't have a normal replica server yet. I'll try...

Thanks. I think this will be important for figuring out whether the problem is 
replication generally, or backupd specific.


I understand. As soon as I can, I'll setup a normal replica.

Thank you

Cheers
Marco

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: backupd and sync_client IOERROR

[ellie timoney]

Hi Marco,

On Tue, Jul 7, 2020, at 12:17 AM, Marco wrote:
> I copied the content of the failing mailbox into another mailbox with a 
> name without dots:

Oh that's very curious.  It suggests that something about the mailbox contents 
is causing it to fail under -A but not under -u.  I was hoping it would just be 
the dot in the name, cause something like that should be fairly easy to 
reproduce and debug.  But it's sounding like it'll be more difficult!

It might be worth running "mbexamine" over some of these mailboxes -- both some 
that work correctly, and the ones that are misbehaving -- and see if any 
interesting patterns emerge?  The man page is here: 
https://www.cyrusimap.org/imap/reference/manpages/systemcommands/mbexamine.html

> > Does it fail in the same way replicating to a normal replica, instead of a 
> > backup server?
> 
> At this moment I don't have a normal replica server yet. I'll try...

Thanks. I think this will be important for figuring out whether the problem is 
replication generally, or backupd specific.

Cheers,

ellie

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: backupd and sync_client IOERROR

[Marco]

Hello,

On 03/07/2020 05:08, ellie timoney has written:

I notice that the users that worked correctly with "sync_client -A" don't have 
dots in their address localparts.  If you create another user that also has a dot, does 
it fail under -A in the same way?


I copied the content of the failing mailbox into another mailbox with a 
name without dots:


springst...@example.com

and the result is the same:

# sync_client -A -n bck -z -v

USER springst...@example.com
MAILBOX example.com!user.springsteen
Error from do_user(springst...@example.com): bailing out!


2020-07-06T15:31:51.538935+02:00 tst-msg03-bck backupd[2234574]: login: 
tst-msg03.example.com [10.102.102.102] cyr_backup LOGIN User logged in
2020-07-06T15:31:51.576283+02:00 tst-msg03-bck backupd[2234574]: 
creating sql_db /var/spool/cyr_backup/s/springsteen@example.com_t8Yjcb.index
2020-07-06T15:36:57.552290+02:00 tst-msg03 cyrus/sync_client[17488]: 
MESSAGE received NO response: IMAP_PROTOCOL_ERROR Protocol error
2020-07-06T15:36:57.753294+02:00 tst-msg03 cyrus/sync_client[17488]: 
do_folders(): update failed: example.com!user.springsteen 'Bad protocol'
2020-07-06T15:36:57.758121+02:00 tst-msg03 cyrus/sync_client[17488]: 
IOERROR: do_user_main: Bad protocol for springst...@example.com to [no 
channel] (tst-msg03-bck.example.com)
2020-07-06T15:36:57.765868+02:00 tst-msg03 cyrus/sync_client[17488]: 
Error in do_user(springst...@example.com): bailing out!


Instead, if I do:

# sync_client -n bck -z -v -u springst...@uc.csi.it

USER springst...@example.com
MAILBOX example.com!user.springsteen
MAILBOX example.com!user.springsteen.#splitconversations
MAILBOX example.com!user.springsteen.ALLARMI
MAILBOX example.com!user.springsteen.ALLARMI.John
[...]

the program replicates as well and without any errors.


Does it fail in the same way replicating to a normal replica, instead of a 
backup server?


At this moment I don't have a normal replica server yet. I'll try...

Thank you very much

Cheers
Marco

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: UID THREAD REFS US-ASCII ALL slow / stalls forever on one folder.

[Jesper Schmitz Mouridsen via Info-cyrus]

Hi

Thanks for the debugging hints!

client_timeout sat to 30M and the UID THREAD REFS US-ASCII ALL actually 
completes.
But first after ~10 mins on a CPU: Intel(R) Celeron(R) CPU  N2930  @ 1.83GHz 
(1833.38-MHz K8-class CPU)
and after 69.484 secs on a CPU: Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz (The 
same jail, moved to faster hardware)
63994 messages in around 40k threads.
What could cause the much spent time building the thread data? version is still 
3.2.2, but 3.0.13 shows the same
behavior. The thread output of the slow folder is here FWIW 
https://pastebin.com/PpiJ8X2E

Regards
/Jesper


On 03.07.2020 01.52, ellie timoney wrote:

Hi,

I think I would do something like:

0) set client_timeout to a big value (see below)
1) let the imapd start normally
2) connect to it with a minimal imap client (like imtest or telnet)
3) check logs to see which imapd process id your client is connected to (if 
there's more than one)
4) use the "gdb /path/to/binary pid" invocation to attach gdb to the running 
imap process
5) repeat #4 in a few different ways if it's unable to find symbols
6) set a breakpoint on the cmd_thread function (since that's what handles the 
THREAD command) and then continue
7) back in your imap client, send the UID THREAD REFS US-ASCII ALL" command and 
step through to see what happens

Note that these are not exhaustive steps, more of a "get started, and then react 
accordingly, depending on what you see"

I would also try variations of that THREAD command -- if you add/remove 
options, does it start working?  Can you isolate the problem to a specific 
combination of options?  Does it only happen for some mailboxes?

You probably want to set client_timeout to a big value.  The default is 10 seconds, so the server will 
probably throw your client off while you're reading output from gdb, and then you'll wind up debugging the 
"disconnect an unresponsive client code" instead.  I usually set this to be 30 minutes or so for 
debugging.  For 3.2 and newer, you can spell this as "30m".  For older versions, the value is in 
seconds, so you want "1800".

Permissions may be awkward.  You will need to be the cyrus user (or root) to 
connect gdb to the running imapd, but you will also need permission to read the 
source that it was built from, which is probably not owned by the cyrus user.  
In my case it's under my user account's home directory, so I add the cyrus user 
to my user account's group, and make sure the path to the source directory is 
g+rx (because I don't like running gdb as root).

Cheers,

ellie

On Fri, Jul 3, 2020, at 5:23 AM, Jesper Schmitz Mouridsen via Info-cyrus wrote:

Hi.

I recently upgraded Cyrus to 3.2.2 from 3.0.13.

Now threading "UID THREAD REFS US-ASCII ALL" on one specific folder with
  >50K mails,

makes imapd process use 100 CPU% without any progress. truss[1] or dtrace

shows no output. The process seems totally stuck.

I installed in a FreeBSD 12.1 jail. Any hints beside the online docs of
how to use gdb to

see what  is going one. I could not make imapd run under gdb even with
the -D option

and debug_command setting or reading
https://www.cyrusimap.org/imap/developer/developer-testing.html?highlight=debug

It is fast enough on other folders also with ~50k mails < 3 secs.

Any hints, highly appreciated :-)

[1] https://www.freebsd.org/cgi/man.cgi?truss

Regards

Jesper


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: backupd and sync_client IOERROR

[ellie timoney]

On Wed, Jul 1, 2020, at 11:57 PM, Marco wrote:
> Uhm...

Wow, that's wierd.

I notice that the users that worked correctly with "sync_client -A" don't have 
dots in their address localparts.  If you create another user that also has a 
dot, does it fail under -A in the same way?

Does it fail in the same way replicating to a normal replica, instead of a 
backup server?

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: UID THREAD REFS US-ASCII ALL slow / stalls forever on one folder.

[ellie timoney]

Hi,

I think I would do something like:

0) set client_timeout to a big value (see below)
1) let the imapd start normally
2) connect to it with a minimal imap client (like imtest or telnet)
3) check logs to see which imapd process id your client is connected to (if 
there's more than one)
4) use the "gdb /path/to/binary pid" invocation to attach gdb to the running 
imap process
5) repeat #4 in a few different ways if it's unable to find symbols
6) set a breakpoint on the cmd_thread function (since that's what handles the 
THREAD command) and then continue
7) back in your imap client, send the UID THREAD REFS US-ASCII ALL" command and 
step through to see what happens

Note that these are not exhaustive steps, more of a "get started, and then 
react accordingly, depending on what you see"

I would also try variations of that THREAD command -- if you add/remove 
options, does it start working?  Can you isolate the problem to a specific 
combination of options?  Does it only happen for some mailboxes?

You probably want to set client_timeout to a big value.  The default is 10 
seconds, so the server will probably throw your client off while you're reading 
output from gdb, and then you'll wind up debugging the "disconnect an 
unresponsive client code" instead.  I usually set this to be 30 minutes or so 
for debugging.  For 3.2 and newer, you can spell this as "30m".  For older 
versions, the value is in seconds, so you want "1800".

Permissions may be awkward.  You will need to be the cyrus user (or root) to 
connect gdb to the running imapd, but you will also need permission to read the 
source that it was built from, which is probably not owned by the cyrus user.  
In my case it's under my user account's home directory, so I add the cyrus user 
to my user account's group, and make sure the path to the source directory is 
g+rx (because I don't like running gdb as root).

Cheers,

ellie

On Fri, Jul 3, 2020, at 5:23 AM, Jesper Schmitz Mouridsen via Info-cyrus wrote:
> Hi.
> 
> I recently upgraded Cyrus to 3.2.2 from 3.0.13.
> 
> Now threading "UID THREAD REFS US-ASCII ALL" on one specific folder with 
>  >50K mails,
> 
> makes imapd process use 100 CPU% without any progress. truss[1] or dtrace
> 
> shows no output. The process seems totally stuck.
> 
> I installed in a FreeBSD 12.1 jail. Any hints beside the online docs of 
> how to use gdb to
> 
> see what  is going one. I could not make imapd run under gdb even with 
> the -D option
> 
> and debug_command setting or reading 
> https://www.cyrusimap.org/imap/developer/developer-testing.html?highlight=debug
> 
> It is fast enough on other folders also with ~50k mails < 3 secs.
> 
> Any hints, highly appreciated :-)
> 
> [1] https://www.freebsd.org/cgi/man.cgi?truss
> 
> Regards
> 
> Jesper
> 
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: backupd and sync_client IOERROR

[Marco]

On 19/06/2020 03:01, ellie timoney has written:

Rolling mode only makes incremental updates, so if you're starting from a 
server that already has existing data, you should do the first manual initial 
backups before enabling the rolling mode.


Hello,

 about this error, I retried more times. It seems really reproducible 
all times.


I don't enable rolling mode. At the initial sync if I do:


$ sync_client -A -n bck -z -v
USER d...@example.com
MAILBOX example.com!user.dan
MAILBOX example.com!user.dan.Sent
QUOTA example.com!user.dan
QUOTA example.com!user.dan.Sent
USER d...@example.com
MAILBOX example.com!user.din
MAILBOX example.com!user.din.Spam
MAILBOX example.com!user.din.Trash
QUOTA example.com!user.din
USER d...@example.com
MAILBOX example.com!user.don
QUOTA example.com!user.don
USER utente.archivi...@example.com
MAILBOX example.com!user.utente^archivista
Error from do_user(utente.archivi...@example.com): bailing out!

and the error log is:

2020-07-01T15:42:14.161553+02:00 tst-msg03 cyrus/sync_client[4861]: 
MESSAGE received NO response: IMAP_PROTOCOL_ERROR Protocol error
2020-07-01T15:42:14.296022+02:00 tst-msg03 cyrus/sync_client[4861]: 
do_folders(): update failed: example.com!user.utente^archivista 'Bad 
protocol'
2020-07-01T15:42:14.296220+02:00 tst-msg03 cyrus/sync_client[4861]: 
IOERROR: do_user_main: Bad protocol for utente.archivi...@example.com to 
[no channel] (tst-msg03-bck.example.com)
2020-07-01T15:42:14.296290+02:00 tst-msg03 cyrus/sync_client[4861]: 
Error in do_user(utente.archivi...@example.com): bailing out!



If I repeat the "sync_client -A -n bck -z -v" the error still occurs 
every time.



If I do:

$ sync_client -n bck -z -v -u utente.archivi...@example.com
USER utente.archivi...@example.com
MAILBOX example.com!user.utente^archivista
MAILBOX example.com!user.utente^archivista.Cest
MAILBOX example.com!user.utente^archivista.Posta archiviata
MAILBOX example.com!user.utente^archivista.Posta archiviata.Archivio
[...]

it works without errors.

Uhm...

Thank you again
Cheers
Marco

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cyrus IMAP 3.2.2 released

[Sergey]

On Wednesday 01 July 2020, ellie timoney wrote:

> Yes, please.  If you don't, I will, but then you won't get
> automatic notifications of updates. 

https://github.com/cyrusimap/cyrus-imapd/issues/3090

> Are they appearing in a log somewhere, or is this output
> from an analysis tool you're running?

Second. This is ALT Linux specific part of rpm-build package:
http://git.altlinux.org/gears/r/rpm-build.git

This is a heavily modified part of the original rpm 4.0.4, but
verify-elf is a sh script and it can be used separately.
3 sh scripts from rpm-build is needed: verify-elf, ldd and functions.

-- 
Regards,
Sergey

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cyrus IMAP 3.2.2 (Re: cyrus imap 2.5.5 build in Linux: problem with pcre)

[ellie timoney]

Hi Sergey,

Quoting out of order, because it's a bit easier to explain that way:

> And there is another problem that is not obvious. -lpcreposix is needed
> in perl/imap/Makefile.PL and in perl/sieve/managesieve/Makefile.PL I seems.

This bit sounds a lot like https://github.com/cyrusimap/cyrus-imapd/issues/2629

There's an experimental patch from me in that issue, which seems to have worked 
around the issue for Fedora package maintainers.  But I can't reproduce the 
problem on Debian, so I can't confirm myself that it fixes things correctly.  
More feedback would be great.

> I returned to this question. 3.2.2 still does not find 
> /usr/include/pcre/pcre.h
> 
> $ pkg-config --cflags libpcre
> -I/usr/include/pcre
> 
> Build is successful with CFLAGS="-I/usr/include/pcre" ./configure ...

Cyrus's configure script does not currently use pkg-config to locate libpcre.  
Instead it uses bespoke detection built from AC_CHECK_HEADER, which I guess 
doesn't know to look in /usr/include/pcre.  There might be a generic way for a 
sysadmin to add this path to the paths that AC_CHECK_HEADER checks, but I don't 
know it offhand.  Otherwise, specifying it in CFLAGS like you have done seems 
like the right thing to do.

The issue linked above also touches on the possibility of changing the libpcre 
detection to use pkg-config instead of bespoke detection, in which case this 
wouldn't be necessary, but it has its own complications.

> Except perl/imap: CFLAGS is not used by Makefile.PL.

Aaand this sounds like its own whole separate bug!  I'm not sure offhand what 
the right way to pass this down will be, but it clearly needs doing.  New issue 
here: https://github.com/cyrusimap/cyrus-imapd/issues/3087

Cheers,

ellie

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cyrus IMAP 3.2.2 released

[ellie timoney]

On Tue, Jun 30, 2020, at 10:10 PM, Sergey wrote:
> On Monday 22 June 2020, ellie timoney wrote:
> 
> > The Cyrus team is proud to announce the immediate availability
> > of a new version of Cyrus IMAP: 3.2.2 
>  
> There was a problem with AC_SYS_LARGEFILE. Most likely it was
> there before, but I was no need to use AC_SYS_LARGEFILE.
> Programs for x32 cannot work with large partitions without
> building with _FILE_OFFSET_BITS 64. AC_SYS_LARGEFILE in configure.ac
> usually helps. This mostly helps when building Cyrus IMAP, but not
> for all components.
> 
> without AC_SYS_LARGEFILE:
> 
> $ cat cyrus-imapd-3.2.2-alt0-SS.log | grep "uses non-LFS functions"| wc -l
> 42
> 
> with AC_SYS_LARGEFILE:
> 
> $ cat cyrus-imapd-3.2.2-alt0-SS.log | grep "uses non-LFS functions"| wc -l
> 7
> 
> verify-elf: WARNING: ./usr/lib/libcyrus_imap.so.0.0.0: uses non-LFS 
> functions: statvfs
> verify-elf: WARNING: ./usr/lib/cyrus/restore: uses non-LFS functions: 
> lseek open
> verify-elf: WARNING: ./usr/lib/cyrus/httpd: uses non-LFS functions: 
> __fxstat __xstat
> verify-elf: WARNING: ./usr/lib/cyrus/cyr_backup: uses non-LFS 
> functions: lseek open
> verify-elf: WARNING: ./usr/lib/cyrus/ctl_backups: uses non-LFS 
> functions: lseek open
> verify-elf: WARNING: ./usr/lib/cyrus/backupd: uses non-LFS functions: 
> lseek open
> verify-elf: WARNING: ./usr/lib/libcyrus.so.0.0.0: uses non-LFS 
> functions: fopen
> 
> Put it in a bugtracker on github?

Yes, please.  If you don't, I will, but then you won't get automatic 
notifications of updates.

Where do you see these warnings?  Are they appearing in a log somewhere, or is 
this output from an analysis tool you're running?  I tried searching for 
"verify-elf" but most of the results seem to be about signing binaries, which I 
don't think is related.

Cheers,

ellie

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cyrus IMAP 3.2.2 released

[Sergey]

On Monday 22 June 2020, ellie timoney wrote:

> The Cyrus team is proud to announce the immediate availability
> of a new version of Cyrus IMAP: 3.2.2 
 
There was a problem with AC_SYS_LARGEFILE. Most likely it was
there before, but I was no need to use AC_SYS_LARGEFILE.
Programs for x32 cannot work with large partitions without
building with _FILE_OFFSET_BITS 64. AC_SYS_LARGEFILE in configure.ac
usually helps. This mostly helps when building Cyrus IMAP, but not
for all components.

without AC_SYS_LARGEFILE:

$ cat cyrus-imapd-3.2.2-alt0-SS.log | grep "uses non-LFS functions"| wc -l
42

with AC_SYS_LARGEFILE:

$ cat cyrus-imapd-3.2.2-alt0-SS.log | grep "uses non-LFS functions"| wc -l
7

verify-elf: WARNING: ./usr/lib/libcyrus_imap.so.0.0.0: uses non-LFS functions: 
statvfs
verify-elf: WARNING: ./usr/lib/cyrus/restore: uses non-LFS functions: lseek open
verify-elf: WARNING: ./usr/lib/cyrus/httpd: uses non-LFS functions: __fxstat 
__xstat
verify-elf: WARNING: ./usr/lib/cyrus/cyr_backup: uses non-LFS functions: lseek 
open
verify-elf: WARNING: ./usr/lib/cyrus/ctl_backups: uses non-LFS functions: lseek 
open
verify-elf: WARNING: ./usr/lib/cyrus/backupd: uses non-LFS functions: lseek open
verify-elf: WARNING: ./usr/lib/libcyrus.so.0.0.0: uses non-LFS functions: fopen

Put it in a bugtracker on github?

-- 
Regards, Sergey

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cyrus IMAP 3.2.2 released

[Sergey]

On Tuesday 30 June 2020, ellie timoney wrote:

> > > The Cyrus team is proud to announce the immediate availability of
> > > a new version of Cyrus IMAP: 3.2.2 
> > 
> > Tests have issued a new warning compared to 3.0.x (building in Linux):
 
> > verify-elf: WARNING: ./usr/lib64/libcyrus.so.0.0.0: found executable 
> > STACK entry:
> >   GNU_STACK  0x00 0x   0x 0x00 
> > 0x00 RWE 0x10
> > 
> > Is executable STACK really needed?

> I have no idea what this refers to or where it comes from.  Any
> further information you could provide would be greatly appreciated! 
 
I didn't go investigate detail yet and remove it from binary by
"execstack -c libcyrus.so.0.0.0". Everything seems to be working.
The problem is probably wrong flags for gcc. I can find out it
later maybe.

-- 
Regards, Sergey

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cyrus IMAP 3.2.2 released

[ellie timoney]

I have no idea what this refers to or where it comes from.  Any further 
information you could provide would be greatly appreciated!

Thanks

On Tue, Jun 30, 2020, at 5:14 AM, Sergey wrote:
> On Monday 22 June 2020, ellie timoney wrote:
> 
> > The Cyrus team is proud to announce the immediate availability of a new 
> > version of Cyrus IMAP: 3.2.2
> 
> Tests have issued a new warning compared to 3.0.x (building in Linux):
> 
> verify-elf: WARNING: ./usr/lib64/libcyrus.so.0.0.0: found executable 
> STACK entry:
>   GNU_STACK  0x00 0x   0x 0x00 
> 0x00 RWE 0x10
> 
> Is executable STACK really needed?
> 
> -- 
> Regards,
> Sergey
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
>

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Cyrus IMAP 3.2.2 (Re: cyrus imap 2.5.5 build in Linux: problem with pcre)

[Sergey]

On Friday 16 October 2015, Sergey wrote:

> I wanted to build Cyrus-IMAP with libpcre but it did not success.
> System libpcre-devel package install headers to /usr/include/pcre.
 
I returned to this question. 3.2.2 still does not find /usr/include/pcre/pcre.h

$ pkg-config --cflags libpcre
-I/usr/include/pcre

Build is successful with CFLAGS="-I/usr/include/pcre" ./configure ...
Except perl/imap: CFLAGS is not used by Makefile.PL.

And there is another problem that is not obvious. -lpcreposix is needed
in perl/imap/Makefile.PL and in perl/sieve/managesieve/Makefile.PL I seems.

-- 
Regards,
Sergey

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cyrus IMAP 3.2.2 released

[Sergey]

On Monday 22 June 2020, ellie timoney wrote:

> The Cyrus team is proud to announce the immediate availability of a new 
> version of Cyrus IMAP: 3.2.2

Tests have issued a new warning compared to 3.0.x (building in Linux):

verify-elf: WARNING: ./usr/lib64/libcyrus.so.0.0.0: found executable STACK 
entry:
  GNU_STACK  0x00 0x   0x 0x00 0x00 
RWE 0x10

Is executable STACK really needed?

-- 
Regards,
Sergey

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: cyrus-imap building: sse extension

[Sergey]

On Monday 29 June 2020, ellie timoney wrote:

> If you _want_ to use the hardware CRC32c algorithm

No. I want for Cyrus-IMAP works on systems without SSE4 when it built
on system with SSE4.

> > Can Cyrus-IMAP be running on systems without SSE4 at this case?
> 
> Yep, it'll work just fine.  The hardware CRC32c code will simply not
> be compiled, which, since it isn't used anyway, will have no effect.

Question about running precompiled binary.

> I don't know if it's even possible to implement the same algorithm
> with SSE2

--disable-hardware-CRC32c would be enough In my case. :-)

> Cyrus doesn't actually use it though

--disable-hardware-CRC32c may be required in the future I think.

-- 
Regards,
Sergey

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: cyrus-imap buildin: sse extention

[Anatoli]

Ellie,

I also had the doubt about this feature, though I'd already seen a mention that 
the result of the hw implementation is incompatible (and before your last mail 
completely forgot about it).

Maybe it makes sense to remove its mention (and detection) from configure 
altogether, until it becomes useful? Just to not confuse those building it from 
sources.

Regards,
Anatoli

On 28/6/20 21:29, ellie timoney wrote:
> Hi Sergey,
> 
>> Hardware support:
>>SSE4.2: yes
> 
> This is detected for a hardware implementation of the CRC32c algorithm.  
> Cyrus doesn't actually use it though, because it's not compatible with the 
> existing CRC32 algorithm: i.e. for the same input, it produces a different 
> checksum, which would break everything on a system with pre-existing data.
> 
> If you _want_ to use the hardware CRC32c algorithm on a brand new deployment 
> (and know what you are doing), I believe at this stage you would need to 
> patch Cyrus to use it -- the code is there, but it is never called.
> 
>> Can Cyrus-IMAP be running on systems without SSE4 at this case?
> 
> Yep, it'll work just fine.  The hardware CRC32c code will simply not be 
> compiled, which, since it isn't used anyway, will have no effect.
> 
>> If no, can I set limit to SSE2 ?
> 
> There's currently no configurability for this at all.  I don't know if it's 
> even possible to implement the same algorithm with SSE2.  At the moment I 
> assume that, since it's looking for SSE42 specifically, then that means that 
> the hardware feature it needs is probably only available in SSE42.
> 
> Cheers,
> 
> ellie
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
> 

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Newly arrived mail is marked as "read"

[ellie timoney]

Hi,

I'm not sure, but it kind of sounds like your mailbox's index version is too 
old?

Around 2.4, the storage of the mailbox owner's seen state was moved from the 
seen databases to the cyrus.index.  (i.e. nowadays the seen database only 
stores the seen state for _other_ users who have been given access to a mailbox 
-- which is often nobody).

If your mailbox indexes have not been updated yet, then they won't have a field 
to record the owner's seen state, which might result in the behaviour you're 
seeing?

To update your mailbox indexes to the latest version, you can use the "-V max" 
option with the reconstruct utility.  The upgrade documentation for 3.0 
(https://www.cyrusimap.org/3.0/imap/download/upgrade.html#reconstruct-databases-and-cache)
 mentions that this may take a long time, so maybe you already saw this but 
haven't gotten to that stage yet.

You should definitely take note of the warning on that page though, and 
consider upgrading to at least 3.0.11 rather than 3.0.7.

Hope this is helpful!

Cheers,

ellie

On Fri, Jun 26, 2020, at 10:43 PM, Cheng-Jih Chen wrote:
> I'm in the process of testing the upgrade from a CentOS 6 box running 
> 2.3.16 to a CentOS 8 box running 3.0.7.
> 
> The upgrade has been less troublesome than expected, but I'm seeing a 
> strange problem with newly received mail being marked as "read" even 
> though they have not been opened by the client.  This is the case for 
> both mail sent locally as well as mail received externally (Gmail).
> 
> Can anyone point me in the right direction on where to look to fix this?
> 
> Thanks.
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
>

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: cyrus-imap buildin: sse extention

[ellie timoney]

Hi Sergey,

> Hardware support:
>SSE4.2: yes

This is detected for a hardware implementation of the CRC32c algorithm.  Cyrus 
doesn't actually use it though, because it's not compatible with the existing 
CRC32 algorithm: i.e. for the same input, it produces a different checksum, 
which would break everything on a system with pre-existing data.

If you _want_ to use the hardware CRC32c algorithm on a brand new deployment 
(and know what you are doing), I believe at this stage you would need to patch 
Cyrus to use it -- the code is there, but it is never called.

> Can Cyrus-IMAP be running on systems without SSE4 at this case?

Yep, it'll work just fine.  The hardware CRC32c code will simply not be 
compiled, which, since it isn't used anyway, will have no effect.

> If no, can I set limit to SSE2 ?

There's currently no configurability for this at all.  I don't know if it's 
even possible to implement the same algorithm with SSE2.  At the moment I 
assume that, since it's looking for SSE42 specifically, then that means that 
the hardware feature it needs is probably only available in SSE42.

Cheers,

ellie

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Upgrade to 3.2.2 and sieve

[Stephan]

Am 26.06.20 um 14:18 schrieb Jean Charles Delépine:

Stephan  écrivait (wrote) :


sieve_rebuild: [path to script] parse failed: script errors:#015#012line
5: header 'resent-from': not a valid header for an address test


I had the same error with a 2.5 to 3.0 migration.

Corrected with 'rfc3028_strict: 0' :

rfc3028_strict: 1
   If enabled, Sieve will be strict (per RFC 3028) with regards to which
   headers are allowed to be used in address and envelope tests.  This means 
that
   only those headers which are defined to contain addresses will be allowed in
   address tests and  only  "to"  and "from" will be allowed in envelope tests.
   When disabled, ANY grammatically correct header will be allowed.



Thank you ! rfc3028_strict was enabled here in 3.0.13 and the scripts
were accepted so I guess it must be one of the sieve bug fixes and
features mentioned in the 3.2 release notes.

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Upgrade to 3.2.2 and sieve

[Jean Charles Delépine]

Stephan  écrivait (wrote) :

> sieve_rebuild: [path to script] parse failed: script errors:#015#012line
> 5: header 'resent-from': not a valid header for an address test

I had the same error with a 2.5 to 3.0 migration.

Corrected with 'rfc3028_strict: 0' :

rfc3028_strict: 1
  If enabled, Sieve will be strict (per RFC 3028) with regards to which
  headers are allowed to be used in address and envelope tests.  This means that
  only those headers which are defined to contain addresses will be allowed in
  address tests and  only  "to"  and "from" will be allowed in envelope tests.
  When disabled, ANY grammatically correct header will be allowed.


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: backupd and sync_client IOERROR

[ellie timoney]

> I think there isn't a all-in-one command for this use case: a user 
> expunged some messages and deleted some folders somewhere. I want to 
> recover all expunged messages and all the deleted folders which are no 
> more present in the original IMAP server (because they were expired from 
> cyr_expire).
> 
> I have to set "-x" to avoid duplication of messages.
> With "-a -x" I recover all expunged messages and all deleted mailboxes. 
> But messages inside deleted mailboxes are not marked as expunged, so 
> these mailboxes are recovered empty in the IMAP server.

I would run restore twice in this case:  once without -x, specifying just the 
deleted mailboxes (you can use "cyr_backup list mailboxes ..." to get a list of 
the mailboxes in the user's backup).  And once with -x to get all the expunged 
stuff.  For the -x invocation only, I would probably also use the -M option to 
dump all the recovered stuff into a new folder, so they can easily tell it 
apart from any new mail that might have arrived coincidentally at the same time.

> Another idea is to recover all in another empty IMAP server, without 
> "-x" at all, and the user can look at the mailbox recovered there...

We kinda had a similar idea!  Putting it all into a folder with -M is much 
easier than setting up a separate server, but it'll lose the folder structure.  
Recovering to a separate server means the folder structure can be preserved.  I 
guess it depends on the specific recovery situation, and how hard it is to spin 
up a server in your environment.

Cheers,

ellie

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: reconstructing mailboxes from backup

[ellie timoney]

Hi Tim,

It's worth observing that, in Cyrus, the user "george"'s IMAP inbox is the 
"user/george" folder.  Which means, on disk, this user has another folder 
called "INBOX" within their inbox.  Depending on the Cyrus version, and maybe 
depending on your server's value of "altnamespace", this is invalid -- and it 
looks like your reconstruct has skipped it and everything under it, 
unsurprisingly.

It's also worth observing that there is both a "Deleted Messages" folder as a 
subdirectory of the bad "INBOX", and an "INBOX^Deleted Messages" directory that 
looks like maybe the result of unixhierarchysep being changed out from under 
the client, or something like that.  Looks like reconstruct has pulled the 
latter one in, cause it's not technically bad (but it will be very 
weird/confusing for the user to have a folder called "INBOX.Deleted Messages" 
in their client that is neither their inbox, nor their Deleted Messages folder, 
nor a directory hierarchy of the two).

So, it looks like reconstruct has found all the valid folders, and skipped the 
invalid INBOX and everything in it.  That seems coherent.

If you can start again from scratch: then I'd suggest renaming, on disk, that 
"INBOX" folder to something like "old inbox", and optionally renaming the 
"INBOX^Deleted Messages" folder to something like "old deleted messages", 
before you run the reconstruct.  Then the reconstruct will be able to find 
everything, and the user can then move the messages from the "old..." folders 
back into wherever they want them to be just over IMAP.

If you can't start again from scratch: then you should only rename the bad 
"INBOX" folder on disk, and then reconstruct. The previous reconstruct already 
found and repaired the "INBOX^Deleted Messages" folder, so renaming it on disk 
now might make a new mess.  But it can be renamed over IMAP, either by an admin 
session or the user.

Hope this helps :)

ellie

On Tue, Jun 23, 2020, at 11:42 PM, Tim Coote wrote:
> Hullo
> 
> I have a cyrus implementation on Fedora for a small (~10) users that’s 
> been migrated through many versions of the various components, 
> including several different of IMAP clients.
> 
> Realising the fragility of the setup, I thought I’d restore from a 
> backup. However, I’m finding that several of the mailboxes are not 
> being recovered. I feel that I am missing somethign obvious, but I 
> cannot spot it. 
> 
> The restoring version of cyrus-imap is: cyrus-imapd-3.0.13-2.fc32.x86_64,
> 
> The restored filesystem layout can be summarised thus:
> 
> `sudo find /var/spool/imap/g | grep cyrus.header`:
> 
> /var/spool/imap/g/user/george/Notes/cyrus.header
> /var/spool/imap/g/user/george/cyrus.header
> /var/spool/imap/g/user/george/Sent Messages/cyrus.header
> /var/spool/imap/g/user/george/Deleted Messages/cyrus.header
> /var/spool/imap/g/user/george/Sent/cyrus.header
> /var/spool/imap/g/user/george/Trash/cyrus.header
> /var/spool/imap/g/user/george/INBOX/Sent Messages/cyrus.header
> /var/spool/imap/g/user/george/INBOX/Deleted Messages/cyrus.header
> /var/spool/imap/g/user/george/INBOX/Drafts/cyrus.header
> /var/spool/imap/g/user/george/INBOX^Deleted Messages/cyrus.header
> /var/spool/imap/g/user/george/Drafts/cyrus.header
> 
> [so I would expect all of the subdirectories to be reconstructed as mailboxes]
> 
> however, using:
> `sudo -u cyrus reconstruct -r -f user/george`
> 
> I only get:
> user/george
> user/george/Deleted Messages
> user/george/Drafts
> user/george/INBOX.Deleted Messages
> user/george/Notes
> user/george/Sent
> user/george/Sent Messages
> user/george/Trash
> 
> ie no subdirectories below the top level, but excluding those 
> directories below INBOX.
> Should there be a file: 
> `/var/spool/imap/g/user/george/INBOX/cyrus.header`? 
> 
> Is there anything that I should be doing/how can I recover the other 
> mailboxes?
> 
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: What you do with old account

[Kenneth Marshall]

On Tue, Jun 23, 2020 at 04:10:56PM +0200, Albert Shih wrote:
> 
> In fact I just notice, I've no idea...how to remove a mailbox in cyrus
> 
> With dovecot it's rm -rf ;-) Something I famillar with.

cyradm deletemailbox user/xxx

Regards,
Ken

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: What you do with old account

[Albert Shih]

Le 14/06/2020 à 21:51:49+0200, Sebastian Hagedorn a écrit

Hi,


Thanks for your answer.

> Am 09.06.20 um 16:32 schrieb Adam Tauno Williams:
> > On Tue, 2020-06-09 at 15:51 +0200, Albert Shih wrote:>
> >> After switching to cyrus imap, I think about how to do that.
> >> If I'm correct I cannot just copy the file somewhere else, because cyrus
> >> database would keep the information about the existance of the mailbox, so
> >> what will the «state of the art» way to remove a mail account and all the
> >> mail.
> >> And how what would be the «state of the art» way to put it back ?
> > I create a calendar event [task] to delete the mailbox and otherwise
> > just leave it. If the account itself is disabled it cannot be accessed.
> >
> > Putting things back-into a mailstore is too much of a pain with current
> > storage prices.
>
> We have about 90,000 accounts, and our current model is that we leave
> expired accounts around for a year. The user can't login, and we don't
> accept new mails, but it's still there in case the account is
> re-activated. After one year the mailbox hierarchy is put into a .tgz
> and written to tape. When that is done the account is permanently
> deleted. If the user should come back, they get a completely new

In fact I just notice, I've no idea...how to remove a mailbox in cyrus

With dovecot it's rm -rf ;-) Something I famillar with.

> account. I can't recall a single instance where the .tgz was ever
> needed, but that's not my problem. After one more year the .tgz is

Absolutly.

> deleted from tape as well.

Ok.

Thanks

Regards.


--
Albert SHIH
Observatoire de Paris
xmpp: j...@obspm.fr
Heure local/Local time:
Tue 23 Jun 2020 04:09:05 PM CEST

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: httpd server signature off

[Ken Murchison]

Are you talking about removing this from the body of error responses?

Currently you can't, but I will patch master so that it obeys the 
serverinfo option.



On 6/23/20 8:19 AM, Zorg wrote:

Hi

for security reason i want to get rid off

Cyrus-HTTP/3.0.6-Debian-3.0.6-6+deb1u1 Cyrus-SASL/2.1.23 OpenSSL/1.1 
Zlib/1.2.10 LibXML2.9.5 SQLite/3.21.1 LibiCal/3.0 ICU4C/63.1 
Jansson/2.12 Server at cyrus.domain.com Port 9443


like apache using

ServerTokens Prod or ServerSignature Off


Have I try serverinfo: off in imapd.conf but it don't work

What should i do

Cyril


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus



--
Kenneth Murchison
Cyrus Development Team
Fastmail US LLC


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: backupd and sync_client IOERROR

[Marco]

Hello,

On 22/06/2020 03:29, ellie timoney has written:
[...]

So like, maybe a user has deleted some stuff, and you don't want to mess around figuring out which 
individual messages they need restored, so you just want to restore everything, and let the user 
figure it out.  This is what -x is for -- so you can say "restore the contents of mailbox foo, 
but only the expunged stuff" or "restore every mailbox (-a), but only the expunged 
stuff".

[...]


 thank you for all these explanations, I think I understand now. My use 
case is "the user accidentally deleted and expunged and now he ask for 
recovery". I have to unexpunge all messages after I recovered it with 
"restore -x".


I think there isn't a all-in-one command for this use case: a user 
expunged some messages and deleted some folders somewhere. I want to 
recover all expunged messages and all the deleted folders which are no 
more present in the original IMAP server (because they were expired from 
cyr_expire).


I have to set "-x" to avoid duplication of messages.
With "-a -x" I recover all expunged messages and all deleted mailboxes. 
But messages inside deleted mailboxes are not marked as expunged, so 
these mailboxes are recovered empty in the IMAP server.


I have to examine the output of the restore command and repeat a restore 
without the "-x" on each DELETED recovered mailboxes at the previous step.



Another idea is to recover all in another empty IMAP server, without 
"-x" at all, and the user can look at the mailbox recovered there...


Thank you very much!!

Cheers
Marco

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: backupd and sync_client IOERROR

[ellie timoney]

Hi Marco,

On Fri, Jun 19, 2020, at 6:44 PM, Marco wrote:
> wow, yes, it works. With this config, after the Cyrus restart the 
> mailboxes.db and the skipstamps dbs are created and the error disappears 
> from syslog.

Great! I've updated the documentation, and the website should update shortly.

> I suggested the possibility to choose the recovery date (ie: all 
> messages and mailboxes deleted or expunged from  to ). I 
> hope you could add this feature.

Yep, I saw that, thanks for the request :)

> I now tried a restore, but I still have some problems. I very appreciate 
> if you could read the following issue.
> 
> My goal is to recover a message that was removed (expunged) from the 
> IMAP server, ignoring already existing messages.
> 
> It seems the "-x" flag could help. It seems it allows to restore and 
> unexpunge a previously expunged messages in the IMAP server.

No, you've misunderstood.  The "-x" flag is to ONLY restore expunged messages.  
If you've specified a list of mailboxes or messages on the command line, and 
some of them are not expunged, the "-x" option will cause it to ignore the ones 
that are not expunged, and only restore the expunged ones.  The "-X" option 
does the inverse -- they're filters.  Without one of these filters, it will 
restore everything you asked for, regardless of its expunged status.

So like, maybe a user has deleted some stuff, and you don't want to mess around 
figuring out which individual messages they need restored, so you just want to 
restore everything, and let the user figure it out.  This is what -x is for -- 
so you can say "restore the contents of mailbox foo, but only the expunged 
stuff" or "restore every mailbox (-a), but only the expunged stuff".

Think about it: if the messages _weren't_ expunged, there would usually be no 
need to restore them from backup!  You would simply remove the \Deleted flag 
over IMAP.  So, you are almost always using restore to bring back expunged 
messages, thus there is no special flag needed to enable this functionality.

(But, if you had lost an entire server to some disaster, and restoring to its 
replacement, then you would need to also restore the unexpunged stuff.)
 
> Yesterday I expunged the message 16ceead9802286784d7a54c5bc782891f76f2f2e:
> 
> 2020-06-18T15:44:37.374556+02:00 tst-msg03 cyrus/imap[9461]: auditlog: 
> expunge sessionid= 
> mailbox= 
> uniqueid= uid=<32092> modseq=<46828> 
> sysflags= guid=<16ceead9802286784d7a54c5bc782891f76f2f2e>
> 
> 
> I tried today to restore it:
> cyr_restore -v -U -x tst-msg03.example.com -u 
> utente.archivi...@example.com 16ceead9802286784d7a54c5bc782891f76f2f2e
> example.com!user.utente^archivista: trying to keep 
> uidvalidity(1550228105), uniqueid(nvf96uirjgfs6xaekwwc7g44), 
> highestmodseq(46828)
> 
> and I see:
> 2020-06-19T09:29:24.030624+02:00 tst-msg03 cyrus/imap[32742]: login: 
> tst-msg03-bck.example.com [10.102.42.169] cyr_restore LOGIN User logged 
> in SESSIONID=
> 
> 2020-06-19T09:29:24.162909+02:00 tst-msg03 cyrus/imap[32742]: auditlog: 
> append sessionid= 
> mailbox= 
> uniqueid= uid=<32096> modseq=<46829> 
> sysflags= guid=<16ceead9802286784d7a54c5bc782891f76f2f2e> 
> messageid=
> 
> 2020-06-19T09:29:24.163058+02:00 tst-msg03 cyrus/imap[32742]: auditlog: 
> expunge sessionid= 
> mailbox= 
> uniqueid= uid=<32096> modseq=<46829> 
> sysflags= guid=<16ceead9802286784d7a54c5bc782891f76f2f2e>
> 
> 2020-06-19T09:29:24.167561+02:00 tst-msg03 cyrus/imap[32742]: auditlog: 
> modseq sessionid= 
> mailbox= 
> uniqueid= highestmodseq=<46829>
> 
> 2020-06-19T09:29:24.215036+02:00 tst-msg03 cyrus/imap[32742]: USAGE 
> cyr_restore user: 0.042492 sys: 0.015435
> 
> 2020-06-19T09:29:24.215191+02:00 tst-msg03 cyrus/imap[32742]: auditlog: 
> traffic sessionid= 
> bytes_in=<2412> bytes_out=<1039>
> 
> 2020-06-19T09:29:32.855379+02:00 tst-msg03 cyrus/sync_client[32566]: 
> IOERROR: zero length response to MAILBOXES (end of file reached)
> 
> 2020-06-19T09:29:32.855682+02:00 tst-msg03 cyrus/sync_client[32566]: 
> IOERROR: zero length response to RESTART (end of file reached)
> 
> 2020-06-19T09:29:32.855713+02:00 tst-msg03 cyrus/sync_client[32566]: 
> Error in do_sync(): bailing out! Bad protocol
> 
> 2020-06-19T09:29:32.855738+02:00 tst-msg03 cyrus/sync_client[32566]: 
> Processing sync log file /var/lib/imap/sync/bck/log-run failed: Bad protocol
> 
> I tried again and I see the same result, but without the IOERROR:
> 
> 2020-06-19T09:36:51.153939+02:00 tst-msg03 cyrus/imap[32762]: login: 
> tst-msg03-bck.example.com [10.102.42.169] cyr_restore LOGIN User logged 
> in SESSIONID=
> 
> 2020-06-19T09:36:51.199742+02:00 tst-msg03 cyrus/imap[32762]: 
> example.com!user.utente^archivista: same message appears twice 32096 32097
> 
> (why twice? It expunges 32096 just above)
> 
> 2020-06-19T09:36:51.225525+02:00 tst-msg03 cyrus/imap[32762]: auditlog: 
> append sessionid= 
> mailbox= 
> uniqueid= uid=<32097> modseq=<46834> 
> sysflags= 

SOLVED Re: cyrus-imapd exporting databases failed on shutdown - deliver.db.skiplist 2048M

[Pongrácz István]

2020. 06.  21, vasárnap keltezéssel 10.53-kor Simon Matter ezt írta:
> 2020. 06.  20, szombat keltezéssel 21.31-kor Simon Matter ezt írta:
> 
> The deliver.db still about 48MB.
> 

I just deleted deliver.db and restart cyrus.
Restarting was pretty quick, takes only some seconds.
New deliver.db created, its size is 8KByte.

So far, so good.

Thank you!

István

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: cyrus-imapd exporting databases failed on shutdown - deliver.db.skiplist 2048M

[Pongrácz István]

2020. 06.  21, vasárnap keltezéssel 10.53-kor Simon Matter ezt írta:
> 2020. 06.  20, szombat keltezéssel 21.31-kor Simon Matter ezt írta:
> Please make sure the options here are also valid for your cyrus version.
> However, I also guess your deliver.db is corrupted somehow. From my own
> experience skiplist dbs are easier to handle than bdb and using skiplist
> only has not shown any issues.
> 
> Regards,
> Simon

Hi,

Thank you the hint Simon.
They are valid.

What do you think, should I just delete deliver.db after I stopped cyrus-imap 
and restart again?
What can I loose?

Cheers,
István

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: cyrus-imapd exporting databases failed on shutdown - deliver.db.skiplist 2048M

[Simon Matter via Info-cyrus]

> 2020. 06.  20, szombat keltezéssel 21.31-kor Simon Matter ezt írta:
>> Hi,
>>
>> The question is why is the deliver db > 2GB in skiplist format? Is it
>> normal or do you have a corrupt BDB db or does your db pruning not work
>> for deliverdb. I think that should be something like 'delprune
>> cmd="cyr_expire -D 7 -E 3 -X 7" at=0400' in cyrus.conf.
>>
>> I think the easiest way would be to make sure you have pruning
>> configured
>> correctly, then change config of deliver db to skiplist, and start
>> without
>> a db so a new, empty deliver db is created.
>>
>> Then have an eye on the db file to see if it grows again to almost 2GB.
>> If
>> it doesn't grow so much, you should be fine.
>>
>> Regards,
>> Simon
>
> Hi,
>
> Something definitely not seems fine:
>
> -bash-3.2$ /usr/lib/cyrus-imapd/cyr_expire -E 3 -D 7 -X 7 -v

Please make sure the options here are also valid for your cyrus version.
However, I also guess your deliver.db is corrupted somehow. From my own
experience skiplist dbs are easier to handle than bdb and using skiplist
only has not shown any issues.

Regards,
Simon

>
> expunged 0 out of 0 messages from 0 mailboxes
>
> The deliver.db still about 48MB.
>
> Tomorrow I will continue.
>
> Thanks,
> István
>



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: cyrus-imapd exporting databases failed on shutdown - deliver.db.skiplist 2048M

[Pongrácz István]

2020. 06.  20, szombat keltezéssel 21.31-kor Simon Matter ezt írta:
> Hi,
> 
> The question is why is the deliver db > 2GB in skiplist format? Is it
> normal or do you have a corrupt BDB db or does your db pruning not work
> for deliverdb. I think that should be something like 'delprune 
> cmd="cyr_expire -D 7 -E 3 -X 7" at=0400' in cyrus.conf.
> 
> I think the easiest way would be to make sure you have pruning configured
> correctly, then change config of deliver db to skiplist, and start without
> a db so a new, empty deliver db is created.
> 
> Then have an eye on the db file to see if it grows again to almost 2GB. If
> it doesn't grow so much, you should be fine.
> 
> Regards,
> Simon

Hi,

Something definitely not seems fine:

-bash-3.2$ /usr/lib/cyrus-imapd/cyr_expire -E 3 -D 7 -X 7 -v

expunged 0 out of 0 messages from 0 mailboxes

The deliver.db still about 48MB.

Tomorrow I will continue.

Thanks,
István

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: cyrus-imapd exporting databases failed on shutdown - deliver.db.skiplist 2048M

[Pongrácz István]

2020. 06.  20, szombat keltezéssel 21.31-kor Simon Matter ezt írta:
> 
> Hi,
> 
> The question is why is the deliver db > 2GB in skiplist format? Is it
> normal or do you have a corrupt BDB db or does your db pruning not work
> for deliverdb. I think that should be something like 'delprune 
> cmd="cyr_expire -D 7 -E 3 -X 7" at=0400' in cyrus.conf.
> 
> I think the easiest way would be to make sure you have pruning configured
> correctly, then change config of deliver db to skiplist, and start without
> a db so a new, empty deliver db is created.
> 
> Then have an eye on the db file to see if it grows again to almost 2GB. If
> it doesn't grow so much, you should be fine.
> 
> Regards,
> Simon
> 
> 

Hi,
Thank you for your answer!
I have this in my config at EVENTS section:

EVENTS {
  # this is required
  checkpointcmd="ctl_cyrusdb -c" period=30

  # this is only necessary if using duplicate delivery suppression
  delprune  cmd="ctl_deliver -E 3" period=1440

  # this is only necessary if caching TLS sessions
  tlsprune  cmd="tls_prune" period=1440
}

Regarding to users, we do not see any anomalies on daily use or restarting the 
server sometimes, only this exporting failed.

I issued the mentioned command on command line (cyr_expire -D 7 -E 3 -X 7) it 
finished in around 1-2 seconds. (At this moment there are no online users as 
this is weekend).

Additional information I forgot to mention:
there are a lot of shared email folder of the catchall account to share common 
mailboxes.

I got this error after some lines, when I run /usr/lib/cyrus-imapd/ctl_deliver 
-d

fatal error: Internal error: assertion failed: duplicate.c: 344: (datalen == 
sizeof(time_t)) || (datalen == sizeof(time_t) + sizeof(unsigned long))

Probably deliver.db itself is corrupted?

Cheers,
István

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: cyrus-imapd exporting databases failed on shutdown - deliver.db.skiplist 2048M

[Simon Matter via Info-cyrus]

> Hi,
>
> I run into a problem on an old clearos server, where the cyrus shutdown
> always failed at step exporting databases.
> As I checked the situation using ps ax on an other console, I found
> that, it was exporting deliver.db.skiplist file, which failed after a
> lng time (some minutes).
> I checked that file on the filesystem, I saw the file size is 2048MB,
> which seems a limit for me and I suspect the problem should be that,
> the 32 bit cyrus cannot write more data to that file and caused the
> problem.
> As I read the db_export.log, that confirmed my theory, file size limit
> exceeded.

Hi,

The question is why is the deliver db > 2GB in skiplist format? Is it
normal or do you have a corrupt BDB db or does your db pruning not work
for deliverdb. I think that should be something like 'delprune 
cmd="cyr_expire -D 7 -E 3 -X 7" at=0400' in cyrus.conf.

I think the easiest way would be to make sure you have pruning configured
correctly, then change config of deliver db to skiplist, and start without
a db so a new, empty deliver db is created.

Then have an eye on the db file to see if it grows again to almost 2GB. If
it doesn't grow so much, you should be fine.

Regards,
Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: backupd and sync_client IOERROR

[Marco]

Hi Ellie,

Il 19/06/2020 03:01, ellie timoney ha scritto:

I think you might need to add the usual recover entry to the START section:
 
recover   cmd="ctl_cyrusdb -r"


I notice this is missing from the backups documentation -- please let me know 
if it this sorts it out, and I'll fix it up!


wow, yes, it works. With this config, after the Cyrus restart the 
mailboxes.db and the skipstamps dbs are created and the error disappears 
from syslog.



2) Is there a way to delete a user from backupd host?

That's an interesting question...

Under normal use, if a user is deleted from their imap server, then, after the deletion has 
replicated to the backup server, and after the "backup_retention" period has elapsed, and 
after a subsequent "ctl_backups compact ..." has been run, then the backup file should be 
approximately empty.  But I think it will still exist; nothing actually deletes it yet.

To get rid of it manually, you can use "ctl_backups list ..." to find the 
filename where it exists on disk.  You can also find it by using cyr_dbtool to look up 
the user's key in backups.db.  You will then need to:

a) use cyr_dbtool to delete that user's key from backups.db
b) delete the named file (it will be like "/some/path/userid_XX")
c) also delete the corresponding "/some/path/userid_XX.index" file

Note that if the user still really exists, and a rolling sync_client replicates 
them to the same backup server again, then the backup file will be recreated 
(with a new XX) -- with the same problems as above wrt no-initial-sync.


It works!
Making some tests and the initial sync, it's easy with sync_client 
insert a wrong non-existent userid (I added "/user" before the userid). 
And that wrong userid is immediately created on the backupd server :)

With the above procedure I can delete the wrong entry.


Definitively I think that this backup feature is a very good idea.

I suggested the possibility to choose the recovery date (ie: all 
messages and mailboxes deleted or expunged from  to ). I 
hope you could add this feature.


Thank you very very much for these detailed answers!!


I now tried a restore, but I still have some problems. I very appreciate 
if you could read the following issue.


My goal is to recover a message that was removed (expunged) from the 
IMAP server, ignoring already existing messages.


It seems the "-x" flag could help. It seems it allows to restore and 
unexpunge a previously expunged messages in the IMAP server.


Yesterday I expunged the message 16ceead9802286784d7a54c5bc782891f76f2f2e:

2020-06-18T15:44:37.374556+02:00 tst-msg03 cyrus/imap[9461]: auditlog: 
expunge sessionid= 
mailbox= 
uniqueid= uid=<32092> modseq=<46828> 
sysflags= guid=<16ceead9802286784d7a54c5bc782891f76f2f2e>



I tried today to restore it:
cyr_restore -v -U -x tst-msg03.example.com -u 
utente.archivi...@example.com 16ceead9802286784d7a54c5bc782891f76f2f2e
example.com!user.utente^archivista: trying to keep 
uidvalidity(1550228105), uniqueid(nvf96uirjgfs6xaekwwc7g44), 
highestmodseq(46828)


and I see:
2020-06-19T09:29:24.030624+02:00 tst-msg03 cyrus/imap[32742]: login: 
tst-msg03-bck.example.com [10.102.42.169] cyr_restore LOGIN User logged 
in SESSIONID=


2020-06-19T09:29:24.162909+02:00 tst-msg03 cyrus/imap[32742]: auditlog: 
append sessionid= 
mailbox= 
uniqueid= uid=<32096> modseq=<46829> 
sysflags= guid=<16ceead9802286784d7a54c5bc782891f76f2f2e> 
messageid=


2020-06-19T09:29:24.163058+02:00 tst-msg03 cyrus/imap[32742]: auditlog: 
expunge sessionid= 
mailbox= 
uniqueid= uid=<32096> modseq=<46829> 
sysflags= guid=<16ceead9802286784d7a54c5bc782891f76f2f2e>


2020-06-19T09:29:24.167561+02:00 tst-msg03 cyrus/imap[32742]: auditlog: 
modseq sessionid= 
mailbox= 
uniqueid= highestmodseq=<46829>


2020-06-19T09:29:24.215036+02:00 tst-msg03 cyrus/imap[32742]: USAGE 
cyr_restore user: 0.042492 sys: 0.015435


2020-06-19T09:29:24.215191+02:00 tst-msg03 cyrus/imap[32742]: auditlog: 
traffic sessionid= 
bytes_in=<2412> bytes_out=<1039>


2020-06-19T09:29:32.855379+02:00 tst-msg03 cyrus/sync_client[32566]: 
IOERROR: zero length response to MAILBOXES (end of file reached)


2020-06-19T09:29:32.855682+02:00 tst-msg03 cyrus/sync_client[32566]: 
IOERROR: zero length response to RESTART (end of file reached)


2020-06-19T09:29:32.855713+02:00 tst-msg03 cyrus/sync_client[32566]: 
Error in do_sync(): bailing out! Bad protocol


2020-06-19T09:29:32.855738+02:00 tst-msg03 cyrus/sync_client[32566]: 
Processing sync log file /var/lib/imap/sync/bck/log-run failed: Bad protocol


I tried again and I see the same result, but without the IOERROR:

2020-06-19T09:36:51.153939+02:00 tst-msg03 cyrus/imap[32762]: login: 
tst-msg03-bck.example.com [10.102.42.169] cyr_restore LOGIN User logged 
in SESSIONID=


2020-06-19T09:36:51.199742+02:00 tst-msg03 cyrus/imap[32762]: 
example.com!user.utente^archivista: same message appears twice 32096 32097


(why twice? It expunges 32096 just above)

Re: backupd and sync_client IOERROR

[ellie timoney]

Hi Marco,

On Thu, Jun 18, 2020, at 10:19 PM, Marco wrote:
> Hello,
> 
>   I'm trying to configure backupd in rolling mode as a final setup. 
> Running a first backup on few users
> 
> sync_client -A -n bck -z -v -v
> 
> after a while the process die with:
> 
> cyrus/sync_client[9540]: MESSAGE received NO response: 
> IMAP_PROTOCOL_ERROR Protocol error
> cyrus/sync_client[9540]: do_folders(): update failed: 
> example.com!user.utente^archivista 'Bad protocol'
> cyrus/sync_client[9540]: IOERROR: do_user_main: Bad protocol for 
> utente.archivi...@example.com to [no channel] (tst-msg03-bck.example.com)
> Error from do_user(utente.archivi...@example.com): bailing out!
> cyrus/sync_client[9540]: Error in 
> do_user(utente.archivi...@example.com): bailing out!
> 
> The backupd host doesn't complain:
> 
> 2020-06-18T11:50:30.528584+02:00 tst-msg03-bck backupd[1308172]: 
> creating sql_db 
> /var/spool/cyr_backup/u/utente.archivista@example.com_cxlsmX.index
> 2020-06-18T11:56:44.971042+02:00 tst-msg03-bck backupd[1308172]: login: 
> tst-msg03.example.com [10.102.42.168] cyr_backup LOGIN User logged in
> 
> So I repeated the same command, but with less verbosity:
> 
> sync_client -A -n bck -z -v
> 
> and it worked well, without errors. Uhm...
> 
> Maybe, before to run the first initial backup (sync_client -A), do I 
> have to stop the sync_client already working in rolling mode?

Rolling mode only makes incremental updates, so if you're starting from a 
server that already has existing data, you should do the first manual initial 
backups before enabling the rolling mode.

It's generally safe to run a manual sync_client side-by-side with a rolling one 
(they will lock things properly and keep out of each other's way).  But the 
exception to this is that a rolling mode replication can't coherently update an 
uninitialised replica (since it only sends new changes recorded in the sync 
log, but not the pre-existing data).  You need to do a full manual replication 
first, to ensure both sides are the same, before rolling mode can work 
sensibly.  This is the same regardless of whether your replica is a normal 
replica or a backupd one.

As you saw, replicating again manually will probably fix you up, at least. :)

> http://www.cyrusimap.org/imap/reference/admin/backups.html says "Update 
> cyrus.conf(5) to add a sync_client(8) invocation to the SERVICES section 
> specifying (at least) the -r and -n channel options.". Really maybe you 
> intend START. In SERVICE I need to specify a listen too, this is not 
> very clear to me.

This is wrong.  It should be the DAEMON section, not the SERVICES section.  I'm 
fixing this now, so the website should be updated shortly.

You can also run it from the START section, but note that if the sync_client 
exits early for some reason, master will not restart it.  If you run it from 
the START section, you will need to monitor/restart it yourself.

> Around of backupd I would ask two other questions:
> 
> 1) The backupd host always claims this:
> 
> backupd[1309970]: DBERROR: reading /var/lib/imap/db/skipstamp, assuming 
> the worst: No such file or directory
> 
> Could you tell me how to fix that DBERROR too?

I think you might need to add the usual recover entry to the START section:

recover   cmd="ctl_cyrusdb -r"

I notice this is missing from the backups documentation -- please let me know 
if it this sorts it out, and I'll fix it up!
 
> 2) Is there a way to delete a user from backupd host?

That's an interesting question...

Under normal use, if a user is deleted from their imap server, then, after the 
deletion has replicated to the backup server, and after the "backup_retention" 
period has elapsed, and after a subsequent "ctl_backups compact ..." has been 
run, then the backup file should be approximately empty.  But I think it will 
still exist; nothing actually deletes it yet.

To get rid of it manually, you can use "ctl_backups list ..." to find the 
filename where it exists on disk.  You can also find it by using cyr_dbtool to 
look up the user's key in backups.db.  You will then need to:

a) use cyr_dbtool to delete that user's key from backups.db
b) delete the named file (it will be like "/some/path/userid_XX")
c) also delete the corresponding "/some/path/userid_XX.index" file

Note that if the user still really exists, and a rolling sync_client replicates 
them to the same backup server again, then the backup file will be recreated 
(with a new XX) -- with the same problems as above wrt no-initial-sync.

Cheers,

ellie

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: CalDav CardDav webmail client ?

[ego...@sarenet.es]

Hi!

Answering below!!


> El 17 jun 2020, a las 15:09, Niels Dettenbach  escribió:
> 
> Am Mittwoch, 17. Juni 2020, 14:39:36 CEST schrieb ego...@sarenet.es:
>> Although we at present, are not running Cyrus Caldav/Carddav, Davical
>> instead (it comes from long time ago), we are running Roundcube. I adapted
>> the Caldav Kolab plugin in order to even support Free/Busy and for fixing
>> some bug… I should have uploaded it to Github or wherever, but I have not
>> have time for getting it ready for that. By the way, it’s not done for
>> Roundcube 1.4. It’s for Roundcube 1.3. We even use Caldavzap (with a lot
>> of js work too) with Roundcube as tasks plugin too.
> Thanks for that tip,
> 
> but it seems requiring PHP5 (which is obsolete today) and don't get any 
> secuity updates anymore.
> 

You could get it working with PHP7…. It has some work… but you can….


Cheers!!


> It seems that even many other famous open source projects for CalDAV/CardDAV 
> servers stacks are still more maintained - including i.e. Apples Darwin 
> Calendar Servers (DCS):
> https://www.calendarserver.org/
> 
> We use Cyrus IMAP with Horde5 to provide CalDAV/CardDAV as "MS Exchange Sync" 
> emulation ("newer ActiveSync") "out of one box" plus a "nice" (responsive) 
> Groupware web GUI, but the Calendar / Adressbook server stack is still used 
> from Horde (on SQL) while Cyrus serves the Emails for Horde only. Horde5 now 
> works on PHP7.x and is still maintained.
> 
> It would be nice to get that Horde5 stack completely run with Cyrus as this 
> would offer even more compatibility / protocols as the very known "Cyrus 
> robustness"...ß) ...and on the longer run Horde out of the path for 
> compatible clients / users.
> 
> just my .02$
> 
> 
> 
> niels.
> 
> -- 
> ---
> Niels Dettenbach
> Syndicat IT & Internet
> http://www.syndicat.com
> PGP: https://syndicat.com/pub_key.asc
> ---
> 
> 
> 
> 
> 
> 
> 


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: CalDav CardDav webmail client ?

[Niels Dettenbach via Info-cyrus]

Am Mittwoch, 17. Juni 2020, 14:39:36 CEST schrieb ego...@sarenet.es:
> Although we at present, are not running Cyrus Caldav/Carddav, Davical
> instead (it comes from long time ago), we are running Roundcube. I adapted
> the Caldav Kolab plugin in order to even support Free/Busy and for fixing
> some bug… I should have uploaded it to Github or wherever, but I have not
> have time for getting it ready for that. By the way, it’s not done for
> Roundcube 1.4. It’s for Roundcube 1.3. We even use Caldavzap (with a lot
> of js work too) with Roundcube as tasks plugin too.
Thanks for that tip,

but it seems requiring PHP5 (which is obsolete today) and don't get any 
secuity updates anymore.

It seems that even many other famous open source projects for CalDAV/CardDAV 
servers stacks are still more maintained - including i.e. Apples Darwin 
Calendar Servers (DCS):
https://www.calendarserver.org/

We use Cyrus IMAP with Horde5 to provide CalDAV/CardDAV as "MS Exchange Sync" 
emulation ("newer ActiveSync") "out of one box" plus a "nice" (responsive) 
Groupware web GUI, but the Calendar / Adressbook server stack is still used 
from Horde (on SQL) while Cyrus serves the Emails for Horde only. Horde5 now 
works on PHP7.x and is still maintained.

It would be nice to get that Horde5 stack completely run with Cyrus as this 
would offer even more compatibility / protocols as the very known "Cyrus 
robustness"...ß) ...and on the longer run Horde out of the path for 
compatible clients / users.

just my .02$



niels.

-- 
 ---
 Niels Dettenbach
 Syndicat IT & Internet
 http://www.syndicat.com
 PGP: https://syndicat.com/pub_key.asc
 ---
 







Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: CalDav CardDav webmail client ?

[ego...@sarenet.es]

Hi,

Although we at present, are not running Cyrus Caldav/Carddav, Davical instead 
(it comes from long time ago), we are running Roundcube. I adapted the Caldav 
Kolab plugin in order to even support Free/Busy and for fixing some bug… I 
should have uploaded it to Github or wherever, but I have not have time for 
getting it ready for that. By the way, it’s not done for Roundcube 1.4. It’s 
for Roundcube 1.3. We even use Caldavzap (with a lot of js work too) with 
Roundcube as tasks plugin too.

The only important thing in the combination of all of them, is that you need to 
use a specific version of Sabre for the contacts plugin and there’s a packaged 
version of the Contacts plugin with the needed Sabredav version which allows 
you to run it combined with the Kolab caldav plugin, although it’s slightly 
limited and slightly buggy (Kolab caldav)…. I hope to have some time for 
uploading our patches next months… I have not tested them and don’t expect them 
to work in Roundcube 1.4. by the way… sadly...

Cheers,


> El 12 jun 2020, a las 9:35, Marco  escribió:
> 
> Hello,
> 
> I'm interesting in Cyrus Caldav/Carddav server. I never used it before, 
> because most webmail solutions implement their own calendar and addressbook.
> 
> I wonder if I can interface Cyrus CalDav/CardDav in some opensource webmail, 
> such as SOGo, Roundcube, or Horde Webmail.
> 
> Could you help me or tell me about your experience about these integrations?
> 
> Thank you very much
> Greetings
> Marco
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: What you do with old account

[Sebastian Hagedorn]

Am 09.06.20 um 16:32 schrieb Adam Tauno Williams:
> On Tue, 2020-06-09 at 15:51 +0200, Albert Shih wrote:> 
>> After switching to cyrus imap, I think about how to do that.
>> If I'm correct I cannot just copy the file somewhere else, because cyrus
>> database would keep the information about the existance of the mailbox, so
>> what will the «state of the art» way to remove a mail account and all the
>> mail.
>> And how what would be the «state of the art» way to put it back ?
> I create a calendar event [task] to delete the mailbox and otherwise
> just leave it. If the account itself is disabled it cannot be accessed.
> 
> Putting things back-into a mailstore is too much of a pain with current
> storage prices.

We have about 90,000 accounts, and our current model is that we leave
expired accounts around for a year. The user can't login, and we don't
accept new mails, but it's still there in case the account is
re-activated. After one year the mailbox hierarchy is put into a .tgz
and written to tape. When that is done the account is permanently
deleted. If the user should come back, they get a completely new
account. I can't recall a single instance where the .tgz was ever
needed, but that's not my problem. After one more year the .tgz is
deleted from tape as well.
-- 
   .:.Sebastian Hagedorn - Weyertal 121 (Gebäude 133), Zimmer 2.02.:.
.:.Regionales Rechenzentrum (RRZK).:.
  .:.Universität zu Köln / Cologne University - ✆ +49-221-470-89578.:.
<>

smime.p7s
Description: S/MIME Cryptographic Signature

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: CalDav CardDav webmail client ?

[Xavier Bestel]

A bit on the heavy side, but Nextcloud's agenda component is CalDAV and
can access external CalDAV/ICS calendars. However its mail client will
only access its own address book (which is CardDAV).

Xav


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: CalDav CardDav webmail client ?

[Niels Dettenbach via Info-cyrus]

Am Freitag, 12. Juni 2020, 09:35:30 CEST schrieb Marco:

from my last experiences / knowledge:
> Roundcube, 
is not able to use "external" CardDav/CalDav server

But there seem to exist external (commercial?) plugins which allow "manual 
configurable" client mode (see bottom):
https://roundcubeplus.com/tutorials/caldav/creating-caldav-connection

some third party developments:
https://github.com/christian-putzke/Roundcube-CardDAV
https://packagist.org/packages/roundcube/carddav


> or Horde Webmail.
Horde 4 has it's own CardDAV/CalDAV server implementation wwhich is default. 
However, it provides usage / connectivity to "special" external IMAP based 
CardDAV/CalDAV within the Kolab project which is developed on/with cyrus, but 
still does not provide the new HTTP standards mechs..

"manual client mode":
=
But Horde is able to "use" CalDAV" als client by "external calendars" over 
HTTP ressources, but must be manually added / configured to a users account 
(with ugly manual auth). So this is probably not what you looking for.

Would be cool if Horde 4 get Cyrus HTTP support too in the future.


cheers,


niels.


-- 
 ---
 Niels Dettenbach
 Syndicat IT & Internet
 http://www.syndicat.com
 PGP: https://syndicat.com/pub_key.asc
 ---
 







Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cyrus Murder Environment Upgrade

[Wolfgang Breyha]

On 10/06/2020 20:47, Miguel Mucio Santos Moreira wrote:
> Wolfgang,
>
> If you don't mind, I'd like to know if when you upgraded backend servers
> from 2.4 version to 2.5 version you have an increase, in metadata size.Here
> we had 30% around of increase

Yes, cyrus.* are larger. Not sure about .cache, but .index for sure. And
the databases will grow as well if you change to twoskip.

We had to go back to skiplist on some hosts after the database reached
~200MB due to long locking times. The frontends started to have troubles
with long locks leading to timeouts on connect.

Greetings, Wolfgang
--
Wolfgang Breyha  | https://www.blafasel.at/
Vienna University Computer Center | Austria

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cyrus Murder Environment Upgrade

[Nic Bernstein]

Miguel,
That's perfectly normal, and I think it's even covered in the release 
notes... Nope, I'm wrong, the release notes mention the larger Memory 
footprint, due to more data and metadata being cached in memory.  But 
the on-disk size increases, too, as there is more information being held 
in indexes, etc.


The same is true, again, when going from 2.5 to 3.X

Cheers,
    -nic

On 6/10/20 1:47 PM, Miguel Mucio Santos Moreira wrote:

Wolfgang,

If you don't mind, I'd like to know if when you upgraded backend 
servers from 2.4 version to 2.5 version you have an increase, in 
metadata size.Here we had 30% around of increase


Thanks one more time

Greetings
--

*Miguel Moreira*
DTE/SRE/GRE - Gerência de Redes
+55(31)3339-1401
PRODEMGE - Companhia de Tecnologia da Informação do Estado de Minas Gerais


Aviso: Esta mensagem é destinada exclusivamente para a(s) pessoa(s) a 
quem é dirigida, podendo conter informação sigilosa e legalmente 
protegida. O uso impróprio será tratado conforme as normas da empresa 
e a legislação em vigor. Caso não seja o destinatário, favor notificar 
o remetente, ficando proibidas a utilização, divulgação, cópia e 
distribuição. Em Terça, Junho 09, 2020 10:39 -03, Wolfgang Breyha 
 escreveu:

Hi!

On 09/06/2020 14:56, Miguel Mucio Santos Moreira wrote:
> Dear Wolfgang,
>
> Firstly thanks for your answer, secondly I have one more doubt, 
during this
> time where the new Mupdate Master is receiving mailboxes 
information from

> backend servers, is necessary stopping comunications between frontend
> servers and mupdate master or none action is necessary besides that one
> you've mentioned before?
> We're concerned if frontend servers will connect to Mupdate Master and
> receive from it an information which there's no mailboxes anymore 
until the
> backend push entirely mailboxes information and this situation to 
cause any

> trouble.

I recommend the follow steps:
*) check that your currently running setup has no conflicts in 
mailboxes.db

by running "ctl_mboxlist -mw". It is possible that you see output if
somebody changes his mailboxes while you run the command, but it should
not appear again if you run the command a second time. If everything is
fine...

*) shut down all running cyrus
frontends first, then backends and mupdate last
*) backup your mailboxes.db on mupdate server
*) replace/update mupdate and start it with empty/removed mailboxes.db.
*) backup your mailboxes.db on the backends
*) do the ctl_mboxlist -m by hand on the backends (only one at the same
time) you can check if everything went ok with "-mw" at any time
afterwards.
*) start cyrus on backends

In our setup the frontends have mupdate running as well. I can't 
currently

remember if this is mandatory. If this is true in your setup then:
*) remove or rename the mailboxes.db database on the frontends and start
them. They will fetch the database immediately from the mupdate server.
This is visible in syslog as well and takes about 30 seconds in our
setup (~250MB mailboxes.db).
otherwise
*) start the frontends

At this point everything should be up and running again.

Watching syslog output all the time usually helps.

IIRC we updated the mupdate server to 2.5 first. Then we had a mix of 2.4
and 2.5 backends for some time. The 2.5 backends had some capabilities
suppressed. Frontends had 2.4 until all backends had 2.5. Last but not
least we upgraded to 2.5 on the frontends.

If you need more info/details feel free to ask.

Greetings, Wolfgang
--
Wolfgang Breyha  | https://www.blafasel.at/
Vienna University Computer Center | Austria



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus


--
Nic Bernstein   n...@nicbernstein.com
https://www.nicbernstein.com
https://www.linkedin.com/in/nic-b-26577a178/


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cyrus Murder Environment Upgrade

[Miguel Mucio Santos Moreira]

Wolfgang,

If you don't mind, I'd like to know if when you upgraded backend servers from 
2.4 version to 2.5 version you have an increase, in metadata size.Here we had 
30% around of increase

Thanks one more time

Greetings
--

Miguel Moreira
DTE/SRE/GRE - Gerência de Redes
+55(31)3339-1401
PRODEMGE - Companhia de Tecnologia da Informação do Estado de Minas Gerais


Aviso: Esta mensagem é destinada exclusivamente para a(s) pessoa(s) a quem é 
dirigida, podendo conter informação sigilosa e legalmente protegida. O uso 
impróprio será tratado conforme as normas da empresa e a legislação em vigor. 
Caso não seja o destinatário, favor notificar o remetente, ficando proibidas a 
utilização, divulgação, cópia e distribuição. Em Terça, Junho 09, 2020 10:39 
-03, Wolfgang Breyha  escreveu:Hi!

On 09/06/2020 14:56, Miguel Mucio Santos Moreira wrote:
> Dear Wolfgang,
>
> Firstly thanks for your answer, secondly I have one more doubt, during this
> time where the new Mupdate Master is receiving mailboxes information from
> backend servers, is necessary stopping comunications between frontend
> servers and mupdate master or none action is necessary besides that one
> you've mentioned before?
> We're concerned if frontend servers will connect to Mupdate Master and
> receive from it an information which there's no mailboxes anymore until the
> backend push entirely mailboxes information and this situation to cause any
> trouble.

I recommend the follow steps:
*) check that your currently running setup has no conflicts in mailboxes.db
by running "ctl_mboxlist -mw". It is possible that you see output if
somebody changes his mailboxes while you run the command, but it should
not appear again if you run the command a second time. If everything is
fine...

*) shut down all running cyrus
frontends first, then backends and mupdate last
*) backup your mailboxes.db on mupdate server
*) replace/update mupdate and start it with empty/removed mailboxes.db.
*) backup your mailboxes.db on the backends
*) do the ctl_mboxlist -m by hand on the backends (only one at the same
time) you can check if everything went ok with "-mw" at any time
afterwards.
*) start cyrus on backends

In our setup the frontends have mupdate running as well. I can't currently
remember if this is mandatory. If this is true in your setup then:
*) remove or rename the mailboxes.db database on the frontends and start
them. They will fetch the database immediately from the mupdate server.
This is visible in syslog as well and takes about 30 seconds in our
setup (~250MB mailboxes.db).
otherwise
*) start the frontends

At this point everything should be up and running again.

Watching syslog output all the time usually helps.

IIRC we updated the mupdate server to 2.5 first. Then we had a mix of 2.4
and 2.5 backends for some time. The 2.5 backends had some capabilities
suppressed. Frontends had 2.4 until all backends had 2.5. Last but not
least we upgraded to 2.5 on the frontends.

If you need more info/details feel free to ask.

Greetings, Wolfgang
--
Wolfgang Breyha  | https://www.blafasel.at/
Vienna University Computer Center | Austria

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: What you do with old account

[Adam Tauno Williams]

On Tue, 2020-06-09 at 15:51 +0200, Albert Shih wrote:> 
> After switching to cyrus imap, I think about how to do that.
> If I'm correct I cannot just copy the file somewhere else, because cyrus
> database would keep the information about the existance of the mailbox, so
> what will the «state of the art» way to remove a mail account and all the
> mail.
> And how what would be the «state of the art» way to put it back ?

I create a calendar event [task] to delete the mailbox and otherwise
just leave it. If the account itself is disabled it cannot be accessed.

Putting things back-into a mailstore is too much of a pain with current
storage prices.

-- 
Adam Tauno Williams, awill...@whitemice.org
Multi-Modal Activists Against Auto Dependent Development
resisting the unAmerican socialists of the Motorist hegemony
http://www.mmaaadd.org 

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cyrus Murder Environment Upgrade

[Miguel Mucio Santos Moreira]

Wolfgang,

I'm sure your help and experience with Cyrus Murder upgrading will save a lot 
of time and reduce the possibility of an eventual problem during the upgrade.

Thankful

--

Miguel Moreira
DTE/SRE/GRE - Gerência de Redes
+55(31)3339-1401
PRODEMGE - Companhia de Tecnologia da Informação do Estado de Minas Gerais


Aviso: Esta mensagem é destinada exclusivamente para a(s) pessoa(s) a quem é 
dirigida, podendo conter informação sigilosa e legalmente protegida. O uso 
impróprio será tratado conforme as normas da empresa e a legislação em vigor. 
Caso não seja o destinatário, favor notificar o remetente, ficando proibidas a 
utilização, divulgação, cópia e distribuição. Em Terça, Junho 09, 2020 10:39 
-03, Wolfgang Breyha  escreveu:Hi!

On 09/06/2020 14:56, Miguel Mucio Santos Moreira wrote:
> Dear Wolfgang,
>
> Firstly thanks for your answer, secondly I have one more doubt, during this
> time where the new Mupdate Master is receiving mailboxes information from
> backend servers, is necessary stopping comunications between frontend
> servers and mupdate master or none action is necessary besides that one
> you've mentioned before?
> We're concerned if frontend servers will connect to Mupdate Master and
> receive from it an information which there's no mailboxes anymore until the
> backend push entirely mailboxes information and this situation to cause any
> trouble.

I recommend the follow steps:
*) check that your currently running setup has no conflicts in mailboxes.db
by running "ctl_mboxlist -mw". It is possible that you see output if
somebody changes his mailboxes while you run the command, but it should
not appear again if you run the command a second time. If everything is
fine...

*) shut down all running cyrus
frontends first, then backends and mupdate last
*) backup your mailboxes.db on mupdate server
*) replace/update mupdate and start it with empty/removed mailboxes.db.
*) backup your mailboxes.db on the backends
*) do the ctl_mboxlist -m by hand on the backends (only one at the same
time) you can check if everything went ok with "-mw" at any time
afterwards.
*) start cyrus on backends

In our setup the frontends have mupdate running as well. I can't currently
remember if this is mandatory. If this is true in your setup then:
*) remove or rename the mailboxes.db database on the frontends and start
them. They will fetch the database immediately from the mupdate server.
This is visible in syslog as well and takes about 30 seconds in our
setup (~250MB mailboxes.db).
otherwise
*) start the frontends

At this point everything should be up and running again.

Watching syslog output all the time usually helps.

IIRC we updated the mupdate server to 2.5 first. Then we had a mix of 2.4
and 2.5 backends for some time. The 2.5 backends had some capabilities
suppressed. Frontends had 2.4 until all backends had 2.5. Last but not
least we upgraded to 2.5 on the frontends.

If you need more info/details feel free to ask.

Greetings, Wolfgang
--
Wolfgang Breyha  | https://www.blafasel.at/
Vienna University Computer Center | Austria

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cyrus Murder Environment Upgrade

[Wolfgang Breyha]

Hi!

On 09/06/2020 14:56, Miguel Mucio Santos Moreira wrote:
> Dear Wolfgang,
>
> Firstly thanks for your answer, secondly I have one more doubt, during this
> time where the new Mupdate Master is receiving mailboxes information from
> backend servers, is necessary stopping comunications between frontend
> servers and mupdate master or none action is necessary besides that one
> you've mentioned before?
> We're concerned if frontend servers will connect to Mupdate Master and
> receive from it an information which there's no mailboxes anymore until the
> backend push entirely mailboxes information and this situation to cause any
> trouble.

I recommend the follow steps:
*) check that your currently running setup has no conflicts in mailboxes.db
   by running "ctl_mboxlist -mw". It is possible that you see output if
   somebody changes his mailboxes while you run the command, but it should
   not appear again if you run the command a second time. If everything is
   fine...

*) shut down all running cyrus
   frontends first, then backends and mupdate last
*) backup your mailboxes.db on mupdate server
*) replace/update mupdate and start it with empty/removed mailboxes.db.
*) backup your mailboxes.db on the backends
*) do the ctl_mboxlist -m by hand on the backends (only one at the same
   time) you can check if everything went ok with "-mw" at any time
   afterwards.
*) start cyrus on backends

In our setup the frontends have mupdate running as well. I can't currently
remember if this is mandatory. If this is true in your setup then:
*) remove or rename the mailboxes.db database on the frontends and start
   them. They will fetch the database immediately from the mupdate server.
   This is visible in syslog as well and takes about 30 seconds in our
   setup (~250MB mailboxes.db).
otherwise
*) start the frontends

At this point everything should be up and running again.

Watching syslog output all the time usually helps.

IIRC we updated the mupdate server to 2.5 first. Then we had a mix of 2.4
and 2.5 backends for some time. The 2.5 backends had some capabilities
suppressed. Frontends had 2.4 until all backends had 2.5. Last but not
least we upgraded to 2.5 on the frontends.

If you need more info/details feel free to ask.

Greetings, Wolfgang
--
Wolfgang Breyha  | https://www.blafasel.at/
Vienna University Computer Center | Austria

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cyrus Murder Environment Upgrade

[Miguel Mucio Santos Moreira]

Dear Wolfgang,

Firstly thanks for your answer, secondly I have one more doubt, during this 
time where the new Mupdate Master is receiving mailboxes information from 
backend servers, is necessary stopping comunications between frontend servers 
and mupdate master or none action is necessary besides that one you've 
mentioned before?
We're concerned if frontend servers will connect to Mupdate Master and receive 
from it an information which there's no mailboxes anymore until the backend 
push entirely mailboxes information and this situation to cause any trouble.

One more time, thanks

Cheers!

--

Miguel Moreira
DTE/SRE/GRE - Gerência de Redes
+55(31)3339-1401
PRODEMGE - Companhia de Tecnologia da Informação do Estado de Minas Gerais


Aviso: Esta mensagem é destinada exclusivamente para a(s) pessoa(s) a quem é 
dirigida, podendo conter informação sigilosa e legalmente protegida. O uso 
impróprio será tratado conforme as normas da empresa e a legislação em vigor. 
Caso não seja o destinatário, favor notificar o remetente, ficando proibidas a 
utilização, divulgação, cópia e distribuição. Em Terça, Junho 09, 2020 05:31 
-03, Wolfgang Breyha  escreveu:On 08/06/2020 17:37, Miguel 
Mucio Santos Moreira wrote:
> Now we're in doubt about how is the best solution to replace the mupdate
> master server for a new one.
> Nowadays we have around 16K mailboxes.

IIRC we simply replaced the mupdate server and did a "ctl_mboxlist -m" on
all backends to fill it. Shouldn't take that long since we did it with 130k
on 8 backends in under 20 minutes (or even shorter).

Greetings, Wolfgang
--
Wolfgang Breyha  | https://www.blafasel.at/
Vienna University Computer Center | Austria

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cyrus Murder Environment Upgrade

[Wolfgang Breyha]

On 08/06/2020 17:37, Miguel Mucio Santos Moreira wrote:
> Now we're in doubt about how is the best solution to replace the mupdate
> master server for a new one.
> Nowadays we have around 16K mailboxes.

IIRC we simply replaced the mupdate server and did a "ctl_mboxlist -m" on
all backends to fill it. Shouldn't take that long since we did it with 130k
on 8 backends in under 20 minutes (or even shorter).

Greetings, Wolfgang
--
Wolfgang Breyha  | https://www.blafasel.at/
Vienna University Computer Center | Austria

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Sieve EditHeaders and Logging

[David Moyes]

On 05/06/2020 05:29, ellie timoney wrote:
> Hi David,
> 
>> Is it possible to enable the "editheaders" sieve extension? if so, how?
> 
> Not in 3.0, but it's available in 3.2

I plan to update to 3.2 once I have 3.0 working in my environment (I'm
migrating an existing legacy 2.5.3 server).

> 
>> Are sieve actions logged anywhere, e.g. to aid with debugging?
> 
> Generally? I don't know.  Maybe if you increase your syslog log level to 
> "debug" and add "debug: yes" to your imapd.conf?
> 
> 3.2 has an experimental "x-cyrus-log" extension, which adds an action that 
> produces a log line.  You probably don't want to allow this for user-supplied 
> sieve scripts, but if you have some sort of "sieve builder" system that your 
> users use (rather than uploading their own handcrafted sieve scripts), then, 
> if you were running 3.2, this extension could be useful.  On the master 
> branch, and therefore in the next major release (2021), this extension will 
> be formalised as "vnd.cyrus.log".

thanks. I have a number of questions around logging; I'll post another
question about that.


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Renaming mailboxes

[Marco]

Hello David,

Il 27/05/2020 16:48, David Moyes ha scritto:


I want to rename a mailbox (and all of its descendants), however I am
struggling with what, on the surface, would appear to be a simple task.

I first tried to use cyradm rename command to rename user.foo to
user@example.com, like this:

 localhost> rename user.foo user@example.com

However the rename command does not appear to be recursive, and I could
not find any argument to tell it to recurse. Do I have this right or am
I doing something wrong?


from what I remember, the rename is "recursive". If you login as an 
admin you can simply rename all users mailboxes from a name to a new 
name in a single step.


If you login as the user, then you have to move all folders as you 
describe below. Pay attention to specialuse folders and to the xapian 
indexes which could increase a lot.


I didn't remember on cyradm, but I currently use Cyrus::IMAP::Admin for 
almost all my scripts. I don't know if something of this could be useful 
for you:


https://github.com/falon/cyr_scripts
https://cloudsmith.io/~csi/repos/cyrus-scripts/packages/

(see at cyr_moveMailboxPart.pl, cyr_moveINBOX.pl)

Greetings
Marco


I decided to try it in a Perl script using Cyrus::IMAP::Admin. By doing
a `list` I could get the mailbox and its descendants like this:

 @mailboxes = $client->list("user/$oldname");
 @mailboxes_sub = $client->list("user/$oldname/*");
 push (@mailboxes , @mailboxes_sub);

(I found that listing "user/$oldname*" got me user.foo but also
user.foobar so I had to combine two calls to prevent that.)

Then I could iterate over the mailboxes doing

 $client->rename( $oldmailbox, $mailbox );

However, nothing works after the first rename; the program just exits,
not even "or die" error messages are displayed. In case it means
anything, the exit status is 141 which I think indicates a SIGPIPE (as
confirmed using `kill -l "$?"` which returns `PIPE`).

I did also notice a similar thing in cyradm, that after doing a rename
that subsequent action caused cyradm to exit:

localhost> rename user/testuser/Trash user/t...@example.net/Trash
renamemailbox:
localhost> lm  # this curiously returns nothing!

localhost> echo $?
0
localhost> lm
[cyrus@fee7fffd9be7 ~]$ echo $?
141


Any help would be greatly appreciated. I am currently using cyrus-imapd
3.0.13-3 on Arch Linux.



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus




Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cyrus IMAP 3.2.1 released

[ellie timoney]

Hi Marco,

On Thu, Jun 4, 2020, at 11:04 PM, Marco wrote:
> On 29/05/2020 06:20, ellie timoney has written:
> > The Cyrus team is proud to announce the immediate availability of a new 
> > version of Cyrus IMAP: 3.2.1
> 
> Hello,
> 
>   in Redhat EL8 I still fail these tests:
> 
> ERRORS:
> Rename.rename_inbox
>   Perl exception: Errors found in syslog
>   at Cassandane/Instance.pm line 1322,  line 91.

You probably saw on github -- I've just pushed a fix to Cassandane which I 
think should fix up the remaining syslog-related failures...

Though, Rename.rename_inbox is a special exception.  This one will probably 
always error on 3.2.  There's a real (niche) bug, which this test is detecting. 
 It's existed for a very long time, but we're not sure how to fix it yet, so 
you might as well just ignore it.

Though... it's very interesting that it FOUND errors in syslog, when the usual 
RedHat problem is not being able to find the syslog output.  Curious?

> FAILS:
> 
> 1) Admin.imap_admins
>   expected 'ok', got 'no' at Cassandane/Cyrus/Admin.pm line 90.

This is interesting.  I'd like to see the full output please!

> 2) Caldav.supports_event
>   Boolean assertion failed at 
> /usr/share/perl5/vendor_perl/Test/Unit/Exception.pm line 13.

I think we discussed Caldav.supports_event before -- it passes here, still no 
idea what the difference is.  Maybe it's a libical thing?  What version is your 
libical?

> 3) ImapTest.append-binary
>   Boolean assertion failed at 
> /usr/share/perl5/vendor_perl/Test/Unit/Exception.pm line 13.
> 4) ImapTest.fetch-binary-mime
>   Boolean assertion failed at 
> /usr/share/perl5/vendor_perl/Test/Unit/Exception.pm line 13.
> 5) ImapTest.urlauth-binary
>   Boolean assertion failed at 
> /usr/share/perl5/vendor_perl/Test/Unit/Exception.pm line 13.

The "ImapTest.*-binary*" tests were known to be not-quite-right, and I believe 
have been removed from ImapTest-- my build is from 20190509 and doesn't contain 
them.  If you can't update ImapTest, it's reasonable to just suppress these 
three instead.

> 10) Cassandane::Test::Core.core_files_*
>didn't match /(?^:Core files found)/ at Cassandane/Test/Core.pm line 96.

The RedHat environment is probably preventing core files from being created, in 
which case the Test::Core tests will always fail.  That being the case, I'd 
suggest just suppressing them.

> The details are attached, if it could be of interest.

There's no attachment, did the mailing list eat it?  Feel free to send to me 
directly

If you update to the latest Cassandane (for the syslog fixes), then you'll 
start seeing failures on 3.2.1 from these tests:

JMAPEmail.blob_get
JMAPEmail.email_query_guidsearch_mixedfilter

These tests have been updated to detect bugs that will be fixed in 3.2.2, but 
of course they fail for 3.2.1 cause the bugs still exist in that version.

Cheers,

ellie

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Sieve EditHeaders and Logging

[ellie timoney]

Hi David,

> Is it possible to enable the "editheaders" sieve extension? if so, how?

Not in 3.0, but it's available in 3.2

> Are sieve actions logged anywhere, e.g. to aid with debugging?

Generally? I don't know.  Maybe if you increase your syslog log level to 
"debug" and add "debug: yes" to your imapd.conf?

3.2 has an experimental "x-cyrus-log" extension, which adds an action that 
produces a log line.  You probably don't want to allow this for user-supplied 
sieve scripts, but if you have some sort of "sieve builder" system that your 
users use (rather than uploading their own handcrafted sieve scripts), then, if 
you were running 3.2, this extension could be useful.  On the master branch, 
and therefore in the next major release (2021), this extension will be 
formalised as "vnd.cyrus.log".

Cheers,

ellie

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Object Storage and Cyrus IMAP

[ellie timoney]

On Fri, Jun 5, 2020, at 4:48 AM, Albert Shih wrote:
> Le 04/06/2020 à 10:23:12+0200, Marco a écrit
> > Hello,
> >
> >I see that Cyrus IMAP 3 can interface with some Object Storage such as
> > Caringo or OpenIO.
> >
> > Is anyone using these solutions?
> >
> > I would like to know how I can find more details about these deployment,
> > other than the brief description in imapd.conf man page.
> >
> > In particular I would like to know if these interfaces are stable and
> > supported in future releases of Cyrus IMAP.
> >
> > Do you plan to add wider support to object storage, maybe by adopting some
> > standard vendor independent?
> >
> > I thank you very much for every info you could provide.
> 
> No sure if my informations are still correct, but long time ago, I ask
> openio about that. They say the connector between openIO and cyrus are
> maintained by openIO, it's not opensource and you need to pay a licence.
> 
> And when Cyrus make a new release openio would make adjustment to make it
> work.
> 
> I not sure who use that, but as I ear they(openIO) got few customer use
> this solution on a very large scale > 1Po.

Our object storage support was contributed by one of those two vendors (i.e 
Caringo or OpenIO, though I don't remember which).  As I understand it, they 
implemented support in Cyrus for both backends, to ensure that the object 
storage support was generalised, not specific to their own product.  This might 
have been a condition we imposed for accepting their patches?

Anyway, in theory, some third backend (whether closed- or open-source) could be 
similarly integrated, now that the abstract support is there.  I don't know if 
such a third backend even exists.

Looking through commit history, the last commits from the vendor developers to 
the object storage code were just before the release of 3.0.0.  So I would 
expect this works okay for 3.0 deployments.  I'm not sure about 3.2 -- we 
haven't had any specific updates, so maybe it works okay and doesn't need any?  
Or maybe it won't work until it's updated, and their customers are simply 
staying on 3.0 for the time being.

I guess this is kind of a long-winded way of saying, you probably need to speak 
to those vendors about this.  Whether Cyrus supports their backend is almost 
incidental compared to the larger question of whether (and how) they support 
their backend being used from Cyrus.

Cheers,

ellie

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Replication and Deleted Files

[Ian Batten via Info-cyrus]

On Thu 04 Jun 2020 at 18:57:37, Michael Menge 
(michael.me...@zdv.uni-tuebingen.de) wrote:


>

you also need to run cyr_expire on the "new_server" to remove the old
expunged mails and deleted folders.


Obvious when you try it!    Thanks so much.  

Expired 23 and expunged 7617 out of 289060 messages from 268 mailboxes

For some reason I had decided that you only ran cyr_expire on the master, and I 
was quite emphatic about it some years ago:

  # expire old stuff: dups 7 days, keep deletions for 3 days
  # XXX XXX XXX expire does not run on replica, does run on master XXX XXX XXX
  # expire        cmd="cyr_expire -E 7 -X 3 -D 3" at=0100

Thank you again,.,.I shall be back in another 25 years with another query :-)

ian

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Object Storage and Cyrus IMAP

[Albert Shih]

Le 04/06/2020 à 10:23:12+0200, Marco a écrit
> Hello,
>
>I see that Cyrus IMAP 3 can interface with some Object Storage such as
> Caringo or OpenIO.
>
> Is anyone using these solutions?
>
> I would like to know how I can find more details about these deployment,
> other than the brief description in imapd.conf man page.
>
> In particular I would like to know if these interfaces are stable and
> supported in future releases of Cyrus IMAP.
>
> Do you plan to add wider support to object storage, maybe by adopting some
> standard vendor independent?
>
> I thank you very much for every info you could provide.

No sure if my informations are still correct, but long time ago, I ask
openio about that. They say the connector between openIO and cyrus are
maintained by openIO, it's not opensource and you need to pay a licence.

And when Cyrus make a new release openio would make adjustment to make it
work.

I not sure who use that, but as I ear they(openIO) got few customer use
this solution on a very large scale > 1Po.

Regards
--
Albert SHIH
DIO bâtiment 15
xmpp: j...@obspm.fr
Heure local/Local time:
Thu 04 Jun 2020 08:44:56 PM CEST

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: imap clients say i have 4K messages but spool has 12894 files

[Brian J. Murrell]

Interestingly, through no action on my (as admin) part, this problem
seems to have resolved itself on May 31.  According to my backup, on
May 29 for my main inbox, user.brian, there were 13339 files on the
disk but on May 31's backup there are only 4136.

IMAP has always reported in the neighborhood of the 4K messages.

Strange.

b.



signature.asc
Description: This is a digitally signed message part

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Replication and Deleted Files

[Michael Menge]

Hi,

Quoting Ian Batten via Info-cyrus :


Hi, long-time Cyrus user (25 years, I think), but stumped on this one…

I have an ancient Cyrus 2.5.11 on Solaris 11 installation I am  
trying to migrate off.  The strategy is to run rolling replication  
onto the new server (3.0.8-6+deb10u4 on Debian 10.4), and then point  
the DNS record at the new server.  With Covid, this has become more  
protracted than I would like, as I don’t want to accidentally mess  
up users who are isolating, so the replication has been running for  
some weeks.


The replication structure is old-server -> new-server -> (backup1,  
backup2) where backup1 and backup2 are configured as separate  
channels on new-server.  This has been running seemingly correctly  
for about three months now.


Today I decided to check all was well by using rsync -an to confirm  
that the replicas have everything that is on the master.  They do,  
in that using 


rsync -anvO --size-only  --exclude='cyrus.*'  
root@mail:/var/imap/partition1/user/ /var/imap/partition1/user 


where “mail” is the old server shows that there are no messages  
missing (—size-only because there’s some time slew in a few places,  
usually only of a few seconds, but up to a day in others).


However, reversing it:

rsync -anvO --size-only  --exclude='cyrus.*'  
/var/imap/partition1/user/ root@mail:/var/imap/partition1/user


Shows that there are a _lot_ of files on the replicas which are not  
on the master, some of them relating to recent deletions, but some  
of them seemingly quite old.  I am using:


delete_mode: delayed
expunge_mode: delayed

everywhere, running cyr_expire on the master but not on the  
replicas.  I have enough bandwidth that sync_reset and re-sync is  
realistic, but I’d rather not have to do that immediately prior to a  
cut-over.   These old files are a worry because if I ever had to  
reconstruct one of the mailboxes, presumably the deleted (I think)  
messages would all reappear.  Does anyone have any suggestions?




you also need to run cyr_expire on the "new_server" to remove the old
expunged mails and deleted folders.

I haven't used replication for backup but I suspect that cyr_expire is
also required your backup servers






M.MengeTel.: (49) 7071/29-70316
Universität Tübingen   Fax.: (49) 7071/29-5912
Zentrum für Datenverarbeitung  mail:  
michael.me...@zdv.uni-tuebingen.de

Wächterstraße 76
72074 Tübingen


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cyrus IMAP 3.2.1 released

[Marco]

On 29/05/2020 06:20, ellie timoney has written:

The Cyrus team is proud to announce the immediate availability of a new version 
of Cyrus IMAP: 3.2.1


Hello,

 in Redhat EL8 I still fail these tests:

ERRORS:
Rename.rename_inbox
 Perl exception: Errors found in syslog
 at Cassandane/Instance.pm line 1322,  line 91.

FAILS:

1) Admin.imap_admins
 expected 'ok', got 'no' at Cassandane/Cyrus/Admin.pm line 90.
2) Caldav.supports_event
 Boolean assertion failed at 
/usr/share/perl5/vendor_perl/Test/Unit/Exception.pm line 13.

3) ImapTest.append-binary
 Boolean assertion failed at 
/usr/share/perl5/vendor_perl/Test/Unit/Exception.pm line 13.

4) ImapTest.fetch-binary-mime
 Boolean assertion failed at 
/usr/share/perl5/vendor_perl/Test/Unit/Exception.pm line 13.

5) ImapTest.urlauth-binary
 Boolean assertion failed at 
/usr/share/perl5/vendor_perl/Test/Unit/Exception.pm line 13.

10) Cassandane::Test::Core.core_files_*
  didn't match /(?^:Core files found)/ at Cassandane/Test/Core.pm line 96.

The details are attached, if it could be of interest.
The Cassandane commit is very recent: 
68e4890346edeb68463d20f394a37f0862bef2ae


I think all these failure are somewhere related to Redhat environment... 
I'll skip these tests in Cyrus IMAP 3.2.1 build.


Regards
Marco

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: imap clients say i have 4K messages but spool has 12894 files

[Patrick Boutilier]

On 6/4/20 7:45 AM, Brian J. Murrell wrote:

On Wed, 2020-06-03 at 19:35 -0400, Ken Murchison wrote:

Brian,

Trying running 'unexpunge -l' on the mailbox in question.


This avenue has already been explored earlier in this thread:

https://lists.andrew.cmu.edu/pipermail/info-cyrus/2020-May/041258.html

To save the effort of re-reading the message:

# sudo -u cyrus bash -c "/usr/lib/cyrus-imapd/unexpunge -l user.brian"
[nothing returned]

So this is looking more like a "bad accounting" problem than something
typically operational.

But how to reconcile it?

It seems to me that a process of comparing what's in the index to
what's on disk to account for the orphans is needed.  I just don't know
what that process is.  I probably just don't know the toolset well
enough to know which tools to apply and how.  mbexamine seems a
candidate but I'm not sure how to interpret it's output to this task.
Or maybe there other/better tools?

Any suggestions?



Have you looked in some of the orphaned messages to see if they are 
emails you have deleted before? My thought would be to move these 
orphaned messages out of /var/spool/imap/b/user/brian . Then delete and 
expunge a few messages using your mail client and see if they are also 
removed from /var/spool/imap/b/user/brian







Cheers,
b.



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

<>
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: imap clients say i have 4K messages but spool has 12894 files

[Ken Murchison]

You could try 'reconstruct -R' which should force a re-parsing of all 
message files in the mailboxes directory.  Note that if this works, you 
will have 8k new messages show up in your mailbox. Adding -n may just 
report what reconstruct will do rather than actually doing it.



On 6/4/20 6:45 AM, Brian J. Murrell wrote:

On Wed, 2020-06-03 at 19:35 -0400, Ken Murchison wrote:

Brian,

Trying running 'unexpunge -l' on the mailbox in question.

This avenue has already been explored earlier in this thread:

https://lists.andrew.cmu.edu/pipermail/info-cyrus/2020-May/041258.html

To save the effort of re-reading the message:

# sudo -u cyrus bash -c "/usr/lib/cyrus-imapd/unexpunge -l user.brian"
[nothing returned]

So this is looking more like a "bad accounting" problem than something
typically operational.

But how to reconcile it?

It seems to me that a process of comparing what's in the index to
what's on disk to account for the orphans is needed.  I just don't know
what that process is.  I probably just don't know the toolset well
enough to know which tools to apply and how.  mbexamine seems a
candidate but I'm not sure how to interpret it's output to this task.
Or maybe there other/better tools?

Any suggestions?

Cheers,
b.



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus


--
Kenneth Murchison
Senior Software Developer
Fastmail US LLC


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: imap clients say i have 4K messages but spool has 12894 files

[Brian J. Murrell]

On Thu, 2020-06-04 at 09:30 +1000, Ian Willis wrote:
> Hi Brian,

Hi Ian,

> The answer to your question is that yes, UID appears to correlate
> with
> the message file name. 

Thanks.

> At a guess something appears significantly awry.

Indeed.

> Have you tried create a separate mail user. Copy your existing
> message
> over via imap to the new folder. 
> Delete and expunge the original mailbox and recreate, recopy.

I have considered that.  But I suspect that is going to cause the
message numbers on the disk to be recreated.  Since this seems to
affect many (all?) folders for many (again, all?) users, that would
result in trashing the efficiency of my incremental backup.

> In the longer term I would be tempted to move to a newer version of
> cyrus

I'm stuck with what my distro vendor supplies.  That said, once the
cause of this problem, or it's reproducibiilty can be confirmed, distro
vendor will be getting a ticket to resolve this in some way.  But the
first step is identifying the issue, resolving it and seeing if it
reproduces.

> or if you have the patience closely monitor the file-system to
> debug how this is occurring.

Right.  I think the first step is to figure out how to get things back
to normal so that it can be monitored more closely and the discrepancy
is no longer a haystack and will be more incrementally obvious when it
does happen.  Figuring out how to get back to normal will also provide
the tools/process to monitor on an ongoing basis to see why it's
happening.

I'm just not sure how to get back to normal at this point.

Cheers,
b.



signature.asc
Description: This is a digitally signed message part

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: imap clients say i have 4K messages but spool has 12894 files

[Brian J. Murrell]

On Wed, 2020-06-03 at 19:35 -0400, Ken Murchison wrote:
> Brian,
> 
> Trying running 'unexpunge -l' on the mailbox in question.

This avenue has already been explored earlier in this thread:

https://lists.andrew.cmu.edu/pipermail/info-cyrus/2020-May/041258.html

To save the effort of re-reading the message:

# sudo -u cyrus bash -c "/usr/lib/cyrus-imapd/unexpunge -l user.brian"
[nothing returned]

So this is looking more like a "bad accounting" problem than something
typically operational.

But how to reconcile it?

It seems to me that a process of comparing what's in the index to
what's on disk to account for the orphans is needed.  I just don't know
what that process is.  I probably just don't know the toolset well
enough to know which tools to apply and how.  mbexamine seems a
candidate but I'm not sure how to interpret it's output to this task. 
Or maybe there other/better tools?

Any suggestions?

Cheers,
b.



signature.asc
Description: This is a digitally signed message part

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: imap clients say i have 4K messages but spool has 12894 files

[Ken Murchison]

Brian,

Trying running 'unexpunge -l' on the mailbox in question.  This will 
list expunged, but not-yet-removed-from-disk messages.


cyr_expire is used to removed expunged messages from disk.



On 6/3/20 7:30 PM, Ian Willis wrote:

Hi Brian,

The answer to your question is that yes, UID appears to correlate with 
the message file name.

At a guess something appears significantly awry.

To resolve the immediate issue.
Have you tried create a separate mail user. Copy your existing message 
over via imap to the new folder.

Delete and expunge the original mailbox and recreate, recopy.

In the longer term I would be tempted to move to a newer version of 
cyrus or if you have the patience closely monitor the file-system to 
debug how this is occurring.


Kind Regards
Ian

-Original Message-
*From*: Brian J. Murrell <mailto:%22brian%20j.%20murrell%22%20%3cbr...@interlinx.bc.ca%3e>>
*To*: info-cyrus@lists.andrew.cmu.edu 
<mailto:info-cyrus@lists.andrew.cmu.edu>
*Subject*: Re: imap clients say i have 4K messages but spool has 12894 
files

*Date*: Mon, 01 Jun 2020 21:45:42 -0400

On Tue, 2020-05-26 at 09:33 -0400, Brian J. Murrell wrote:
Hi.
Every IMAP client I query my cyrus imapd 2.4.17 server with says I
have
~4K messages in my INBOX.  However when I do a listing of
/var/spool/imap/b/user/brian/ it shows almost 13K files.
None of these include messages which have been deleted but not
expunged.  I manually expunge my mailbox many times per day.
If I'm understanding mbexamine's output correctly, I have files on
disk
that are not being displayed by mbexmine.  My understanding of
mbexamine's output is that on a line formatted as such:
01> UID:00089183   INT_DATE:[redacted] SENTDATE:[redacted]
SIZE:1537
that the 00089183 is the reference to the file on the spool in
/var/spool/imap/b/user/brian/89183.
Is that correct?  If so, I definitely have files on the disk which
are
not found in any "01> UID" line from mbexamine.  ~9600 of them.
That seems to make up the difference between what an IMAP client sees
and how many files are on disk.
I also have multiple occurrences of the same "01> UID:" and where
there are no matching files on the disk.  Should that be possible?
So how come the huge discrepancies and how do I reconcile them?
No other thoughts on how I can reconcile this gross discrepancy?
Ultimately I have an IMAP spool that is growing without bound due to
messages continuing to live on the spool beyond their life in the index
and getting orphaned.
Cheers,
b.

Cyrus Home Page:http://www.cyrusimap.org/
List Archives/Info:http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus


--
Kenneth Murchison
Senior Software Developer
Fastmail US LLC


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: imap clients say i have 4K messages but spool has 12894 files

[Ian Willis]

Hi Brian,

The answer to your question is that yes, UID appears to correlate with
the message file name. 
At a guess something appears significantly awry.

To resolve the immediate issue.
Have you tried create a separate mail user. Copy your existing message
over via imap to the new folder. 
Delete and expunge the original mailbox and recreate, recopy.

In the longer term I would be tempted to move to a newer version of
cyrus or if you have the patience closely monitor the file-system to
debug how this is occurring.

Kind Regards 
Ian

-Original Message-
From: Brian J. Murrell 
To: info-cyrus@lists.andrew.cmu.edu
Subject: Re: imap clients say i have 4K messages but spool has 12894
files
Date: Mon, 01 Jun 2020 21:45:42 -0400

On Tue, 2020-05-26 at 09:33 -0400, Brian J. Murrell wrote:
Hi.
Every IMAP client I query my cyrus imapd 2.4.17 server with says
Ihave~4K messages in my INBOX.  However when I do a listing
of/var/spool/imap/b/user/brian/ it shows almost 13K files.
None of these include messages which have been deleted but
notexpunged.  I manually expunge my mailbox many times per day.
If I'm understanding mbexamine's output correctly, I have files
ondiskthat are not being displayed by mbexmine.  My understanding
ofmbexamine's output is that on a line formatted as such:
01> UID:00089183   INT_DATE:[redacted]
SENTDATE:[redacted]SIZE:1537  
that the 00089183 is the reference to the file on the spool
in/var/spool/imap/b/user/brian/89183.

Is that correct?  If so, I definitely have files on the disk
whicharenot found in any "01> UID" line from mbexamine.  ~9600 of
them. That seems to make up the difference between what an IMAP client
seesand how many files are on disk.
I also have multiple occurrences of the same "01> UID:" and
wherethere are no matching files on the disk.  Should that be possible?
So how come the huge discrepancies and how do I reconcile them?
No other thoughts on how I can reconcile this gross discrepancy?
Ultimately I have an IMAP spool that is growing without bound due
tomessages continuing to live on the spool beyond their life in the
indexand getting orphaned.
Cheers,b.

Cyrus Home Page: http://www.cyrusimap.org/List Archives/Info: 
http://lists.andrew.cmu.edu/pipermail/info-cyrus/To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: imap clients say i have 4K messages but spool has 12894 files

[Brian J. Murrell]

On Tue, 2020-05-26 at 09:33 -0400, Brian J. Murrell wrote:
> Hi.
> 
> Every IMAP client I query my cyrus imapd 2.4.17 server with says I
> have
> ~4K messages in my INBOX.  However when I do a listing of
> /var/spool/imap/b/user/brian/ it shows almost 13K files.
> 
> None of these include messages which have been deleted but not
> expunged.  I manually expunge my mailbox many times per day.
> 
> If I'm understanding mbexamine's output correctly, I have files on
> disk
> that are not being displayed by mbexmine.  My understanding of
> mbexamine's output is that on a line formatted as such:
> 
> 01> UID:00089183   INT_DATE:[redacted] SENTDATE:[redacted]
> SIZE:1537  
> 
> that the 00089183 is the reference to the file on the spool in
> /var/spool/imap/b/user/brian/89183.
> 
> Is that correct?  If so, I definitely have files on the disk which
> are
> not found in any "01> UID" line from mbexamine.  ~9600 of them. 
> That seems to make up the difference between what an IMAP client sees
> and how many files are on disk.
> 
> I also have multiple occurrences of the same "01> UID:" and where
> there are no matching files on the disk.  Should that be possible?
> 
> So how come the huge discrepancies and how do I reconcile them?

No other thoughts on how I can reconcile this gross discrepancy?

Ultimately I have an IMAP spool that is growing without bound due to
messages continuing to live on the spool beyond their life in the index
and getting orphaned.

Cheers,
b.



signature.asc
Description: This is a digitally signed message part

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cyrus LMTP Delivery Error

[Infraestructura TIC - UNNOBA]

Hello, David.
The code is in imap/lmtp_err..et, according to this bug report:

https://github.com/cyrusimap/cyrus-imapd/issues/3035

Good luck!

Javier.-



El 28/5/20 a las 13:20, David Faller escribió:
>
> Thanks for your solution, could you provide detailed which line did
> you change?
>
> under imap/lmptd.c or lmtpd.h which line did you adjust?
>
>  
>
> Best Regards,
>
> David Faller
>


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cyrus LMTP Delivery Error

[Infraestructura TIC - UNNOBA]

Finally, I downloaded source from

https://github.com/cyrusimap/cyrus-imapd/releases/download/cyrus-imapd-3.2.0/cyrus-imapd-3.2.0.tar.gz

untar into /usr/src
installed dependencies  (pkg-config, libsasl2-dev, libicu-dev, 
libjansson-dev, libssl-dev, bison, flex...)


edit /usr/src/cyrus-imapd-3.2.0/imap/lmtpd
and change the first line for

ec LMTP_OK,
   "250 2.1.5 Ok SESSIONID=<*%s*>"

./configure
./make

and

mv /usr/lib/cyrus/bin/lmtpd /usr/lib/cyrus/bin/lmtpd.old
ln -s /usr/src/cyrus-imapd-3.2.0/imap/lmtpd /usr/lib/cyrus/bin/lmtpd


And now, it's working until upgraded Debian package 3.2.1 is delivered.


Thanks!



El 28/5/20 a las 10:58, David Faller escribió:


Dear all, we’re running into the same issue after upgrade cyrus to 
3.2.0-5~bpo10+1


We had try to downgrade cyrus but after this we got kernel issues 
which prevent thunderbird connections.


May 28 15:43:26 CGSG cyrus/master[2136]: process type:SERVICE 
name:lmtpchroot path:/usr/lib/cyrus/bin/lmtpd age:0.057s pid:2694 
signaled to death by signal 6 (Aborted)


May 28 15:43:26 CGSG postfix/lmtp[2693]: 446EEDE17EF: 
to=, 
relay=groupware.uc-central.net[/local/socket/lmtp], delay=4.2, 
delays=4.1/0.01/0.02/0.04, dsn=4.4.2, status=deferred (lost connection 
with groupware.uc-central.net[/local/socket/lmtp] while sending end of 
data -- message may be sent more than once)


Best Regards



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus




Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus