Re: cyrus-imapd exporting databases failed on shutdown - deliver.db.skiplist 2048M

[Simon Matter via Info-cyrus]

> 2020. 06.  20, szombat keltezéssel 21.31-kor Simon Matter ezt írta:
>> Hi,
>>
>> The question is why is the deliver db > 2GB in skiplist format? Is it
>> normal or do you have a corrupt BDB db or does your db pruning not work
>> for deliverdb. I think that should be something like 'delprune
>> cmd="cyr_expire -D 7 -E 3 -X 7" at=0400' in cyrus.conf.
>>
>> I think the easiest way would be to make sure you have pruning
>> configured
>> correctly, then change config of deliver db to skiplist, and start
>> without
>> a db so a new, empty deliver db is created.
>>
>> Then have an eye on the db file to see if it grows again to almost 2GB.
>> If
>> it doesn't grow so much, you should be fine.
>>
>> Regards,
>> Simon
>
> Hi,
>
> Something definitely not seems fine:
>
> -bash-3.2$ /usr/lib/cyrus-imapd/cyr_expire -E 3 -D 7 -X 7 -v

Please make sure the options here are also valid for your cyrus version.
However, I also guess your deliver.db is corrupted somehow. From my own
experience skiplist dbs are easier to handle than bdb and using skiplist
only has not shown any issues.

Regards,
Simon

>
> expunged 0 out of 0 messages from 0 mailboxes
>
> The deliver.db still about 48MB.
>
> Tomorrow I will continue.
>
> Thanks,
> István
>



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: cyrus-imapd exporting databases failed on shutdown - deliver.db.skiplist 2048M

[Simon Matter via Info-cyrus]

> Hi,
>
> I run into a problem on an old clearos server, where the cyrus shutdown
> always failed at step exporting databases.
> As I checked the situation using ps ax on an other console, I found
> that, it was exporting deliver.db.skiplist file, which failed after a
> lng time (some minutes).
> I checked that file on the filesystem, I saw the file size is 2048MB,
> which seems a limit for me and I suspect the problem should be that,
> the 32 bit cyrus cannot write more data to that file and caused the
> problem.
> As I read the db_export.log, that confirmed my theory, file size limit
> exceeded.

Hi,

The question is why is the deliver db > 2GB in skiplist format? Is it
normal or do you have a corrupt BDB db or does your db pruning not work
for deliverdb. I think that should be something like 'delprune 
cmd="cyr_expire -D 7 -E 3 -X 7" at=0400' in cyrus.conf.

I think the easiest way would be to make sure you have pruning configured
correctly, then change config of deliver db to skiplist, and start without
a db so a new, empty deliver db is created.

Then have an eye on the db file to see if it grows again to almost 2GB. If
it doesn't grow so much, you should be fine.

Regards,
Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: imap clients say i have 4K messages but spool has 12894 files

[Simon Matter via Info-cyrus]

> On Tue, 2020-05-26 at 08:47 -0500, Nic Bernstein wrote:
>>
>> |expunge_mode:| delayed
>>
>> The mode in which messages (and their corresponding cache
>> entries) are expunged. “semidelayed” mode is the old behavior
>> in
>> which the message files are purged at the time of the
>> EXPUNGE,
>> but index and cache records are retained to facilitate
>> QRESYNC.
>> In “delayed” mode, which is the default since Cyrus 2.5.0,
>
> So this doesn't apply to my 2.4.17 then does it?

How did you install your 2.4.17 server? Because you don't run the latest
of the 2.4 series I guess you have installed it from packages?

If so it's possible that you distribution uses different configuration
than the default, at build time. You should check this.

Regards,
Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: [Help] Cyrus 2.4.17 segfault

[Simon Matter via Info-cyrus]

> Hello,
>
>   we are experiencing a strange problem on cyrus-imapd 2.4.17.
>
> An user user/a has full ACL to another mailbox user/b. When the user/a
> SELECT a folder on user/b where he has access the imap process crashes.
>
> $ telnet cyrus.example.com 143
> Trying 10.10.10.10 ...
> Connected to cyrus.example.com.
> Escape character is '^]'.
> * OK [CAPABILITY IMAP4rev1 LITERAL+ ID ENABLE AUTH=PLAIN SASL-IR]
> cyrus.example.com Cyrus IMAP v2.4.17-Invoca-RPM-2.4.17-6.el6 server ready
> a authenticate plain dsfrrFwaWNlQGNzaS5pdABveGN5cnVzAENdfsdfMx
> a OK [CAPABILITY IMAP4rev1 LITERAL+ ID ENABLE ACL RIGHTS=kxte QUOTA
> MAILBOX-REFERRALS NAMESPACE UIDPLUS NO_ATOMIC_RENAME UNSELECT CHILDREN
> MULTIAPPEND BINARY CATENATE CONDSTORE ESEARCH SORT SORT=MODSEQ
> SORT=DISPLAY THREAD=ORDEREDSUBJECT THREAD=REFERENCES ANNOTATEMORE
> LIST-EXTENDED WITHIN QRESYNC SCAN XLIST URLAUTH URLAUTH=BINARY
> X-NETSCAPE LOGINDISABLED COMPRESS=DEFLATE IDLE] Success (no protection)
> SESSIONID=
> a SELECT "user/b/SOME THINGS/WATER"
> Connection closed by foreign host.
>
>
> Could you suggest me some way to resolve this problem?

I don't know but if it's a bug then it's probably solved in current
version 2.4.20.

Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: script to check quota failed

[Simon Matter via Info-cyrus]

> Hi,
>
> seems that there is a problem with your cyrus quota tree
> (/var/lib/cyrus/quota) or at least some of its files.

In the case of RHEL6 this will be /var/lib/imap/quota/.

I suggest to look at the quota files there, they have names like
user. and are text files with two lines. Should be easy to find
the broken file.

Regards,
Simon

>
> Maybe strace command may help you find which files are corrupt:
> su - cyrus -c "strace /usr/lib/cyrus-imapd/quota user"
>
> Anthony Prades
>
> On 3/18/19 2:32 PM, Stephane Branchoux wrote:
>> Hello,
>>
>> We run cyrus-imapd-2.3.16-6 on a Red Hat Enterprise Linux Server
>> release 6.10.
>>
>> 2 days ago, a power crash occured in our Datacenter.
>>
>> After reboot of cyrus imap, all seems ok except our script to check
>> quota :
>>
>> su - cyrus -c "/usr/lib/cyrus-imapd/quota user"
>> failed building quota list for 'user': System I/O error: %m
>>
>> I tried :
>>
>>  su - cyrus -c "/usr/lib/cyrus-imapd/quota -f"
>> failed building quota list for '*': System I/O error: %m
>>
>> Any idea ?
>>
>> Many thanks in advance
>>
>>
>> 
>> Cyrus Home Page: http://www.cyrusimap.org/
>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>> To Unsubscribe:
>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Which imap command to rename a root mailbox while maintaining its partition

[Simon Matter via Info-cyrus]

> Hi,
>
> seems that there is a problem with your cyrus quota tree
> (/var/lib/cyrus/quota) or at least some of its files.

In the case of RHEL6 this will be /var/lib/imap/quota/.

I suggest to look at the quota files there, they have names like
user. and are text files with two lines. Should be easy to find
the broken file.

Regards,
Simon

>
> Maybe strace command may help you find which files are corrupt:
> su - cyrus -c "strace /usr/lib/cyrus-imapd/quota user"
>
> Anthony Prades
>
>
> On 3/18/19 10:13 AM, Marco wrote:
>> Il 18/03/2019 08:25, Thomas Cataldo ha scritto:
>>> Hi,
>>>
>>> On cyrus 2.4.x we used to run
>>> A1 RENAME mbox1 mbox2 our_partition
>>>
>>> When we run that on cyrus 3.0.8 we get :
>>> A1 NO Cross-server or cross-partition move w/rename not supported
>>>
>>> Is there a way to do this rename with one command with cyrus3.
>>> If we run A1 RENAME mbox1 mbox2
>>> the mailbox is renamed as expected but it moves from our_partition to
>>> default which is not what we want.
>>>
>>> Any suggestion ?
>>
>> Hello, no suggestions, I verified too. :( But if you have a mailbox
>> with:
>>
>> folder1 --> part1
>> folder2 --> part2
>>
>> then you can do:
>>
>> A1 RENAME folder1 folder2/folder1
>>
>> and folder1 is now on part2. So, under some circumstances, a cross
>> partition rename seems to be still supported...
>>
>> Bye
>> Marco
>>
>> Ps: on the above example unixhierarchysep: 1
>> 
>> Cyrus Home Page: http://www.cyrusimap.org/
>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>> To Unsubscribe:
>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
>



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Mailbox hierarchy determination?

[Simon Matter via Info-cyrus]

> On 3/14/19 9:46 AM, Simon Matter wrote:
>> I guess you're missing the fact that these options have possibly changed
>> between releases. Another thing is that distribution packages can also
>> alter the defaults and if they don't do it correct, they may "forget" to
>> also change the docs accordingly.
>>
>
> Thank you for your help with this.  I think what you're saying is that
> for my configuration I should have
>
>fulldirhash:   1
>hashimapspool: 1
>
>
> I'm not 100% certain how the defaults are set for my installation -- is
> there any way to check this?

Our RPMs create two files at build time:

/usr/share/cyrus-imapd/rpm/imapd.conf.dist
This is what the source distribution uses.

/usr/share/cyrus-imapd/rpm/imapd.conf.default
This is what the patched build uses.

Unfortunately I don't know of a way to get the same info at run time.

Regards,
Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Mailbox hierarchy determination?

[Simon Matter via Info-cyrus]

> On 3/14/19 3:50 AM, Simon Matter wrote:
>>
>> Dirhashing is controlled by "fulldirhash" and "hashimapspool".
>>
>
>
> Right.  This is what it says in imapd.conf:
>
> 
> fulldirhash: 0
>If enabled, uses an improved directory hashing scheme which hashes on
> the entire username instead of using just the first letter as the hash.
> This changes hash algorithm used for quota and user directories and if
> hashimapspool is enabled, the entire mail spool.
>
> Note that this option CANNOT be changed on a live system. The server
> must be quiesced and then the directories moved with the rehash utility.
>
> hashimapspool: 0
>   If enabled, the partitions will also be hashed, in addition to the
> hashing done on configuration directories. This is recommended if one
> partition has a very bushy mailbox tree.
> 
>
> I don't have these options enabled, and my interpretation of this
> description is that enabling them would facilitate my configuration, not
> the other way around.  This is what confused me in the first place.
>
> What am I missing?

I guess you're missing the fact that these options have possibly changed
between releases. Another thing is that distribution packages can also
alter the defaults and if they don't do it correct, they may "forget" to
also change the docs accordingly.

Regards,
Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Mailbox hierarchy determination?

[Simon Matter via Info-cyrus]

> The first couple of times I set up cyrus-imapd I used Debian packages
> which separated mailboxes in the default partition into folders based on
> the starting letter of the mailbox; i.e. something like:
>
>/var/spool/cyrus/mail/{[A-Z,a-z,0-9]}/user
>
> Since I have fewer than 50 users and since I found myself frequently
> "going to the files" so to speak to definitely determine the presence or
> absence of some message a user was concerned about, this hierarchy was
> something of a annoyance, so during the last major upgrade I switched
> this to:
>
> /srv/imap/{user1,user2,...,userN}
>
> i.e. flattening the folder structure and moving the default partition to
> a large non-system disk unaffected by /var volatility.
>
> Now for the life of me I can't figure out what setting in imapd.conf (if
> any) facilitated this structure (not the location of the default
> partition, but rather not using the [A-Z,a-z,0-9] subfolders.
>
> Last time I left the sieve folders divided by letter; in the next
> iteration I'd like to flatten that folder structure to.
>
> Looking at the Arch package post_install script, it pre-creates a ton of
> folders:
>
> --
>  # all kinds of directories needed for the IMAP spool
>  for subdir in imap/{,db,log,msg,proc,socket,sieve}
> spool/imap/{,stage.}; do
>  if [ ! -d /var/${subdir} ]; then
>  mkdir -m 0750 -p "/var/${subdir}"
>  fi
>  done
>  for subdir in imap/{user,quota,sieve} spool/imap; do
>  for i in a b c d e f g h i j k l m n o p q r s t v u w x y z \
>   A B C D E F G H I J K L M N O P Q R S T U V W X Y Z \
>   0 1 2 3 4 5 6 7 8 9; do
>  if [ ! -d /var/${subdir}/${i} ]; then
>  mkdir -m 0750 -p "/var/${subdir}/${i}"
>  fi
>  done
> --
>
> I'm wondering how much or if any of this is really necessary. Will cyrus
> autocreate the necessary folders if missing?

Dirhashing is controlled by "fulldirhash" and "hashimapspool".

IIRC creating the dirs like above is not needed, they are created on demand.

Regards,
Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cyrus-imapd 2.4.17: processes stick on mailbox locking resulting in total mailsystem failure

[Simon Matter via Info-cyrus]

> Hi Ellie
>
> Thanks a lot, I will try to build and test 2.4.20

Maybe try this:

http://www.invoca.ch/pub/packages/cyrus-imapd/RPMS/ils-7/SRPMS/cyrus-imapd-2.4.20-2.el7.src.rpm

Regards,
Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: sieve runtime error

[Simon Matter via Info-cyrus]

> Thanks, this got me looking into sendmail a little closer. I've never used
> the program and didn't realize a dummy sendmail binary was installed on my
> system. Actually installed sendmail and it works now, messages are being
> filtered.

You don't have to install sendmail, also postfix has a compatible sendmail
binary. Only the sendmail config in imapd.conf may not point to it.

Regards,
Simon

>
> On Tue, Dec 18, 2018 at 4:32 AM Simon Matter 
> wrote:
>
>> > I'm trying to setup sieve and getting the following error in my logs:
>> >
>> > Dec 17 10:36:07 bllmail01 cyrus/lmtp[14530]: sieve runtime error for
>> > jschaef...@harmonywave.net id
>> > :
>> > Reject: Sendmail process terminated normally, exit status 255
>> >
>> >
>> > I'm following the documentation here:
>> >
>> https://www.cyrusimap.org/imap/reference/admin/sieve.html?highlight=sieve#testing-the-sieve-server
>> >
>> > I'm trying to get sieve working on my IMAP server. Using Ubuntu 16.04
>> with
>> > the cyrus-imapd 2.4.18-3 package. I'm using the test sieve script
>> shown
>> in
>> > the documentation to reject everything from my personal email:
>> >
>> > require ["reject","fileinto"];
>> > if address :is :all "From" "jschaeffer0...@gmail.com"
>> > {
>> > reject "testing";
>> > }
>> >
>> > I then connect using sieveshell, upload the file, and activate it:
>> >
>> > root@bllmail01:~# sieveshell -u jschaef...@harmonywave.net -a
>> > jschaef...@harmonywave.net mail.harmonywave.cloud
>> > connecting to mail.harmonywave.cloud
>> > Please enter your password:
>> >> put /tmp/testing.sieve testing
>> >> activate testing
>> >> list
>> > testing  <- active script
>> >> quit
>> >
>> > However when I send a test email from my personal account to the email
>> > I have setup on the IMAP server it always comes through and I get this
>> > in mail.log:
>> >
>> > Dec 17 10:36:07 bllmail01 cyrus/lmtp[14530]: sieve runtime error for
>> > jschaef...@harmonywave.net id
>> > :
>> > Reject: Sendmail process terminated normally, exit status 255
>>
>> I think sieve tries to send mail using the configured sendmail binary
>> and
>> that doesn't work for some reason. You may check the sendmail config in
>> your imapd.conf and also consult the mail logs to learn more.
>>
>> Regards,
>> Simon
>>
>>
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: sieve runtime error

[Simon Matter]

> I'm trying to setup sieve and getting the following error in my logs:
>
> Dec 17 10:36:07 bllmail01 cyrus/lmtp[14530]: sieve runtime error for
> jschaef...@harmonywave.net id
> :
> Reject: Sendmail process terminated normally, exit status 255
>
>
> I'm following the documentation here:
> https://www.cyrusimap.org/imap/reference/admin/sieve.html?highlight=sieve#testing-the-sieve-server
>
> I'm trying to get sieve working on my IMAP server. Using Ubuntu 16.04 with
> the cyrus-imapd 2.4.18-3 package. I'm using the test sieve script shown in
> the documentation to reject everything from my personal email:
>
> require ["reject","fileinto"];
> if address :is :all "From" "jschaeffer0...@gmail.com"
> {
> reject "testing";
> }
>
> I then connect using sieveshell, upload the file, and activate it:
>
> root@bllmail01:~# sieveshell -u jschaef...@harmonywave.net -a
> jschaef...@harmonywave.net mail.harmonywave.cloud
> connecting to mail.harmonywave.cloud
> Please enter your password:
>> put /tmp/testing.sieve testing
>> activate testing
>> list
> testing  <- active script
>> quit
>
> However when I send a test email from my personal account to the email
> I have setup on the IMAP server it always comes through and I get this
> in mail.log:
>
> Dec 17 10:36:07 bllmail01 cyrus/lmtp[14530]: sieve runtime error for
> jschaef...@harmonywave.net id
> :
> Reject: Sendmail process terminated normally, exit status 255

I think sieve tries to send mail using the configured sendmail binary and
that doesn't work for some reason. You may check the sendmail config in
your imapd.conf and also consult the mail logs to learn more.

Regards,
Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: suddenly 'User unknown'?

[Simon Matter]

> Dan
>
> I have, and have always had, an empty /var/log/imapd.log so I'm not
> going to make progress until I fix that.
>
> In n /etc/rsyslog.conf
>
> # cyrus imapd
> #local6.*                              /var/log/imapd.log   - tried this
> first.
> local6.debug                        /var/log/imapd.log
> auth.debug                          /var/log/auth.log
>
> and in /etc/imapd.conf
>
> syslog_prefix: cyrus
> syslog_facility: LOCAL6
>
> If I remove the file /etc/imapd.log then
>
> # /etc/init.d/rsyslog restart
>
> # logger local6.debug 'test log message'
>
> # cat /var/log/imapd.log
>
> Nov 29 15:06:42 dell2600-1 brad: test log message
>
> Obviously syslog is working local6. But still no messages from cyrus!
> Therefor I'm now stuck with this secondary problem.
>
> I have followed the cyrus instructions as best I can, but no go. I say
> again this has all worked for years, albeit with an always empty imapd.log
>
> There must be some missing cyrus syslog configuration.

From README.RPM in the cyrus-imapd package:
12) Check your syslog configuration. This RPM uses the mail facility to log
messages. On busy sites you may want to limit the mail facility to the
info priority with something like 'mail.info/var/log/maillog' in
/etc/syslog.conf.

So, I guess your cyrus-imapd messages go to /var/log/maillog then, not?

Regards,
Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Thunderbird and "Login to server failed".

[Simon Matter]

> Hi,
>
> I'm using cyrus-imapd (now 2.4.x) and Thunderbird. The problem is, when
> I click fast on the e-mails, I often get the "Login to server foo.bar
> failed" (when I click like one -email per 3 seconds everything is fine).
> My password is okay, so I just press "Cancel". I live with this problem
> for a really long time, and I can say the this is specific to the TB: it
> doesn't happen in Outlook. I've read a lot about this, and, besides the
> mailbox being improperly configured and failing authrization 100% of the
> time (which is not my case), there's an urban legend that TB uses by
> default  5 concurrent connections to the IMAP server (and this part is
> true), while cyrus permits less. But the thing is, even with 1 connecton
> configured in TB (in advanced settings), I still get this error when
> clicking messages fast. No errors are logged on the cyrus side, just the
> normal log stating that I was allowed to examine my mailbox.
>
> Is there any way to get rid of these boring messages ?
>
> Cyrus ML may not be the correct place to ask for this, but BT/Mozilla
> community just gives me futile answers like "are you sure you type your
> passwords right" and so on.

Interesting, I've never seen this problem over the years.

Are there firewall rules active on the cyrus-imapd host? Probably limiting
connection rate?

Regards,
Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: 2.4.12 vs 3.0.2 different protocol behaviour

[Simon Matter]

> I looked deeper into the problem.
> Looks like the command:
> . LIST "" "Other Users"
> responds abnormally, until I run :
> . LIST "" "Other Users/%"
> that actually list the shared folders.
> Here's a trace of the imap protocol:
> * OK [CAPABILITY IMAP4rev1 LITERAL+ ID ENABLE STARTTLS AUTH=PLAIN SASL-IR]
> www Cyrus IMAP 3.0.2 server ready
> . login gabriele.bul...@sonicle.com Nrdstg88!

I hope the login credentials are only for your test here?

Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Update from cyrus-imapd 2.5 to 3.0

[Simon Matter]

> In my "dhill" mailbox, I no longer see my folders and if I create a new
> folder in that mailbox, let's say "Sent", it'll create it this way:
>
> [root@zappa Sent]# ls -atlr
> total 20
> drwx--. 3 cyrus mail 4096 Jun 20 04:44 ..
> -rw---. 1 cyrus mail  166 Jun 20 04:44 cyrus.header
> -rw---. 1 cyrus mail  336 Jun 20 04:45 cyrus.annotations
> drwx--. 2 cyrus mail 4096 Jun 20 04:45 .
> -rw---. 1 cyrus mail  128 Jun 27 10:57 cyrus.index
> [root@zappa Sent]# pwd
> /var/spool/imap/s/Sent
>
> Instead of
> /var/spool/imap/d/user/dhill/Sent ...

You should check the 'fulldirhash' and 'hashimapspool' settings in
imapd.conf. They influence it.

Regards,
Simon

> Dave
>
>
>
> On 2017-06-27 12:20 PM, Jason L Tibbitts III wrote:
>>> "DH" == David Hill  writes:
>> DH> Also, if a folder is created, it is created in
>> DH> /var/spool/imap/[first_letter_of_folder]/Junk instead of
>> DH> /var/spool/imap/d/user/dhill/Junk ...
>>
>> In your bug you indicated that things were being stored under
>> /var/spool/[first letter]/ in the previous version.  Here you say they
>> were being stored under /var/spool/imap/first letter]/.  Perhaps that's
>> why my responses to you in the bugzilla ticket are so confused.
>>
>> Maybe it would be better if you could provide more explicit information
>> in some way.  Like, a directory listing or something.
>>
>>   - J<
>
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
>



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: lmtpd segfaults after update to 2.4.19, seems it wants to act as proxy

[Simon Matter]

> On Wed, May 24, 2017, at 04:17 PM, Don Lewis wrote:
>> This first version of the patch worked for me.  A problem with
>>
>> the second version of the patch is that mlookup() gets called in
>>
>> a bunch of different places and the server variable would need
>>
>> to be initialized in all of them.  In verify_user(), mlookup is called
>>
>> several times, so server would have to be set to NULL before
>>
>> each call.
>>
>>
>
> Oh, good catch, I didn't look for other occurrences.
>
> Attached is an updated version of my patch that initialises these
> variables correctly.

Thanks, I can confirm that the updated patch fixes the issue for me.

Kind regards,
Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: lmtpd segfaults after update to 2.4.19, seems it wants to act as proxy

[Simon Matter]

>
> Quoting Simon Matter <simon.mat...@invoca.ch>:
>
>> Hi,
>>
>> I'm resending this with new subject and one again attached the straces
>> of
>> a delivery with 2.4.18 and 2.4.19.
>>
>>>>> I've just updated our cyrus-imapd rpms from 2.4.18 -> 2.4.19. All
>> seems well with just some small changes to the build.
>>>>>
>>>>> However, I just can't get it to work: lmtpd segfaults on mail
>>>>> delivery.
>>>>>
>>>>> The logs show something like this:
>>>>> May 17 14:21:42 mh1 kernel: lmtpd[7433]: segfault at 5ebc24d7 ip
>> 7f216292973b sp 7ffe5ebc1f58 error 4 in
>>>>> lmtpd[7f21628c1000+169000]
>>>>> May 17 14:21:42 mh1 master[7423]: process 7433 exited, signaled to
>>>>> death
>>>>> by 11
>>>>>
>>>>> First I've rebuilt without some of the custom patches in the package
>>>>> like
>>>>> the autocreate patches. But without a change.
>>>>>
>>>>> I've then straced an lmtpd during mail delivery, once with 2.4.18 and
>> once
>>>>> with 2.4.19 in the hope to find something. The traces are attached.
>>>>>
>>>>> I see that there were quite some change concerning lmtp, does anyone
>> of the developers have an idea where the problem in lmtpd changes
>> could be?
>>>>
>>>> It turns out that lmtpd wants to act as proxy, because only then it
>> creates the spoolfile in /tmp/. But why?
>>>> Regards,
>>>> Simon
>>>>
>>>
>>> Are you running a murder setup? Can you produce a coredump and brovide
>>> a
>> backtrace with symbols loaded?
>>
>> Hi,
>>
>> I did two things meanwhile:
>>
>> 1) Built a completely vanilla cyrus-imapd-2.4.19 without any patches to
>> make sure it's not anything wrong with additional patches. The crash was
>> the same.
>>
>> 2) Ran the normal build with an unstripped "lmtpd" binary and produced
>> core and backtrace. So below:
>>
>> Core was generated by `lmtpd'.
>> Program terminated with signal 11, Segmentation fault.
>> #0  strlcpy (dst=0x7ffaa64e00d0 "", src=0x5d73c6c7 > out
>> of bounds>, len=1024) at xstrlcpy.c:65
>> 65   if ((dst[n] = src[n]) == '\0') break;
>> Missing separate debuginfos, use: debuginfo-install
>> cyrus-sasl-gssapi-2.1.23-15.el6_6.2.x86_64
>> cyrus-sasl-lib-2.1.23-15.el6_6.2.x86_64
>> cyrus-sasl-md5-2.1.23-15.el6_6.2.x86_64
>> cyrus-sasl-plain-2.1.23-15.el6_6.2.x86_64 db4-4.7.25-22.el6.x86_64
>> glibc-2.12-1.209.el6_9.1.x86_64 keyutils-libs-1.4-5.el6.x86_64
>> krb5-libs-1.10.3-65.el6.x86_64 libcom_err-1.41.12-23.el6.x86_64
>> libselinux-2.0.94-7.el6.x86_64
>> nss-softokn-freebl-3.14.3-23.3.el6_8.x86_64
>> openssl-1.0.1e-57.el6.x86_64 pcre-7.8-7.el6.x86_64
>> tcp_wrappers-libs-7.6-58.el6.x86_64 zlib-1.2.3-29.el6.x86_64
>> (gdb) bt
>> #0  strlcpy (dst=0x7ffaa64e00d0 "", src=0x5d73c6c7 > out
>> of bounds>, len=1024) at xstrlcpy.c:65
>> #1  0x7ffaa5150c2e in proxy_adddest (dlist=0x7ffc5d73c288,
>> rcpt=0x7ffaa64d26d0 "si...@imap.bi.corp.invoca.ch", rcpt_num=0,
>> server=0x5d73c6c7 , authas=0x0) at
>> proxy.c:81
>
> here the bt is already in the wrong path and server has not been set
> wich leads to the SIGSEV
>
>> #2  0x7ffaa514fbae in deliver (msgdata=0x7ffaa64d0950, authuser=0x0,
>> authstate=0x7ffc5d73c280) at lmtpd.c:749
>
> This is the point where the wrong path is taken.
>
>
> r = mlookup(namebuf, , NULL, NULL);
> if (!r && server) {
>  /* remote mailbox */
>  proxy_adddest(, rcpt, n, server, authuser);
>  status[n] = nosieve;
> }
>
> mlookup did return something that is not 0, and server has not been set
> in mlookup. In 2.4.17 there has was "if (server) *server = NULL;"

Hi Michael,

Thanks a lot for pointing me into the right direction. Indeed attached
patch makes it work for me.

I don't know if this is the proper fix for the issue. Hopefully someone
who knows better than me can comment.

Regards,
Simon--- cyrus-imapd-2.4.19/imap/lmtpd.c.orig	2017-05-15 07:43:30.0 +0200
+++ cyrus-imapd-2.4.19/imap/lmtpd.c	2017-05-20 08:36:47.629758869 +0200
@@ -392,6 +392,8 @@
 char *c;
 struct mboxlist_entry mbentry;
 
+if (server) *server = NULL;
+
 /* do a local lookup and kick the slave if necessary */
 r = mboxlist_lookup(name, , tid);
 if (r == IMAP_MAILBOX_NONEXISTENT && config_mupdate_server) {
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

lmtpd segfaults after update to 2.4.19, seems it wants to act as proxy

[Simon Matter]

Hi,

I'm resending this with new subject and one again attached the straces of
a delivery with 2.4.18 and 2.4.19.

>>> I've just updated our cyrus-imapd rpms from 2.4.18 -> 2.4.19. All
seems well with just some small changes to the build.
>>>
>>> However, I just can't get it to work: lmtpd segfaults on mail delivery.
>>>
>>> The logs show something like this:
>>> May 17 14:21:42 mh1 kernel: lmtpd[7433]: segfault at 5ebc24d7 ip
7f216292973b sp 7ffe5ebc1f58 error 4 in
>>> lmtpd[7f21628c1000+169000]
>>> May 17 14:21:42 mh1 master[7423]: process 7433 exited, signaled to death
>>> by 11
>>>
>>> First I've rebuilt without some of the custom patches in the package like
>>> the autocreate patches. But without a change.
>>>
>>> I've then straced an lmtpd during mail delivery, once with 2.4.18 and
once
>>> with 2.4.19 in the hope to find something. The traces are attached.
>>>
>>> I see that there were quite some change concerning lmtp, does anyone
of the developers have an idea where the problem in lmtpd changes
could be?
>>
>> It turns out that lmtpd wants to act as proxy, because only then it
creates the spoolfile in /tmp/. But why?
>> Regards,
>> Simon
>>
>
> Are you running a murder setup? Can you produce a coredump and brovide a
backtrace with symbols loaded?

Hi,

I did two things meanwhile:

1) Built a completely vanilla cyrus-imapd-2.4.19 without any patches to
make sure it's not anything wrong with additional patches. The crash was
the same.

2) Ran the normal build with an unstripped "lmtpd" binary and produced
core and backtrace. So below:

Core was generated by `lmtpd'.
Program terminated with signal 11, Segmentation fault.
#0  strlcpy (dst=0x7ffaa64e00d0 "", src=0x5d73c6c7 , len=1024) at xstrlcpy.c:65
65  if ((dst[n] = src[n]) == '\0') break;
Missing separate debuginfos, use: debuginfo-install
cyrus-sasl-gssapi-2.1.23-15.el6_6.2.x86_64
cyrus-sasl-lib-2.1.23-15.el6_6.2.x86_64
cyrus-sasl-md5-2.1.23-15.el6_6.2.x86_64
cyrus-sasl-plain-2.1.23-15.el6_6.2.x86_64 db4-4.7.25-22.el6.x86_64
glibc-2.12-1.209.el6_9.1.x86_64 keyutils-libs-1.4-5.el6.x86_64
krb5-libs-1.10.3-65.el6.x86_64 libcom_err-1.41.12-23.el6.x86_64
libselinux-2.0.94-7.el6.x86_64 nss-softokn-freebl-3.14.3-23.3.el6_8.x86_64
openssl-1.0.1e-57.el6.x86_64 pcre-7.8-7.el6.x86_64
tcp_wrappers-libs-7.6-58.el6.x86_64 zlib-1.2.3-29.el6.x86_64
(gdb) bt
#0  strlcpy (dst=0x7ffaa64e00d0 "", src=0x5d73c6c7 , len=1024) at xstrlcpy.c:65
#1  0x7ffaa5150c2e in proxy_adddest (dlist=0x7ffc5d73c288,
rcpt=0x7ffaa64d26d0 "si...@imap.bi.corp.invoca.ch", rcpt_num=0,
server=0x5d73c6c7 , authas=0x0) at
proxy.c:81
#2  0x7ffaa514fbae in deliver (msgdata=0x7ffaa64d0950, authuser=0x0,
authstate=0x7ffc5d73c280) at lmtpd.c:749
#3  0x7ffaa5152f41 in lmtpmode (func=0x7ffaa54b0020, pin=, pout=0x7ffaa64cf850, fd=) at
lmtpengine.c:1325
#4  0x7ffaa514f4ec in service_main (argc=1, argv=0x7ffaa64c9010,
envp=) at lmtpd.c:282
#5  0x7ffaa514dd95 in main (argc=, argv=, envp=0x7ffc5d740718) at service.c:585


The imapd.conf used for the test was:
configdirectory: /var/lib/imap
partition-default: /var/spool/imap
admins: cyrus
sievedir: /var/lib/imap/sieve
sendmail: /usr/sbin/sendmail
hashimapspool: true
sasl_pwcheck_method: saslauthd
sasl_mech_list: PLAIN
tls_cert_file: /etc/pki/cyrus-imapd/cyrus-imapd.pem
tls_key_file: /etc/pki/cyrus-imapd/cyrus-imapd.pem
tls_ca_file: /etc/pki/tls/certs/ca-bundle.crt
sieve_maxscriptsize: 128
allowplaintext: 1
delete_mode: delayed
expunge_mode: delayed
flushseenstate: 1
lmtp_strict_rfc2821: 0
mboxname_lockpath: {configdirectory}/volatile/lock
proc_path: {configdirectory}/volatile/proc
statuscache_db_path: {configdirectory}/volatile/statuscache.db
tlscache_db_path: {configdirectory}/volatile/tls_sessions.db

Any help would be much appreaciated!

Regards,
SimonProcess 2830 attached
accept(4, 0, NULL)  = 11
fcntl(10, F_SETLKW, {type=F_UNLCK, whence=SEEK_SET, start=0, len=0}) = 0
alarm(0)= 0
getpeername(11, {sa_family=AF_LOCAL, NULL}, [2]) = 0
write(3, "\2\0\0\0\16\v\0\0", 8)= 8
dup2(11, 0) = 0
dup2(11, 1) = 1
dup2(11, 2) = 2
close(11)   = 0
write(3, "\3\0\0\0\16\v\0\0", 8)= 8
getpeername(0, {sa_family=AF_LOCAL, NULL}, [2]) = 0
open("/var/lib/imap/log/postman/lmtpunix-2830", O_WRONLY|O_CREAT|O_APPEND, 
0644) = -1 ENOENT (No such file or directory)
select(1, [0], NULL, NULL, {0, 0})  = 0 (Timeout)
write(1, "220 mh1.bi.corp.testmh.ch Cyrus "..., 83) = 83
select(1, [0], NULL, NULL, {360, 0})= 1 (in [0], left {359, 97})
read(0, "LHLO mh1.bi.corp.testmh.ch\r\n", 4096) = 28
select(1, [0], NULL, NULL, {0, 0})  = 0 (Timeout)
write(1, "250-mh1.bi.corp.testmh.ch\r\n250-8"..., 128) = 128
select(1, [0], NULL, NULL, {360, 0})= 1 (in [0], left {359, 97})
read(0, "MAIL FROM: SIZE=2"..., 4096) = 83

Re: Cyrus IMAP 2.4.19 released

[Simon Matter]

> Hi,
>
> Quoting Simon Matter <simon.mat...@invoca.ch>:
>
>>>> The Cyrus team is proud to announce the immediate availability of a
>>>> new
>>>> version of Cyrus IMAP: 2.4.19.
>>>
>>> I've just updated our cyrus-imapd rpms from 2.4.18 -> 2.4.19. All seems
>>> well with just some small changes to the build.
>>>
>>> However, I just can't get it to work: lmtpd segfaults on mail delivery.
>>>
>>> The logs show something like this:
>>> May 17 14:21:42 mh1 kernel: lmtpd[7433]: segfault at 5ebc24d7 ip
>>> 7f216292973b sp 7ffe5ebc1f58 error 4 in
>>> lmtpd[7f21628c1000+169000]
>>> May 17 14:21:42 mh1 master[7423]: process 7433 exited, signaled to
>>> death
>>> by 11
>>>
>>> First I've rebuilt without some of the custom patches in the package
>>> like
>>> the autocreate patches. But without a change.
>>>
>>> I've then straced an lmtpd during mail delivery, once with 2.4.18 and
>>> once
>>> with 2.4.19 in the hope to find something. The traces are attached.
>>>
>>> I see that there were quite some change concerning lmtp, does anyone of
>>> the developers have an idea where the problem in lmtpd changes could
>>> be?
>>
>> It turns out that lmtpd wants to act as proxy, because only then it
>> creates the spoolfile in /tmp/. But why?
>> Regards,
>> Simon
>>
>
> Are you running a murder setup? Can you produce a coredump and brovide
> a backtrace with symbols loaded?

Hi,

I did two things meanwhile:

1) Built a completely vanilla cyrus-imapd-2.4.19 without any patches to
make sure it's not anything wrong with additional patches. The crash was
the same.

2) Ran the normal build with an unstripped "lmtpd" binary and produced
core and backtrace. So below:

Core was generated by `lmtpd'.
Program terminated with signal 11, Segmentation fault.
#0  strlcpy (dst=0x7ffaa64e00d0 "", src=0x5d73c6c7 , len=1024) at xstrlcpy.c:65
65  if ((dst[n] = src[n]) == '\0') break;
Missing separate debuginfos, use: debuginfo-install
cyrus-sasl-gssapi-2.1.23-15.el6_6.2.x86_64
cyrus-sasl-lib-2.1.23-15.el6_6.2.x86_64
cyrus-sasl-md5-2.1.23-15.el6_6.2.x86_64
cyrus-sasl-plain-2.1.23-15.el6_6.2.x86_64 db4-4.7.25-22.el6.x86_64
glibc-2.12-1.209.el6_9.1.x86_64 keyutils-libs-1.4-5.el6.x86_64
krb5-libs-1.10.3-65.el6.x86_64 libcom_err-1.41.12-23.el6.x86_64
libselinux-2.0.94-7.el6.x86_64 nss-softokn-freebl-3.14.3-23.3.el6_8.x86_64
openssl-1.0.1e-57.el6.x86_64 pcre-7.8-7.el6.x86_64
tcp_wrappers-libs-7.6-58.el6.x86_64 zlib-1.2.3-29.el6.x86_64
(gdb) bt
#0  strlcpy (dst=0x7ffaa64e00d0 "", src=0x5d73c6c7 , len=1024) at xstrlcpy.c:65
#1  0x7ffaa5150c2e in proxy_adddest (dlist=0x7ffc5d73c288,
rcpt=0x7ffaa64d26d0 "si...@imap.bi.corp.invoca.ch", rcpt_num=0,
server=0x5d73c6c7 , authas=0x0) at
proxy.c:81
#2  0x7ffaa514fbae in deliver (msgdata=0x7ffaa64d0950, authuser=0x0,
authstate=0x7ffc5d73c280) at lmtpd.c:749
#3  0x7ffaa5152f41 in lmtpmode (func=0x7ffaa54b0020, pin=, pout=0x7ffaa64cf850, fd=) at
lmtpengine.c:1325
#4  0x7ffaa514f4ec in service_main (argc=1, argv=0x7ffaa64c9010,
envp=) at lmtpd.c:282
#5  0x7ffaa514dd95 in main (argc=, argv=, envp=0x7ffc5d740718) at service.c:585


The imapd.conf used for the test was:
configdirectory: /var/lib/imap
partition-default: /var/spool/imap
admins: cyrus
sievedir: /var/lib/imap/sieve
sendmail: /usr/sbin/sendmail
hashimapspool: true
sasl_pwcheck_method: saslauthd
sasl_mech_list: PLAIN
tls_cert_file: /etc/pki/cyrus-imapd/cyrus-imapd.pem
tls_key_file: /etc/pki/cyrus-imapd/cyrus-imapd.pem
tls_ca_file: /etc/pki/tls/certs/ca-bundle.crt
sieve_maxscriptsize: 128
allowplaintext: 1
delete_mode: delayed
expunge_mode: delayed
flushseenstate: 1
lmtp_strict_rfc2821: 0
mboxname_lockpath: {configdirectory}/volatile/lock
proc_path: {configdirectory}/volatile/proc
statuscache_db_path: {configdirectory}/volatile/statuscache.db
tlscache_db_path: {configdirectory}/volatile/tls_sessions.db

Any help would be much appreaciated!

Regards,
Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cyrus IMAP 2.4.19 released

[Simon Matter]

> Hi,
>
> Quoting Simon Matter <simon.mat...@invoca.ch>:
>
>>>> The Cyrus team is proud to announce the immediate availability of a
>>>> new
>>>> version of Cyrus IMAP: 2.4.19.
>>>
>>> I've just updated our cyrus-imapd rpms from 2.4.18 -> 2.4.19. All seems
>>> well with just some small changes to the build.
>>>
>>> However, I just can't get it to work: lmtpd segfaults on mail delivery.
>>>
>>> The logs show something like this:
>>> May 17 14:21:42 mh1 kernel: lmtpd[7433]: segfault at 5ebc24d7 ip
>>> 7f216292973b sp 7ffe5ebc1f58 error 4 in
>>> lmtpd[7f21628c1000+169000]
>>> May 17 14:21:42 mh1 master[7423]: process 7433 exited, signaled to
>>> death
>>> by 11
>>>
>>> First I've rebuilt without some of the custom patches in the package
>>> like
>>> the autocreate patches. But without a change.
>>>
>>> I've then straced an lmtpd during mail delivery, once with 2.4.18 and
>>> once
>>> with 2.4.19 in the hope to find something. The traces are attached.
>>>
>>> I see that there were quite some change concerning lmtp, does anyone of
>>> the developers have an idea where the problem in lmtpd changes could
>>> be?
>>
>> It turns out that lmtpd wants to act as proxy, because only then it
>> creates the spoolfile in /tmp/. But why?
>> Regards,
>> Simon
>>
>
> Are you running a murder setup? Can you produce a coredump and brovide
> a backtrace with symbols loaded?

No murder involved. Unfortunately I'm unable to produce a coredump right
now. It will take some time for me to do so...

Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cyrus IMAP 2.4.19 released

[Simon Matter]

>> The Cyrus team is proud to announce the immediate availability of a new
>> version of Cyrus IMAP: 2.4.19.
>
> I've just updated our cyrus-imapd rpms from 2.4.18 -> 2.4.19. All seems
> well with just some small changes to the build.
>
> However, I just can't get it to work: lmtpd segfaults on mail delivery.
>
> The logs show something like this:
> May 17 14:21:42 mh1 kernel: lmtpd[7433]: segfault at 5ebc24d7 ip
> 7f216292973b sp 7ffe5ebc1f58 error 4 in lmtpd[7f21628c1000+169000]
> May 17 14:21:42 mh1 master[7423]: process 7433 exited, signaled to death
> by 11
>
> First I've rebuilt without some of the custom patches in the package like
> the autocreate patches. But without a change.
>
> I've then straced an lmtpd during mail delivery, once with 2.4.18 and once
> with 2.4.19 in the hope to find something. The traces are attached.
>
> I see that there were quite some change concerning lmtp, does anyone of
> the developers have an idea where the problem in lmtpd changes could be?

It turns out that lmtpd wants to act as proxy, because only then it
creates the spoolfile in /tmp/. But why?
Regards,
Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cyrus IMAP 2.4.19 released

[Simon Matter]

> The Cyrus team is proud to announce the immediate availability of a new
> version of Cyrus IMAP: 2.4.19.

I've just updated our cyrus-imapd rpms from 2.4.18 -> 2.4.19. All seems
well with just some small changes to the build.

However, I just can't get it to work: lmtpd segfaults on mail delivery.

The logs show something like this:
May 17 14:21:42 mh1 kernel: lmtpd[7433]: segfault at 5ebc24d7 ip
7f216292973b sp 7ffe5ebc1f58 error 4 in lmtpd[7f21628c1000+169000]
May 17 14:21:42 mh1 master[7423]: process 7433 exited, signaled to death
by 11

First I've rebuilt without some of the custom patches in the package like
the autocreate patches. But without a change.

I've then straced an lmtpd during mail delivery, once with 2.4.18 and once
with 2.4.19 in the hope to find something. The traces are attached.

I see that there were quite some change concerning lmtp, does anyone of
the developers have an idea where the problem in lmtpd changes could be?

Thanks,
Simon

strace-2.4.18.txt.gz
Description: application/gzip


strace-2.4.19.txt.gz
Description: application/gzip

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: cyr_expire suddenly segfaulting

[Simon Matter]

> So everything is 2.4 defaults, which are:
>
> brong@wot:~/src/cyrus-imapd$ grep '_db"' lib/imapoptions
> { "annotation_db", "skiplist", STRINGLIST("berkeley", "berkeley-hash",
> "skiplist")}
> { "duplicate_db", "skiplist", STRINGLIST("berkeley", "berkeley-nosync",
> "berkeley-hash", "berkeley-hash-nosync", "skiplist", "sql")}
> { "mboxkey_db", "skiplist", STRINGLIST("berkeley", "skiplist") }
> { "mboxlist_db", "skiplist", STRINGLIST("flat", "berkeley",
> "berkeley-hash", "skiplist")}
> { "ptscache_db", "skiplist", STRINGLIST("berkeley", "berkeley-hash",
> "skiplist")}
> { "quota_db", "quotalegacy", STRINGLIST("flat", "berkeley",
> "berkeley-hash", "skiplist", "sql", "quotalegacy")}
> { "seenstate_db", "skiplist", STRINGLIST("flat", "berkeley",
> "berkeley-hash", "skiplist")}
> { "subscription_db", "flat", STRINGLIST("flat", "berkeley",
> "berkeley-hash", "skiplist")}
> { "statuscache_db", "skiplist", STRINGLIST("berkeley", "berkeley-nosync",
> "berkeley-hash", "berkeley-hash-nosync", "skiplist") }
> { "tlscache_db", "skiplist", STRINGLIST("berkeley", "berkeley-nosync",
> "berkeley-hash", "berkeley-hash-nosync", "skiplist", "sql")}
> { "userdeny_db", "flat", STRINGLIST("flat", "berkeley", "berkeley-hash",
> "skiplist", "sql")}
>
> So you shouldn't have any berkeley databases at all.  It should be
> perfectly safe to shut down Cyrus, delete the $confdir/db folder, and
> restart Cyrus.

I suggest to make sure first that the ClearOS build really uses those
defaults - because they could change defaults in their builds.

Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: What happened to normalizeuid?

[Simon Matter via Info-cyrus]

> Hi,
>
> I considered opening a Github issue, but the question seems too trivial
> for
> that.
>
> I'm playing around with cyrus-imapd-3.0.0-rc1. When I ran "cyr_info
> conf-lint" on the conf files from our 2.4.x production server, I got a few
> items I knew how to deal with, but also this:
>
> normalizeuid: 1
> tls_require_cert: off
>
> tls_require_cert was off by default anyway, and all the tls options were
> reworked a lot, so I figured that one could just be dropped.
>
> But in 2.4 normalizeuid 1 was off by default. I couldn't find the string
> normalizeuid anywhere in the source or the release notes. I'd say that's
> bad. If you get rid of an option, that should be documented. What is the
> new default?

Hi,

We and others had this as a patch in our RPMs but I think it has never
been part of vanilla cyrus-imapd.

Regards,
Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: imap impersonate

[Simon Matter via Info-cyrus]

> Hi,
> is there any mechanism with Cyrus imap to impersonate another user?
> I've seen other imap servers scenarios where one may use plain
> authentication and sending user as mailboxuser plus a separator plus
> adminuser and use only adminpassword, to get access to the mailboxuser as
> is (dovecot, exchange).
> Anything like this in Cyrus?
> Gabriele

Hi Gabriele,

Check the "proxyservers" directive in imapd.conf.

Regards,
Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Could not connect to socket /var/imap/socket/lmtp: Connection refused by localhost

[Simon Matter via Info-cyrus]

> Hi.
>
> On 17.01.2017 19:09, Andy Dorman via Info-cyrus wrote:
>>
>> I am not an expert by any means and I hope someone corrects me if I
>> make a bad suggestion...but I have two questions:
>>
>> 1. It sounds like you have a heavily used server, so why do you have
>> Cyrus listening on both "localhost:lmtp" AND a unix socket
>> "/var/imap/socket/lmtp"?
>>
>> From the log entry it looks like your MTA uses a unix socket. Unless
>> you have something else (mail clients or other MTAs running on your
>> Cyrus server?) that need to communicate via the localhost:lmtp port,
>> you could comment out the unneeded lmtp service line and save those
>> resources.
> Well, on one hand you are right, seems like noone uses network lmtp
> connections, but on the other hand how can the idle processes save
> resources ? They only can save the memory, which doesn't seem to be the
> problem. However, I will try you advice.
>>
>> 2. You say "increasing this value can make the situation even worse".
>> Which value?  There are 5 values on those two lines that you could
>> increase.  And by "even worse" do you mean even more refused
>> connections?
> The maxchild number.
>>
>> While I am not a Cyrus guru, I have seen my share of overloaded mail
>> servers and if you are running into a disk IO limit, adding more
>> processes fighting over a limited resource is very likely to make
>> things worse.  So you should also confirm a hardware limitation is not
>> at play here.
> Yup, this is exaclty what happens when increasing the maxchild number:
> more messages start to bounce. And yes, the disks iops seems to be the
> limiting factor. So, are there any other approaches besides scaling out
> the disks iops ?

I remember a situation more than a decade ago where we had to tune this.
The problem with LMTP deliveries was that a lot of mails with hundreds of
recipients on the same server were sent. Our Postfix MTA has sent every
mail by a single LMTP transfer which resulted in high LMTP load and
prevented effective usage of single instance store. The problem was solved
by setting "local_destination_recipient_limit = X00" on the Postfix MTA (I
don't remember the exact number).

Regards,
Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: FreeBSD 9.3 STABLE, cyrus-imapd 2.5.9, Listen queue overflow

[Simon Matter via Info-cyrus]

Hi,

I'm not an expert in this but just a wild guess: Could it be that you have
an issue with /dev/random or /dev/urandom?

Maybe others on this list can tell more about it and you can search the
list archives to find some information on the topic.

Regards,
Simon

> Hello.
>
> I have strange problem with cyrus-imapd 2.5.9, after upgrade my server
> OS from FreeBSD 7 to FreeBSD 9.3 STABLE and cyrus-imapd from 2.3.x to
> 2.5.9. All packages reinstalled after upgrade OS.
> A problem that, after a 2 or 4 or 7 days (differently) of normal work of
> service cyrus-imapd, he freeze. Users can't connect to mailboxes.
> In the dmesg log i see it - sonewconn: pcb 0xfe0101da7c40: Listen
> queue overflow: 49 already in queue awaiting acceptance (12 occurrences).
> And netstat -LAan show me it - fe0081a747a0 tcp4  49/0/32
> *.993.
> Why queue overflow, i'm not understand. I think that a problem in the
> process idled, because after stoped cyrus-imapd process
> ("/usr/local/etc/rc.d/imapd stop"), several processes of idled are
> visible in the list of processes (ps -ax).
> I run command "kiilall -9 idled" and "/usr/local/etc/rc.d/imapd start" -
> service normal work.
>
> Sorry about my english, my english is hard.
> Maybe someone help me.
>
> i have provided below detailed information about system and from logs
> (service cyrus-imapd freeze).
>
> System information:
> # uname -a
> FreeBSD mail.spectrum.ru 9.3-STABLE FreeBSD 9.3-STABLE #0: Sat Jul  2
> 17:48:04 MSK 2016
> old...@mail.spectrum.ru:/usr/obj/usr/src/sys/SRVKERNEL  amd64
>
> # pkg info | grep cyrus
> cyrus-imapd25-2.5.9The cyrus mail server, supporting POP3
> and IMAP4 protocols
> cyrus-sasl-2.1.26_12   RFC  SASL (Simple Authentication and
> Security Layer)
> cyrus-sasl-saslauthd-2.1.26_3  SASL authentication server for cyrus-sasl2
>
>
> Conf files
> # cat /usr/local/etc/cyrus.conf
> START {
> idled   cmd="idled"
> recover cmd="ctl_cyrusdb -r"
> #   syncclient  cmd="sync_client -r"
> }
>
> SERVICES {
> imapcmd="imapd" listen="imap" prefork=0
> imaps   cmd="imapd -s" listen="imaps" prefork=0
> pop3cmd="pop3d" listen="pop3" prefork=0
> pop3s   cmd="pop3d -s" listen="spop3" prefork=0
> sieve   cmd="timsieved" listen="sieve" prefork=0
> lmtpunixcmd="lmtpd" listen="/data/imap/socket/lmtp"
> prefork=0
> smmap   cmd="smmapd" listen="/data/imap/socket/smmap"
> prefork=1
> #   syncclient  cmd="sync_client -r" listen="csync" prefork=1
> }
>
> EVENTS {
> checkpoint  cmd="ctl_cyrusdb -c" period=30
> delprunecmd="cyr_expire -v -E 3" at=0400
> tlsprunecmd="tls_prune" at=0400
> squattercmd="squatter -i" at=0100
> }
>
> # cat /usr/local/etc/imapd.conf
> admins: cyrus
> allowanonymouslogin: no
> allowplaintext: yes
> altnamespace: yes
> annotation_db: skiplist
> configdirectory: /data/imap
> defaultpartition: default
> duplicate_db: berkeley-nosync
> imapidresponse: yes
> imapmagicplus: 0
> lmtp_downcase_rcpt: 1
> lmtpsocket: /data/imap/socket/lmtp
> mboxlist_db: skiplist
> munge8bit: 0
> partition-default: /data/spool/imap
> postmaster: postmaster
> ptscache_db: berkeley
> rfc2046_strict: 0
> sasl_auxprop_plugin: sasldb
> sasl_log_level: 1
> sasl_mech_list: plain cram-md5 digest-md5 login
> sasl_pwcheck_method: auxprop
> seenstate_db: skiplist
> sendmail: /usr/sbin/sendmail
> sievedir: /data/imap/sieve
> subscription_db: flat
> # sync_authname: cyrus
> # sync_batch_size: 0
> # sync_host: support.spectrum.ru
> # sync_log: 1
> # sync_password: pDkazwok
> # sync_repeat_interval: 5
> # sync_shutdown_file: /data/imap/socket/syncshutdown
> tls_server_ca_file: /etc/ssl/imapserver.pem
> tls_server_cert: /etc/ssl/imapserver.pem
> tls_server_key: /etc/ssl/imapserver.pem
> tls_session_timeout: 1440
> unixhierarchysep: no
> virtdomains: userid
>
>
> Information from logs:
> # netstat -LAan
> Current listen queue sizes (qlen/incqlen/maxqlen)
> TcpcbProto Listen Local Address
> fe01257a33d0 tcp4  0/0/32 *.4190
> fe0125e45b70 tcp4  0/0/32 *.995
> fe00a4a38b70 tcp4  0/0/32 *.110
> fe0081a747a0 tcp4  49/0/32*.993
> fe00a434e000 tcp4  0/0/32 *.143
> fe00a4f453d0 tcp4  0/0/10 *.587
> fe0007ccd3d0 tcp4  0/0/10 *.465
> fe0125e453d0 tcp4  0/0/10 *.25
> fe0007ccd7a0 tcp4  0/0/512*.113
> fe0007ccdb70 tcp4  0/0/512*.79
> fe0007acc3d0 tcp4  0/0/128*.22
> fe00079947a0 tcp4  0/0/512127.0.0.1.8891
> fe00079943d0 tcp4  0/0/128*.888
> fe0007acd3d0 tcp4  0/0/128127.0.0.1.953
> fe0007994b70 tcp4  0/0/10 127.0.0.1.53
> unix  0/0/32 /data/imap/socket/smmap
> unix  0/0/32 /data/imap/socket/lmtp
> unix  0/0/32 /var/run/saslauthd/mux
> 

Re: stock centos 7 based Cyrus-imapd not working - troubleshooting suggestions?

[Simon Matter via Info-cyrus]

> Trying to troubleshoot a centos 7/Cyrus-imap configuration.   The process
> is not running correctly - the /var/log/maillog fills up with messages and
> once this grep string is executed this is what is seen:
>
> grep -v Fatal maillog | grep -v abnormally | grep -v "ptions not presen"
>
> Jun 18 12:23:42 mail master[20269]: setrlimit: Unable to set file
> descriptors limit to -1: Operation not permitted
> Jun 18 12:23:42 mail master[20269]: retrying with 4096 (current max)
> Jun 18 12:23:42 mail ctl_cyrusdb[20277]: recovering cyrus databases
> Jun 18 12:23:42 mail ctl_cyrusdb[20277]: done recovering cyrus databases
> Jun 18 12:23:42 mail master[20269]: unable to setsocketopt(IP_TOS):
> Operation not supported
> Jun 18 12:23:42 mail ctl_cyrusdb[20314]: checkpointing cyrus databases
> Jun 18 12:23:42 mail ctl_cyrusdb[20314]: done checkpointing cyrus
> databases

AFAIK those messages are normal and don't show any real problem. What
exactly does not work?

Regards,
Simon

>
>
>
> The firewall is off (to attempt to see if the setsocketopt would go away):
> # systemctl status firewalld
>   firewalld.service - firewalld - dynamic firewall daemon
> Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled;
> vendor preset: enabled)
> Active: inactive (dead)
>
> And SELinux is set to not stop things.
> # getenforce
> Permissive
>
>
>
> strace /usr/lib/cyrus-imapd/cyrus-master isn't really being helpful either
>
> (lots snipped)
> access("/etc/system-fips", F_OK)= -1 ENOENT (No such file or
> directory)
> open("/etc/imapd.conf", O_RDONLY)   = 3
> fstat(3, {st_mode=S_IFREG|0644, st_size=692, ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
> 0x7f259e1b8000
> read(3, "configdirectory: /var/lib/imap\np"..., 4096) = 692
> read(3, "", 4096)   = 0
> close(3)= 0
> munmap(0x7f259e1b8000, 4096)= 0
> uname({sys="Linux", node="mail.example.com", ...}) = 0
> close(0)= 0
> open("/dev/null", O_RDWR)   = 0
> close(1)= 0
> open("/dev/null", O_RDWR)   = 1
> close(2)= 0
> open("/dev/null", O_RDWR)   = 2
> close(3)= -1 EBADF (Bad file descriptor)
> dup(0)  = 3
> close(4)= -1 EBADF (Bad file descriptor)
> dup(0)  = 4
> setrlimit(RLIMIT_NOFILE, {rlim_cur=RLIM64_INFINITY,
> rlim_max=RLIM64_INFINITY}) = -1 EPERM (Operation not permitted)
> open("/etc/localtime", O_RDONLY|O_CLOEXEC) = 5
> fstat(5, {st_mode=S_IFREG|0644, st_size=3559, ...}) = 0
> fstat(5, {st_mode=S_IFREG|0644, st_size=3559, ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
> 0x7f259e1b8000
> read(5, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\6\0\0\0\6\0\0\0\0"...,
> 4096) = 3559
> lseek(5, -2272, SEEK_CUR)   = 1287
> read(5, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\7\0\0\0\7\0\0\0\0"...,
> 4096) = 2272
> close(5)= 0
> munmap(0x7f259e1b8000, 4096)= 0
> (lots more snipped)
>
> Nothing in the strace output for setsocketopt.
>
>
> Suggestions on what to do to figure out why on
> # cat /etc/redhat-release
> CentOS Linux release 7.2.1511 (Core)
>
> the installed packages
>
> # rpm -q -a | grep cyrus
> cyrus-imapd-2.4.17-8.el7_1.x86_64
> cyrus-sasl-devel-2.1.26-20.el7_2.x86_64
> cyrus-sasl-lib-2.1.26-20.el7_2.x86_64
> cyrus-imapd-utils-2.4.17-8.el7_1.x86_64
> cyrus-imapd-devel-2.4.17-8.el7_1.x86_64
> cyrus-sasl-plain-2.1.26-20.el7_2.x86_64
> cyrus-sasl-2.1.26-20.el7_2.x86_64
> cyrus-sasl-scram-2.1.26-20.el7_2.x86_64
> cyrus-sasl-md5-2.1.26-20.el7_2.x86_64
>
> are not working.
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
>



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: lmtpd triggering a delivery.db checkpointing (Cyrus 2.3.16)

[Simon Matter via Info-cyrus]

> Hi,
>
>
> Several times a month our server freezes up on deliveries and the system
> load average shoots up into the hundreds. Things quickly return to normal
> between one and two minutes later but this has always puzzled me.
>
> Today I was watching the system from up close when it happened.
>
>
> May 17 10:59:14  lmtp[24980]: skiplist: checkpointed
> /ssd/cyrs/imap/deliver.db (223062 records, 25295200 bytes) in 119 seconds
>
>
>
> I took a quick dive into the code but could not find where and when lmtpd
> is supposed to trigger a delivery.db checkpointing action.

Isn't it controlled by 'checkpointcmd="ctl_cyrusdb -c" period=30' in
cyrus.conf?

Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cannot connect with cyradm

[Simon Matter via Info-cyrus]

> That did it, but why did I have to specify PLAIN? None of the docs mention
> having to do that with a default install.

I don't know, but you don't have a default install but a configuration
tailored by fedora.

Simon

>
>
> On Fri, May 6, 2016 at 2:16 AM, Simon Matter <simon.mat...@invoca.ch>
> wrote:
>
>> > I am trying to set up a basic system with cyrus-imap and postfix on
>> amazon
>> > linux
>> >
>> > I can connect using imtest, but cannot connect with cyradm:
>> >
>> > Additionally, when I connect via a client with a user I know has mail,
>> > it's
>> > saying that no INBOX exists. Postfix's mail spool is /var/spool/mail,
>> but
>> > I
>> > don't see anywhere to verify that imapd or lmtpd is looking there.
>> >
>> > Sorry if these are dumb questions. I'm completely new to cyrus-imap
>> and
>> > the
>> > documentation at cyrusimap.org is extremely sketchy.
>> >
>> > Any advice appreciated.
>> >
>> > $ more /etc/imapd.conf
>> > configdirectory: /var/lib/imap
>> > partition-default: /var/spool/imap
>> > admins: cyrus
>> > sievedir: /var/lib/imap/sieve
>> > sendmail: /usr/sbin/sendmail
>> > hashimapspool: true
>> > sasl_pwcheck_method: saslauthd
>> > sasl_mech_list: PLAIN LOGIN
>> > tls_cert_file: /etc/pki/cyrus-imapd/cyrus-imapd.pem
>> > tls_key_file: /etc/pki/cyrus-imapd/cyrus-imapd.pem
>> > tls_ca_file: /etc/pki/tls/certs/ca-bundle.crt
>> >
>> > $ grep MECH /etc/init.d/saslauthd
>> > MECH=pam
>> >
>> > $ imtest -t "" -u cyrus -a cyrus localhost
>> > S: * OK [CAPABILITY IMAP4 IMAP4rev1 LITERAL+ ID STARTTLS LOGINDISABLED
>> > COMPRESS=DEFLATE] ip-172-31-1-214 Cyrus IMAP
>> > v2.3.16-Fedora-RPM-2.3.16-6.9.amzn1 server ready
>> > C: S01 STARTTLS
>> > S: S01 OK Begin TLS negotiation now
>> > verify error:num=18:self signed certificate
>> > TLS connection established: TLSv1 with cipher DHE-RSA-AES256-SHA
>> (256/256
>> > bits)
>> > C: C01 CAPABILITY
>> > S: * CAPABILITY IMAP4 IMAP4rev1 LITERAL+ ID AUTH=PLAIN AUTH=LOGIN
>> SASL-IR
>> > COMPRESS=DEFLATE ACL RIGHTS=kxte QUOTA MAILBOX-REFERRALS NAMESPACE
>> UIDPLUS
>> > NO_ATOMIC_RENAME UNSELECT CHILDREN MULTIAPPEND BINARY SORT SORT=MODSEQ
>> > THREAD=ORDEREDSUBJECT THREAD=REFERENCES ANNOTATEMORE CATENATE
>> CONDSTORE
>> > SCAN IDLE LISTEXT LIST-SUBSCRIBED X-NETSCAPE URLAUTH
>> > S: C01 OK Completed
>> > Please enter your password:
>> > C: A01 AUTHENTICATE PLAIN Y3lydXMAY3lydXMAaGVsbC1oYXRoLW5vLWZ1cnk=
>> > S: A01 OK [CAPABILITY IMAP4 IMAP4rev1 LITERAL+ ID LOGINDISABLED
>> > COMPRESS=DEFLATE ACL RIGHTS=kxte QUOTA MAILBOX-REFERRALS NAMESPACE
>> UIDPLUS
>> > NO_ATOMIC_RENAME UNSELECT CHILDREN MULTIAPPEND BINARY SORT SORT=MODSEQ
>> > THREAD=ORDEREDSUBJECT THREAD=REFERENCES ANNOTATEMORE CATENATE
>> CONDSTORE
>> > SCAN IDLE LISTEXT LIST-SUBSCRIBED X-NETSCAPE URLAUTH] Success (tls
>> > protection)
>> > Authenticated.
>> > Security strength factor: 256
>> > . LIST "" "*"
>> > . OK Completed (0.000 secs 1 calls)
>> > . LOGOUT
>> > * BYE LOGOUT received
>> > . OK Completed
>> > Connection closed.
>> >
>> > $ cyradm --user cyrus --authz cyrus localhost
>> > Login disabled.
>> > cyradm: cannot authenticate to server with  as cyrus
>> > $ cyradm --user cyrus --authz cyrus --auth pam localhost
>> > verify error:num=18:self signed certificate
>> > cyradm: cannot authenticate to server with pam as cyrus
>> > $ cyradm --user cyrus --authz cyrus --auth shadow localhost
>> > verify error:num=18:self signed certificate
>> > cyradm: cannot authenticate to server with shadow as cyrus
>>
>> What does it do if you run with "--auch PLAIN" instead?
>>
>> Regards,
>> Simon
>>
>>
>



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cannot connect with cyradm

[Simon Matter via Info-cyrus]

> I am trying to set up a basic system with cyrus-imap and postfix on amazon
> linux
>
> I can connect using imtest, but cannot connect with cyradm:
>
> Additionally, when I connect via a client with a user I know has mail,
> it's
> saying that no INBOX exists. Postfix's mail spool is /var/spool/mail, but
> I
> don't see anywhere to verify that imapd or lmtpd is looking there.
>
> Sorry if these are dumb questions. I'm completely new to cyrus-imap and
> the
> documentation at cyrusimap.org is extremely sketchy.
>
> Any advice appreciated.
>
> $ more /etc/imapd.conf
> configdirectory: /var/lib/imap
> partition-default: /var/spool/imap
> admins: cyrus
> sievedir: /var/lib/imap/sieve
> sendmail: /usr/sbin/sendmail
> hashimapspool: true
> sasl_pwcheck_method: saslauthd
> sasl_mech_list: PLAIN LOGIN
> tls_cert_file: /etc/pki/cyrus-imapd/cyrus-imapd.pem
> tls_key_file: /etc/pki/cyrus-imapd/cyrus-imapd.pem
> tls_ca_file: /etc/pki/tls/certs/ca-bundle.crt
>
> $ grep MECH /etc/init.d/saslauthd
> MECH=pam
>
> $ imtest -t "" -u cyrus -a cyrus localhost
> S: * OK [CAPABILITY IMAP4 IMAP4rev1 LITERAL+ ID STARTTLS LOGINDISABLED
> COMPRESS=DEFLATE] ip-172-31-1-214 Cyrus IMAP
> v2.3.16-Fedora-RPM-2.3.16-6.9.amzn1 server ready
> C: S01 STARTTLS
> S: S01 OK Begin TLS negotiation now
> verify error:num=18:self signed certificate
> TLS connection established: TLSv1 with cipher DHE-RSA-AES256-SHA (256/256
> bits)
> C: C01 CAPABILITY
> S: * CAPABILITY IMAP4 IMAP4rev1 LITERAL+ ID AUTH=PLAIN AUTH=LOGIN SASL-IR
> COMPRESS=DEFLATE ACL RIGHTS=kxte QUOTA MAILBOX-REFERRALS NAMESPACE UIDPLUS
> NO_ATOMIC_RENAME UNSELECT CHILDREN MULTIAPPEND BINARY SORT SORT=MODSEQ
> THREAD=ORDEREDSUBJECT THREAD=REFERENCES ANNOTATEMORE CATENATE CONDSTORE
> SCAN IDLE LISTEXT LIST-SUBSCRIBED X-NETSCAPE URLAUTH
> S: C01 OK Completed
> Please enter your password:
> C: A01 AUTHENTICATE PLAIN Y3lydXMAY3lydXMAaGVsbC1oYXRoLW5vLWZ1cnk=
> S: A01 OK [CAPABILITY IMAP4 IMAP4rev1 LITERAL+ ID LOGINDISABLED
> COMPRESS=DEFLATE ACL RIGHTS=kxte QUOTA MAILBOX-REFERRALS NAMESPACE UIDPLUS
> NO_ATOMIC_RENAME UNSELECT CHILDREN MULTIAPPEND BINARY SORT SORT=MODSEQ
> THREAD=ORDEREDSUBJECT THREAD=REFERENCES ANNOTATEMORE CATENATE CONDSTORE
> SCAN IDLE LISTEXT LIST-SUBSCRIBED X-NETSCAPE URLAUTH] Success (tls
> protection)
> Authenticated.
> Security strength factor: 256
> . LIST "" "*"
> . OK Completed (0.000 secs 1 calls)
> . LOGOUT
> * BYE LOGOUT received
> . OK Completed
> Connection closed.
>
> $ cyradm --user cyrus --authz cyrus localhost
> Login disabled.
> cyradm: cannot authenticate to server with  as cyrus
> $ cyradm --user cyrus --authz cyrus --auth pam localhost
> verify error:num=18:self signed certificate
> cyradm: cannot authenticate to server with pam as cyrus
> $ cyradm --user cyrus --authz cyrus --auth shadow localhost
> verify error:num=18:self signed certificate
> cyradm: cannot authenticate to server with shadow as cyrus

What does it do if you run with "--auch PLAIN" instead?

Regards,
Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

RE: Disappearing Mailbox Content

[Simon Matter]

>
>
> --On September 9, 2015 at 13:49:40 -0500 "Robert T. Covell"
>  wrote:
>
>> That is the problem.  I cannot reproduce (reliably or at all).  It might
>> be months before we hear about it.  It has been happening for about two
>> years.  Always chalked it up to user error.  But I can't say that it is
>> or isn't.
>>
>
> Check the ACL on the mailbox and make sure no user has permission to
> delete
> the mailbox-- no "x" right. Loss of all content including the cyrus* files
> is mailbox deletion. (This is the voice of experience talking)
>
> Too bad you didn't implement delayed delete. You can just rename the
> deleted folder to its original name.

But, since he's using our Invoca RPM, delayed delete should be used by
default. So yes, deleted folders should be there, are they?

Regards,
Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Migrate from 2.2.13 to 2.4.17 disasters

[Simon Matter]

 Am Montag, 24. August 2015, 07:44:42 schrieb bs...@vsvinc.com:
  From what I have read in the documentation, you must have each user's
 password.
 This is incorrect for cyrus.

 Is there something I'm missing in the docs?
 It seem so.

 You can use a single cyrus admin user instead too which have access to any
 account content usually - imapsync provides this authentication scheme (by
 an
 option auth-user or so) too...

And also check the proxyservers option in imapd.conf which can be used to
...to  proxy  for  other users...

Regards,
Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cyrus IMAP 3.0.0-beta1 released

[Simon Matter]

 The Cyrus team is proud to announce the immediate availability of the
 first beta from the Cyrus IMAP 3.0 series: 3.0.0-beta1.

Hi,

As I'm currently working on a 2.5.x release of our Invoca RPMs I'm
wondering if it would make sense to just go with 3.0 beta? The work on
2.5.x will still take some more time because it's a big change with the
new build system, new functionality and support for new systems with
systemd.

Thanks for any insights which help me to make a decision.

Regards,
Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

RE: RPM for Cyrus-IMAPd 2.5.x?

[Simon Matter]

 RHEL6 doesn't use systemd. Or at least ours doesn't.

I know that. But our packages build on all supported RHEL versions from a
single source RPM, that means since RHEL7 it has to support both, sysv and
systemd.


 -Original Message-
 From: Michael Gratton [mailto:m...@vee.net]
 Sent: Tuesday, May 05, 2015 12:58 AM
 To: Simon Matter
 Cc: Rosenbaum, Larry M.; 'info-cyrus@lists.andrew.cmu.edu'
 Subject: Re: RPM for Cyrus-IMAPd 2.5.x?


 I hear systemd also eats babies.

 Anyone have a PPA with 2.5 for Ubuntu 14.04 LTS (or later)?

 --
 ⊨ Michael Gratton, Percept Wrangler.
 ⚙ http://mjog.vee.net/


 On Tue, 5 May, 2015 at 2:46 PM, Simon Matter simon.mat...@invoca.ch
 wrote:
   Where can I get an RPM for Cyrus IMAPd 2.5.1 (or even 2.5.0)? We
 are
   running RHEL6.
 
  Doing RPMs for current Cyrus IMAPd is on my RPM TODO list.
  Unfortunately
  this list is quite long (~30) at the moment. The reason is simple and
  is
  called systemd.
 
  Regards,
  Simon
 
  
  Cyrus Home Page: http://www.cyrusimap.org/
  List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
  To Unsubscribe:
  https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus






Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: RPM for Cyrus-IMAPd 2.5.x?

[Simon Matter]

 Where can I get an RPM for Cyrus IMAPd 2.5.1 (or even 2.5.0)? We are
 running RHEL6.

Doing RPMs for current Cyrus IMAPd is on my RPM TODO list. Unfortunately
this list is quite long (~30) at the moment. The reason is simple and is
called systemd.

Regards,
Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Problem with quota

[Simon Matter]

 /Hi,
 we
 - stoped cyrus-imapd
 - reconstructed a mailbox (/usr/lib/cyrus-imapd/reconstruct -r -f
 user.account)
 ///- started cyrus-imapd/
 - deleted file /var/lib/imap/quota/a/user.account//
 - set quota (sq user.account  3000)
 -//recalculated the quota (/usr/lib/cyrus-imapd/quota -f
 user.///account)/

 The output of cyrus quota is differente to real space of mailbox on
 disk.


 lq///user.account/
   STORAGE 9789922/3000 (32.63307%)

 du -s /var/spool/imap/a/user/account/
 7308012var/spool/imap/a/user/account//

 There are a difference//greater than  2.0 GB (9.8 GB - 7.3 GB)

 //Why?

Hi,

I guess that's because of single instance store. It's not a bug then but a
feature if duplicatesuppression: 1. Duplicate messages are hardlinked on
disk, they don't consume space there, but are still calculated in quota
usage.

Regards,
Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Problem with quota

[Simon Matter]

 On 04/04/2015 09:02 AM, Patrick Boutilier wrote:
 Quoting Simon Matter simon.mat...@invoca.ch, Sat, 04 Apr 2015:

 I guess that's because of single instance store. It's not a bug then
 but a
 feature if duplicatesuppression: 1. Duplicate messages are
 hardlinked on
 disk, they don't consume space there, but are still calculated in
 quota
 usage.

 Isn't that singleinstancestore:1 ? duplicatesuppression is where lmtpd
 will suppress delivery of a message to a mailbox if a message with the
 same message-id (or resent-message-id) is recorded as having already
 been delivered to the mailbox.

 Yes Patrick, you're correct.  However there is a connection, in that
 singleinstancestore requires the duplicate DB in order to do its work,
 so people often conflate the two.

Thanks Nic, that's correct. I shouldn't post before having a useful
coffeine level.

Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: saslauthd question

[Simon Matter]

 On 12/11/2014 12:45 PM, Andrew Morgan wrote:
 I only have PAM files for imap, lmtp, and sieve
 although I have other service names for some of them.


 I don't understand why you have PAM files for lmtp and sieve, but most
 particularly lmtp.  lmtpd is just a local daemon that transfers stuff
 from your smtp server to cyrus.  Are you running cyrus and smtpd on
 different servers?  If so, what does the PAM lmtp configuration look like?

If you do lmtp over the network, you may want to authenticate who can
deliver mails. Otherwise you can just use lmtpd -a if the environment is
considered secure.


 I don't know anything about sieve, but thought the filters where all
 internal, too; hence not in need of authentication.

Sieve rules have to be managed per user, and therefore you need
authentication. The server itself doesn't need to auth anything to run the
filters.

Regards,
Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Strange load issue with 2.4.17

[Simon Matter]

 Hi,

 for the last week we have seen strange load issues on our Cyrus server.
 All
 of a sudden the load increases to several thousands, user CPU goes down to
 basically zero, system CPU spikes. In the past we've had trouble with poor
 I/O performance, but that went along with an increase in Wait I/O. We
 don't
 see that now. vmstat shows a massive increase in context switches. When
 the
 system reaches this state, all we can do is restart Cyrus or reboot the
 machine if that doesn't work anymore.

 I'm attaching a Ganglia screenshot that shows the problem clearly. When
 the
 problem exists, there's not much we can do to analyze it. A colleague
 suggested that what we see could be related to this bug:

 https://bugzilla.cyrusimap.org/show_bug.cgi?id=3744

 It was reported for 2.4.16, and it sounds as if it has been fixed, but is
 that fix really part of 2.4.17? Any other ideas?

Is this a physical host or running virtualized?

Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Strange load issue with 2.4.17

[Simon Matter]

 --On 13. Oktober 2014 17:35:25 +0200 Simon Matter simon.mat...@invoca.ch
 wrote:

 Hi,

 for the last week we have seen strange load issues on our Cyrus server.
 All
 of a sudden the load increases to several thousands, user CPU goes down
 to basically zero, system CPU spikes. In the past we've had trouble
 with
 poor I/O performance, but that went along with an increase in Wait I/O.
 We don't
 see that now. vmstat shows a massive increase in context switches. When
 the
 system reaches this state, all we can do is restart Cyrus or reboot the
 machine if that doesn't work anymore.

 I'm attaching a Ganglia screenshot that shows the problem clearly. When
 the
 problem exists, there's not much we can do to analyze it. A colleague
 suggested that what we see could be related to this bug:

 https://bugzilla.cyrusimap.org/show_bug.cgi?id=3744

 It was reported for 2.4.16, and it sounds as if it has been fixed, but
 is
 that fix really part of 2.4.17? Any other ideas?

 Is this a physical host or running virtualized?

 It's virtualized, but it's been that way for more than a year.

Is this by any chance running on KVM, maybe on an AMD cpu?


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Weird CMD and ENVIRON

[Simon Matter]

 Hi!

 Since sometime in 2.4, we've seen odd behaviour in the CMD and ENVIRON of
 imapd
 processes. Like this:


 --klip--
 [jmmpelto@pcn1 ~]$ ps -ef|grep imapd|head -n 1
 cyrus  300 15885  0 13:30 ?00:00:00 imapd:
 webmail-3.mappi.helsinki.fi [128.214.20.217] ajokela
 [jmmpelto@pcn1 ~]$ sudo cat /proc/300/cmdline
 imapd: webmail-3.mappi.helsinki.fi [128.214.20.217] ajokela


Hi,

I see the same behaviour. Could it be that it has something to do with
setproctitle()?

If I strings /proc/20488/environ on preforked processes the ENV looks
good until a process actually gets used.

Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: cvt_cyrusdb_all

[Simon Matter]

 Simon, while I think you're the best contact for my question, I'm
 including the list to be helpful to others that may have a similar
 question.

 We run multiple instances of Cyrus on a server and I noticed that your
 cvt_cyrusdb_all script cites mult-instance support several places in the
 change log. However, there's no man page for the script and it's not
 immediately obvious to me by looking at the code how to appropriately
 pass an instance name to the script as an argument. As is, this script
 attempts to operate on /var/lib/imap, which is empty in my case. Can you
 please explain how to pass an instance name to your script?

Hi, the script takes the instance name in the var ${INSTANCE}. You should
find how to use it in
/usr/share/doc/cyrus-imapd-2.4.17/README.multi-instance. Please note that
the script should only be used by the init scripts from the RPM package.

Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Problem creating / finding mailboxes

[Simon Matter]

 CentOS 6.5, cyrus-imapd 2.3.16-6

 So, I gather you're supposed to create mailboxes called user.username
 But I wind up with user^username, and mailbox can't be found.

 [joliver@localhost ~]$ cyradm -u testuser1 localhost
 Password:
 localhost cm user.testuser2
 localhost cm user.testuser3
 localhost cm user.textuser4

 [joliver@localhost ~]$ sudo ls -l /var/vmail/u/
 total 12
 drwx--. 2 cyrus mail 4096 Sep 15 15:07 user^testuser2
 drwx--. 2 cyrus mail 4096 Sep 15 14:51 user^testuser3
 drwx--. 2 cyrus mail 4096 Sep 15 15:07 user^textuser4

 Sep 15 15:05:18 localhost lmtpunix[15704]: verify_user(user.testuser2)

I guess you should send mail to testuser2, not user.testuser2?

Simon

 failed: Mailbox does not exist
 Sep 15 15:05:18 localhost master[15929]: about to exec
 /usr/lib/cyrus-imapd/lmtpd
 Sep 15 15:05:18 localhost lmtpunix[15929]: executed
 Sep 15 15:05:18 localhost postfix/lmtp[15928]: 4B2E61ED5:
 to=testus...@domain.com,
 relay=localhost.localdomain[/var/lib/imap/socket/lmtp], delay=32,
 delays=32/0.01/0/0.02, dsn=5.1.1, status=bounced (host
 localhost.localdomain[/var/lib/imap/socket/lmtp] said: 550-Mailbox
 unknown.  Either there is no mailbox associated with this 550-name or
 you do not have authorization to see it. 550 5.1.1 User unknown (in
 reply to RCPT TO command))


 What did I do wrong?

 --
 ***
 * John Oliver http://www.john-oliver.net/ *
 * *
 ***
 
 Cyrus Home Page: http://www.cyrusimap.org/
 List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
 To Unsubscribe:
 https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus




Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Problem creating / finding mailboxes

[Simon Matter]

 CentOS 6.5, cyrus-imapd 2.3.16-6

 So, I gather you're supposed to create mailboxes called user.username
 But I wind up with user^username, and mailbox can't be found.

 [joliver@localhost ~]$ cyradm -u testuser1 localhost
 Password:
 localhost cm user.testuser2
 localhost cm user.testuser3
 localhost cm user.textuser4

 [joliver@localhost ~]$ sudo ls -l /var/vmail/u/
 total 12
 drwx--. 2 cyrus mail 4096 Sep 15 15:07 user^testuser2
 drwx--. 2 cyrus mail 4096 Sep 15 14:51 user^testuser3
 drwx--. 2 cyrus mail 4096 Sep 15 15:07 user^textuser4

 Sep 15 15:05:18 localhost lmtpunix[15704]: verify_user(user.testuser2)

 I guess you should send mail to testuser2, not user.testuser2?

Sorry, I think that was wrong.
However, you should check your unixhierarchysep, hashimapspool and 
fulldirhash settings.

Regards,
Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cyrus-imapd can't find my mail

[Simon Matter]

 I'm pretty certain this is because it just doesn't know where to look,
 but I'm not sure how to tell it :-(

No, it will not work that way because Cyrus uses a different message store
format. You may search the list archives for more on this topic.

Regards,
Simon


 I was using http://www.linuxmail.info/postfix-dovecot-ldap-centos-5/ as
 a guide, but needed to switch to cyrus-imapd  I think cyrus doesn't know
 how to look in /var/vmail/ as the vmail user.

 --
 ***
 * John Oliver http://www.john-oliver.net/ *
 * *
 ***
 
 Cyrus Home Page: http://www.cyrusimap.org/
 List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
 To Unsubscribe:
 https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus




Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: backup rsync

[Simon Matter]

 Am Freitag, den 29.08.2014, 07:56 -0700 schrieb David R Bosso:
 --On August 29, 2014 at 4:27:57 PM +0200 Marcus Schopen
 li...@localguru.de wrote:

  Hi,
 
  I'm planing to use lvm snaps and rsync for a daily disaster recovery
  backup on my master cyrus (2.4.12 Ubuntu 12.04 LTS):

 Have you tested lvm snaps for this purpose yet?  When I tried to use lvm
 snapshots in this way a few years ago, it absolutely killed performance
 on
 the volume with the active snapshots.  Hopefully things have improved,
 but
 I wouldn't bet on it.

 Yes, I've tested it and don't see any performance problems while the
 snaphot is active. My problem is how to backup about 140 GB mailspool in
 an acceptable time slot. What tools are recommended?

 If using rsync what options are best practice?

I suggest -aH to preserve single instance storage in the backup.

Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: problems with deleted messages and reconstruct

[Simon Matter]

 Hello all, I’m having the following issue with cyrus imapd 2.4.13: a user
 deletes a large amount of mails but many of those mails remain in the
 mailbox folder. I double checked and imapd reports no mails prior to, say,
 2008. However, there are mail files in the mailbox older than 2008. I
 tried to reconstruct -r user.xpto, as well as “reconstruct -r -O
 user.xpto”, but this doesn’t seem to solve the problem. Also, I didn’t
 find a solution/explanation in neither FAQ nor googling.

 Any clues on how to diagnose this problem?

Does your server use expunge_mode: delayed?

Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: problems with deleted messages and reconstruct

[Simon Matter]

Depending on how you installed the defaults could have been changed by a
patch.

Simon


 No, it does not. I do not specify it in imapd.conf, so I assume the
 default behavior (not delayed).

 Rodrigo

 On 19/06/2014, at 10:47, Simon Matter simon.mat...@invoca.ch wrote:


 Hello all, I’m having the following issue with cyrus imapd 2.4.13: a
 user
 deletes a large amount of mails but many of those mails remain in the
 mailbox folder. I double checked and imapd reports no mails prior to,
 say,
 2008. However, there are mail files in the mailbox older than 2008. I
 tried to reconstruct -r user.xpto, as well as “reconstruct -r -O
 user.xpto”, but this doesn’t seem to solve the problem. Also, I didn’t
 find a solution/explanation in neither FAQ nor googling.

 Any clues on how to diagnose this problem?

 Does your server use expunge_mode: delayed?

 Simon


 Rodrigo Ventura
 Institute for Systems and Robotics
 Instituto Superior Técnico
 Lisbon, Portugal
 http://users.isr.ist.utl.pt/~yoda




Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: problems with deleted messages and reconstruct

[Simon Matter]

Then I guess unexpunge -l should show them.

Simon


 I didn’t apply any patch. This is a vanilla cyrus imap clean install.

 Shouldn’t reconstruct -O take care of non indexed files? Should I assume
 that, if it didn’t delete them, they are still indexed? How can I list all
 expunged/non-deleted mails?

 Thank you,
 Rodrigo

 On 19/06/2014, at 11:51, Simon Matter simon.mat...@invoca.ch wrote:

 Depending on how you installed the defaults could have been changed by a
 patch.

 Simon


 No, it does not. I do not specify it in imapd.conf, so I assume the
 default behavior (not delayed).

 Rodrigo

 On 19/06/2014, at 10:47, Simon Matter simon.mat...@invoca.ch wrote:


 Hello all, I’m having the following issue with cyrus imapd 2.4.13: a
 user
 deletes a large amount of mails but many of those mails remain in the
 mailbox folder. I double checked and imapd reports no mails prior to,
 say,
 2008. However, there are mail files in the mailbox older than 2008. I
 tried to reconstruct -r user.xpto, as well as “reconstruct -r -O
 user.xpto”, but this doesn’t seem to solve the problem. Also, I
 didn’t
 find a solution/explanation in neither FAQ nor googling.

 Any clues on how to diagnose this problem?

 Does your server use expunge_mode: delayed?

 Simon


 Rodrigo Ventura
 Institute for Systems and Robotics
 Instituto Superior Técnico
 Lisbon, Portugal
 http://users.isr.ist.utl.pt/~yoda




 Rodrigo Ventura
 Institute for Systems and Robotics
 Instituto Superior Técnico
 Lisbon, Portugal
 http://users.isr.ist.utl.pt/~yoda




Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: cyrus super user -- able to login in as any user with a superuser password

[Simon Matter]

 hi all,

 i have a cyrus instance with an ldap backend -- works beautifully ...

 i was wondering if there is a way to be able to login as any user with a
 `superuser` password without messing with the users `real` password.

 i am assuming this would have to involve some saslauthd magic ...

Not really saslauthd magic, just check the proxyservers option in
imapd.conf.

Regards,
Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Re[2]: Postfix with Cyrus Imap

[Simon Matter]

 Ok, well thanks for the info.
 There is only one problem with my Cyrus Imapd 2.4.16... I have not found
 autocrate path for this version of Cyrus.

Maybe the patches from here will work
http://slackware.org.uk/slackbuilds.org/14.0/network/cyrus-imapd/patches/

Regards,
Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re:

[Simon Matter]

Hi,

Did you try running reconstruct -r -f ...?

BTW, could you change your mailer so you don't send HTML only?

Regards,
Simon

 htmlhead/headbodydiv style=font-family: Verdana;font-size:
 12.0px;divHello,/div

 divnbsp;/div

 divI had a cyrus installation running that worked perfectly. After a
 crash of a hard drive I had to reinstall the system. The hard drive
 contained the cyrus installation. However, the cyrus
 quot;partition-defaultquot; was on a different hard drive./div

 divnbsp;/div

 divI reinstalled cyrus with the quot;partition-defaultquot; as
 before.nbsp; The user mailbox can be seen in cyradmin [user.paul
 (#92;HasChildren)]. However all subfolders exept user.paul.Drafts,
 user.paul.Muuml;lleimer, user.paul.Versendete_Nachrichten can NOT be
 seen. The directories and the messages in the storage partion under
 /var/spool/imap/user/paul/ are all there./div

 divnbsp;/div

 divWith a squirrelmail I do see all messages in INBOX and I do see the
 three subfolders. However, the subfolders are being shown to be empty. The
 files in the corresponding directories exist./div

 divnbsp;/div

 divI did a reconstruct -r with the cyrus admin allready, however I still
 would have the data as they were after the hard drive crash./div

 divnbsp;/div

 divIs there anybody who can help to get the old subfolders and messages
 back?/div

 divnbsp;/div

 divBest regards/div

 divnbsp;/div

 divStefan/div

 divnbsp;/div/div/body/html
 
 Cyrus Home Page: http://www.cyrusimap.org/
 List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
 To Unsubscribe:
 https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Squatter crash with statusdb

[Simon Matter]

 Hi Andy, could you file a bug for this?  Then it will not be forgotten...

Or, could you check this bug here
http://bugzilla.cyrusimap.org/show_bug.cgi?id=3757

The patch below was the fix, could you verify if it also fixes your issue?

Thanks,
Simon

From 1661683d453ea444aae5832b4a2cb7fd54489672 Mon Sep 17 00:00:00 2001
From: Bron Gondwana br...@opera.com
Date: Sun, 09 Dec 2012 19:42:17 +
Subject: Bug #3757 - don't segfault on mailbox close with no user

Broke squatter and possibly other users as well.

This is probably a 2.4 only fix - the code has changed
for statuscache a bit since.
---
diff --git a/imap/statuscache_db.c b/imap/statuscache_db.c
index fadc58d..41dfd2b 100644
--- a/imap/statuscache_db.c
+++ b/imap/statuscache_db.c
@@ -150,9 +150,11 @@ static char *statuscache_buildkey(const char
*mailboxname, const char *userid,

 /* Build statuscache key */
 len = strlcpy(key, mailboxname, sizeof(key));
+/* double % is a safe separator, it can't exist in a mailboxname */
 key[len++] = '%';
 key[len++] = '%';
-len += strlcpy(key + len, userid, sizeof(key) - len);
+if (userid)
+   len += strlcpy(key + len, userid, sizeof(key) - len);

 *keylen = len;

@@ -410,11 +412,9 @@ int statuscache_invalidate(const char *mboxname,
struct statusdata *sdata)
 drock.db = statuscachedb;
 drock.tid = NULL;

-key = statuscache_buildkey(mboxname, , keylen);
+key = statuscache_buildkey(mboxname, /*userid*/NULL, keylen);

-/* strip off the second NULL that buildkey added, so we match
- * the entires for all users */
-r = DB-foreach(drock.db, key, keylen - 1, NULL, delete_cb,
+r = DB-foreach(drock.db, key, keylen, NULL, delete_cb,
drock, drock.tid);
 if (r != CYRUSDB_OK) {
syslog(LOG_ERR, DBERROR: error invalidating: %s (%s),



 Quoting Andy Fiddaman cy...@fiddaman.net, Mon, 24 Jun 2013:

 FWIW, this gets it working again:

 --- cyrus-imapd-2.4.17.dist/imap/statuscache_db.c   2013-06-24
 10:10:08.219203100 +
 +++ cyrus-imapd-2.4.17/imap/statuscache_db.c2013-06-24
 10:10:20.537711377 +
 @@ -152,7 +152,7 @@
  len = strlcpy(key, mailboxname, sizeof(key));
  key[len++] = '%';
  key[len++] = '%';
 -len += strlcpy(key + len, userid, sizeof(key) - len);
 +len += strlcpy(key + len, userid ? userid : cyrus, sizeof(key) -
 len);

  *keylen = len;


 On Mon, 24 Jun 2013, Andy Fiddaman wrote:

 ;
 ; Hi,
 ;
 ; I've just upgraded my Cyrus installation to 2.4.17 and squatter is
 ; crashing in statuscache_buildkey() because userid is NULL.
 ;
 ; I'm not sure what the best fix for this is. Should squatter even be
 using
 ; the statuscache or should it populate cyrus as the username when
 ; initialising the index, or something else?
 ;
 ; Thanks,
 ;
 ; Andy
 ;
 ; Program received signal SIGSEGV, Segmentation fault.
 ; [Switching to Thread 1 (LWP 1)]
 ; 0xfd7ffe3dccb0 in .strlenalign16_loop () from /lib/64/libc.so.1
 ; (gdb) where
 ; #0  0xfd7ffe3dccb0 in .strlenalign16_loop () from
 /lib/64/libc.so.1
 ; #1  0xfd7ffe414149 in strlcpy () from /lib/64/libc.so.1
 ; #2  0x004610ac in statuscache_buildkey (
 ; mailboxname=0x5ab8b0 example.net!user.silo, userid=0x0,
 ; keylen=0xfd7fffdfe0cc) at statuscache_db.c:155
 ; #3  0x0046169a in statuscache_update_txn (
 ; mboxname=0x5ab8b0 example.net!user.silo,
 sdata=0xfd7fffdfe290,
 ; tidptr=0xfd7fffdfe218) at statuscache_db.c:326
 ; #4  0x004619ad in statuscache_invalidate (
 ; mboxname=0x5ab8b0 example.net!user.silo,
 sdata=0xfd7fffdfe290)
 ; at statuscache_db.c:425
 ; #5  0x00434099 in mailbox_unlock_index (mailbox=0x5db998,
 ; sdata=0xfd7fffdfe290) at mailbox.c:1637
 ; #6  0x00422feb in index_unlock (state=0x5af6a0) at
 index.c:1232
 ; #7  0x00420c98 in index_open (name=0x5ac960
 ; example.net!user.silo,
 ; init=0x0, stateptr=0xfd7fffdfec18) at index.c:246
 ; #8  0x00420395 in index_me (name=0x5ac960
 example.net!user.silo,
 ; matchlen=20, maycreate=0, rock=0xfd7fffdffcac) at
 squatter.c:594
 ; #9  0x004208f9 in main (argc=3, argv=0xfd7fffdffcf8)
 ; at squatter.c:745
 ;
 ;
 ; (gdb) frame 7
 ; #7  0x00420c98 in index_open (name=0x5ac960
 ; example.net!user.silo,
 ; init=0x0, stateptr=0xfd7fffdfec18) at index.c:246
 ; 246 in index.c
 ; (gdb) print *state
 ; $4 = {mailbox = 0x5db998, num_records = 6933, oldexists = 0, exists =
 ; 6933,
 ;   last_uid = 6934, highestmodseq = 1606, delayed_modseq = 0, map =
 ; 0x5dd740,
 ;   mapsize = 7168, internalseen = 0, skipped_expunge = 0, seen_dirty =
 0,
 ;   keepingseen = 0, examining = 0, myrights = 0, numrecent = 0,
 ;   numunseen = 6933, firstnotseen = 1, flagname = {0x0 repeats 128
 ; times},
 ;   userid = 0x0, out = 0x0, qresync = 0, authstate = 0x0}
 ;
 ; 
 ; Cyrus Home Page: http://www.cyrusimap.org/
 ; List Archives/Info: 

Re: namesppace trouble

[Simon Matter]

 Hello,
 the cyrus namespace base is located in my default installation in

 /var/spool/imap/user/

 I expected afterrunning the mkimap script to find the alphabetical
 letters of the usernames in that directory:

 a
 b
 c
 d
 e

 and so on

What's you 'hashimapspool' setting? Is it 0|false?

Regards
Simon


 and have every user mailbox created in the dorresponding alphabetical
 subdirectory

 for example user foo would be in

 /var/spool/imap/user/f/foo

 but it is not like that

 when I Create user foo or user bar their mailbox are set up as

 /var/spool/imap/user/foo
 /var/spool/imap/user/bar

 How can I change this behaviour  and have the classical schema ?

 /var/spool/imap/user/a/
 /var/spool/imap/user/b/
 /var/spool/imap/user/c/
 /var/spool/imap/user/d/
 .
 .
 .
 /var/spool/imap/user/z/

 I am using cyrus-imapd-2.4.17 on FreeBSD 9.1


 thank you

 Rick


 
 Cyrus Home Page: http://www.cyrusimap.org/
 List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
 To Unsubscribe:
 https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus




Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Safe to delete lock files in /var/imap/lock ?

[Simon Matter]

 I am moving my Cyrus server to new hardware using rsync. I noticed that
 the /var/imap/lock has 2.7G worth of lock files in it. Can the files in
 /var/imap/lock be deleted once Cyrus is shutdown? If so, I will not
 bother rsyncing them to the new server at all.

I'd say yes, you don't have to sync them. But, I'm wondering why you have
2.7G size in lock/? Since all the locks are 0 byte files I'm wondering
where the size comes from?

Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: small patch to disable openssl compression

[Simon Matter]

 Hi - This patch disables openssl compression - not sure if this is a
 security risk or not... but, I don't think I like the encryption library
 performing compression anyway, it's complicated already. Maybe.

Just FYI, with latest RHEL5+6 openssl now disables compression by default.
I guess the end result for cyrus is the same like with your patch? Maybe
other distributions do the same.

Simon


 Chris


 diff -rupN cyrus-imapd-2.4.17/imap/tls.c cyrus-imapd-2.4.17.f/imap/tls.c
 --- cyrus-imapd-2.4.17/imap/tls.c 2012-12-01 19:57:54.0 +
 +++ cyrus-imapd-2.4.17.f/imap/tls.c   2013-01-18 15:27:58.0 +
 @@ -667,6 +667,11 @@ int tls_init_serverengine(const char
   off |= SSL_OP_NO_SSLv3;
  }
  SSL_CTX_set_options(s_ctx, off);
 +
 +#ifdef SSL_OP_NO_COMPRESSION
 +SSL_CTX_set_options(s_ctx, SSL_OP_NO_COMPRESSION);
 +#endif
 +
  SSL_CTX_set_info_callback(s_ctx, (void (*)())
 apps_ssl_info_callback);

  /* Don't use an internal session cache */


 
 Cyrus Home Page: http://www.cyrusimap.org/
 List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
 To Unsubscribe:
 https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus




Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Exchange to cyrus-imapd mail migration query

[Simon Matter]

 On 02/21/13 15:59 +0530, jayesh shinde wrote:
Hi all ,

I am migrating the Exchange 2003 server to Cyrus-imapd + postfix  server
Due to some reason , on Exchange server imap protocol is not open and
only mapi protocol is available.
Along with this end users are not allowing the share the passwords.

In Exchange by giving the administrator access ( i.e ACL ) on
particular mailbox , admin ID can see other account's emails.  But
its only opening on Mapi enable outlook.

 You might see if it's possible to enable IMAP on your exchange server.
 IMAP
 runs on port 993 on our's (imaps).

Is there any tool ( like imapsync ) by which I can connect to
Exchange server over Mapi protocol
and sync the emails in cyrus-impad's imap ( 143 ) enable server ?

I want to know , how peoples are handing such situation and migrating
the emails ?

 Try configuring an Outlook client simultaneously with both a MAPI
 connection, and an IMAP connection, and then copy the messages between the
 two.

 Another approach would be to export your messages, using Outlook, to a
 .pst
 file, and figure out a way to import those messages into cyrus, or back
 into outlook after IMAP has been configured.

If IMAP is enabled on the exchange side, then imapsync should also work.

Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: problems with quota files

[Simon Matter]

 Hello I am running cyrus 2.4.17

 I realized that quota shown for users with quota command is much less
 than real filesystem usage

 an example:

 /usr/lib/cyrus-imapd/quota user.username
 Quota   % Used Used Root
   1500   34  5189501 user.username

 du -sh /var/spool/imap/m/user/username
 11G/var/spool/imap/m/user/username

 I am a bit worried about it.

 I am using ext4

 the same problem was also on the previous cyrus version I was using
 2.3.16  because I have the same problems.

 I fixed the quota files with quota -f but it did not resolve my problem.

 any hints ?

Maybe single instance store is causing it?

Regards,
Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: multiple statuscache db (only Invoca rpm?)

[Simon Matter]

 Hello cyrus user,

   on my cyrus-imapd server I see this:

 man imapd.conf:

 statuscache: 0
  Enable/disable the imap status cache.

 statuscache_db: skiplist
  The cyrusdb backend to use for the imap status cache.

  Allowed values: berkeley, berkeley-nosync, berkeley-hash,
 berkeley-hash-nosync, skiplist

 statuscache_db_path: {configdirectory}/volatile/statuscache.db
  The absolute path to the statuscache db file.


 In my imapd.conf I set:
 statuscache: 1

 Every time I restart cyrus-imapd, a copy of
 /var/lib/imap/volatile/statuscache.db appears on /var/lib/imap path:

What kind of problem do you see?
The idea behind the volatile/ directory is that you can put it on tmpfs
for performance reason if you like and the files there are saved to
/var/lib/imap/ on shutdown and restored from there on startup.
And yes, that's Invoca rpm specific.


 #ls -l /var/lib/imap/
 total 68
 -rw--- 1 cyrus mail  852 Feb  8 09:15 annotations.db
 drwx-- 2 cyrus mail 4096 Jan 14 08:43 backup
 drwx-- 2 cyrus mail 4096 Feb  8 08:52 db
 drwx-- 2 cyrus mail 4096 Feb  8 09:15 db.backup1
 drwx-- 2 cyrus mail 4096 Feb  8 09:10 db.backup2
 drwx-- 2 cyrus mail 4096 Jan 14 08:43 log
 -rw--- 1 cyrus mail  144 Feb  8 09:15 mailboxes.db
 drwx-- 2 cyrus mail 4096 Jan 14 08:43 meta
 drwx-- 2 cyrus mail 4096 Jan 14 08:43 msg
 drwx-- 2 cyrus mail 4096 Jan 14 08:43 ptclient
 drwx-- 2 cyrus mail 4096 Jan 14 08:43 quota
 drwx-- 2 cyrus mail 4096 Feb  8 09:15 rpm
 drwx-- 2 cyrus mail 4096 Jan 14 08:43 sieve
 drwxr-x--- 2 cyrus mail 4096 Feb  8 09:15 socket
 -rw--- 1 cyrus mail  144 Feb  8 09:10 statuscache.db
 drwx-- 2 cyrus mail 4096 Jan 14 08:43 sync
 drwx-- 2 cyrus mail 4096 Jan 14 08:43 user
 drwx-T 2 cyrus mail   60 Feb  8 09:15 volatile

 I have to remove it by hand...

Why? Are there any problems?

Regards,
Simon


 I don't know if this happens only on Invoca rpm which I use:

 name   : Cyrus IMAPD
 version: v2.4.17-Invoca-RPM-2.4.17-6.el6 d1df8aff 2012-12-01
 vendor : Project Cyrus
 support-url: http://www.cyrusimap.org
 os : Linux
 os-version : 2.6.32-279.el6.x86_64
 environment: Built w/Cyrus SASL 2.1.23
   Running w/Cyrus SASL 2.1.23
   Built w/OpenSSL 1.0.0-fips 29 Mar 2010
   Running w/OpenSSL 1.0.0-fips 29 Mar 2010
   Built w/zlib 1.2.3
   Running w/zlib 1.2.3
   CMU Sieve 2.4
   TCP Wrappers
   NET-SNMP
   mmap = shared
   lock = fcntl
   nonblock = fcntl
   idle = idled

 Regards
 Marco

 
 Cyrus Home Page: http://www.cyrusimap.org/
 List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
 To Unsubscribe:
 https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus




Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: How can this happen?

[Simon Matter]

 On Thu, 24 Jan 2013 13:12:33 -0800 (PST) Andrew Morgan wrote:
 On Thu, 24 Jan 2013, Frank Elsner wrote:

 
  Hello,
 
  we have the strange situation with our murder environment that
 
  mailbackend has
 
  user.x.Sent2 default x lrswipkxtecda
 
  but the folder is non-existent in the filesystem.
 
  The mupdate server doesn't know this folder (not in mailboxes.db)
 
  How can this happen?

 Some bug in Cyrus?

 Don't know.

 If you want to fix this, you can try creating the proper structure on
 the
 filesystem, run reconstruct to get Cyrus to sync back up with it, then
 delete the folder using cyradm or an IMAP client.

 I didn't ask for a fix. I ask for a possible reason.

I suggest to at lest report the version of Cyrus used.

Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: How can this happen?

[Simon Matter]

 On Fri, 25 Jan 2013 10:48:28 +0100 Simon Matter wrote:
  On Thu, 24 Jan 2013 13:12:33 -0800 (PST) Andrew Morgan wrote:
  On Thu, 24 Jan 2013, Frank Elsner wrote:
 
  
   Hello,
  
   we have the strange situation with our murder environment that
  
   mailbackend has
  
   user.x.Sent2 default x  lrswipkxtecda
  
   but the folder is non-existent in the filesystem.
  
   The mupdate server doesn't know this folder (not in mailboxes.db)
  
   How can this happen?
 
  Some bug in Cyrus?
 
  Don't know.
 
  If you want to fix this, you can try creating the proper structure on
  the
  filesystem, run reconstruct to get Cyrus to sync back up with it,
 then
  delete the folder using cyradm or an IMAP client.
 
  I didn't ask for a fix. I ask for a possible reason.

 I suggest to at lest report the version of Cyrus used.

 Oh, shit. Shame on me.
 2.3.16 on RHEL.

OK, the current version is 2.4.17. If it's a bug there's a good chance it
has been fixed since 2.3.16 :)

Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Load spikes when new email arrives

[Simon Matter]

 In another email discussion on the Redhat mailing list, I've confirmed
 we have
 an issue with partition alignment.  This is getting to be quite the mess
 out there.  I saw one posting where it is speculated there are
 thousands of
 poorly set up disk partitions for their RAID stripe size.  fdisk and
 OS installers were late getting updated for the new TB disks
 and SSD disks as well.  Partition alignment might account
 for 5 to 30% of a performance hit.

 I've checked and my cyrus lmtpd process count
 never exceeds 11 under work load.
 await jumps up to 150-195 at worst.

 If I'm already at IO saturation, I can't see how a higher lmtpd limit
 would help.

 My goal is to keep the system load reasonable so it is responsive for
 mailbox access by the end users.  Right now we get nagios alerts
 about 6 times a day for excessive load.  If I can move the mail
 queue workload into a hill instead of a sharp peak on the cacti
 load graph, it would be good.  There are minutes around the peaks
 where the queue is emptied and we have only 5 messages
 inbound per minute.

 In hind sight, I agree RAID 10 should have been implemented.
 At the time, four years ago, getting lots of space was the
 priority as space needs always grow.  We've never seen load
 issues until this month, and it seems to coincide with a
 general increase of all email volume and traffic.  Our primary
 MX is also getting hit more than normal.



 There are a couple suggestions I'd like to put forth. First, improper
 partition alignment is generally masked by the controller cache. I
 strongly encourage you to check that your RAID array is making use of
 this cache by enabling the WriteBack caching option on this array,
 especially if your PERC card has a BBU (I think this was optional on
 perc 5). You can install the MegaCLI tool from LSI to verify this (can
 also be checked from OpenManage or reboot into the controller BIOS).

I strongly suggest to do that *ONLY* with proper BBU in place!


 MegaCLI Link:
 http://www-947.ibm.com/support/entry/portal/docdisplay?lndocid=migr-5082327
 The relevant commands are as follows:
 MegaCli -AdpBbuCmd -aALL
 MegaCli -LDInfo -Lall -aALL

 Second, the PERC card does support RAID level migration, so if you want
 to add a spindle or even change RAID levels, you can. This can be done
 via either OpenManage (hit or miss) or the MegaCLI tool (daunting, but
 there are cheat sheets). You could also add a separate array to act as a
 dedicated mail spool. You can also replace the existing disks with
 faster (and/or larger) disks for additional performance without ever
 touching the software.


 To directly answer your question of If I can move the mail queue
 workload into a hill instead of a sharp peak on the cacti load graph, it
 would be good. , then lowering the LMTP limit in cyrus (or the upstream
 MX server) to turn the mail flow into a trickle, rather than a flood,
 would do this. You can adjust the concurrency rate of LMTP deliveries in
 postfix using lmtp_destination_concurrency_limit (default 20).  The
 cyrus method has already been mentioned. You may also look at other ways
 to reduce IO wait, such as disk defragmentation or utilizing hard links
 in cyrus (singleinstancestore: 1).

Another thing is to check partitioning here. Using separate spindles for
/var/lib/imap seems a good idea, RAID1 on two small but fast disks has
always worked fine for me.

Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: cyrus-imap configuration question SOLVED

[Simon Matter]

 Thanks All,

 Well yes, ahem.. obviously!  Since I'm a complete beginner at this, who,
 what
 and how did the ownership of mailboxes.db get set wrongly in the first
 place?

 The script /usr/lib/cyrus-imapd/mkimap was run as user cyrus, as per the
 instructions.

I think you should follow the instructions mentioned in the Fedora rpm
docs and not the generic instructions of the cyrus docs. IMHO things like
mkimap should not be needed with a packaged version of cyrus (but of
course it should not break things). If it still fails after following the
package docs you may report this to Fedora as the package could have a bug
then.

Regards,
Simon


 Surely cyrus CANNOT create a root owned file ??

 Observation:

 Just as a quick experiment I changed mailboxes.db permissions to 666 and
 low
 it works! But something knows and changed the ownership to cyrus:mail with
 permissions 600

 Something very fishy there.

  ~ o ~

 While on the subject of incorrect permissions. The file
 /var/lib/imap/db/skipstamp was also root:root and causing this from
 systemctl
 status cyrus-imapd after a SUCCESSFUL start.

 Jan 20 12:15:44 blanked.com ctl_cyrusdb[1652]: DBERROR: writing
 /var/lib/imap/db/skipstamp: Permission denied

 I have know idea why these Permission denied messages are not in
 /var/log/messages, perhaps it has something to do with SELinux !!

 Thanks for all the help, Charles Bradshaw

 On: Sun, 20 Jan 2013 07:20:18 -0400, Patrick wrote:
 On 01/20/2013 06:40 AM, Charles Bradshaw wrote:
  Thanks Dan, Here are the details you asked for.
 
  My /etc/imapd.conf :
 
  configdirectory: /var/lib/imap
  partition-default: /var/spool/imap
  admins: cyrus
  sievedir: /var/lib/imap/sieve
  sendmail: /usr/sbin/sendmail
  hashimapspool: true
  sasl_pwcheck_method: auxprop
  sasl_mech_list: PLAIN DIGEST-MD5 CRAM-MD5
  sasl_auxprop_plugin:sasldb
  #allowplaintext: no
  #defaultdomain: mail
  tls_cert_file: /etc/pki/cyrus-imapd/cyrus-imapd.pem
  tls_key_file: /etc/pki/cyrus-imapd/cyrus-imapd.pem
  tls_ca_file: /etc/pki/tls/certs/ca-bundle.crt
  # uncomment this if you're operating in a DSCP environment (RFC-4594)
  # qosmarking: af13
 
  I have played with allowplaintext and defaultdomain.
  For the record the 3 sasl_... values are the same as the entries in
  /etc/sasl2/Sendmail.conf
 
  Directory ownership:
 
  ls -ld /var/lib/imap
  drwxr-x---. 18 cyrus mail 4096 Jan 19 19:37 lib/imap
 
  ls -l lib/imap
  total 72
  drwx--. 2 cyrus mail 4096 Jan 20 03:35 backup
  drwx--. 2 cyrus mail 4096 Jan 19 19:37 db
  drwx--. 2 cyrus mail 4096 Jan 19 19:37 db.backup1
  drwx--. 2 cyrus mail 4096 Jan 19 16:45 db.backup2
  -rw---. 1 cyrus mail  144 Jan 19 12:20 deliver.db
  drwx--. 2 cyrus mail 4096 Mar 15  2012 log
  -rw---. 1 root  root  144 Jan 14 07:13 mailboxes.db
  drwx--. 2 cyrus mail 4096 Mar 15  2012 md5
  drwx--. 2 cyrus mail 4096 Mar 15  2012 meta
  drwx--. 2 cyrus mail 4096 Mar 15  2012 msg
  drwx--. 2 cyrus mail 4096 Mar 15  2012 proc
  drwx--. 2 cyrus mail 4096 Mar 15  2012 ptclient
  drwx--. 2 cyrus mail 4096 Mar 15  2012 quota
  drwx--. 2 cyrus mail 4096 Jan 19 15:29 rpm
  drwx--. 2 cyrus mail 4096 Mar 15  2012 sieve
  drwxr-x---. 2 cyrus mail 4096 Jan 19 19:37 socket
  drwx--. 2 cyrus mail 4096 Mar 15  2012 sync
  drwx--. 2 cyrus mail 4096 Mar 15  2012 user
 
  ls -ld /var/spool/imap
  drwxr-x---. 4 cyrus mail 4096 Jan 19 11:39 /var/spool/imap
 
  ls -l /var/spool/imap
  total 8
  drwxr-xr-x. 2 cyrus mail 4096 Jan 19 11:39 stage.
  drwxr-xr-x. 2 cyrus mail 4096 Jan 19 11:39 sync.
 
  stage. and sync. are empty
 
  The only possibility I see above is mailboxes.db is root:root  IS
 THIS THE
  PROBLEM?

 mailboxes.db should be owned by cyrus user.

 
  I assume this was created during:
  [root@dell2600 ~]# su cyrus
  bash-4.2$ /usr/lib/cyrus-imapd/mkimap
 
  Charles Bradshaw
 
 
  -- Original Message ---
 
  On: Sat, 19 Jan 2013 18:46:38 -0600, Dan wrote
 
  On 01/19/13 17:51 +, Charles Bradshaw wrote:
  I'm tying to configure cyrus-imap on a Fedora 17 system.
 
  cyrus-imapd version cyrus-imapd.i686 2.4.14-1.fc17
 
  I have sendmail and saslauthd working using DIGEST-MD5 and CRAM-MD5
 working.
 
  I have gone through the cyrus-imap configuration procedure, but when
 I
 try to
  start the server:
 
  # systemctl start cyrus-imapd.service
  Job failed. See system journal and 'systemctl status' for details.
 
  # systemctl status cyrus-imapd.service
  cyrus-imapd.service - Cyrus-imapd IMAP/POP3 email server
 Loaded: loaded (/usr/lib/systemd/system/cyrus-imapd.service;
 disabled)
 Active: failed (Result: exit-code) since Sat, 19 Jan 2013
 13:29:32 +;
  28s ago
Process: 2049 ExecStartPre=/usr/lib/cyrus-imapd/cyr_systemd_helper
 start
  (code=exited, status=75)
 CGroup: name=systemd:/system/cyrus-imapd.service
 
  If I start the master process manually or in debug mode:
  # /usr/lib/cyrus-imapd/cyrus-master -D  

Re: Stripping of attachments using Horde 4/IMP 5.

[Simon Matter]

 - Message from awill...@whitemice.org -
  Date: Sun, 30 Dec 2012 18:56:36 -0500
  From: Adam Tauno Williams awill...@whitemice.org
 Reply-To: awill...@whitemice.org
   Subject: Re: Stripping of attachments using Horde 4/IMP 5.
To: info-cyrus@lists.andrew.cmu.edu


 On Sun, 2012-12-30 at 12:23 +0530, an...@isac.gov.in wrote:
 Dear experts,
 I had extensive discussion on this issue in Horde/IMP mailing list.
 Later I reported the problem in RedHat Bugzilla.  The details of the
 problem are also part of bugzilla.
 https://bugzilla.redhat.com/show_bug.cgi?id=885620
 Though patch for cyrus-imapd was given to me for testing, I was
 suppose to reproduce the problem in another server.  I am unable to
 reproduce the problem.  Now, I have a feeling that, the problem may be
 due to /var/lib/imap residing in NetApp storage (though not NFS).
 I have a feeling, the problem may solve, if I make /var/lib/imap part
 of OS disk.  Your opinion please...

 Are you using Berkley databases?  If it does seg-fault what does the
 backtrace look like? [I don't see a backtrace in the bug report, just a
 core file; but the core isn't really useful unless one has the same
 version of the software].

 --
 Adam Tauno Williams  GPG D95ED383
 Systems Administrator, Python Developer, LPI / NCLA

 
 Cyrus Home Page: http://www.cyrusimap.org/
 List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
 To Unsubscribe:
 https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus


 The databases used are:

 -rw--- 1 cyrus mail 144 Dec  9 09:29 ./annotations.db
 ./annotations.db: Cyrus skiplist DB

 -rw--- 1 cyrus mail 144 Dec 30 11:30 ./db.backup1/annotations.db
 ./db.backup1/annotations.db: Cyrus skiplist DB

 -rw--- 1 cyrus mail 1882668 Dec 30 11:30 ./db.backup1/mailboxes.db
 ./db.backup1/mailboxes.db: Cyrus skiplist DB

 -rw--- 1 cyrus mail 144 Dec 30 11:00 ./db.backup2/annotations.db
 ./db.backup2/annotations.db: Cyrus skiplist DB

 -rw--- 1 cyrus mail 1882668 Dec 30 11:00 ./db.backup2/mailboxes.db
 ./db.backup2/mailboxes.db: Cyrus skiplist DB

 -rw--- 1 cyrus mail 18038784 Dec 30 11:30 ./deliver.db
 ./deliver.db: Berkeley DB (Btree, version 9, native byte-order)

 -rw--- 1 cyrus mail 1882668 Dec 29 14:33 ./mailboxes.db
 ./mailboxes.db: Cyrus skiplist DB


 I don't have backtrace?  I am using RHEL rpms.

Unfortunately we can also not see what has been changed in the patched
RPMs you got from RH.


 But, is it OK to use NetApp Storage for /var/lib/imap file system?

NetApp often sounds like NFS but you told us you are not using NFS but FC
attached disks? If so I don't know why it shouldn't work exactly as local
disks would.

But when I hear Horde/IMP I remember a problem that some people hit after
upgrading Horde/IMP. I don't remember what it was but you should find it
in the archives. IIRC it has been fixed in the latest version of
cyrus-imapd.

Regards,
Simon

Regards,
Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Stripping of attachments using Horde 4/IMP 5.

[Simon Matter]

 - Message from awill...@whitemice.org -
  Date: Sun, 30 Dec 2012 18:56:36 -0500
  From: Adam Tauno Williams awill...@whitemice.org
 Reply-To: awill...@whitemice.org
   Subject: Re: Stripping of attachments using Horde 4/IMP 5.
To: info-cyrus@lists.andrew.cmu.edu


 On Sun, 2012-12-30 at 12:23 +0530, an...@isac.gov.in wrote:
 Dear experts,
 I had extensive discussion on this issue in Horde/IMP mailing list.
 Later I reported the problem in RedHat Bugzilla.  The details of the
 problem are also part of bugzilla.
 https://bugzilla.redhat.com/show_bug.cgi?id=885620
 Though patch for cyrus-imapd was given to me for testing, I was
 suppose to reproduce the problem in another server.  I am unable to
 reproduce the problem.  Now, I have a feeling that, the problem may be
 due to /var/lib/imap residing in NetApp storage (though not NFS).
 I have a feeling, the problem may solve, if I make /var/lib/imap part
 of OS disk.  Your opinion please...

 Are you using Berkley databases?  If it does seg-fault what does the
 backtrace look like? [I don't see a backtrace in the bug report, just a
 core file; but the core isn't really useful unless one has the same
 version of the software].

 --
 Adam Tauno Williams  GPG D95ED383
 Systems Administrator, Python Developer, LPI / NCLA

 
 Cyrus Home Page: http://www.cyrusimap.org/
 List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
 To Unsubscribe:
 https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus


 The databases used are:

 -rw--- 1 cyrus mail 144 Dec  9 09:29 ./annotations.db
 ./annotations.db: Cyrus skiplist DB

 -rw--- 1 cyrus mail 144 Dec 30 11:30 ./db.backup1/annotations.db
 ./db.backup1/annotations.db: Cyrus skiplist DB

 -rw--- 1 cyrus mail 1882668 Dec 30 11:30 ./db.backup1/mailboxes.db
 ./db.backup1/mailboxes.db: Cyrus skiplist DB

 -rw--- 1 cyrus mail 144 Dec 30 11:00 ./db.backup2/annotations.db
 ./db.backup2/annotations.db: Cyrus skiplist DB

 -rw--- 1 cyrus mail 1882668 Dec 30 11:00 ./db.backup2/mailboxes.db
 ./db.backup2/mailboxes.db: Cyrus skiplist DB

 -rw--- 1 cyrus mail 18038784 Dec 30 11:30 ./deliver.db
 ./deliver.db: Berkeley DB (Btree, version 9, native byte-order)

 -rw--- 1 cyrus mail 1882668 Dec 29 14:33 ./mailboxes.db
 ./mailboxes.db: Cyrus skiplist DB


 I don't have backtrace?  I am using RHEL rpms.

 Unfortunately we can also not see what has been changed in the patched
 RPMs you got from RH.


 But, is it OK to use NetApp Storage for /var/lib/imap file system?

 NetApp often sounds like NFS but you told us you are not using NFS but FC
 attached disks? If so I don't know why it shouldn't work exactly as local
 disks would.

 But when I hear Horde/IMP I remember a problem that some people hit after
 upgrading Horde/IMP. I don't remember what it was but you should find it
 in the archives. IIRC it has been fixed in the latest version of
 cyrus-imapd.

Sorry forgot to mention, if you make any tests on a test system make sure
it's the same arch as the production system. I mean, don't test on a i386
box if production is running x86_64 or so.

Regards,
Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Making sub-folder email reappear after recovery from backup

[Simon Matter]

 On 11/2/2012 12:09 PM, Adam Tauno Williams wrote:
 On Fri, 2012-11-02 at 11:16 -0400, Gordon Marler wrote:
 Can't find this mentioned in the docs or with any search I've done on
 the mailing list, so here goes:
 - Upgraded from 2.4.10 to 2.4.16 after losing my 2.4.10 system, but
 having mail spool backed up
 - After moving to the 2.4.16 system, able to see Inbox for accounts
 - Had to make sub-folders reappear with command like (Thank you mailing
 list for that tidbit!):
 reconstruct -p default -rf user.gmarler.Subfolder
 - Now I can see the Inbox, Subfolders, Sub-subfolders, etc in
 Thunderbird
 BUT - Only Inbox has visible emails - All of the subfolders appear to
 be empty, but there are definitely mails in them on the mail spool.
 Looks like I forgot a step somewhere.  Where have I gone wrong?
 reconstruct, with the -r [recursive] and -f [examine filesystem for
 mailboxes] options.
 Ah, more involved than that - the crash on 2.4.10 seems to have left the
 spool in an odd state, like it was completely reorganizing the spool and
 got interrupted:
 - $SPOOL is located in /opt/csw/var/cyrus/mail/
 - Original emails were left in $SPOOL/.u/g/user/gmarler/
 - A partial copy (subfolders all empty) was here:
$SPOOL/g/user/gmarler/ (I wasn't looking here)
 - Another complete copy was here:
$SPOOL/user/gmarler/ (This is what I was looking at, but Cyrus wasn't)

 - Merged everything from $SPOOL/.u/g/user/gmarler/ and
 $SPOOL/user/gmarler/ into $SPOOL/g/user/gmarler/
 - Ran reconstruct -p default -rf user.gmarler
 - Started cyrus and started getting these errors:
DBERROR db4: PANIC: fatal region error detected; run recovery
 - Killed off the master and cleaned up the DB4 *.db databases (except
 for db/)
 - Ran reconstruct -rf user.gmarler
 - Restarted master

 Now everything from 1995 shows up.  Thanks for that extra tip!

I'm not sure but did you look at the hashimapspool and fulldirhash
options (and defaults)? Can it be that the new server didn't have the same
configs?

Simon


 GM

 
 Cyrus Home Page: http://www.cyrusimap.org/
 List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
 To Unsubscribe:
 https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

 
 Cyrus Home Page: http://www.cyrusimap.org/
 List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
 To Unsubscribe:
 https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus




Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Another 2.4 upgrade horror story

[Simon Matter]

 On 2012-09-25 19:05, Simon Beale wrote:
 The only gotcha I experienced was I forgot that cyrus was configured to
 hardlink mail, which of course was no longer the case after each mailbox
 was migrated, so my disk usage exploded. (But easily fixed/restored once
 identified).

 What did you use for restoring the hardlinks? freedup as well?

 I'm asking because I found a bug in freedup causing dataloss. I already
 sent a patch fixing it to the author of freedup last november, but he
 didn't release a new version yet.

 In case cyr_expire is running while freedup tries to hardlink files it is
 possible to loose both the source freedup wants to link to and the copy
 freedup still removes on error. Running cyr_expire and freedup (up to
 1.6-2) together is a really bad idea.

 If it's of interest I can provide my patch here, too.

Hi,

I have not used freedup for restoring the hardlinks but I'm interested in
the patch. If it's not big could you post it here?

Thanks,
Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: IMAP error reported by server. Invalid body section.

[Simon Matter]

 On Jun 22, 2012, at 5:10 PM, Simon Matter simon.mat...@invoca.ch wrote:

 On 06/22/2012 09:43 AM, Dave McMurtrie wrote:
 On 06/22/2012 06:35 AM, Adam Tauno Williams wrote:
 On Thu, 2012-06-21 at 13:07 -0300, Rodrigo Abantes Antunes wrote:
 The source from horde3 is exactly the same as horde4

 That is expected.  It isn't the message but the interpretation of the
 message.  These evil messages contain many named parts separated by a
 boundry (the boundry value is declared in the header of the message).
 Then parts of a message can refer to other parts of the message.  So
 either H4 can't correctly [or incorrectly!] parse the message into
 parts
 by boundry or one part references another part that isn't found.

 It would be useful to ask this question on the Horde / IMP mail list.

 I think this originated as a bug report to Horde and they think it's
 the
 IMAP server's fault.

 Rodrigo, can you forward the message to me?

 Hi.  Rodrigo sent me the message.  I wanted to confirm that the MIME
 structure was correct so I used munpack which was able to successfully
 unpack all the message parts.  This isn't a guarantee that the MIME
 structure is correct, but at the very least I can't definitely say the
 message is malformed.

 I then imported the message into my mailstore.  reconstruct was not
 pleased with it from the start:

 Jun 22 15:29:48 cyrusbe-d04 reconstruct[28021]: ERROR: message has more
 than 1000 header lines, not caching any more

 I did the same test on my box and reconstruct worked fine and I can view
 the message with Squirrelmail and Thunderbird without any problems.

 What's your version of cyrus-imapd you tested with? I have tested with a
 2.4.16 server.


 Interesting.  The server I'm testing on isn't a released version, but
 rather a snapshot build from the caldav-2.4 Git branch.  It should be

Without looking at it closely, I have two things in my setup which could
make the difference?

1) I have set lmtp_strict_rfc2821: 0

2) I have this patch:
--- cyrus-imapd-2.3.7/imap/message.c2006-10-28 22:18:08.0 +0200
+++ cyrus-imapd-2.3.7/imap/message.c.nobarenewlinescheck2006-10-28
22:21:55.0 +0200
@@ -256,8 +256,9 @@
r = IMAP_MESSAGE_CONTAINSNULL;
}
else if (*p == '\n') {
-   if (!sawcr  (inheader || !allow_null))
-   r = IMAP_MESSAGE_CONTAINSNL;
+   /* Do *NOT* check for RFC compliant line breaks (bare
newlines) */
+   /* if (!sawcr  (inheader || !allow_null))
+   r = IMAP_MESSAGE_CONTAINSNL; */
sawcr = 0;
if (blankline) {
inheader = 0;


Regards,
Simon

 fairly close to 2.4.16.  Can you grab telemetry and see what
 Squirrelmail/tbird is requesting?



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: IMAP error reported by server. Invalid body section.

[Simon Matter]

 On 06/22/2012 09:43 AM, Dave McMurtrie wrote:
 On 06/22/2012 06:35 AM, Adam Tauno Williams wrote:
 On Thu, 2012-06-21 at 13:07 -0300, Rodrigo Abantes Antunes wrote:
 The source from horde3 is exactly the same as horde4

 That is expected.  It isn't the message but the interpretation of the
 message.  These evil messages contain many named parts separated by a
 boundry (the boundry value is declared in the header of the message).
 Then parts of a message can refer to other parts of the message.  So
 either H4 can't correctly [or incorrectly!] parse the message into
 parts
 by boundry or one part references another part that isn't found.

 It would be useful to ask this question on the Horde / IMP mail list.

 I think this originated as a bug report to Horde and they think it's the
 IMAP server's fault.

 Rodrigo, can you forward the message to me?

 Hi.  Rodrigo sent me the message.  I wanted to confirm that the MIME
 structure was correct so I used munpack which was able to successfully
 unpack all the message parts.  This isn't a guarantee that the MIME
 structure is correct, but at the very least I can't definitely say the
 message is malformed.

 I then imported the message into my mailstore.  reconstruct was not
 pleased with it from the start:

 Jun 22 15:29:48 cyrusbe-d04 reconstruct[28021]: ERROR: message has more
 than 1000 header lines, not caching any more

I did the same test on my box and reconstruct worked fine and I can view
the message with Squirrelmail and Thunderbird without any problems.

What's your version of cyrus-imapd you tested with? I have tested with a
2.4.16 server.

Regards,
Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: mailboxes.db vs IMAP client irregularities

[Simon Matter]

 On 05/19/2012 01:51 PM, Stephen Ingram wrote:
 I'm running 2.4.13 from the invoca rpms on CentOS 5.8. I recently had
 an issue with a folder in a mailbox that would not show any
 subfolders. I created a new folder 'folder2' and moved all of the
 subfolders to it and then performed a reconstruct on the new set of
 folders and everything worked. Now I deleted the old folder 'folder'
 from the file system and then (after it wouldn't go away from the
 cyradm listing) used cyr_dbtool to manually remove it (and the
 subfolders) from the mailboxes.db file. The old folders and subfolders
 are now gone, however, I can't (using the IMAP client) rename
 'folder2' back to 'folder' as when I do, the subfolders are not
 visible.

 I've dumped the mailboxes.db file to a flat file to look and see if
 there is anything in there that wasn't visible in cyradm or using
 cyr_dbtool show. Everything is as expected except there are some
 DELETED.user.xxx.folder entries at the top. Are you not allowed the
 create folders with the same name you've just deleted? Where are these
 DELETED folders actually stored and how long does it take them to go
 away? (I'm not using delayed expunge.)

 Sounds like you are using delayed delete. Mine show up in
 /imap/mail/C/DELETED/  . How long they stay around depends on when you
 run cyr_expire and what parameters you give it.


 Man page entries:

 deletedprefix: DELETED
  If  delete_mode  set  to  be  delayed,  the  prefix for
 the deleted mailboxes hierarchy.  The hierarchy delimiter will be
 automatically appended.

 delete_mode: immediate
  The manner in which mailboxes are deleted. immediate mode
 is the default  behavior  in  which  mailboxes  are  removed
 immediately.   In delayed mode, mailboxes are renamed to a special
 hiearchy defined by the deletedprefix option to be removed later by
 cyr_expire.

Just to clear this up, if he's using our invoca rpms then his man page reads:

   delete_mode: delayed
The manner in which mailboxes are deleted. immediate mode is
the
the mode in which mailboxes are removed immediately.  In
delayed
mode, mailboxes are renamed to a special hiearchy defined  by 
the
deletedprefix option to be removed later by cyr_expire.

   Note:  This Invoca RPM build uses delayed by default instead of
immedi-
   ate for delete_mode.

   Allowed values: immediate, delayed

Regards,
Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: (important) cyrus-imapd 2.4.16 released

[Simon Matter]

 On Thu, 2012-04-19 at 11:00 +0100, Jeroen van Meeuwen wrote:
 Hi there,

 I'm forwarding this message posted to the announcement mailing list
 originally, to let you know any upgrades should target 2.4.16 as opposed
 to
 2.4.15.

 We are pleased to announce the release of Cyrus IMAPd 2.4.16.
 [1] https://bugzilla.cyrusimap.org/show_bug.cgi?id=3651

 The bug contains the comment:
 Can we please confirm/deny this only breaks systems with fulldirhash: 1
 configured?

 Is the answer to that question Yes;  sites that do not specify
 fulldirhash or have a fulldirhash: 0 in their imapd.conf are not
 affected and do not need to rehash.

I think that's correct. At least we have hashimapspool: true as default in
our RPMs and fulldirhash is disabled by default and there are no problems
AFAIK.

Regards,
Simon



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: Lock problem with mupdate

[Simon Matter]

 On Tue, 17 Apr 2012 05:43:41 -0400 Dave McMurtrie wrote:
 strace the server and find out what it's doing.  Look at the logs on the
 server.  Run netstat to confirm both sides still have an established
 connection and that iptables isn't silently dropping packets, etc.

 No iptables active, due to private net.

SELinux? Same answer as above I guess?

Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

RE: RPMs for cyrus-imapd v2.4.13?

[Simon Matter]

 These are source RPMs. Are there any binary RPMs available?  I'm kinda new
 at Linux and don't know how to install a source RPM.  (Also, will it use
 the same directory structure as the binary RPMs?)

Hi Larry,

We do have binary RPMs for RHEL3, 4, 5 and 6 for ix86 and x86_64 in our
YUM repositories which are only accessible for our paying customers.

But, if you're interested in a current set of binary packages just let me
know and I'll send them to you in private mail.

And yes, the directory structure should still be compatible, as the
packages provided by RedHat, Fedora or Kolab Systems are based on a old
version of our RPMs - despite the fact that in certain RPMs the changelogs
have been cleaned up :)

Regards,
Simon


 Thanks, Larry

 From: Chris Conn [mailto:cc...@abacom.com]
 Sent: Thursday, January 19, 2012 12:03 PM
 To: Rosenbaum, Larry M.
 Cc: info-cyrus@lists.andrew.cmu.edu
 Subject: Re: RPMs for cyrus-imapd v2.4.13?

 On 2012-01-19 11:40, Rosenbaum, Larry M. wrote:
 Are there any RPMs available yet for cyrus-imapd v2.4.13?  We're running
 RHEL6.

 Larry M. Rosenbaum
 Oak Ridge National Laboratory

 http://www.invoca.ch/pub/packages/cyrus-imapd/

 I built them on a Centos 6.2 server and they worked well in my setup.

 Chris







 

 Cyrus Home Page: http://www.cyrusimap.org/

 List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

 
 Cyrus Home Page: http://www.cyrusimap.org/
 List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: anysievefolder and autosievefolders gone in 2.4?

[Simon Matter]

 Hello,

 Sorry - I've been planning to do it for ages, and it just hasn't
 happened
 due to other things always being more pressing.  It's on the MUST HAVE
 list for 2.5.

 ok, but where can I find a patch for the current 2.4.9 release of Ubuntu
 11.10? The latest release I can find at
 http://email.uoa.gr/projects/cyrus/autosievefolder/ is from 2009 for the
 2.3.16 release.

Check here:
http://www.vx.sk/download/patches/cyrus-imapd/

If they don't apply, then you can extract the version in our current rpm:
http://www.invoca.ch/pub/packages/cyrus-imapd/RPMS/ils-5/SRPMS/cyrus-imapd-2.4.13-1.el5.src.rpm

I think I had to rework them.

Simon


 Kind regards
 Marten
 
 Cyrus Home Page: http://www.cyrusimap.org/
 List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/




Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: High cpu usage for imapd processes on 2.4.12 (Invoca rpm)

[Simon Matter]

 Hi,

 I've got an imap server being hammered by a program that is constantly
 connecting via IMAP, searching, reading and deleting emails. At a very
 rough estimate there'll be 10 to 20 new emails every minute that
 arrive, are checked and deleted.

 The original machine is running 2.3.16 without replication and the
 average load rarely went over 1. On the new server running 2.4.12
 (details below) I'm seeing the average load hover around 5 constantly.
 The new box is running replication and this appears to be set up
 correctly as the logs look pretty clean. top shows CPU percentage for
 each imapd process running quite high. The largest mailbox contains
 about 3000 emails and is usually at 50 to 60% CPU.

 I/O for the machine looks fine, I'm not seeing any iowait or high
 usage with iostat. It's using local disks so not something I would
 expect to be a problem.

 Can anyone suggest where I could start looking to find the cause? Or
 point me in the direction of configuration options that are known to
 have such an effect after upgrading between the two version?

 Thanks
 Guy


 New server:
 name   : Cyrus IMAPD
 version: v2.4.12-Invoca-RPM-2.4.12-2el5_2 63b65d1c 2011-10-04
 vendor : Project Cyrus
 support-url: http://www.cyrusimap.org
 os : Linux
 os-version : 2.6.18-238.el5
 environment: Built w/Cyrus SASL 2.1.22
  Running w/Cyrus SASL 2.1.22
  Built w/OpenSSL 0.9.8e-fips-rhel5 01 Jul 2008
  Running w/OpenSSL 0.9.8e-fips-rhel5 01 Jul 2008
  Built w/zlib 1.2.3
  Running w/zlib 1.2.3
  CMU Sieve 2.4
  TCP Wrappers
  mmap = shared
  lock = fcntl
  nonblock = fcntl
  idle = idled

 imapd.conf:
 admins: cyradmin cyradmin-repl
 configdirectory: /var/lib/imap
 delete_mode: immediate
 duplicate_db: skiplist
 expunge_mode: default
 hashimapspool: 1
 improved_mboxlist_sort: 1
 lmtp_downcase_rcpt: 1
 munge8bit: 0
 partition-default: /var/spool/imap
 ptscache_db: skiplist
 sasl_pwcheck_method: saslauthd
 sendmail: /usr/sbin/sendmail
 sievedir: /var/lib/imap/sieve
 statuscache_db: skiplist
 sync_authname: cyradmin-repl
 server101_sync_host: 10.10.10.10
 server201_sync_host: 10.11.10.11
 sync_log: 1
 sync_log_channels: server101 server201
 sync_password: replpass
 tls_ca_file: /etc/ssl/certs/domain.com.ca-bundle
 tlscache_db: skiplist
 tls_cert_file: /etc/ssl/certs/domain.com.crt
 tls_key_file: /etc/ssl/certs/domain.com.key
 unix_group_enable: 0
 normalizeuid: 1

Hi,

I've not seen any such issues on my 2.4.x systems after upgrading from
2.3.x and I'm also wondering what's going on.
So, is it the same hardware, os and everything and only cyrus changed? And
then of course I don't know about replication how it affects it.

What comes to mind is, did you try with expunge_mode: delayed? That can
make quite a difference IIRC.

Regards,
Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: imapd 2.4.12 hangs

[Simon Matter]

 I see, and I remember reading about it.
 But, what will happen if I make install a skiplist version onto an older
 BDB deployment
 with running users? Will I need to run a reconstruct? Will I need to get
 rid of older bdb files
 manually? Also, do you think I will find any benefits by using skiplist on
 NFS instead of BDB?

Sorry, I can't help you here as we only do packaging on build hosts and
then install resulting RPMs on production systems.
What I'm sure is that you have to convert away from BDB before installing
new binaries, because they won't do anything with any BDB file.
I have never used NFS in this case.

Simon


 Inviato da iPad

 Il giorno 21/ott/2011, alle ore 14:07, Simon Matter
 simon.mat...@invoca.ch ha scritto:

 Well, I've been using BDB for years on Cyrus.
 Now I'm rebuilding a new distribution with the latest Cyrus, and I just
 went on with the same
 scheme.
 How do I get rid of BDB? What will it use instead?

 skiplist, most databases default to skiplist anyway. Those which still
 default to BDB can be configured to use skiplist instead. Works fine for
 me and many others.

 Of course, I don't say your problems have to do with BDB, it's just
 possible.

 Simon

 Gabriele.
 --
 Da: Simon Matter
 A: Gabriele Bulfon
 Cc: info-cyrus@lists.andrew.cmu.edu
 Data: 21 ottobre 2011 12.17.33 CEST
 Oggetto: Re: imapd 2.4.12 hangs
 Hi, I'm still having trouble running 2.4.12, once started, if I connect
 to
 port 143, I never
 get the hello response, and the launched imapd goes in a loop.
 These is the output of ldd against master, maybe some libraries are bad
 versions?
 Hi,
 Do you build yourself? If so, do you really need BDB? I not, then you
 may
 want to build without BDB just to make sure to get rid of a possible
 source of problems.
 Regards,
 Simon
 linux-gate.so.1 =(0x009ff000)
 libgssapi_krb5.so.2 =/sonicle/lib/libgssapi_krb5.so.2 (0x00a81000)
 libkrb5.so.3 =/sonicle/lib/libkrb5.so.3 (0x00923000)
 libk5crypto.so.3 =/sonicle/lib/libk5crypto.so.3 (0x00b76000)
 libcom_err.so.3 =/sonicle/lib/libcom_err.so.3 (0x004a3000)
 libkrb5support.so.0 =/sonicle/lib/libkrb5support.so.0 (0x00cc8000)
 libresolv.so.2 =/lib/libresolv.so.2 (0x00a2e000)
 libssl.so.0.9.8 =/sonicle/lib/libssl.so.0.9.8 (0x0030)
 libcrypto.so.0.9.8 =/sonicle/lib/libcrypto.so.0.9.8 (0x0011)
 libdb-4.8.so =/sonicle/lib/libdb-4.8.so (0x004a6000)
 libpcre.so.0 =/sonicle/lib/libpcre.so.0 (0x00257000)
 libpcreposix.so.0 =/sonicle/lib/libpcreposix.so.0 (0x00cdf000)
 libz.so.1 =/sonicle/lib/libz.so.1 (0x00277000)
 libc.so.6 =/lib/libc.so.6 (0x00655000)
 libdl.so.2 =/lib/libdl.so.2 (0x0028f000)
 libpthread.so.0 =/lib/libpthread.so.0 (0x00294000)
 /lib/ld-linux.so.2 (0x00421000)
 
 Cyrus Home Page: http://www.cyrusimap.org/
 List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/








Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: imapd 2.4.12 hangs

[Simon Matter]

 Hi, I'm still having trouble running 2.4.12, once started, if I connect to
 port 143, I never
 get the hello response, and the launched imapd goes in a loop.
 These is the output of ldd against master, maybe some libraries are bad
 versions?

Hi,

Do you build yourself? If so, do you really need BDB? I not, then you may
want to build without BDB just to make sure to get rid of a possible
source of problems.

Regards,
Simon

 linux-gate.so.1 =(0x009ff000)
 libgssapi_krb5.so.2 =/sonicle/lib/libgssapi_krb5.so.2 (0x00a81000)
 libkrb5.so.3 =/sonicle/lib/libkrb5.so.3 (0x00923000)
 libk5crypto.so.3 =/sonicle/lib/libk5crypto.so.3 (0x00b76000)
 libcom_err.so.3 =/sonicle/lib/libcom_err.so.3 (0x004a3000)
 libkrb5support.so.0 =/sonicle/lib/libkrb5support.so.0 (0x00cc8000)
 libresolv.so.2 =/lib/libresolv.so.2 (0x00a2e000)
 libssl.so.0.9.8 =/sonicle/lib/libssl.so.0.9.8 (0x0030)
 libcrypto.so.0.9.8 =/sonicle/lib/libcrypto.so.0.9.8 (0x0011)
 libdb-4.8.so =/sonicle/lib/libdb-4.8.so (0x004a6000)
 libpcre.so.0 =/sonicle/lib/libpcre.so.0 (0x00257000)
 libpcreposix.so.0 =/sonicle/lib/libpcreposix.so.0 (0x00cdf000)
 libz.so.1 =/sonicle/lib/libz.so.1 (0x00277000)
 libc.so.6 =/lib/libc.so.6 (0x00655000)
 libdl.so.2 =/lib/libdl.so.2 (0x0028f000)
 libpthread.so.0 =/lib/libpthread.so.0 (0x00294000)
 /lib/ld-linux.so.2 (0x00421000)
 
 Cyrus Home Page: http://www.cyrusimap.org/
 List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: imapd 2.4.12 hangs

[Simon Matter]

 Well, I've been using BDB for years on Cyrus.
 Now I'm rebuilding a new distribution with the latest Cyrus, and I just
 went on with the same
 scheme.
 How do I get rid of BDB? What will it use instead?

skiplist, most databases default to skiplist anyway. Those which still
default to BDB can be configured to use skiplist instead. Works fine for
me and many others.

Of course, I don't say your problems have to do with BDB, it's just possible.

Simon

 Gabriele.
 --
 Da: Simon Matter
 A: Gabriele Bulfon
 Cc: info-cyrus@lists.andrew.cmu.edu
 Data: 21 ottobre 2011 12.17.33 CEST
 Oggetto: Re: imapd 2.4.12 hangs
 Hi, I'm still having trouble running 2.4.12, once started, if I connect to
 port 143, I never
 get the hello response, and the launched imapd goes in a loop.
 These is the output of ldd against master, maybe some libraries are bad
 versions?
 Hi,
 Do you build yourself? If so, do you really need BDB? I not, then you may
 want to build without BDB just to make sure to get rid of a possible
 source of problems.
 Regards,
 Simon
 linux-gate.so.1 =(0x009ff000)
 libgssapi_krb5.so.2 =/sonicle/lib/libgssapi_krb5.so.2 (0x00a81000)
 libkrb5.so.3 =/sonicle/lib/libkrb5.so.3 (0x00923000)
 libk5crypto.so.3 =/sonicle/lib/libk5crypto.so.3 (0x00b76000)
 libcom_err.so.3 =/sonicle/lib/libcom_err.so.3 (0x004a3000)
 libkrb5support.so.0 =/sonicle/lib/libkrb5support.so.0 (0x00cc8000)
 libresolv.so.2 =/lib/libresolv.so.2 (0x00a2e000)
 libssl.so.0.9.8 =/sonicle/lib/libssl.so.0.9.8 (0x0030)
 libcrypto.so.0.9.8 =/sonicle/lib/libcrypto.so.0.9.8 (0x0011)
 libdb-4.8.so =/sonicle/lib/libdb-4.8.so (0x004a6000)
 libpcre.so.0 =/sonicle/lib/libpcre.so.0 (0x00257000)
 libpcreposix.so.0 =/sonicle/lib/libpcreposix.so.0 (0x00cdf000)
 libz.so.1 =/sonicle/lib/libz.so.1 (0x00277000)
 libc.so.6 =/lib/libc.so.6 (0x00655000)
 libdl.so.2 =/lib/libdl.so.2 (0x0028f000)
 libpthread.so.0 =/lib/libpthread.so.0 (0x00294000)
 /lib/ld-linux.so.2 (0x00421000)
 
 Cyrus Home Page: http://www.cyrusimap.org/
 List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/




Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: singleinstancestore obsolete?

[Simon Matter]

 Ramprasad wrote, on 19.10.2011 15:37:
 I think , writing a standalone index upgrade utility , like the ipurge ,
 seems to be a reasonable thing to do


 If there was a light enough index upgrade possible ( only for inboxes ..
 not subfolders ) Then I could stop cyrus , fork probably around 100
 parallel upgrades take a 2-3 hour downtime and then start services again

 Currently I'm considering the following way:
 *) build a new backend with same partition count/names
 *) output something like
find ../partxx -type -f -links +2 -ls |sort to a file for every
 partition
on the old backend. -ls maybe replaced by special -printf.
 *) move mailboxes from 2.3 to 2.4 with rename keeping partitions the same
 *) write a script reading find-output and relink all the files with same
inodes. This can be done at any time and with low impact.

 I think that should be pretty safe if the script has enough safty belts in
 place. Mails moved or deleted in the meantime are a special case. Don't
 know
 if it's worth to try hard to find them.

I think for the singleinstancestore, you can redo it after migration with
tools like hardlink or http://www.freedup.org/. IIRC I did this once and
it worked fine - I think I was using a simple bash script as you suggested
above. The only problem could be that you need the extra space while
migration is going on.

Simon



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: Never-ending bug with recontruct - MISSING FOLDERS

[Simon Matter]

 Dear Jeroen,

 Le 14.09.2011 10:54, Jeroen van Meeuwen (Kolab Systems) a écrit :
 I think I've posted already here about this some years ago about this
 problem and I'm disappointed that cyrus reconstruct seems to still
 have
 the same bug.

 You wouldn't happen to have a ticket in bugzilla.cyrusimap.org about
 this problem you are experiencing, do you?

 No sorry, at that time I only posted here.

 Let me explain :
 I have to switch to a new server and moved the cyrus mailboxes on it.
 Most of the migration is working, but some folders are missing !!!

 Could you please state the version of Cyrus IMAP on both the old as
 well as the new server?

 New server : v2.3.7-Invoca-RPM-2.3.7-7.el5_6.4 2006/07/10 13:46:20
 Old server : don't know (but I don't think it plays any role because
 most of the data was recovered by reconstruct)

You missed the point, your new server is using a 5 year old version of
cyrus-imapd, which isn't exactly new. So, either try a current version,
which is 2.4.11, or ask your distributor to backport fixes if they exist.

Regards,
Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: Deliverdb in a memcached

[Simon Matter]

 On a very busy Imap server , duplicate suppression sometimes becomes the
 bottleneck
 I have seen that If I disable duplicate suppression , my lmtp deliveries
 are speeded up.

 Duplicate suppression is important , but the database need not persist
 for very long.
 I have seen in most of the cases if there is a duplicate mail ( due to
 forwards , groups etc ), it arrives within 10 minutes of the first mail
 ( Any exception to this is too minor and can be ignored )


 IMHO There should be a configuration that the deliverdb can be,
 optionally,  stored in memcached or directly in memory.
 Of course there are cons .. like loss of data on restart etc. But these
 are OK.

I think storing it in tmpfs would be what you want. The init script could
then save a permanent copy on shutdown and put it in place on startup.
That way you won't loose anything on normal restart procedure.

Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: autocreate / autosieve patches on 2.4.10

[Simon Matter]

 seems the patches at http://email.uoa.gr/projects/cyrus/ don't have
 anything for any 2.4.x version and looking at the man page for
 imapd.conf, there is only autocreatequota which I think has always been
 a base cyrus implementation and not part of the other patches.

 I have always been spoiled by Simon's packages but I have moved to
 Ubuntu server so I've been building from source but deeply miss the
 autocreate  autosieve patches.

 Is it just a choice of use 2.3.16 or live without?

Hi,

Did you try the patches from here
http://blog.vx.sk/index.php?archives/13-German.htmluser_language=en

Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: IOERROR: zero index/expunge record

[Simon Matter]

 Am 04.08.2011 10:06, schrieb Bron Gondwana:
 Pretty much - and the patch would be super-invasive even if someone
 did write it.

 I opened a bugreport at redhat:
 https://bugzilla.redhat.com/show_bug.cgi?id=728148
 Maybe they will fix it.


Hi,

The interesting point is why do you see those problems and others do not.
Since you are running a rebuild of stock RHEL6 cyrus-imapd package it
seems like others should have the same problem since it's out for quite
some time.


 Everything should migrate itself automatically. It will be VERY slow for
 a while while it basically does a reconstruct of each mailbox first time
 you open it to upgrade some of the datastructures.

 I'll setup a test system during the next days and try switching to 2.4.10.



 Just because of being curious: If I switch to the latest version now, is
 it
 possible to switch back to the distribution version, if Redhat fix it? Or
 is a
 downgrade not possible?

You can not directly go back because of the upgrade some of the
datastructures. While I can suggest running 2.4.10 I'm quite sure it
should also work fine with 2.3.16. So there may really be something wrong
with your system. I'd try to find out first what's going on.

I could send you my 2.4.10 rpms for EL6 (i686 or x86_64) for testing, but
I don't think it's a good idea to upgrade your production box if you may
want to go back to 2.3.16 later.

Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: IOERROR: zero index/expunge record

[Simon Matter]

 Am 03.08.2011 06:55, schrieb Simon Matter:
 Something is wrong with the database because Cyrus-imapd uses a number
 of databases. So I guess here is a problem.

 I removed cyrus.cache, cyrus.index and cyrus.header out of the users
 broken
 mailbox and run a reconstruct to recreate them. For some mailboxes this
 fixes
 the issue. But for some it comes back after hours or days.

 I also read that I should run
 db_recover -h /var/lib/imap/db
 while the cyrus services are stopped. But this doesn't help either.

 What other databases could be affected and how can I fix them?


Well, I don't know what configuration EL6 ships with. At least the
db_recover command above will only work on BDB but not on skiplist
databases. What does file /var/lib/imap/*.db show?

Simon


 Do you not see any other errors in the Cyrus logs?

 No. The
   imap[21918]: IOERROR: user.xxx zero index/expunge record 41/47
 errors are the only ones.




 Regards,
 Marc




Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: IOERROR: zero index/expunge record

[Simon Matter]

 Am 03.08.2011 09:47, schrieb Simon Matter:
 Well, I don't know what configuration EL6 ships with. At least the
 db_recover command above will only work on BDB but not on skiplist
 databases. What does file /var/lib/imap/*.db show?

 The configuration is the same that we used on Centos 5 before for years.


 # file /var/lib/imap/*.db
 /var/lib/imap/annotations.db:  Cyrus skiplist DB
 /var/lib/imap/deliver.db:  Berkeley DB (Btree, version 9, native
 byte-order)
 /var/lib/imap/mailboxes.db:Cyrus skiplist DB
 /var/lib/imap/tls_sessions.db: Berkeley DB (Btree, version 9, native
 byte-order)

So, you are still using Berkeley DB but the versions are different. How
did you upgrade those files? I suggest to switch to skiplist for all
databases. And make sure you clean up /var/lib/imap/db. BTW, deliver.db
and tls_sessions.db are not so important, you could configure skiplist and
just remove those dbs and also the content of /var/lib/imap/db.

Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: IOERROR: zero index/expunge record

[Simon Matter]

 Does anybody have an idea what cause this problem and how to finally fix
 it?



 Am 22.07.2011 10:19, schrieb Marc Muehlfeld:
 Hi,

 two weeks ago we migrated from Centos 5.6 (cyrus 2.3.7) to Scientific
 Linux
 6.0 (cyrus 2.3.16). Since that day some users are having problems with
 their
 mailbox. Thunderbird shows them IOERRORS and some are seeing
 undeletable
 mails with no content.

 The imapd's logfile contains the following:

 imap[21918]: IOERROR: user.xxx zero index/expunge record 41/47
 imap[16600]: IOERROR: user.yyy zero index/expunge record 16/17

 Also the mailbox (sub-)folder contains *.NEW files when this happens:
 -rw---   1 cyrus mail  13660 22. Jul 09:22 cyrus.cache.NEW
 -rw---   1 cyrus mail800 22. Jul 09:22 cyrus.index.NEW

 Reconstructing the mailbox removes the *.NEW files and it's working for
 some
 hours or a day. But the problem and the *NEW files are always comming
 back.
 But reconstructing several mailboxes every day can't be the solution. On
 client side nothing had changed.

 For the migration the database was converted to text on the old system
 and

Something is wrong with the database because Cyrus-imapd uses a number
of databases. So I guess here is a problem.

Do you not see any other errors in the Cyrus logs?

Simon



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: pop3d connection problemi

[Simon Matter]

 Hi,

 with cyrus imapd 2.4.10 i can connect with imap but not with pop3
 only when i start the service cyrus i can connet with the pop3 for a few
 seconds and then non responds

Did you try to strace it while running? Maybe you can see where it hangs.

Simon


 Example .if i try with the port 143 :

 Trying 127.0.0.1...
 Connected to localhost.
 Escape character is '^]'.
 * OK [CAPABILITY IMAP4rev1 LITERAL+ ID ENABLE AUTH=PLAIN SASL-IR]
 dns.digicolor.net http://dns.digicolor.net Cyrus IMAP v2.4.10 server
 ready

 if i try with the port 110

 Trying 127.0.0.1...
 Connected to localhost.
 Escape character is '^]'.

 he stay wait and the pop3d daemons grow critically and the machine uptime
 :

 top - 12:32:09 up 225 days, 16:25, 2 users, load average: 20.45, 26.21,
 24.55

 My /etc/imapd.conf :

 postmaster: postmaster
 configdirectory: /var/imap
 partition-default: /var/spool/imap
 admins: cyrus
 allowanonymouslogin: no
 allowplaintext: yes
 sasl_mech_list: PLAIN
 servername: dns.digicolor.net http://dns.digicolor.net
 autocreatequota: 1
 reject8bit: no
 quotawarn: 90
 timeout: 30
 poptimeout: 10
 dracinterval: 0
 drachost: localhost
 sasl_pwcheck_method: saslauthd
 sievedir: /usr/sieve
 sendmail: /usr/sbin/sendmail
 sieve_maxscriptsize: 32
 sieve_maxscripts: 5
 lmtp_over_quota_perm_failure: 1
 #unixhierarchysep: yes
 hashimapspool: yes


 /etc/cyrus.conf :

 START {
 # do not delete this entry!
 recover cmd=ctl_cyrusdb -r

 # this is only necessary if using idled for IMAP IDLE
 # idled cmd=idled
 # Oberdan Alertoni replication per farlo in tempo reale
 # syncclient cmd=/usr/cyrus/bin/sync_
 client -r
 }


 # UNIX sockets start with a slash and are put into /var/imap/socket
 SERVICES {
 # add or remove based on preferences
 # imap cmd=imapd listen=62.123.105.181:imap prefork=0
 imap cmd=imapd listen=imap prefork=0
 # imaplocal cmd=imapd -C /etc/imapd-local.conf listen=127.0.0.1:imap
 prefork=0
 # imaps cmd=imapd -s listen=62.123.105.183:imaps prefork=0
 # imapslocal cmd=imapd -C /etc/imapd-local.conf
 listen=127.0.0.1:imaps prefork=0
 pop3 cmd=pop3d listen=pop3 prefork=0
 # pop3s cmd=pop3d -s listen=pop3s prefork=0
 sieve cmd=timsieved listen=sieve prefork=0
 # sievelocal cmd=timsieved -C /etc/imapd-local.conf
 listen=127.0.0.1:sieve prefork=0

 # at least one LMTP is required for delivery
 # lmtp cmd=lmtpd listen=lmtp prefork=0
 lmtpunix cmd=lmtpd listen=/var/imap/socket/lmtp prefork=0

 # this is only necessary if using notifications
 # notify cmd=notifyd listen=/var/imap/socket/notify proto=udp
 prefork=1
 }

 EVENTS {
 # this is required
 checkpoint cmd=ctl_cyrusdb -c period=30

 # this is only necessary if using duplicate delivery suppression
 delprune cmd=ctl_deliver -E 3 period=1440

 # this is only necessary if caching TLS sessions
 tlsprune cmd=tls_prune period=1440
 }


 --
 Oberdan Albertoni

 ---

 DIGICOLOR.NET s.r.l. Via Ala Ponzone, 30 - 26100 CREMONA (CR) Tel. +39
 (0372) 30669 - 1965101 - 1951509
 Fax +39 (0372) 464840 E-mail i...@digicolor.net - Website
 http://www.digicolor.net
 C.F / P.IVA 01227520192 - R.E.A. CR n. 153382 - Capitale Sociale € 10.400
 i.v.

 Filiale di Codogno
 Via Galilei, 17 – 26845 CODOGNO (LO) Tel. +39 (0377) 741006 – 431931 Fax
 +39 (0372) 464840
 E-mail info.codo...@digicolor.net - Website http://www.digicolor.net

 ---

 Ai sensi del D.lgs n.196 del 30.06.03 (Codice Privacy) si precisa che le
 informazioni contenute in questo messaggio sono riservate e ad uso
 esclusivo
 del destinatario. Qualora il messaggio in parola Le fosse pervenuto per
 errore, La preghiamo di eliminarlo senza copiarlo e di non inoltrarlo a
 terzi, dandocene gentilmente comunicazione. Grazie

 This message, for the D.lgs n.196 / 30.06.03 (Privacy Code), may contain
 confidential and/or privileged information. If you are not the addressee
 or
 authorized to receive this for the addressee, you must not use, copy,
 disclose or take any action based on this message or any information
 herein.
 If you have received this message in error, please advise the sender
 immediately by reply e-mail and delete this message. Thank you for your
 cooperation.

 ---

 
 Cyrus Home Page: http://www.cyrusimap.org/
 List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/




Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: ctl_cyrusdb db version error

[Simon Matter]

 On 28/07/2011 08:44, Simon Matter wrote:
 Helo,
 on a server ( debian 5.0 ) I installed three years ago cyrus 3.2.11 but
 I have never used
 now I wanted to install cyrus 2.4.10 but when I run the command
 /usr/cyrus/bin/ctl_cyrusdb -r I get :

 Fatal error: wrong db version
 segmentation fault

 it is possible that ctl_cyrus version 2.4.10 compiled with version 4.6
 of libdb give me error because' /usr/lib/libbd.so is a symbolic link to
 version 4.4?

 How can I fix this problem ?

 Since I will have to do the same operation on a production server
 should
 I switch from berkeley db to skiplist db, before upgrade cyrus ?
 I can only recommend to switch to skiplist before upgrade. That way you
 can get rid of the BDB version issues. You can then even build 2.4.x
 without BDB support.

 I  recompiled  cyrus-imap with:

   ./configure --prefix=/usr --with-sasl=/usr/local/lib --with-perl
 --with-openssl=/usr/lib/ssl --enable-replication

 I remove all file *.db ( is only a test server with no mailbox ) but when
 I run imapd in the /var/log/messages I find :

 Jul 28 16:08:55 dnsback kernel: ctl_cyrusdb[21801]: segfault at 170 ip
 080863f1 sp bff25050 error 4 in ctl_cyrusdb[8048000+12d000]
 Jul 28 16:09:56 dnsback kernel: ctl_cyrusdb[21943]: segfault at 170 ip
 080863f1 sp bfb2b240 error 4 in ctl_cyrusdb[8048000+12d000]

 and if I run /usr/cyrus/bin/ctl_cyrusdb -r I get :

 fatal error: wrong db version
 Segmentation fault

 Why it  tells me that  even  the version of the  db and  iswrong?
 Should not  create a  new file  mailboxes.db in the skiplist format?

1) I don't think you have built cyrus-imapd without BDB support. Did you
check the resulting binaries with ldd?
2) You also have to clean up the $CONFIGDIR/db directory

Simon



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: Problems with the quota command (2.3.16-5)

[Simon Matter]

 Hello,

 I'm using cyrus-imapd-2.3.16-5 (Packager: Simon Matter) on CentOS 5.6
 (I'm currently not updated to 2.4 because there are know bugs in the quota
 command)

Hi,

1) the quota bug has existed for years, a workaraound is to run quota -f
twice. That may also be the reason why you see strange behaviour like
below.

2) the quota bug has been fixed in the latest 2.4 release, so you really
should upgrade.

Regards,
Simon



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: Problems with the quota command (2.3.16-5)

[Simon Matter]

 On Tue, July 26, 2011 9:57 am, Simon Matter wrote:
 Hello,


 I'm using cyrus-imapd-2.3.16-5 (Packager: Simon Matter) on CentOS 5.6
 (I'm currently not updated to 2.4 because there are know bugs in the
 quota
 command)

 Hi,


 1) the quota bug has existed for years, a workaraound is to run quota
 -f
 twice. That may also be the reason why you see strange behaviour like
 below.

 2) the quota bug has been fixed in the latest 2.4 release, so you
 really
 should upgrade.


 Simon,
 Bron,


 Does the quota bug expose itself independently of the underlying quota
 database structure ?
 I'm running a Cyrus 2.2 and 2.3 server since 2006 for tens of thousands
 of users and we regularly rename top level mailboxes, run reconstruct
 and quota -f also on other occasions. Never ran into the bug.
 Quotalegacy format (= one file per user).

I have always used quotalegacy format and I remember to have seen this
issue from time to time, even years ago. Maybe you also ran into the bug
but just didn't notice? I mean if we say running quota twice it means
things get fixed automatically after some time if you run quota from time
to time.

Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: cyradm +reconstruct all mailboxes (2.3.xx)

[Simon Matter]

Hello,
after copying mails to the new server and reconstructing emails
 with a command sudo -u cyrus /usr/lib/cyrus/bin/reconstruct -r -f
 user.username seems to be all fine. But after some fights with
 plugin for check quota in the Squirrelmail I tried a command for
 listing quota in the cyradm. Command mentioned above didn't
 reconstructed usage. Reconstruction of the usage works fine only in
 the cyradm. But it doesn't accept wildcard * to reconstruct all
 mailboxes. So there is a question - how to reconstruct all ? Any ideas ?

Did you try simply reconstruct -r -f without mailbox name?

Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: Solved: Re: cannot retrieve mail in desktop client

[Simon Matter]

 On Fri, 2011-07-01 at 12:56 +0800, JonL wrote:
 Finally,  I have it fixed!!!

 What I had to do to resolve this problem was:
 1) make sure mail was in the postfix group
 2) I added 1 to
 lmtpunix  cmd=lmtpd listen=/var/lib/imap/socket/lmtp prefork=1

 Thanks everyone for your assistance.

 Couple of question however,
 a) since this is a SLES 10 SP1 server and I plan to move to SP4 soon, is
 there a rpm for the cyrus-imapd 2.4.9? and to upgrade it manually, if
 there
 is no cyrus-imapd rpm, the instructions seems very vague are there any
 good
 step-by-step instructions.
 
 use Simon's rpm's (invoca.ch)
 http://www.invoca.ch/pub/packages/cyrus-imapd/

But he's running Suse :(
I don't think that will work well.

Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: Cyrus 2.4.9beta1 Released

[Simon Matter]

 We are pleased to announce the release of Cyrus IMAPd 2.4.9beta1

 You can download it from the snapshots directory:

 http://www.cyrusimap.org/releases/snapshots/cyrus-imapd-2.4.9beta1.tar.gz

 This is an beta release in the 2.4.x series.  This fixes
 a selection of bugs, including one first reported years
 ago that slipped through the cracks then.

Thanks, I'm running it on my own small server and can not find any issues
so far, nice work!

Regards,
Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: deliver.db

[Simon Matter]

 Hi,

 One of our scanner produces non-unique message-ids when sending mails.
 Ids are repeated in about a week.

 So I decided to have deliver.db purged at least once a day, because I
 have duplicatesuppression on in imapd.conf

 cyrus.conf
 EVENTS {

   checkpointcmd=ctl_cyrusdb -c period=30
   delprune  cmd=cyr_expire -E 1 -a -v at=1015
   tlsprune  cmd=tls_prune period=1440
 }
 ##


 deliver.db had 114 lines, I tried to set at to the time 5 min in the
 future. I waited for 10 mins. Log reported that cyr_expire ran, but
 deliver.db size and number of lines did not change.

 I have stopped cyrus, renamed deliver.db, started cyrus. And now
 deliver.db is quite small and not changing.

 -rw--- 1 cyrus mail 8,0K May 11 10:55 /var/imap/deliver.db


 What should I do anyway to fight that ugly scanner while keeping
 duplicatesuppression on ?

I suggest to generate a valid message ID on the receiving MTA for just
those mails. How to do it really depends on how your config looks like and
there may be a lot of ways to do that. If you could remove the message ID
before it is injected into the MTA, the MTA may happily generate a new
unique one.

Regards,
Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: POLL: what should reconstruct -f do?

[Simon Matter]

 The question came up from the following bug report:

 http://bugzilla.cyrusimap.org/bugzilla3/show_bug.cgi?id=3449

 Where there were spool files on disk, but no meta data left.
 Reconstruct gave no information about the files on disk at
 all.

 I see 4 options, can I'd like some opinions on what people
 think reconstruct should do.  Speak now(ish) or hold your
 peace!

 1) what we do now - require a cyrus.header in the directory
or ignore it.

 2) like (1) but warn about the directory with no cyrus.header

 3) add the mailbox if there's a directory, don't require
cyrus.header.

 4) like (3) - but check that there's at least one cyrus.* file
OR at least one message file in the directory before
creating the mailbox.  (so an empty directory doesn't generate
a bogus mailbox, and neither does one containing nothing that
looks like it belongs in a mailbox)

For me 3) sounds ok. I'm only wondering what could happen if there is crap
in the directory which Cyrus doesn't expect?

Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: POLL: what should reconstruct -f do?

[Simon Matter]

 On Saturday 23 April 2011 01:07:00 Bron Gondwana wrote:
 The question came up from the following bug report:

 http://bugzilla.cyrusimap.org/bugzilla3/show_bug.cgi?id=3449

 Where there were spool files on disk, but no meta data left.
 Reconstruct gave no information about the files on disk at
 all.

 I see 4 options, can I'd like some opinions on what people
 think reconstruct should do.  Speak now(ish) or hold your
 peace!

 1) what we do now - require a cyrus.header in the directory
or ignore it.

 2) like (1) but warn about the directory with no cyrus.header

 3) add the mailbox if there's a directory, don't require
cyrus.header.

 4) like (3) - but check that there's at least one cyrus.* file
OR at least one message file in the directory before
creating the mailbox.  (so an empty directory doesn't generate
a bogus mailbox, and neither does one containing nothing that
looks like it belongs in a mailbox)


 Alright, cast your votes!  I'll come back to this thread in a week
 or so and implement the winner.  (4) is the hardest to implement,
 but even that's not very tricky.


 I vote for 3.

 Mainly as I am wondering what will happen in the following situation:
 A folder X exists on the filesystem, but isn't recorded as a folder.

 What happens when a user now tries to create folder X?

 If the reconstruct had recorded it, any possible contents will already
 be
 known and no further decision about what to do with that folder has to be
 made.

What I was wondering, isn't it possible with IMAP to have a folder with no
parentfolder? If yes, does a reconstruct then create the parent folder
which was not intended to be there?

Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: cvt_cyrusdb before upgrade

[Simon Matter]

 Hello All,

 I am looking at upgrading from Cyrus 2.3.16 to 2.4.8. If I convert all
 the dbs from berkeley to skiplist before hand, is there any need to even
 build 2.4.8 with berkeley db support? (Using FreeBSD ports)

 I know 2.4.8 will automatically update the dbs, but I figure if I did it
 before hand, I could remove one more dependency from my system.

 If so, is there any special trick to running cvt_cyrusdb, or do I just
 shut things down and run it as the cyrus user?

There is no special trick, at least not one I remember. Just to be sure I
suggest to backup the whole configdirectory firtst.

Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

Re: murder and mailbox rename

[Simon Matter]

 hi all,

 I've configured 4 imap server as murder frontend/backend unified with
 one murder server. All seem to work, but when users rename a mailbox
 on the same backend the mails disappear, and I've to rebuild the maildir
 loosing all flags.

 There is some thing to configure or to check on unified or on murder ?

Maybe what you see is related to this?
http://asg.andrew.cmu.edu/archive/message.php?mailbox=archive.info-cyrusmsg=52544

Simon


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/