Re: LMTP said: 550-Mailbox unknown or you do not have authorization to see it
Le 14/10/2020 à 14:30:31-0500, Ezsra McDonald a écrit > I am building a new mail server to replace an older EL6 server. The new server > is Centos 8. I keep getting this response when trying to deliver email to a > local account stored in LDAP. > > host mail.example.org[/var/lib/imap/socket/lmtp] said: > 550-Mailbox unknown. Either there is no mailbox associated with this > 550-name or you do not have authorization to see it. > 550 5.1.1 User unknown (in reply to RCPT TO command)) > > I have tried replacing the new configs with my old working configs from the > EL6 > server but they get the same result. > > a postmap -q against the LDAP table config returns the appropriate > information. > I am wondering if the key is the 'or you do not have authorization to see it` > part of the message. What exactly does LMTP need to authorize the delivery? > > Enabling verbose logging on LMTP and LDAP did not give any clues. If you run getent passwd what you got ? Personnaly I don't run the lmtp against ldap, to risky IMHO, if you got any problem with the connection betwen your postfix/cyrus server and the ldap server your are going to loose email. So for me I'm using a script who dump the ldap inside the /etc/passwd, so the all account are local. Regards -- Albert SHIH Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Wed Oct 14 11:13:14 PM CEST 2020 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: What you do with old account
Le 14/06/2020 à 21:51:49+0200, Sebastian Hagedorn a écrit Hi, Thanks for your answer. > Am 09.06.20 um 16:32 schrieb Adam Tauno Williams: > > On Tue, 2020-06-09 at 15:51 +0200, Albert Shih wrote:> > >> After switching to cyrus imap, I think about how to do that. > >> If I'm correct I cannot just copy the file somewhere else, because cyrus > >> database would keep the information about the existance of the mailbox, so > >> what will the «state of the art» way to remove a mail account and all the > >> mail. > >> And how what would be the «state of the art» way to put it back ? > > I create a calendar event [task] to delete the mailbox and otherwise > > just leave it. If the account itself is disabled it cannot be accessed. > > > > Putting things back-into a mailstore is too much of a pain with current > > storage prices. > > We have about 90,000 accounts, and our current model is that we leave > expired accounts around for a year. The user can't login, and we don't > accept new mails, but it's still there in case the account is > re-activated. After one year the mailbox hierarchy is put into a .tgz > and written to tape. When that is done the account is permanently > deleted. If the user should come back, they get a completely new In fact I just notice, I've no idea...how to remove a mailbox in cyrus With dovecot it's rm -rf ;-) Something I famillar with. > account. I can't recall a single instance where the .tgz was ever > needed, but that's not my problem. After one more year the .tgz is Absolutly. > deleted from tape as well. Ok. Thanks Regards. -- Albert SHIH Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Tue 23 Jun 2020 04:09:05 PM CEST Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Maintenance of cyrus
Hi everyone. Just like to know if some can tell me until when the cyrus 3.0.x will be maintained ? The mail are super top critical for me, so I don't like to change major version during the life of the hardware. I see they are still update on 2.x so...maybe I'm lucky and the 3.0.x will ben maintained until end of time ;-) Of course I can understand the cyrus team don't want to do something like that, so no misunderstanding I don't want to blame anyone if you tell me tomorrow it's the end of cyrus 3.0.x (just I rush to the supermarket to buy a bottle of whisky if it is ;-) ). Regards -- Albert SHIH Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Tue 23 Jun 2020 03:37:50 PM CEST Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
What you do with old account
Hi I like to know what you do in time with the olds accounts. Long time ago (well 2 years) we using dovecot as imap server. And when some accounts are closed we create a archive of the person mailbox (with tar), put that in some backup server where the file going to stay about 2 years. After that the data are destroy The reason we keep that is some time (30% of the time) the person going to come back and event it's not mandatory for us, the person are very happy to go it old mailbox. After switching to cyrus imap, I think about how to do that. If I'm correct I cannot just copy the file somewhere else, because cyrus database would keep the information about the existance of the mailbox, so what will the «state of the art» way to remove a mail account and all the mail. And how what would be the «state of the art» way to put it back ? Regards. JAS -- Albert SHIH DIO bâtiment 15 Observatoire de Paris Heure local/Local time: Tue 09 Jun 2020 03:44:13 PM CEST Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Object Storage and Cyrus IMAP
Le 04/06/2020 à 10:23:12+0200, Marco a écrit > Hello, > >I see that Cyrus IMAP 3 can interface with some Object Storage such as > Caringo or OpenIO. > > Is anyone using these solutions? > > I would like to know how I can find more details about these deployment, > other than the brief description in imapd.conf man page. > > In particular I would like to know if these interfaces are stable and > supported in future releases of Cyrus IMAP. > > Do you plan to add wider support to object storage, maybe by adopting some > standard vendor independent? > > I thank you very much for every info you could provide. No sure if my informations are still correct, but long time ago, I ask openio about that. They say the connector between openIO and cyrus are maintained by openIO, it's not opensource and you need to pay a licence. And when Cyrus make a new release openio would make adjustment to make it work. I not sure who use that, but as I ear they(openIO) got few customer use this solution on a very large scale > 1Po. Regards -- Albert SHIH DIO bâtiment 15 xmpp: j...@obspm.fr Heure local/Local time: Thu 04 Jun 2020 08:44:56 PM CEST Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: ACL on root mailbox
Le 07/02/2020 à 11:35:10+0100, Paolo Cravero a écrit > Hello Albert. > > > I would like to known what kind of ACL you give on a shared mailbox. My > > problem is : > > > > I have some users who are admin on the mailbox, it's needed because those > > admin-user can create sub folder. > > > > But some user make mistake time to time and move the mailbox under some > > other mailbox > > [...] > > > So how can I give enough rights to someone to let he manage their mailbox > > but prevent this kind of wrong manipulation. > > See > https://www.cyrusimap.org/imap/reference/admin/access-control/rights-reference.html > for an explanation of ACL levels. > > I think "lrswc" could do in your case, for non-admin users. You may add "i" > and/or "t" and "e", depending on the way they operate on the shared mailbox > (copy mails into | delete mail from the folder). > Yes...I follow this documentation. Currently I run with lrswipktecd it seem that do the trick but the problem is when the user(admin) create a submailbox he cannot delete it if he make a mistake. Regards. -- Albert SHIH Observatoire de Paris France xmpp: j...@obspm.fr Heure local/Local time: Tue 11 Feb 2020 03:04:12 PM CET Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
ACL on root mailbox
Hi everyone, I would like to known what kind of ACL you give on a shared mailbox. My problem is : I have some users who are admin on the mailbox, it's needed because those admin-user can create sub folder. But some user make mistake time to time and move the mailbox under some other mailbox shared/mailbox1 shared/mailbox2 ---> shared/mailbox1/mailbox2 and the problem of course is the mail for mailbox2 go to /dev/null. So how can I give enough rights to someone to let he manage their mailbox but prevent this kind of wrong manipulation. Regards. -- Albert SHIH Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Thu 06 Feb 2020 05:24:57 PM CET Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
sieve authentication.
Hi everyone I'm using SOGo as webui for the mail. SOGo as the capability to manage sieve filters and it's work well with cyrus. The only problem, is SOGo as the capability to disable automatically the sieve filter, but for that it need a account who has the right to modify sieve filter for everyone. I didn't find how to do that with cyrus. If someone have a idea... Regards -- Albert SHIH Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Tue 05 Nov 2019 11:04:19 AM CET Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Issues with replication and folder/Sieve subscription
Le 09/07/2019 à 22:49:01+0200, Egoitz Aurrekoetxea a écrit > By the way, for your case I would recommend doing a script that does a get > from > dovecot and a put to Cyrus instead of copying Sieve files directly… it’s a > much > more cleaner way… Yes, it is what I did, before I try de sync I event do a /usr/local/cyrsus/sievec on each file to by absolute sure the sieve file compile correctly Regards. -- Albert SHIH DIO bâtiment 15 Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Wed 10 Jul 2019 09:20:09 AM CEST Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Issues with replication and folder/Sieve subscription
Le 09/07/2019 à 22:44:19+0200, Egoitz Aurrekoetxea a écrit Hi, > > If instead of -A you used -u for each of your users did it worked? Or did it If I remember correctly (but I not sure) this is how I find out the problem. Try time -A, notice it crash, so try -u first_user, notice it work try again -A, notice it crash again (second user) try -u second_user, notice it work, try -A, notice it crash but for the nth>>1 user try -u (n+1)th user notice it work try to find the difference between nth user who crash and the (n-1)th user who work, find out the only difference is the presence of sieve Try the infinite loop with -A. So in fact I never really try the -u with a non working user. > crashed in the same user as with -A?. Which Cyrus version were you running?. Don't sure but something like 3.0.4 (or 3.0.5) -- Albert SHIH DIO bâtiment 15 Observatoire de Paris France xmpp: j...@obspm.fr Heure local/Local time: Wed 10 Jul 2019 09:14:51 AM CEST Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Issues with replication and folder/Sieve subscription
Le 09/07/2019 à 14:10:49+0200, Egoitz Aurrekoetxea a écrit > Good morning, > > > After we upgraded to Cyrus 3.0.8, we saw that some users in the replicas > didn't > have some folders (or all) subscribed the same way they had in previous env in > Cyrus 2.3. Same happened for some users with Sieve scripts. It seemed the > content itself was perfectly copied. It was like, if the copy between versions > would not had fully succeed. We fixed it easily by creating some scripts for > checking folder subscriptions and Sieve scripts existence. We though it could > perhaps had something to do with some issue replicating folders subscriptions > and sieve scripts from 2.3 to 3.0.8. After that, as we fixed it easily and was > nothing related to content, we just didn’t go in deep in this topic. After my transfering all mail from my old server (dovecot) to the new one (under cyrus), I try to initialize the sync, so I launch the synchronization, and find out everytime the user got a sieve, the sync processus crash, butevent the it crash it still create « something », so the next time I launch the sync it pass (and email a actually synchronize). So for the first synchro I juste launch a infinite loop in bash to synchronize all user. I known it's not a very satisfying method but it work. With new user I don't have any problem. I already send a email here but don't get any solution https://lists.andrew.cmu.edu/pipermail/info-cyrus/2018-May/040186.html Don't know if that help Regards -- Albert SHIH DIO bâtiment 15 Observatoire de Paris 5 Place Jules Janssen 92195 Meudon Cedex France xmpp: j...@obspm.fr Heure local/Local time: Tue 09 Jul 2019 02:57:46 PM CEST Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Sync crash with bailing out
Hi everyone, My sync_client crash just after start with this message [root@imap-mirror-p /var/imap/sync/log]# /usr/local/cyrus/sbin/sync_client -S imap-mirror-m-tmp -A -v MAILBOXES shared.some_folder MAILBOX shared.some_folder Error from do_user(shared.some_folder): bailing out! [root@imap-mirror-p /var/imap/sync/log]# And indeed I don't have a user name shared.some_folder Any help. ? Regards -- Albert SHIH Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Fri Jan 18 15:21:12 CET 2019 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Big problem with replication
Le 16/01/2019 à 17:10:30+0100, Egoitz Aurrekoetxea a écrit > Good afternoon, > > > I would try doing it user by user (with -u). This way you would have all > synced > except the problematic mailbox. Hi, thanks for the help. I got some progress in my problem : > [root@imap-mirror-p /bals/DELETED]# /usr/local/cyrus/sbin/sync_client -S > slave_3 -A -v > MAILBOXES DELETED.DIO.5AEAD6F9 > Error from do_user(DELETED.DIO.5AEAD6F9): bailing out! > [root@imap-mirror-p /bals/DELETED]# > > and the DIO folder don't event exist > > [root@imap-mirror-p /bals/DELETED]# ls DIO > ls: DIO: No such file or directory > [root@imap-mirror-p /bals/DELETED]# > For some strange reason the I was unable to destroy the mailbox either (with cyradm), so I copy some junk mailbox on the filesystem, and run reconstruct and finally I'm was able to destroy those mailbox. But that's not really solve my problem because now when I run the sync_client he crash at the beginning with a shared mailbox. It stop with [root@imap-mirror-p /usr/home/jas-adm]# /usr/local/cyrus/sbin/sync_client -S imap-mirror-m-tmp -A -v MAILBOXES shared.* MAILBOX shared.* Error from do_user(shared.*): bailing out! [root@imap-mirror-p /usr/home/jas-adm]# I've no idea if it's normal or not. I don't think so, because the first level (Master -- replica --> slave_1) work well event with thoses shared_mailbox. Any help would be very welcome. Regards. -- Albert SHIH Heure local/Local time: Wed Jan 16 18:11:53 CET 2019 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Big problem with replication
Hi everyone. I've got some big issue with replication. I've master --- replica ---> slave_1 --- replica ---> slave_2 The replication between master and slave_1 work nice. Between slave_1 and slave_2 I've got some issue (log to big after network failure and work nagios_supervision). So now I'm trying to build a new slave_3 to replace slave_2. And I'm unable to launch sync_client. Each time I try to manually launch I got [root@imap-mirror-p /bals/DELETED]# /usr/local/cyrus/sbin/sync_client -S slave_3 -A -v MAILBOXES DELETED.DIO.5AEAD6F9 Error from do_user(DELETED.DIO.5AEAD6F9): bailing out! [root@imap-mirror-p /bals/DELETED]# and the DIO folder don't event exist [root@imap-mirror-p /bals/DELETED]# ls DIO ls: DIO: No such file or directory [root@imap-mirror-p /bals/DELETED]# Any help would be *very* welcome ;-) Regards. -- Albert SHIH DIO bâtiment 15 Observatoire de Paris Heure local/Local time: Wed Jan 16 16:08:47 CET 2019 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
What's sync. folder ?
Hi,
I would like to know what's the folder inside
{partition-default}/sync.
I've got some folder and have no idea what's.
I understand it's related to synchro between server, but my concerne is
I've got some very old folder with lot of email inside.
Regards
--
Albert SHIH
DIO bâtiment 15
Observatoire de Paris
xmpp: j...@obspm.fr
Heure local/Local time:
Mon Nov 12 10:16:00 CET 2018
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Quotas
Le 07/11/2018 à 08:52:20-0500, James B. Byrne via Info-cyrus a écrit > Cyrus-IMAPD-3.0.7 on FreeBSD-11.2p4 > > We have users who, having deleted email, evidently do not recover the > quota allocated to those messages. We do not believe that we have Well...I don't think so. With my configuration (same as you cyrus + FreeBSD 11) when someone delete a mail the quota is recover. > enabled delayed deletion. All the user mailboxes previous had been with delayed deletion. For example : [root /usr/local/cyrus/sbin]# ./quota user.zog Quota % Used Used Resource Root 20971520 13 2889569 STORAGE user.zog 137037 MESSAGE user.zog 0 X-ANNOTATION-STORAGE user.zog 299X-NUM-FOLDERS user.zog [root /usr/local/cyrus/sbin]# cd /bals/user/zog [root /bals/user/zog]# du -s -h . 3.9G. [root /bals/user/zog]# find . -type f|wc 179261 179264 2760840 [root /bals/user/zog]# So you can see the quota show with cyrus command say I use 2889569 (~2.7 Go) and 137037 messages, but on the filesystem (ZFS) I use 3.9Go and 179000 files (event It's not precise because I didn't exclude cyrus*) Regards -- Albert SHIH DIO bâtiment 15 Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Wed Nov 7 16:25:13 CET 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Problem with sync
Le 08/10/2018 à 10:36:10+0200, Albert Shih a écrit Hi everyone, > I still got my problem : > I got two level of synchro: > > master --- sync --> imap-mirror-1 --- sync --> imap-mirror-2 > > The first level work fine, the second level (imap-mirror-1 --> imap-mirror-2) > crash sometime ago. > > Now I try to restart the sync, and currently I'm not sure it's working, in > the imap-mirror-1 I got > > [root@imap-mirror-1 /var/imap/sync/log]# ls -l > total 133519 > -rw--- 1 cyrus cyrus 16080624 Oct 8 10:27 log > -rw--- 1 cyrus cyrus 256631941 Oct 3 16:39 log-run > [root@imap-mirror-1 /var/imap/sync/log]# top > > When I restart the imap daemon on both mirror, I see a sync_client on the > imap-mirror-1 and imapd on the imap-mirror-2 at 100% during sometime ( 4 > days), now I still got the > > /var/imap/sync/log/log-run > > file and the > > /var/imap/sync/log/log > > who still growing up, but very few activity on those mirror. But it seem the sync working, but it's very slow and stop very often. So how can I speedup me sync, knowning at this speed I will never end the sync because the /var/imap/sync/log/log still growing up. So if it wait the end of /var/imap/sync/log/log-run I don't see how this going to happen. So is they are any way to synchronise what's in the /var/imap/sync/log/log-run manually without waiting the imap daemon to launch the sync. Other question, at this time (during the sync of log-run) can I restart the imap daemon without breaking everything ? Regards. -- Albert SHIH DIO bâtiment 15 Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Mon Oct 15 13:12:05 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: inaccessible mailbox for one user
Le 10/10/2018 à 07:41:52+0200, Lists Nethead a écrit > > Quoting Albert Shih : > > > Le 09/10/2018 à 21:26:28+0200, Lists Nethead a écrit > >> > >> Quoting Lists Nethead : > >> > >> > > Hi, > >> > > > >> > > A user clicks on a mail in Horde IMP and next the inbox appears > >> > > completely empty. This message files all still there of course, but > >> > > the problem is that I cannot run any cyrus command on that mailbox, > >> > > reconstruct, chk_cyrus or anything. > >> > > > >> > > Is there a lock possibly sitting somewhere that prohibits running > >> > > any commands? Any other hints as to where I should start? > >> > > > >> > >> Restored users seen and sub files from backup, ran reconstruct again and > >> then I had to run unexpunge. Now user files are back but I see the > >> following > >> error for this user in the logs > >> > >> IOERROR: conversations_audit on store > >> > >> Should I try sbin/ctl_conversationsdb -b ? > > > > I'm not a cyrus guru but, are you sure your FS are clean ? I'm running > > FreeBSD and long time ago I got a FS error without crash. It was a standard > > server (apache) and I got very strange error, like unable to remove file. > > I eventually restart the server and run a fsck -f manually to fix de > > problem. I never find out where and how I got that problem > > > > The FS was UFS2. > > > > I'm absolutly not sure it's the problem, just try to help. > > Thank you for your response. You'r welcome. And seem...my answer was not very usefull ;-) > Yes, I think the initial fault was not a Cyrus issue and the locking > problem vanished after the jail was restarted. Mail works ok now > except the error above. Did you find a solution ? > We run zfs so fsck is not of much help, but none of the pools report a > problem. We too, run FreeBSD + jail + ZFS (two, one for the mailbox on mechanical disk and another on ssd). Regards. -- Albert SHIH Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Fri Oct 12 18:22:32 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: inaccessible mailbox for one user
Le 09/10/2018 à 21:26:28+0200, Lists Nethead a écrit > > Quoting Lists Nethead : > > > > Hi, > > > > > > A user clicks on a mail in Horde IMP and next the inbox appears > > > completely empty. This message files all still there of course, but > > > the problem is that I cannot run any cyrus command on that mailbox, > > > reconstruct, chk_cyrus or anything. > > > > > > Is there a lock possibly sitting somewhere that prohibits running > > > any commands? Any other hints as to where I should start? > > > > > Restored users seen and sub files from backup, ran reconstruct again and > then I had to run unexpunge. Now user files are back but I see the following > error for this user in the logs > > IOERROR: conversations_audit on store > > Should I try sbin/ctl_conversationsdb -b ? I'm not a cyrus guru but, are you sure your FS are clean ? I'm running FreeBSD and long time ago I got a FS error without crash. It was a standard server (apache) and I got very strange error, like unable to remove file. I eventually restart the server and run a fsck -f manually to fix de problem. I never find out where and how I got that problem The FS was UFS2. I'm absolutly not sure it's the problem, just try to help. Regards. -- Albert SHIH DIO bâtiment 15 Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Tue Oct 9 22:07:55 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Problem with sync
Hi everyone, I got two level of synchro: master --- sync --> imap-mirror-1 --- sync --> imap-mirror-2 The first level work fine, the second level (imap-mirror-1 --> imap-mirror-2) crash sometime ago. Now I try to restart the sync, and currently I'm not sure it's working, in the imap-mirror-1 I got [root@imap-mirror-1 /var/imap/sync/log]# ls -l total 133519 -rw--- 1 cyrus cyrus 16080624 Oct 8 10:27 log -rw--- 1 cyrus cyrus 256631941 Oct 3 16:39 log-run [root@imap-mirror-1 /var/imap/sync/log]# top When I restart the imap daemon on both mirror, I see a sync_client on the imap-mirror-1 and imapd on the imap-mirror-2 at 100% during sometime ( 4 days), now I still got the /var/imap/sync/log/log-run file and the /var/imap/sync/log/log who still growing up, but very few activity on those mirror. What should I do after long period of missing sync ? How can I re-sync everything ? Regards -- Albert SHIH DIO bâtiment 15 Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Mon Oct 8 10:30:46 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Conversations and search
Le 16/08/2018 à 14:04:22+0200, Sebastian Hagedorn a écrit Hi > > I have a question regarding the conversations db and how it affects (Xapian) > search. In this GitHub issue > (<https://github.com/cyrusimap/cyrus-imapd/issues/2376>) I was dealing with > Xapian search always failing. The underlying reason turned out to be an > empty conversations db. When I regenerate that user's conversations db, > Xapian search works fine. So far, so good. But then I noticed this piece of > documentation: > > <https://www.cyrusimap.org/imap/concepts/deployment/databases.html?highlight=conversations#conversations-userid-conversations> > > Quote: "This file contains all the message-id fields from every email that > has been seen in the ***past three months***, mapping to the conversation > IDs in which this message ID has been seen, and the timestamp when it was > last seen." > > This raises a number of questions. What does "has been seen" mean in this > context? And does it mean that Xapian search will always fail to find emails > that arrived more than three months ago?? That doesn't sound very useful ... If I'm correct (I'm new with cyrus too), the conversations DB as nothing to do with the database of xapian. The xapian database is some file with .glass extension and contain all the index of all your mail. Those files stand inside t1searchpartition-default/FIRST_LETTER_OF_LOGIN/user/LOGIN/ The conversation database is some cyrus internal database use by cyrus to *create* the xapian index. In other word the three months means, if you stop the xapian index now, you have three months to restart it or you're going to need to regenerated the xapian database from the beginning (and that take very loooong time). Please if it's incorrect please someone correct me. Regards -- Albert SHIH DIO bâtiment 15 Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Fri Aug 17 16:01:52 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Donate to cyrus team
Le 06/07/2018 à 14:03:49+1000, Bron Gondwana a écrit Hi, > > At the moment the donations to the project are in the form of people's time or > services that the project needs, but we don't have any facility for handling > cash donations. We've talked about moving the project out from CMU into an > organisation which specialises in supporting open source projects, and could > arrange for donations to handled by said organisation. > > But for now - there's no way to donate money. Just to donate bug reports or > code :) I take this opportunity to say a *big* thanks to the cyrus_team for their development and people on this mailing list who answer my question. The migration from dovecot are now complete and everything since. Regards -- Albert SHIH DIO bâtiment 15 Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Fri Jul 6 08:33:12 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
admin for sieve
Hi, I've got a script from a crontab who deactivated the vacation (with sieve). So I need a account who was able to deactivat every sieve-vacation. Is they're something sieve-admin who was different with admin ? Just preventing to put a admin (for cyrus) password (not encrypted) in a script. Regards. -- Albert SHIH DIO bâtiment 15 Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Fri Jun 29 17:12:32 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: sos on cyrusimapd
Le 27/06/2018 à 12:58:22-0400, Eric W. Bates a écrit Hi, Thanks you *very* much for your answer. I will try that. But I'm know pretty sure that the problem doesn't come from network but from I/O of the disk. I manage to configure my mutt (my old MUA) to keep the connection, he lost the connection because by default he keep the connection only for 15 secondes. Changing that to 600 secondes and I never get disconnected. But I still get some hang (10-30 sec) and I think it's the I/O because the server hang too or at least very slow down (when I'm connected by ssh) and If I use atop I'm see the disk are busy. Another point is during those time (~1500 connections for ~500 uniques users), the ARC go down to 20G (by configuration it's get 96Go). I will try the zfs tweaking. > > kern.ipc.maxsockbuf=4194304 # (default 2097152) > net.inet.tcp.sendbuf_max=4194304 # (default 2097152) > net.inet.tcp.recvbuf_max=4194304 # (default 2097152) > net.inet.tcp.mssdflt=1460 # (default 536) > net.inet.tcp.minmss=1300 # (default 216) > net.inet.tcp.cc.algorithm=htcp # (default newreno) > net.inet.tcp.cc.htcp.adaptive_backoff=1 # (default 0 ; disabled) > net.inet.tcp.cc.htcp.rtt_scaling=1 # (default 0 ; disabled) > net.inet.tcp.syncache.rexmtlimit=0 # (default 3) > net.inet.ip.rtexpire=10 # (default 3600) > net.inet.tcp.syncookies=0 # (default 1) > net.inet.tcp.tso=0 # (default 1) > > kern.ipc.soacceptqueue=32768 # (default 128 ; same as kern.ipc.somaxconn) > net.inet.tcp.delayed_ack=0 > > Been quite a while since we set all that up and it's been performing well. well, from thunderbird point of vu everything seem fine, only some *normal* slow down, users doesn't complain. I complain because it's big server with lot of disk (24 disks, 2 SSD for /var/imap, RAM 192Go), soI like to make it very efficient And again thanks you for your help. Regards. -- Albert SHIH DIO bâtiment 15 Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Thu Jun 28 22:35:43 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: sos on cyrusimapd
Le 27/06/2018 à 11:27:48-0400, Eric W. Bates a écrit > Yah. You need to crank up some buffer sizes. > > This is an excellent document (written for FreeBSD 10): > https://calomel.org/freebsd_network_tuning.html Yes I already find that page > > I think the one you're bumping your head on is: > kern.ipc.maxsockbuf but the problem is It's already high according to what netstat -m say I got [root@zenobe /usr/home]# sysctl -a kern.ipc.maxsockbuf kern.ipc.maxsockbuf: 2097152 [root@zenobe /usr/home]# netstat -m 35091/25314/60405 mbufs in use (current/cache/total) 33072/11170/44242/12180860 mbuf clusters in use (current/cache/total/max) 33072/7914 mbuf+clusters out of packet secondary zone in use (current/cache) 1254/4242/5496/6090429 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/1804571 9k jumbo clusters in use (current/cache/total/max) 0/0/0/1015071 16k jumbo clusters in use (current/cache/total/max) 79932K/45636K/125569K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0 sendfile syscalls and both mbufs denied/delayed are at 0. > > but you shouldn't crank it up by itself. Do you think I should still increase that maxsockbuf ? Do you have anything special tweaking you kernel ? Regards. > > > > So today I switch all my user from my old server to the new one running > > cyrus-imapd 3.0.7 with FreeBSD 11.1-p11. > > > > The server get 192Go of Ram. > > > > Currently I got lot of disconnection from MUA (mutt), the client going to > > keep the connection during let's say few minutes and then I lost the > > connection (nothing to do with the network). > > > > If I relaunch just after de disconnection, I can get > > > >back maibox pretty quickly, > >back maibox very slowly > >connection error (strange SSL I/O error on the client) > > > > On the server I didn't see lot of message, only this strange > > > >onewconn: pcb 0xf8276c40e570: Listen queue overflow: 49 already in > > queue awaiting acceptance (209 occurrences) > > > > knowing I'm not able to find the pcb f8276c40e570 with netstat or lsof, > > event it not change in time. > > > > I guessing I mis configure something...but what ? > > > > I know the server are under heavy load because all my user's client are > > re-synchronize their mailbox. But well It's not very good... > > > > Is anybody running cyrusimapd on FreeBSD have tuning some special variable > > in the kernel through sysctl ? > > > > Regards > > > > -- > > Albert SHIH > > Observatoire de Paris > > xmpp: j...@obspm.fr > > Heure local/Local time: > > Wed Jun 27 16:03:10 CEST 2018 > > > > Cyrus Home Page: http://www.cyrusimap.org/ > > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > > To Unsubscribe: > > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus > > > > > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > To Unsubscribe: > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus -- Albert SHIH DIO bâtiment 15 Observatoire de Paris 5 Place Jules Janssen 92195 Meudon Cedex France ☏ +33 1 45 07 76 26/+33 6 86 69 95 71 xmpp: j...@obspm.fr Heure local/Local time: Wed Jun 27 17:41:04 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
sos on cyrusimapd
Hi everyone, So today I switch all my user from my old server to the new one running cyrus-imapd 3.0.7 with FreeBSD 11.1-p11. The server get 192Go of Ram. Currently I got lot of disconnection from MUA (mutt), the client going to keep the connection during let's say few minutes and then I lost the connection (nothing to do with the network). If I relaunch just after de disconnection, I can get back maibox pretty quickly, back maibox very slowly connection error (strange SSL I/O error on the client) On the server I didn't see lot of message, only this strange onewconn: pcb 0xf8276c40e570: Listen queue overflow: 49 already in queue awaiting acceptance (209 occurrences) knowing I'm not able to find the pcb f8276c40e570 with netstat or lsof, event it not change in time. I guessing I mis configure something...but what ? I know the server are under heavy load because all my user's client are re-synchronize their mailbox. But well It's not very good... Is anybody running cyrusimapd on FreeBSD have tuning some special variable in the kernel through sysctl ? Regards -- Albert SHIH Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Wed Jun 27 16:03:10 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Prefork or not prefork
Hi everyone, I would like to know when that's become usefull to use prefork (for imap) and how much you set the prefork (vs nb of users). Regards JAS -- Albert SHIH DIO bâtiment 15 Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Mon Jun 25 14:38:33 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Kill daemon
Hi everyone. I got a small issue for killing imapd when I want to restart the server. >From time to time when I reboot the server the imapd daemon just don't want to die so the reboot just wait indefinitly. I've to poweroff the server. I'm running FreeBSD 11.1 Anyone have this problem ? Regards. JAS -- Albert SHIH DIO bâtiment 15 Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Mon Jun 25 14:29:07 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: CentOS 6, cyrus-imapd 2.3.16-15.el6, postfix 2.6.6-8.el6, procmail 3.22-25.1.el6_5.1, squirrelmail 1.4.22-5.el6
Le 21/06/2018 à 13:28:38-0400, Robert Heller a écrit Sorry when I answer you I forget to put the mailing list on CC, Maybe they are other answers... So I add info-cyrus@lists.andrew.cmu.edu on cc for this answer. Hope you don't mind. > At Thu, 21 Jun 2018 17:23:56 +0200 Albert Shih wrote: > > > > > Le 21/06/2018 à 09:58:15-0400, Robert Heller a écrit > > > I am having trouble getting this all to work properly together. > > > > > > I have a CentOS 6 server, that is mostly a webserver and one main E-Mail > > > account (mine), plus a couple of E-Mail accounts that use procmail to do > > > spam > > > filtering and then forward thing along. I also run Mailman for some > > > mailling > > > lists. Everything is working, except cyrus-imapd. I have postfix set up > > > to > > > use procmail to deliver mail to me, through the filter and forward > > > accounts > > > and to Mailman. I would like to add a *few* webbased e-mail accounts > > > using > > > Squirrelmail and (local for now) imapd. But I cannot figure out how to > > > either > > > get cyrus-imapd to either access the local mail spool (/var/spool/mail) > > > or to > > > get procmail to deliver to imap spool (/var/spool/imap). The postfix > > > config > > > > I'm pretty sure you cannot do that. > > > > > says that you can only use mailbox_command or mailbox_transport, but not > > > both. > > > I want to use procmail + /var/spool/mail for some E-Mail accounts and (I > > > guess) deliver + /var/spool/imap for others. > > > > I'm not sure that's a good idea to use postfix + procmail + cyrus. I thinking of what you asking, well, maybe you can do that (event I find that a little strange inside the procmailrc: SHELL="/bin/bash" SENDMAIL="/usr/sbin/sendmail -oi -t" LOGFILE="/var/log/procmail.log" DEFAULT="$HOME/Maildir/" MAILDIR="$HOME/Maildir/" DELIVER="/usr/local/cyrus/deliver -d $LOGNAME" # If it's for me :0 * To: poweruser@\.* $DEFAULT # deliver to other user :0 w | $DELIVER I didn't test it (I don't use procmail anymore) > > > > I think that would be much better to use > > > > postfix + sieve + cyrus. > > > > I don't say sieve is better than procmail, just say it's integrated. > > If I use sieve instead of procmail, what happens with my existing UNIX Mail > accounts? Does this mean I *have* to use imap to access ALL E-Mail on my > server? I don't want to do that. I use QWK to access my personal mail, > currently delivered to /var/mail/spool via procmail and use procmail recipies > to pre-process it. I need to do this in part because I have dial-up internet > and cannot relaistly use a (so called) "modern" E-Mail client like > Thunderbird, since I don't really have enough bandwidth for that. Part of my > procmail recipies involve stripping off attachments (so I don't have to > download large attachment over a dialup connection) and to strip off HTML Well as I know you cannot do that with sieve. > (since I don't use [and refuse to use] a HTML capable E-Mail client). Well, I'm also in the exact situation than you, but I use mutt as main E-Mail client. It's working very well thought imap (whatever the server is), and you can also use offlineimap to synchronise a local copy of your email. So you can use mutt online by connecting to your imap server and you can also use offlineimap to get a local copy so you can access your email when you're offline. And yes mutt work well on very low bandwidth connection. Regards. -- Albert SHIH DIO bâtiment 15 Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Thu Jun 21 21:27:11 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
cyr_expire in log
Hi Everyone, When I start with cyrusimapd I first create a prototype with cyrus-imapd-3.0.5. Now on the futur production server I run cyrus-imapd 3.0.7 I notice in the 3.0.5 I got message like Jun 18 04:20:21 FQDN cyr_expire[78395]: Expired 0 and expunged 0 out of 24761820 messages from 72060 mailboxes Jun 18 04:21:20 FQDN cyr_expire[78395]: Expired 0 entries of 28143158 entries seen in 1860 conversation databases Jun 18 04:21:20 FQDN cyr_expire[78395]: duplicate_prune: pruning back 3.00 days Jun 18 04:21:22 FQDN cyr_expire[78395]: duplicate_prune: purged 5402 out of 15191 entries but those message doesn't exist in the 3.0.7. So I upgrade the prototype to 3.0.7 and indeed I loose those message. I event cannot give them back with debug: 1 It's not very important thing, but is they're any option to bring those message back ? Regards -- Albert SHIH Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Tue Jun 19 17:26:40 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Monitoring cyrusimapd
Hi everyone, I would like to know what kind of monitoring you perform on a cyrus-imapd. Beside classic (check_imap, check_disk, check_cpu etc...) do you have any special thing to monitor about cyrus-imapd. For example do you launch any check on each database ? How can I check if the replication work fine ? Regards -- Albert SHIH Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Tue Jun 19 17:21:11 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Restart from....? (DRP)
Le 18/06/2018 à 10:22:03+0200, Niels Dettenbach via Info-cyrus a écrit > Am Montag, 18. Juni 2018, 09:46:02 CEST schrieb Albert Shih: > > What do you think ? What's your DRP ? > I shoot snapshots from the underlying FS of the spool partition(s) and the > main DB files (skiplist) - incl. (incremental) filesystem dumps of them. How you do that ? Because at the beginning my plan was to do both (replication and snapshot). The problem is currently I'm encounter big issue with the snapshot. I don't know if this is the right place because I don't know if it's related to Cyrus, so that's why I didn't talk about at the first time. But I got a server (Dell PowerEdge, 192Go, 28 mechanicals disk, 2 ssd, 2 SAS (for the OS)). The system is FreeBSD 11 running on the 2 SAS disk on UFS The cyrus imap run inside a jail on the 2 ssd ( on zfs pool) The mailbox and xapian index are on two zfs dataset on a zpool with 28 mechanicals disk. Everything seem working fine, until I try to send the dataset on other server. I just cannot send a zfs snapshot from this server to another. If the dataset are small that's OK, but with the mailbox (~4To) the zfs command just hang after 10-40 minutes during 1-10 minutes, come back work during 1 or 2 hours and hang again etc. > in a desaster scenario it usually works well to reinstantiate the last > snapshot and start the server(s) with a forced full reconstruct run. But this > only offers "low resolution" recovery (mails / mods since last snapshot are > gone then). > > Beside this we run daily FS backups (incl. cyrus DB dumps) which allows us to How you do that ? Because cyrus got a lot of DB > reinstall from zero (i.e. autmated by ansible or similiar) on system and FS Yes we using puppet, reinstalling the system and configuration are easy. The hard part are the data. > level. > > I'm a bit new to the new included backup mechs and repo features in cyrus 3 > and interested in experiences with setups, allowing a efficient "lossless" > recovery too. I'm a bit new with cyrus so... ;-) All I can say is the replication seem to works well. I got master --> first slave (same room) --> second slave (distant datacenter). I'll will try today to see if it's easy or not to restart with a slave by cloning it. Best regards. -- Albert SHIH DIO bâtiment 15 Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Mon Jun 18 10:36:19 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Restart from....? (DRP)
Hi everyone I've a question about DRP (Disaster Recovery Plan), what's the easiest (= fastest) way to rebuild a server (with the data) after a server « disappear » (fire, water flood, etc.). I see three way to « backup » the data : Replication, Backup service (inside cyrusimapd 3), Filesystem backup (whatever the technic) For replication my concern is the speed of the replication, the main server (I got only one server) got lots of RAM, got SSD, and SAS disk, the replication got SATA disks (lots of RAM too). When I check I think everything are indeed replicated on the « slave » but with some delays (1/2 days). What do you think ? What's your DRP ? Regards. JAS -- Albert SHIH Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Mon Jun 18 09:37:59 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Body in sieve.
Le 02/06/2018 à 13:29:53-0400, Ken Murchison a écrit > It’s disabled by default. Enable it by adding it to the sieve_extensions > option > in imapd.conf Ok. Thanks. But do you know why ? Is because some performance issue ? Regards -- Albert SHIH DIO bâtiment 15 xmpp: j...@obspm.fr Heure local/Local time: Sat Jun 2 20:43:30 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Body in sieve.
Hi everyone. On the website https://www.cyrusimap.org/imap/reference/admin/sieve.html it's say cyrus sieve support body extension. But on my installation (FreeBSD 11 + Cyrus-3.0.7) I didn't find body, if I do a telnet Escape character is '^]'. "IMPLEMENTATION" "Cyrus timsieved 3.0.7" "VERSION" "1.0" "SASL" "" "SIEVE" "comparator-i;ascii-numeric fileinto reject vacation vacation-seconds imapflags notify envelope imap4flags date mailbox mboxmetadata servermetadata relational regex subaddress copy index variables" "STARTTLS" "UNAUTHENTICATE" OK Is this is a problem with the FreeBSD packages ? Regards. -- Albert SHIH DIO bâtiment 15 xmpp: j...@obspm.fr Heure local/Local time: Sat Jun 2 17:23:55 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Backup vs replication
Hi everyone, I'm not sure I really understand what's the benefice of backup (cyrus>3.x) vs replication ? Is the main goal are to save disk space with compression ? Less inode (with large file) ? I believe the add backup feature to cyrus-imapd was/still very lots of work. So what's the advantage of backup vs replication_over_compress_fs. Regards -- Albert SHIH DIO bâtiment 15 Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Fri May 18 22:42:34 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Xapian/Cyrus/Thunderbird
Le 11/05/2018 à 13:32:29+0200, Robert Stepanek a écrit > > On Fri, May 11, 2018, at 13:04, Sebastian Hagedorn wrote: > > For my understanding: does that mean the Xapian index is only used > > for clients that support RFC 6203? If that is the case, how are > > "traditional" IMAP searches handled? > > For non-FUZZY text SEARCH, Cyrus attempts to match the string on its > own [1]. Off the top of my head, I don't think Cyrus falls back to the > search-engine index for large corpus text matches. Using FUZZY always > is the better choice, but if there's a real-world issue, please let me > know. It might be possible to switch to Xapian also for non-FUZZY > search. So I think that would be a very very great idea to make every search through xapian. Because currently I'm testing cyrus-imapd-3.0.6 and activated the xapian index. After turn on telemetry it's seem Thunderbird just do a « classic » search. So that's mean when I do search from Thunderbird it's very slow, and I build the xapian index for almost nothing because almost 90% of my users use Thunderbird as MUA. Do you want I create a issue on github ? Regards. -- Albert SHIH DIO bâtiment 15 Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Fri May 18 22:20:14 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: sync_client crash with sieve
Le 18/05/2018 à 09:54:14+1000, Bron Gondwana a écrit > It sounds like the sieve file might be incorrect in some way? Either it Some are very very simple (forward -> external address). > doesn't parse with the current version of sievec, or it's got the wrong > permissions, or even the wrong naming. On our servers we have: > > websieve.script > websievebc > defaultbc -> websievebc I not sure, but all the sieve script was copied with : rsync from old server /usr/local/cyrus/sbin/sievec sogo.sieve sogo.bc ln -s sogo.bc defaultbc I'm even try to use it by sending a email and those sieve seem working. And more strange some time if I launch again sync_client they pass the sieve, sometime I need to delete the sieve to make sycn_client work again. > > If you have sieve scripts which aren't correctly laid out to how Cyrus > expects, > that could maybe cause sync to abort because it doesn't know how to replicate > that layout. Ok. Is they are any way to exclude sieve in the replication process ? Thanks Regards -- Albert SHIH DIO bâtiment 15 Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Fri May 18 21:34:55 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Xapian/Cyrus/Thunderbird
Le 14/05/2018 à 14:35:21+0200, Sebastian Hagedorn a écrit > > On Fri, May 11, 2018, at 14:25, Sebastian Hagedorn wrote: > >> --On 11. Mai 2018 um 13:32:29 +0200 Robert Stepanek > >> <r...@fastmailteam.com> wrote: > >> > For non-FUZZY text SEARCH, Cyrus attempts to match the string on its > >> > own [1]. > >> > >> That sounds strange to me, because Cyrus 2.4 and earlier don't support > >> FUZZY, and there the SQUAT index was used, if present. Only messages > >> that were added after the last squatter run were searched directly. Why > >> would that have changed? > > > > Right, it hasn't. SQUAT is still the backend for non-FUZZY text search. > > But search_engine is either squat or xapian. That would mean that one would > have to run squatter with two separate configurations in order to cover > both search types. That's at the very least counterintuitive ... Sorry to ask some stupid question, but in the case I choose xapian with only one configuration, does that mean the search is ... what neither SQUAT or XAPIAN if the client don't use FUZZY SEARCH ? > > I thought Xapian was added to improve all IMAP BODY searches, but I guess > the only reason was to enable IMAP FUZZY searches. In my opinion this needs > to be stated explicitly in the documentation, if that's the way it will be agree;-) Regards. -- Albert SHIH DIO bâtiment 15 Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Thu May 17 22:11:13 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
sync_client crash with sieve
Hi, I've got some issue with sync_client (first run, before rolling sync). When I launch sync_client -A the synchronization start and after some time crash. I activate the verbose mode, and It seem sync_client crash over some sieve file. So I not sure, but I pretty sure, everytime I got a sieve file for the user (in /var/imap/sieve) the sync crash. Any idea ? Regards -- Albert SHIH DIO bâtiment 15 Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Thu May 17 17:51:04 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Archive over NFS.
Hi everyone. I would like to know if it's possible to put the archive partition over NFS. Same question about "DELETED", for example can I make /MAIL_BOX_DIR/DELETED as a mountpoint for a NFS partition. If it's yes do I have to take extra precaution about the something (sync, async, etc..) Regards JAS -- Albert SHIH DIO bâtiment 15 Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Wed May 16 22:48:56 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
How to use backup/replica
Hi everyone. Currently I'm trying to use replication to do the backup. When it's working, I would like to know ... how to use the backup. The question maybe sound totaly stupid, but well, with dovecot I know how to use the backup. But with cyrus imap ... I don't know. Three cases : 1/ What should I do if someone loose a mail and warn me long after delayed delete time. But If I got the file on the backup (cheat disk so can keep the much longer deleted mail). 2/ What should I do if I got a corruption a database, how should I fix that with the backup. 3/ What should I do if my server loose everything. Regards -- Albert SHIH DIO bâtiment 15 Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Wed May 16 21:56:19 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Backup methods
Le 10/05/2018 à 16:08:32-0300, Anatoli a écrit Hi. > In both cases, a copy of the master data is made, which requires twice the > space of real usage (Cyrus Backups tries to apply compression on stored data, > not sure how well it works). In ZFS with lz4 (standard compression on ZFS) you got 1.18 ratio (3.57 To on disk for 4.05To of data) so not very good. I use lz4 because it's got same performance than no compression. I didn't try gzip on mail but gzip can be very impressif on ratio but eat lot of cpu. > > What is really needed, IMO, for SME environments is the ability for Cyrus to > sync to disk all data, so one can take a hot copy of that data with standard > UNIX tools and then handle it accordingly. Once a recovery is needed, one just > copies a backup to the Cyrus dir and starts the service. The data would be in > the exact same state as when the backup took place. This is discussed in the > github issue mentioned in the previous mail. I fully agree. In fact 7 years ago when we renew our mail server I already try cyrus and dovecot (we come from courier-imap), and we choose dovecot because it's very easy to backup (and manage) for old_unix_admin. Just put in the crontab some rsync that's all, one mail = one file, etc. Now we choose cyrus-imap over dovecot (so for next 7 years) because all the feature cyrus got. But yes if cyrus got something like mysql_dump or pg_dump_all that would be super nice. Regards. -- Albert SHIH DIO bâtiment 15 Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Thu May 10 22:43:47 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Backup methods
Le 10/05/2018 à 10:38:28-0300, Anatoli a écrit > > Not very sure to understand that. It's always true isn't ? If you have XTo > > of > data and you want n backups you will need X*(n+1) To ? > > The replication as it is designed means that you create an additional > (replica) > instance of Cyrus that will be in sync with the master instance, so when you > need to make a backup, you turn of the replica, take a backup from its data, > then turn it on again so it comes in sync with the master. In this case > there's > no interruption to the service, you just stop a replica. But the replica will > use the same amount of space as your master, so without even making a backup, > you'll use 2x space. + you have to understand how the replication works, then > set it up, control that the sync process is always working and the replica has > the same information as the master... That's a great solution for ISP-level or > public mail service operations, but IMO an absolute overkill for small > deployments. For me, if I put a replica in place it's get the role of backup. Meanning I will put two replica and do not make another backup. > When it comes to making a backup, the best policy IMO is to make incremental > backups. In this case you only store the new mails + binary indexes. Once in a > while (e.g. every month) you make a full backup, then, say, once a week a > level > 1 backup (that stores changes from the previous week, reset at lower level > backup, i.e. every month), then daily level 2 backups and hourly level 3. This > way you can restore up to hourly changes without using excessive amount of > space. Of course you can compress them too (xz -9 gives a pretty good ratio). Using ZFS, do no need to do that. Just use zfs snapshot and he going to keep the differential at block level (much better than file level). Same as compression. Just need to activate compression on the dataset. > > Uhh don't do that. Your Postfix has no problem in retaining mails if Cyrus is > not reachable, then attempt their delivery again. I was referring to that, > depending on the configuration of your incoming MTA, the next delivery attempt > may be in, say, 15 minutes, so you postpone incoming mail for that time if you > turn off Cyrus to take a backup. If you turn off your incoming MTA, the source > MTA may have issues with delivery at all (you don't control it, you don't know > how it's configured, when the next delivery attempt will occur, etc.), never > turn off your incoming MTA. Don't be a problem, I've got 2 public incoming MTA, 4 privates and the postfix on the cyrus-server. So incoming mail, let's say gmail.com going from gmail.com_MX to our MX, then send to cyrus-server. So if I stop the postfix on the cyrus_server, the incoming mail going to stay on the our MX. -- Albert SHIH DIO bâtiment 15 Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Thu May 10 22:27:22 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Backup methods
Le 10/05/2018 à 02:44:18-0300, Anatoli a écrit Hi, > > The replication is reasonable only if you have more than one server in your > deployment (and both servers with the same level of security, if not you risk > to compromise the user data) or "spool size/available disk space" is low, > otherwise you'd need to dedicate 2 times more space than needed to store user > data, only to take a periodic backup (+ the space needed to store the backup > itself). Not very sure to understand that. It's always true isn't ? If you have XTo of data and you want n backups you will need X*(n+1) To ? I don't see how you can avoid that, of course you can activate heavy compression on the backup but beside of that > I suggest you take a look at this issue: https://github.com/cyrusimap/ > cyrus-imapd/issues/1763, where backups for small deployments were already Thanks for the link I will read that. > Answering the OP's question, I'm using Cyrus for 4 years now and I don't know > about any reliable and reasonable strategy for backups of Cyrus data in SME > environments. Summing it up: > > • FS snapshots without stopping the server: a possibility of a corrupted > backup. > • FS snapshots after stopping the server: service downtime, breaking open > connections, delivery issues for incoming MTAs, etc. - reasonable for > daily Well that's is easy to avoid, you just have to stop postfix before stopping the VM, when postfix is stop all incoming messages will stay on the parent smtp server, so no loosing incoming mail. > backups in a 8/5 office, unreasonable for 24/7 deployments (e.g. users > distributed in different time zones) or for intra-day backups. I check, stopping postfix, stopping the VM, take a snapshot, starting the VM, take about 10-15 secondes. So I agree with you it's not a very good solution because user still can loose the connection, but I think without replication it's acceptable. > • Replication: unreasonable requirements for disk space, setup overkill. For the setup the overkill is for me a small price vs loosing dataand as for the disk space that's not a issue at all for me. Currently I run dovecot and have 2 backups, so when I say to my boss « we got X To of mail » I already got 3 * X To of disk. Say in other way, if I can afford X To, I will say I can give you X/3 To of mail. Regards. -- Albert SHIH DIO bâtiment 15 Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Thu May 10 09:03:31 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Xapian/Cyrus/Thunderbird
Le 05/05/2018 à 16:01:31+0200, Robert Stepanek a écrit
Hi,
>
> On Fri, May 4, 2018, at 14:48, Albert Shih wrote:
>
> > Is they are any way to make a test to know if xapian work or not ?
>
> Cyrus comes with two binaries that will tell you its build and
> runtime configuration:
>
> E.g. running
>
> $ /usr/cyrus/sbin/cyr_buildinfo
>
> on my development server yields a JSON-formatted output about Cyrus
> build config. Look for something like
>
>"search": {
> "squat": true, "xapian": true, "xapian_flavor": "cyruslibs"
> },
>
So I get
"search": {
"squat": true,
"sphinx": false,
"xapian": true,
"xapian_flavor": "vanilla"
},
I don't know if vanilla are a correct value of xapian_flavor, I would say
yes because vanilla are ... a flavor...and a flavor of xapian.
> to check if Xapian is enabled in the build (presumably, yes, since
> you have *.glass databases in your directories).
>
> Next, cyr_info will tell you, if xapian got enabled indeed at
> runtime:
>
> $ /usr/cyrus/sbin/cyr_info -C /imapd.conf
> conf-all | grep search
>
> should yield something like
>
> [...] search_engine: xapian
I got that too
search_batchsize: 8192
search_engine: xapian
search_index_headers: no
>
> Next, make sure that the IMAP commands submitted by your clients
> are using SEARCH FUZZY. You might want to inspect the IMAP telemetry
> to check this.
I will check that.
>
> If all these checks passed, there shouldn't be any reason why Cyrus
> should not use Xapian during search. One might want to enable verbose
> logging for search then, but unfortunately, that's currently not
> a runtime-option.
Ok.
Just one question, are the xapian handle all search ? Including body search ?
Thanks a lot
Regards
JAS
--
Albert SHIH
DIO bâtiment 15
Observatoire de Paris
xmpp: j...@obspm.fr
Heure local/Local time:
Wed May 9 09:32:58 CEST 2018
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Cyrus + Xapian
Le 27/04/2018 à 14:30:10+0200, Albert Shih a écrit > Le 24/04/2018 à 21:13:20+0200, Albert Shih a écrit > > Le 24/04/2018 à 13:21:09+0200, Robert Stepanek a écrit > > Hi, > > > > > > > > we just had a chat on the issue you are reporting in the regular > > > Cyrus call: there have been a few fixes for squatter since > > > 3.0.5. Would you be able to compile from the current development > > > snapshot [1]? If not, 3.0.6 will be released soon, which will also > > > contain the fixes. > > > > > > This probably will not resolve the issue, but it would give us > > > one more data point if squatter turns out to run cleanly on the > > > development release. > > > > it's related to my problem with xapian (the xapian are not on the same disk > > than the OS). > > But anyway I need to fix that in the first place, after that I will > > retry...everythingI will keep you informif I'm able to compile the > > src (I'm using the ports, so not my work ;-) ). > > So I was able to fix my problem (wrong driver on the OS). > > Now I can confirm the problem with squatter are not related. Event with a > server who run normaly I got the same issue. > > After sync (imapsync) some mailbox, each time I launch squatter I will get > a first index for one user, after that seg fault (just after jump to next > user), and if I re-launch squatter will index the first user not index, so > If I have 100 users I must launch squatter 100 times. > > I also recheck every option available to get a coredump file but without > succes. > > I try to compile the cyrus-imapd3 from the github but my lack of skill not > allow me to complete the task. I'm sorry. > > Soon the 3.0.6 will be package inside the OS I will try and tell you if > the problem are still here. So on FreeBSD 11, the 3.0.6 are release. After upgrade, I can confirm the Segmentation fault disapear, now everything work fine. . Thanks a lot. Regards. -- Albert SHIH DIO bâtiment 15 Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Wed May 9 09:19:30 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Xapian/Cyrus/Thunderbird
Le 04/05/2018 à 14:48:49+0200, Albert Shih a écrit > Hi, > > So I've installed cyrus-imapd 3.0.5 with xapian. > > I configure and launch xapian according to > > https://www.cyrusimap.org/imap/developer/install-xapian.html > > and I currently launch in a console > > /usr/local/cyrus/sbin/squatter > > to indexe all the mail I got (imapsync). > > The problem is on two larges user.mailbox the first one with a > /xapian-index/user/firstuser/*.glass and the second without (not indexes > already), on those two mailbox I make a search from thunderbird and don't > see any difference about the delay of the answer. just to be very clear, I select the option in thunderbird to make the search on server. I also try those search throught a webmail (SOGO4), to be absolutly sure it's not a search on the client. Regards -- Albert SHIH DIO bâtiment 15 Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Fri May 4 14:52:13 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Xapian/Cyrus/Thunderbird
Hi, So I've installed cyrus-imapd 3.0.5 with xapian. I configure and launch xapian according to https://www.cyrusimap.org/imap/developer/install-xapian.html and I currently launch in a console /usr/local/cyrus/sbin/squatter to indexe all the mail I got (imapsync). The problem is on two larges user.mailbox the first one with a /xapian-index/user/firstuser/*.glass and the second without (not indexes already), on those two mailbox I make a search from thunderbird and don't see any difference about the delay of the answer. So I'm wondering if the xapian work Is they are any way to make a test to know if xapian work or not ? Regards -- Albert SHIH DIO bâtiment 15 Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Fri May 4 14:28:28 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Size of partition.
Le 30/04/2018 à 08:30:44+0200, Robert Stepanek a écrit > On Fri, Apr 27, 2018, at 15:48, Albert Shih wrote: > > I activated squatter but put the index not on ssd. The index > > created by squatter are (in my case) about 5-8% of the size of > > the mailbox. > > > > But I got lot of « conversation » index. What's the purpose > > of those « conversations » ? > > conversations.db initially was used to keep an index of email > conversations (e.g. the threaded views typically found in most email > readers nowadays). But today it also helps for most use-cases where > we need to keep track of messages across mailboxes (e.g. if you have > two copies of an email in two mailboxes, you'll see that reflected in > conversations.db). It confusingly is still named conversations.db for > historic reasons. But for any recent (v3) Cyrus IMAP installation, > including Xapian and JMAP, it's a very critical piece in the puzzle. Ok. Thanks you. Still one question, I see in the configuration file conversations_expire_days : 90 why do I need to keep those information during 90 days ? For example what would be the impact if I say conversations_expire_days : 7 Thanks for you help Regards. -- Albert SHIH DIO bâtiment 15 Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Mon Apr 30 20:00:10 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Size of partition.
Le 27/04/2018 à 15:04:40+0200, Eric Luyten a écrit
>
>
> On 27/04/2018 14:33, Albert Shih wrote:
> > Hi,
> >
> > Some question about the grow speed of the size of
> >
> > /var/imap
> >
> > because I would put that partition on SSD, and SSD are expensive. I would
> > like to know what would be the ratio between the size of mailbox and
> > /var/imap.
> >
> > On my new server I have ~20-25To space for the mailbox. How much /var/imap
> > should I get.
> >
> >
>
> Albert,
>
>
> We have a Cyrus server with 82,000 users, 500,000 (sub)mailboxes and about
> 8 Terabyte of mail messages in 9 ZFS filesystems built on an iSCSI SAN
> offering.
> Each filesystem holds somewhere between 6 and 11 million messages.
>
> Our /var/imap is 1.5 GB large, it holds the delivery and mailboxes
> databases,
> the Sieve scripts, seen information and some local administration stuff.
> It has been sitting on the same pair of 73 GB SLC SSDs since August 2010.
> The server absorbs between (on average) 80,000 (Saturday) and 250,000
> (Monday)
> deliveries per day. At one time we peaked at close to 400,000 deliveries in
> 24 hours,
Very impressif (well...at least for me ;-) )
> We're even thinking of, on our next generation, putting all the Cyrus
> metadata
> (cache, header, index) on solid state as these represent about 2% of the
> messages
> volume (will be more if you have activated delayed expunge and squatter, I
> think).
I activated squatter but put the index not on ssd. The index created by
squatter are (in my case) about 5-8% of the size of the mailbox.
But I got lot of « conversation » index. What's the purpose of those «
conversations » ?
> I just checked ... we have 90 GB of cyrus.{cache, header, index}
Ok thanks a log.
>
> SSDs can now be had in capacities of 3.2 TB
Yes, but my boss will kill me if I present to him the price of this SSD ;-)
and I will very guilty if after 5 years only 1 % of this ssd are used ...
Thanks you for you help.
Regards
--
Albert SHIH
DIO bâtiment 15
Observatoire de Paris
xmpp: j...@obspm.fr
Heure local/Local time:
Fri Apr 27 15:42:06 CEST 2018
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Size of partition.
Hi, Some question about the grow speed of the size of /var/imap because I would put that partition on SSD, and SSD are expensive. I would like to know what would be the ratio between the size of mailbox and /var/imap. On my new server I have ~20-25To space for the mailbox. How much /var/imap should I get. Regards. -- Albert SHIH DIO bâtiment 15 Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Fri Apr 27 14:30:44 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Cyrus + Xapian
Le 24/04/2018 à 21:13:20+0200, Albert Shih a écrit > Le 24/04/2018 à 13:21:09+0200, Robert Stepanek a écrit > Hi, > > > > > we just had a chat on the issue you are reporting in the regular > > Cyrus call: there have been a few fixes for squatter since > > 3.0.5. Would you be able to compile from the current development > > snapshot [1]? If not, 3.0.6 will be released soon, which will also > > contain the fixes. > > > > This probably will not resolve the issue, but it would give us > > one more data point if squatter turns out to run cleanly on the > > development release. > > it's related to my problem with xapian (the xapian are not on the same disk > than the OS). > But anyway I need to fix that in the first place, after that I will > retry...everythingI will keep you informif I'm able to compile the > src (I'm using the ports, so not my work ;-) ). So I was able to fix my problem (wrong driver on the OS). Now I can confirm the problem with squatter are not related. Event with a server who run normaly I got the same issue. After sync (imapsync) some mailbox, each time I launch squatter I will get a first index for one user, after that seg fault (just after jump to next user), and if I re-launch squatter will index the first user not index, so If I have 100 users I must launch squatter 100 times. I also recheck every option available to get a coredump file but without succes. I try to compile the cyrus-imapd3 from the github but my lack of skill not allow me to complete the task. I'm sorry. Soon the 3.0.6 will be package inside the OS I will try and tell you if the problem are still here. Regards. -- Albert SHIH DIO bâtiment 15 Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Fri Apr 27 14:24:28 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Cyrus + Xapian
Le 24/04/2018 à 13:21:09+0200, Robert Stepanek a écrit Hi, > > we just had a chat on the issue you are reporting in the regular > Cyrus call: there have been a few fixes for squatter since > 3.0.5. Would you be able to compile from the current development > snapshot [1]? If not, 3.0.6 will be released soon, which will also > contain the fixes. > > This probably will not resolve the issue, but it would give us > one more data point if squatter turns out to run cleanly on the > development release. Currently I cannot. I just find out on my server I got some very weird hardware issue. It's a brand new Dell PowerEdge R740Xd with plenty of Ram (192Go, drive etc..) and he take 17 times (yes seventeen) more time to install a simple pkg than another almost same server, so I don't know if it's related to my problem with xapian (the xapian are not on the same disk than the OS). But anyway I need to fix that in the first place, after that I will retry...everythingI will keep you informif I'm able to compile the src (I'm using the ports, so not my work ;-) ). Regards -- Albert SHIH DIO bâtiment 15 Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Tue Apr 24 21:09:07 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Cyrus + Xapian
Le 20/04/2018 à 16:18:23+0200, Robert Stepanek a écrit Hi, > > Which Cyrus version are you using? Is it a packaged distribution I'm using cyrus-imapd30-3.0.5_1 xapian-core-1.4.5,1 > or are you building from source? Would you be able to inspect a No it's packages from the distribution (in FreeBSD way). > core dump of squatter? Do you see squatter log anything before No dump (don't know why). > the segfault? No, I activate the log (syslog), and verbose. But nothing special. The « funny » thing is, know it's absolutly sure, the first time I get a Seg Fault, but the second time it's working perfectly I don't think that can be a issue but the mailbox are on ZFS partition and everything running inside a jail (something like systemd-nspwan, lxc). > > I haven't seen the issue you are describing, and it certainly > shouldn't happen. From first glance I would suspect an issue in > the build setup or library environment, but it's hard to narrow Ok I will review the options I choose. > down the root cause with the current information. One question about the way squatter/xapian/cyrus interact, but is it possible to loose email if squatter/xapian goes to segmentation during a incomming mail ? Thanks for the help. Regards. > > > > > > I'm trying to configure xapian and cyrus-imapd, so I put this in > > my config > > > > sync_log: on sync_log_channels: squatter search_engine: > > xapian search_index_headers: no search_batchsize: 16384 > > defaultsearchtier: t1 t1searchpartition-default: /var/imap/search > > conversations: on conversations_db: twoskip > > > > but when I launch squatter they crash randomly, meaning I can > > launch squatter, it going to run during 10sec, sometime 30 sec, > > after that they give a me > > > > squatter[34140]: indexing mailbox user.... > > Indexing mailbox user.... Segmentation fault > > > > It seems (human watch) this happen only when beginning indexing > > a new user, so I tough it's because he need to create the > > > > /var/imap/search/*/user/*/xapian > > > > so I create all this directorybut don't seem give any > > difference. > > > > But I pretty certain squatter give a segmentation fault every > > *FIRST* time he run on a user. > > > > So I guessing I do something wrong dans squatter need some « > > initialize »... > > > > Regards. > > > > > > > > > > -- Albert SHIH DIO bâtiment 15 xmpp: j...@obspm.fr Heure > > local/Local time: Fri Apr 20 15:30:07 CEST 2018 Cyrus > > Home Page: http://www.cyrusimap.org/ List Archives/Info: > > http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: > > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus > Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: > http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus -- Albert SHIH DIO bâtiment 15 Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Sat Apr 21 07:57:04 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Cyrus + Xapian
Hi, I'm trying to configure xapian and cyrus-imapd, so I put this in my config sync_log: on sync_log_channels: squatter search_engine: xapian search_index_headers: no search_batchsize: 16384 defaultsearchtier: t1 t1searchpartition-default: /var/imap/search conversations: on conversations_db: twoskip but when I launch squatter they crash randomly, meaning I can launch squatter, it going to run during 10sec, sometime 30 sec, after that they give a me squatter[34140]: indexing mailbox user.... Indexing mailbox user.... Segmentation fault It seems (human watch) this happen only when beginning indexing a new user, so I tough it's because he need to create the /var/imap/search/*/user/*/xapian so I create all this directorybut don't seem give any difference. But I pretty certain squatter give a segmentation fault every *FIRST* time he run on a user. So I guessing I do something wrong dans squatter need some « initialize »... Regards. -- Albert SHIH DIO bâtiment 15 xmpp: j...@obspm.fr Heure local/Local time: Fri Apr 20 15:30:07 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Synchro for migration
Le 15/04/2018 à 01:17:00+0200, Michael Menge a écrit Hi, > > Quoting Albert Shih <albert.s...@obspm.fr>: > > > Hi everyone. > > > > Totally new in the world of cyrusimapd, I'm trying to migrate my current > > imap server from dovecot to cyrusimapd. > > > > welcome to the list. Thanks. > > > I'm still in the prototype stage. > > > > I've ~ 4To of mailbox for ~ 2000 users. > > > > I would like to know what's the best way to migrate all mailbox and > > subscription. > > > > Using IMAP is the best way I can think of to migrate the Mails. > Depending of the tool you use it might not be the fastest way but > you can migrate all information with out worrying about the on disk > formats. May years ago (~10), at the time we migrated form UW-Imap to Cyrus > we uses imapcopy (it was part of the UW-Imap library) because it > was much faster as imapsync aat that time. Ok. That's also my guessing. > > Cyrus allows admins to proxy authenticate for users, if the sasl mech > support it (PLAIN does suppot it, LOGIN does not) > https://www.cyrusimap.org/sasl/sasl/authentication_mechanisms.html > > It seems like the following imapsync options allow you to use proxy > authentication > > --authmech1 --authmech2 > --authuser1 --authuser2 > --proxyauth1 --proxyauth2 I'm not sure to understand how proxyauth works. I google it and find lots of doc about...proxy imap. Currently I need to authenticate my user against a LDAP(openldap), so I configure saslauthd with ldap and works perfectly. So one of my solution is to have a getpwent (local passwd) with real login and fake password. And after the migration switch to ldap auth. Thanks for your help. -- Albert SHIH Observatoire de Paris France xmpp: j...@obspm.fr Heure local/Local time: Mon Apr 16 08:57:17 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Synchro for migration
Hi everyone. Totally new in the world of cyrusimapd, I'm trying to migrate my current imap server from dovecot to cyrusimapd. I'm still in the prototype stage. I've ~ 4To of mailbox for ~ 2000 users. I would like to know what's the best way to migrate all mailbox and subscription. I already try imapsync, but how can I do that without knowing the password of my user ? I think they are something call master user in dovecot, but is they are anything similar in cyrusimapd ? And if yes how I choose the user name in imapsync ? Regards. -- Albert SHIH Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Fri Apr 13 17:33:23 CEST 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
