Re: Xapian partition definition when using archiving?
Totally! The name "archive" is overused. It could be called something else easily enough. Bron. On Thu, 21 Dec 2017, at 01:05, Nic Bernstein wrote: > Bron, > Thanks for the swift reply. So if I understand this correctly, the > "archivesearchpartition-default" is named such because it's for the > archive location of Xapian search data, not because it's Xapian > search data from an Archive partition. Is that correct? In other > words, this is just another circumstance where seemingly obvious > partition names (like default-default) get us into documentation > trouble. Right?> > Thanks again, > -nic > > > On 12/20/2017 05:39 AM, Bron Gondwana wrote: >> Hi Nic, >> >> The Xapian partitions are entirely separate from archive partitions. >> The indexing code will find the file in the correct location if it >> needs to read it.>> >> Here's an example from one of our servers: >> >> defaultpartition: default >> defaultsearchtier: temp >> partition-default: /mnt/ssd21d2/sloti21d2t40/store254/spool >> tempsearchpartition-default: /tmpfs-search/sloti21d2t40 >> metasearchpartition-default: >> /mnt/ssd21d2/sloti21d2t40/store254/search>> datasearchpartition-default: >> /mnt/i21d2search/sloti21d2t40/store254/search>> >> archivesearchpartition-default: >> /mnt/i21d2search/sloti21d2t40/store254/search- >> archive>> archivepartition-default: >> /mnt/i21d2t40/sloti21d2t40/store254/spool- >> archive>> >> (clearly auto-generated!) >> >> Bron. >> >> >> On Wed, 20 Dec 2017, at 07:32, Nic Bernstein wrote: >>> Bron, et al., >>> We're about to set up a whole new bunch of partitions in support of >>> Xapian indexing, for a 2.5.10-to-3.0.4 upgrade, and then will be >>> introducing archive functionality, too. How do archive partitions >>> and Xapian partitions interact?>>> >>> For example, the server currently has the following in imapd.conf: >>> defaultpartition: default partition-default: /var/mailstores/default metapartition-default: /var/imapmeta/default partition-1: /var/mailstores/1 partition-2: /var/mailstores/2 partition-3: /var/mailstores/3 partition-4: /var/mailstores/4 ... partition-29: /var/mailstores/29 partition- 30: /var/mailstores/30 partition-100: /var/mailstores/100 partition- temp: /var/mailstores/temp ... # non-default metapartitions metapartition- 1: /var/imapmeta/1 metapartition-2: /var/imapmeta/2 metapartition- 3: /var/imapmeta/3 metapartition-4: /var/imapmeta/4 ... metapartition- 29: /var/imapmeta/29 metapartition-30: /var/imapmeta/30 metapartition- 100: /var/imapmeta/100 metapartition-temp: /var/imapmeta/temp >>> Going by the documentation, which I wrote with help from you good >>> folk at Fastmail, the Archive partition scheme might look something >>> like this:>>> >>> https://www.cyrusimap.org/imap/reference/admin/locations/archive-partitions.html>>> archivepartition-default: /var/mailarchives/default archivepartition- 1: /var/mailarchives/1 archivepartition-2: /var/mailarchives/2 archivepartition- 3: /var/mailarchives/3 archiveartition-4: /var/mailarchives/4 ... archivepartition-29: /var/mailarchives/29 archivepartition-30: /var/mailarchives/30 archivepartition-100: /var/mailarchives/100 archivepartition-temp: /var/mailarchives/temp >>> And the Xapian partition structure to mate with this would look >>> something like this (again, from the docs):>>> >>> https://www.cyrusimap.org/imap/developer/install-xapian.html >>> defaultpartition: default partition-default: /var/mailstores/default search_engine: xapian search_index_headers: no search_batchsize: 8192 defaultsearchtier: t1 1searchtier: t1 2searchtier: t1 3searchtier: t1 4searchtier: t1 ... 29searchtier: t1 30searchtier: t1 100searchtier: t1 tempsearchtier: t1 ... t1searchpartition- default: /var/search/default t1searchpartition-1: /var/search/1 t1searchpartition- 2: /var/search/2 t1searchpartition-3: /var/search/3 t1searchpartition- 4: /var/search/4 ... t1searchpartition-29: /var/search/29 t1searchpartition- 30: /var/search/30 t1searchpartition-100: /var/search/100 t1searchpartition- temp: /var/search/temp >>> First question, since there's no examples to work from; Is this >>> Xapian layout correct?>>> >>> Do I need to define & create Xapian partitions for the metadata >>> partitions, as is indirectly implied in Bron's original email on >>> this topic:>>> >>> https://lists.tartarus.org/pipermail/xapian-discuss/2014-October/009112.html>>> >>> >>> Also, how do these Xapian and Archive, interact? Do I need to add a >>> separate Xapian partition for each Archive partition, or will the >>> Archive partition be treated like a child of the non-Archive >>> partition? (again, implied but not directly addressed in that >>> email).>>> >>> Any guidance gladly accepted, and whatever I learn will be >>> repackaged into more complete documentation
Re: SASL 2.1.27 rc6
Ken, I'll try to lab up my original test case (for bug 3480) tomorrow evening. On 12/20/17 11:00 -0500, Ken Murchison wrote: We haven't had much, if any, feedback on this release candidate. Do the GSSAPI/LDAP folks have any further comments on https://github.com/cyrusimap/cyrus-sasl/issues/419 I'd really like to make a final release by Christmas as promised, but I also don't want to make a release that folks will have to patch immediately. -- Dan White
Re: SASL 2.1.27 rc6
We haven't had much, if any, feedback on this release candidate. Do the GSSAPI/LDAP folks have any further comments on https://github.com/cyrusimap/cyrus-sasl/issues/419 I'd really like to make a final release by Christmas as promised, but I also don't want to make a release that folks will have to patch immediately. On 12/11/2017 08:01 AM, Ken Murchison wrote: All, I have built a sixth (and hopefully last) release candidate of SASL 2.1.27 which can be downloaded from here: HTTP: http://www.cyrusimap.org/releases/cyrus-sasl-2.1.27-rc6.tar.gz http://www.cyrusimap.org/releases/cyrus-sasl-2.1.27-rc6.tar.gz.sig FTP: ftp://ftp.cyrusimap.org/cyrus-sasl/cyrus-sasl-2.1.27-rc6.tar.gz ftp://ftp.cyrusimap.org/cyrus-sasl/cyrus-sasl-2.1.27-rc6.tar.gz.sig MD5 Sum: cyrus-sasl-2.1.27-rc6.tar.gz : de083cc2e5c1cc3a1b88f7d85332a3ff cyrus-sasl-2.1.27-rc6.tar.gz.sig: 868cc9f5feee63ca2bd91279f5ac043b Note that the distro has been signed by my colleague Partha Susarla at FastMail. We didn't receive much feedback to Alexey's post on the GSSAPI/LDAP issue, so hopefully this release candidate will provoke some discussion leading to a resolution. As stated previously, we would like to make a final release before Christmas. If we have some last minute activity on the GSSAPI issue or any other showstoppers, we could push the release back to the end of the year as a last resort. The (mostly) complete list of changes from 2.1.26 are these: * Added support for OpenSSL 1.1 * Added support for lmdb (from Howard Chu) * Lots of build fixes (from Ignacio Casal Quinteiro and others) * Treat SCRAM and DIGEST-MD5 as more secure than PLAIN when selecting client mech * DIGEST-MD5 plugin: o Fixed memory leaks o Fixed a segfault when looking for non-existent reauth cache o Prevent client from going from step 3 back to step 2 o Allow cmusaslsecretDIGEST-MD5 property to be disabled * GSSAPI plugin: o Added support for retrieving negotiated SSF o Fixed GSS-SPNEGO to use flags negotiated by GSSAPI for SSF o Properly compute maxbufsize AFTER security layers have been set * SCRAM plugin: o Added support for SCRAM-SHA-256 o Allow SCRAM-* to be used by HTTP * LOGIN plugin: o Don’t prompt client for password until requested by server * NTLM plugin: o Fixed crash due to uninitialized HMAC context * saslauthd: o cache.c: + Don’t use cached credentials if timeout has expired + Fixed debug logging output o ipc_doors.c: + Fixed potential DoS attack (from Oracle) o ipc_unix.c: + Prevent premature closing of socket o auth_rimap.c: + Added support LOGOUT command + Added support for unsolicited CAPABILITY responses in LOGIN reply + Properly detect end of responses (don’t needlessly wait) + Properly handle backslash in passwords o auth_httpform: + Fix off-by-one error in string termination + Added support for 204 success response o auth_krb5.c: + Added krb5_conv_krb4_instance option + Added more verbose error logging At this point any major changes (e.g. API, wire protocol) will be pushed out to 2.1.28 or 2.2.0. I believe that this is close to being a final release which I would like to get out by the end of December. -- Kenneth Murchison Cyrus Development Team FastMail Pty Ltd -- Kenneth Murchison Cyrus Development Team FastMail Pty Ltd
Re: Xapian partition definition when using archiving?
Hi Nic, The Xapian partitions are entirely separate from archive partitions. The indexing code will find the file in the correct location if it needs to read it. Here's an example from one of our servers: defaultpartition: default defaultsearchtier: temp partition-default: /mnt/ssd21d2/sloti21d2t40/store254/spool tempsearchpartition-default: /tmpfs-search/sloti21d2t40 metasearchpartition-default: /mnt/ssd21d2/sloti21d2t40/store254/search datasearchpartition-default: /mnt/i21d2search/sloti21d2t40/store254/searcharchivesearchpartition-default: /mnt/i21d2search/sloti21d2t40/store254/search-archivearchivepartition-default: /mnt/i21d2t40/sloti21d2t40/store254/spool-archive (clearly auto-generated!) Bron. On Wed, 20 Dec 2017, at 07:32, Nic Bernstein wrote: > Bron, et al., > We're about to set up a whole new bunch of partitions in support of > Xapian indexing, for a 2.5.10-to-3.0.4 upgrade, and then will be > introducing archive functionality, too. How do archive partitions > and Xapian partitions interact?> > For example, the server currently has the following in imapd.conf: > >> defaultpartition: default partition-default: /var/mailstores/default >> metapartition-default: /var/imapmeta/default partition-1: >> /var/mailstores/1 partition-2: /var/mailstores/2 partition-3: >> /var/mailstores/3 partition-4: /var/mailstores/4 ... partition-29: >> /var/mailstores/29 partition-30: /var/mailstores/30 partition-100: >> /var/mailstores/100 partition-temp: /var/mailstores/temp ... # non- >> default metapartitions metapartition-1: /var/imapmeta/1 metapartition- >> 2: /var/imapmeta/2 metapartition-3: /var/imapmeta/3 metapartition-4: >> /var/imapmeta/4 ... metapartition-29: /var/imapmeta/29 metapartition- >> 30: /var/imapmeta/30 metapartition-100: /var/imapmeta/100 metapartition- >> temp: /var/imapmeta/temp >>> Going by the documentation, which I wrote with help from you good > folk at Fastmail, the Archive partition scheme might look something > like this:> > https://www.cyrusimap.org/imap/reference/admin/locations/archive-partitions.html> >> archivepartition-default: /var/mailarchives/default archivepartition- >> 1: /var/mailarchives/1 archivepartition-2: /var/mailarchives/2 >> archivepartition- >> 3: /var/mailarchives/3 archiveartition-4: /var/mailarchives/4 ... >> archivepartition- >> 29: /var/mailarchives/29 archivepartition-30: /var/mailarchives/30 >> archivepartition-100: /var/mailarchives/100 archivepartition-temp: >> /var/mailarchives/temp >>> And the Xapian partition structure to mate with this would look > something like this (again, from the docs):> > https://www.cyrusimap.org/imap/developer/install-xapian.html > >> defaultpartition: default partition-default: /var/mailstores/default >> search_engine: xapian search_index_headers: no search_batchsize: 8192 >> defaultsearchtier: t1 1searchtier: t1 2searchtier: t1 3searchtier: t1 >> 4searchtier: t1 ... 29searchtier: t1 30searchtier: t1 100searchtier: >> t1 tempsearchtier: t1 ... t1searchpartition-default: >> /var/search/default t1searchpartition-1: /var/search/1 t1searchpartition- >> 2: /var/search/2 t1searchpartition-3: /var/search/3 t1searchpartition- >> 4: /var/search/4 ... t1searchpartition-29: /var/search/29 t1searchpartition- >> 30: /var/search/30 t1searchpartition-100: /var/search/100 t1searchpartition- >> temp: /var/search/temp> First question, since there's no examples to work >> from; Is this Xapian > layout correct?> > Do I need to define & create Xapian partitions for the metadata > partitions, as is indirectly implied in Bron's original email on > this topic:> > https://lists.tartarus.org/pipermail/xapian-discuss/2014-October/009112.html> > Also, how do these Xapian and Archive, interact? Do I need to add a > separate Xapian partition for each Archive partition, or will the > Archive partition be treated like a child of the non-Archive > partition? (again, implied but not directly addressed in that > email).> > Any guidance gladly accepted, and whatever I learn will be repackaged > into more complete documentation on same.> > Cheers, > -nic > > -- Nic Bernstein n...@onlight.com > Onlight, Inc. www.onlight.com 6525 W > Bluemound Road, Suite 24 v. 414.272.4477 Milwaukee, > Wisconsin 53213-4073 >> Email had 1 attachment: > * nic.vcf > 1k (text/x-vcard) -- Bron Gondwana br...@fastmail.fm
