Notes 20 May

2019-05-20 Thread Bron Gondwana
Present: Bron, Robert, Ken, ellie

Bron in Vienna with Robert!

ellie:
* now has proper internet after many months of unreliable provider issues!
* not much else to report

Ken:
* libical and multi-value parameters/properties discussion (there's a FastMail 
issue about it)
 - what's the expected behaviour on the client side?
 - not a blocker right now, but has been allocated over from Robert to Ken
* Mailbox/get handling of ACL changes
 - this is long and complex because it requires keeping modseq per ACL entry, 
so it's a mailboxes.db format change
 - will work on that this week
* did some bugfixing last week
* nearly finished with vacation support
 - might have to go back to forcing user to include the vacation script, 
because otherwise STOP rules will mean that vacation doesn't happen.
 - plan is to parse active script and check if our custom jmapvacation.script 
is included - if not, reject the vacation object.
 - we're using :fcc at FastMail, may need to add a custom extension to allow 
setting that via JMAP.
 - current spec doesn't have a way to specify how many days to suppress 
responses for -> will need to fall back to server default. Could also add this 
in FM extension, but it's odd that it's not supported in upstream.
 - addresses: how to know it was specifically addressed to you rather than a 
bulk email. Maybe need to add Identity support to Cyrus too! Right now it only 
returns authenticated ID. Maybe add a property to INBOX with a list of email 
addresses!
 - https://github.com/cyrusimap/cyrus-imapd/issues/2760 One per identity. 
Probably uuid as the key.
* promised to write a draft for discovery of shared calendars before next week.

Robert:
* Finally landed Xapian language indexing on upstream master
* Spent a lot of time on JSCalendar RFC based on feedback from the mailing list
* Now started working on annotations API changes. Will be working on this for 
the next week.

Bron:
* working on caching jmap objects in the dav database.


Bron won't be available at the standard time next week, but will be around 
working.

--
 Bron Gondwana, CEO, FastMail Pty Ltd
 br...@fastmailteam.com



Re: Prepending Xapian Tiers

2019-05-20 Thread Bron Gondwana
On Fri, May 17, 2019, at 23:52, Дилян Палаузов wrote:
> Hello,
> 
> I set up a Cyrus system with one tier. I think it works. The .xapianactive 
> files contain 'tiername: 0'.
> 
> How can I insert a second tier?

I have never tried this on a live server! Clearly the right thing to do is to 
build a cassandane search which implements doing this so that we can make sure 
it works.

> Adding a XYZsearchpartition-default to imapd.conf, together with 
> defaultsearchtier: XYZ does not utilize the new directory: it stays empty and 
> the .xapianactive files do not get updated to mention the new tier.

That looks like it should work. I assume you have restarted your cyrus since 
making the change? I'm not certain that a rolling squatter will discover a new 
config in the way that imapd does.

Also - you'll need to run squatter in compact mode in order to add a new 
xapianactive entry. The simplest could be:

squatter -z tiername -t tiername -o

I believe that given your current setup, this will just copy the entry from 
tiername:0 to tirename:1 and also create XYZ:0 in the xapianactive file at the 
same time.

> Besides, if a message is MOVEd over IMAP, is any optimization utilized, to 
> avoid reindexing the message, but just change the address of the document?

Yes, both XAPINDEXED mode where the GUID is read from xapian, and CONVINDEXED 
mode where the GUID is looked up via user.conversations and then mapped into 
the cyrus.indexed.db files in each xapian tier allow Xapian to skip reindexing 
when a message is already indexed. This works for both MOVE and for 
re-uploading of an identical message file via IMAP.

Cheers,

Bron.

--
 Bron Gondwana, CEO, FastMail Pty Ltd
 br...@fastmailteam.com