Re: Removing email from Xapian tier databases

2019-02-12 Thread Sebastian Hagedorn
No, I hadn't, but I have now – just in case. Thanks, Sebastian --On 11. Februar 2019 um 11:40:46 -0500 Bron Gondwana wrote: Excellent - I hope you grabbed both that commit and the one afterwards where I fixed the order of CID parsing. Actually, it might not be as big a deal on 3.0, but

Re: Removing email from Xapian tier databases

2019-02-11 Thread Bron Gondwana
Excellent - I hope you grabbed both that commit and the one afterwards where I fixed the order of CID parsing. Actually, it might not be as big a deal on 3.0, but not calculating the CID first did break one JMAP case on future. Cheers, Bron. On Tue, Feb 12, 2019, at 02:22, Sebastian Hagedorn

Re: Removing email from Xapian tier databases

2019-02-11 Thread Sebastian Hagedorn
Thanks! I rolled my own RPM with that patch, and I can confirm that it works. --On 11. Februar 2019 um 09:12:14 -0500 Bron Gondwana wrote: Yep, it's fixed in git now, so the next release will automatically create G keys for messages, even if they don't have a threadid! Bron. On Mon, Feb

Re: Removing email from Xapian tier databases

2019-02-11 Thread Egoitz Aurrekoetxea
Many many many many thanks a lot Brong!! :) :) :) :) :) :) :) --- EGOITZ AURREKOETXEA Departamento de sistemas 944 209 470 Parque Tecnológico. Edificio 103 48170 Zamudio (Bizkaia) ego...@sarenet.es www.sarenet.es [1] Antes de imprimir este correo electrónico piense si es necesario

Re: Removing email from Xapian tier databases

2019-02-11 Thread Egoitz Aurrekoetxea
Hi mates! Just for finishing this thread... Two squatter proccesses then... one in rolling mode and another one for info movement between Xapian databases and repacking databases as Brong said... can be running without known issues?. I say for avoid damaging something... thanks a lot mates!

Re: Removing email from Xapian tier databases

2019-02-11 Thread Bron Gondwana
It's definitely safe to have one rolling mode writing and one repacking. I wouldn't run multiple repacks in parallel, as they can wind up doing duplicate work (though the end result should always be correct and safe). Here's what we run: # Any time the disk gets over 50%, compress -o single

Re: Removing email from Xapian tier databases

2019-02-11 Thread Bron Gondwana
Yep, it's fixed in git now, so the next release will automatically create G keys for messages, even if they don't have a threadid! Bron. On Mon, Feb 11, 2019, at 21:30, Sebastian Hagedorn wrote: > So running ctl_conversationsdb -z followed by -b would assign thread ids to > those messages?

Re: Removing email from Xapian tier databases

2019-02-11 Thread Egoitz Aurrekoetxea
Now I'm noticing for instance, for moving data between Xapian databases.. you need to launch something like : sudo -u cyrus /usr/cyrus/bin/squatter -C /usr/local/etc/imapd.conf -v -z archive -t temp,meta,data,archive -u user/ego...@sarenet.es perhaps would be better to do : sudo -u cyrus

Re: Removing email from Xapian tier databases

2019-02-11 Thread Sebastian Hagedorn
So running ctl_conversationsdb -z followed by -b would assign thread ids to those messages? Because it works when I do that. Clearly this is an edge case, but IMO it should be handled somehow other than silently failing ;-) --On 11. Februar 2019 um 05:16:47 -0500 Bron Gondwana wrote: That

Re: Removing email from Xapian tier databases

2019-02-11 Thread Bron Gondwana
That sounds like the source messages have no thread id, and hence they aren't being stored. This is an interesting question actually, should we still store G keys for messages without thread identifier (CID)? Bron. On Mon, Feb 11, 2019, at 21:11, Sebastian Hagedorn wrote: > Hi Bron, > > --On

Re: Removing email from Xapian tier databases

2019-02-11 Thread Bron Gondwana
Conversations.db is an index over lots of interesting bits of the message, but the key part that's used by Xapian is the mapping from G key (aka: GUID, aka: sha1 of the message RFC822 data) to individual email. It's used for deduplication and for mapping from results to messages. The data in

Removing email from Xapian tier databases

2019-02-09 Thread Egoitz Aurrekoetxea
Good morning, As far as I understood, for Xapian you first create it's conversation database in order to work. Later you create database(s) for each mailbox where Xapian can search in. You can move data between them, new mails become indexed for instance Squatter in rolling mode... that's ok...