Re: Notes Oct 1st

2018-10-01 Thread Howard Chu via Cyrus-devel
Bron Gondwana wrote: > Partha: > * Ran a bunch of YCSB benchmarks, but looked at LevelDB's benchmarks instead. If you're using LevelDB's benchmark tool, you might find this easier to use instead: https://github.com/lmdb/dbbench >  - compare to what they run against leveldb, kyoto, redis, etc.

Re: MMAP performance and using mmap writes

2018-11-30 Thread Howard Chu via Cyrus-devel
Bron Gondwana wrote: > Hi All, > > We were debugging the CPU usage in a ctl_conversationsdb rebuild yesterday, > and noticed an interesting thing.  70% of the CPU utilisation for this one > process > was inside the kernel!  Mostly with dirty pages. > > ctl_conversationsdb -R is particularly hea

Re: time for cyrus-imap v3.2?

2019-11-08 Thread Howard Chu via Cyrus-devel
Bron Gondwana wrote: > On Thu, Nov 7, 2019, at 15:46, Anatoli via Cyrus-devel wrote: >> Bron, >> >> Thanks for your detailed reply and the work the FM team is doing! >> >> > This is not easy unfortunately with all the different datastructures, >> > because it means that everything else which takes

Re: Sophia database => fail

2016-06-18 Thread Howard Chu via Cyrus-devel
Bron Gondwana via Cyrus-devel wrote: So I wrote a backend for this thing: http://sophia.systems/ Sorry you wasted your time. Even if you got that thing to "behave" as you intended, it would have crashed on you anyway. https://github.com/pmwkaa/sophia/issues/54 https://github.com/pmwkaa/soph

Re: Sophia database => fail

2016-06-18 Thread Howard Chu via Cyrus-devel
Howard Chu wrote: Bron Gondwana via Cyrus-devel wrote: So I wrote a backend for this thing: http://sophia.systems/ Sorry you wasted your time. Even if you got that thing to "behave" as you intended, it would have crashed on you anyway. https://github.com/pmwkaa/sophia/issues/54 https://githu

Re: Sophia database => fail

2016-06-18 Thread Howard Chu via Cyrus-devel
Bron Gondwana wrote: On Sun, Jun 19, 2016, at 07:49, Howard Chu wrote: Bron Gondwana via Cyrus-devel wrote: So I wrote a backend for this thing: http://sophia.systems/ Sorry you wasted your time. Even if you got that thing to "behave" as you intended, it would have crashed on you anyway. ht

Re: Sophia database => fail

2016-06-18 Thread Howard Chu via Cyrus-devel
Bron Gondwana wrote: On Sun, Jun 19, 2016, at 08:16, Howard Chu wrote: Not quite sure I understand your use case. It sounds like you'll maintain a local store, with recent records, and a remote store with older records. Is that correct? For the database I'd keep everything local, but I want to

Re: Sophia database => fail

2016-06-18 Thread Howard Chu via Cyrus-devel
Howard Chu via Cyrus-devel wrote: Bron Gondwana wrote: A good example of what I want is the way that the xapianactive file works in Cyrus search at FastMail: https://blog.fastmail.com/2014/12/01/email-search-system/ Because only the most recent database is writable (in this case on tmpfs

Re: Sophia database => fail

2016-06-20 Thread Howard Chu via Cyrus-devel
Thomas Jarosch via Cyrus-devel wrote: On Sunday, 19. June 2016 07:55:55 Bron Gondwana via Cyrus-devel wrote: I've also been looking at Rocks DB. Unfortunately their C wrapper (it's C++) doesn't seem to include the transaction support, so I'll have to either extend it or write my own. One major

Re: Sophia database => fail

2016-06-21 Thread Howard Chu via Cyrus-devel
Bron Gondwana via Cyrus-devel wrote: I'm going to try to build an LMDB backend. I do have some questions for Howard about how best to do it... :) The CyrusDB abstraction layer allows opening multiple databases by filename. Should we be opening one environment per file, or just a single environm