Re: First draft of logging functionality.
On Mon, 11 Oct 2010, da...@tethera.net wrote: The patches following this message are my first attempt at implementing atomic logging for notmuch. The idea is that such logs could be useful in synchronizing notmuch instances. Feedback of any kind is welcome. I'm particularly interested in comments about the log format and performance. Hi David, I'm not sure whether implementing logging facility outside of notmuch library is a good thing. If somebody will use a third-party tool (such as python bindings) to manipulate his tags, they won't appear in the log. -Michal ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: First draft of logging functionality.
On Tue, 12 Oct 2010, David Bremner wrote: On Tue, 12 Oct 2010 11:06:17 +0200, Michal Sojka sojk...@fel.cvut.cz wrote: I'm not sure whether implementing logging facility outside of notmuch library is a good thing. If somebody will use a third-party tool (such as python bindings) to manipulate his tags, they won't appear in the log. Yeah, thanks for that. I had been worrying about the same thing, but your message helped clarify things for me. I guess log files should be opened in notmuch_database_open, and the actual logging in this case could happen from notmuch_message_add_tag. From an atomicity point of view it might make more sense to store up a list of log lines, and dump them all from _notmuch_message_sync. One could attach a log buffer to a message, and flush that atomically when syncing the message back to the database. Yes, the above sounds good to me. In this context, it is a little more tedious to have more than one log file. Why multiple log files? You may have the buffers for message logs in memory, right? Another thing to keep in mind is how it will behave with 'notmuch restore'. If do dump followed by resore you will have a very long log with no useful information in it. Some optimization may be employed here. Bye -Michal ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: First draft of logging functionality.
da...@tethera.net writes: The patches following this message are my first attempt at implementing atomic logging for notmuch. The idea is that such logs could be useful in synchronizing notmuch instances. For this to be completely safe, I suspect it may need to be adjusted to do write ahead logging or something similar. Otherwise operations could be lost. i.e. notmuch would atomically and safely write the intended (tag) operation to the log, then perform the tag. On startup, notmuch would need to scan the log to detect and apply operations that hadn't been fully completed (presumably due to a crash). More generally, while thinking about sync/logging a few days ago, I wondered about using sqlite. That would help with atomicity, rollback, synchronizing multiple readers/writers, etc. It might also make operations more efficient once we implement all the features we want. For example, with the log information in sqlite, a separate notmuch sync to another machine could be reading from the log (and with limitations, writing) in parallel with normal notmuch operations. Depending on how we decide to handle sync with multiple peers, the log may also need to track which peers have seen what, prune appropriately, etc. Of course sqlite may not be appropriate, and would require performance testing, etc., but we should probably think about the features we'll eventually want, and consider how much work they're likely to require with any given approach, regardless. Hope this helps -- Rob Browning rlb @defaultvalue.org and @debian.org GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4 ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch