[notmuch] Segfault searching for tags
Getting the following segfault with 306635c2 on Fedora 12. Seems to
be happening with any 'tag:' search that returns results. For
example, 'notmuch search tag:inbox' and 'notmuch search tag:unread'
segfault but 'notmuch search tag:nosuchtag', 'notmuch search
subject:logwatch' and 'notmuch search video' seem to work fine.
Core was generated by `/usr/bin/notmuch search --sort=oldest-first tag:inbox'.
Program terminated with signal 11, Segmentation fault.
\#0 Xapian::TermIterator::operator* (this=)
at api/omtermlistiterator.cc:78
78 RETURN(internal->get_termname());
Current language: auto
The current source language is "auto; currently c++".
Thread 1 (Thread 15005):
\#0 Xapian::TermIterator::operator* (this=)
at api/omtermlistiterator.cc:78
No locals.
\#1 0x0040d213 in _notmuch_message_get_in_reply_to (message=0x1594f70)
at lib/message.cc:288
prefix = 0x415b77 "XREPLYTO"
prefix_len = 0
i = {internal = {dest = 0x0}}
in_reply_to = ""
\#2 0x0040f842 in _resolve_thread_relationships (thread=0x1595a00)
at lib/thread.cc:157
node = 0x1596130
message = 0x1594f70
parent = 0x7fff2cade9c8
prev = 0x1595cd0
in_reply_to =
\#3 _notmuch_thread_create (thread=0x1595a00) at lib/thread.cc:285
thread = 0x1595a00
thread_id_query = 0x158beb0
matched_query =
messages = 0x7fff2cade948
message =
thread_id_query_string =
matched_query_string =
\#4 0x0040f3d0 in notmuch_query_search_threads (
query=, first=,
max_threads=) at lib/query.cc:217
threads = 0x158b5f0
thread = 0x6e0077
messages = 0x158b7c0
message = 0x158c580
thread_id = 0x158b890 "2065b08615b4cbbb22d9ee874bb84d3e"
seen = 0x15454a0
messages_seen = 0
threads_seen = 0
\#5 0x004089a1 in do_search_threads (ctx=0x1543140, query=
0x7fff2cade8a0, sort=NOTMUCH_SORT_OLDEST_FIRST,
first=, max_threads=)
at notmuch-search.c:40
thread =
threads = 0x1551290
tags = 0x2
date =
relative_date = 0x2
\#6 0x00408ddd in notmuch_search_command (ctx=,
argc=1, argv=) at notmuch-search.c:156
config =
query = 0x158b510
query_str =
i = 1
first =
max_threads =
opt =
end = 0x0
sort =
\#7 0x0040636f in main (argc=4, argv=0x7fff2cadec98) at notmuch.c:400
local = 0x1543140
command =
--
Jeff Ollie
[notmuch] [PATCH] notmuch: Add tag archive when archiving the mails
On Wed, Nov 18, 2009 at 1:07 PM, Keith Packard wrote: > >> This enables us to search for archived mails later > > I think you can just use 'not tag:inbox' as 'archived' really just means > 'doesn't have the inbox tag'. That resulted in an error for me: $ notmuch search not tag:inbox A Xapian exception occurred: Syntax: NOT A Xapian exception occurred: Syntax: NOT -- Jeff Ollie
[notmuch] RPM spec file
On Wed, Nov 18, 2009 at 7:03 PM, Carl Worth wrote: > On Wed, 18 Nov 2009 12:31:53 -0600, "Jeffrey C. Ollie" > wrote: >> I prefere managing all of the software on my system with RPM, so I >> came up with a quick spec file for building RPMS. ?Once notmuch >> settles down and I continue using it I may submit notmuch to Fedora as >> an official package. ?If you'd like your own RPMS build your own, I >> don't plan on offering a repository of binary packages. > > This sounds great, Jeffrey. > > What makes the most sense for this file? Shall we just leave it in the > central repository? Or put it in its own rpm branch? I'm open to your > suggestions here. I don't think that a separate Git branch makes sense, but maybe stuff like this should be in a subdirectory like "packaging/fedora", since I imagine that there will eventually be a need for "packaging/debian" etc. as well. -- Jeff Ollie
[notmuch] [PATCH 2/2] .gitignore: Add common editor droppings
On Wed, Nov 18, 2009 at 11:39 PM, Karl Wiberg wrote: > > Since these patterns are specific to a user rather than a project, > shouldn't they go in the user's ignore file (~/.gitignore) rather than > the project's? If every user could be counted on to put them in their personal .gitignore probably not, but I think that it's good practice to at least ignore the stuff that emacs and vim use for temporary files and backups. -- Jeff Ollie
[notmuch] Segfault searching for tags
On Thu, Nov 19, 2009 at 9:45 AM, Adrian Perez de Castro wrote: > On Wed, 18 Nov 2009 12:00:10 -0600, Jeffrey wrote: > >> Getting the following segfault with 306635c2 on Fedora 12. ?Seems to >> be happening with any 'tag:' search that returns results. ?For >> example, 'notmuch search tag:inbox' and 'notmuch search tag:unread' >> segfault but 'notmuch search tag:nosuchtag', 'notmuch search >> subject:logwatch' and 'notmuch search video' seem to work fine. >> >> Core was generated by `/usr/bin/notmuch search --sort=oldest-first >> tag:inbox'. >> Program terminated with signal 11, Segmentation fault. >> \#0 ?Xapian::TermIterator::operator* (this=) >> ? ? at api/omtermlistiterator.cc:78 >> 78 ? ? ? ?RETURN(internal->get_termname()); >> Current language: ?auto >> The current source language is "auto; currently c++". > > I have hit what I believe is exactly the same problem. In my case, some > results are printed when I execute "notmuch search tag:inbox", and then > the program crashes in the same exact place. > > The thing is that in notmuch_message_get_in_reply_to(), line 288, a NULL > instance of Xapian::TermIterator is dereferenced. In my particular case, > the culpript is a cache file of Claws-Mail, as seen in the following GDB > session: > [...] > As you can see, there "filename" points to a Claws-Mail cache file, which > is a binary file (I can provide a copy if needed). I suspect that this is > related to the fact that the iterator ends up being NULL somehow. I straced some of the crashes, and the last file that was read before the crash was a malformed message. I've attached one of the messages. I've been using offlineimap to sync my gmail mailbox to my laptop so that I can use notmuch. offlineimap isn't the most stable program, but I'm not sure yet if offlineimap is causing the problem or if that's the way the message is in gmail. -- Jeff Ollie -- next part -- Delivered-To: jeff at ollie.clive.ia.us Received: by 10.90.86.18 with SMTP id j18cs228556agb; Thu, 5 Nov 2009 22:23:50 -0800 (PST) Received: by 10.90.16.38 with SMTP id 38mr7468290agp.112.1257488620374; Thu, 05 Nov 2009 22:23:40 -0800 (PST) Return-Path: Received: from 209.85.223.101 ([116.208.64.100]) by mx.google.com with SMTP id 41si9046203iwn.112.2009.11.05.22.23.38; Thu, 05 Nov 2009 22:23:40 -0800 (PST) Received-SPF: neutral (google.com: 116.208.64.100 is neither permitted nor denied by best guess record for domain of xhutteesjj at yahoo.com.au) client-ip=116.208.64.100; Authentication-Results: mx.google.com; spf=neutral (google.com: 116.208.64.100 is neither permitted nor denied by best guess record for domain of xhutteesjj at yahoo.com.au) smtp.mail=xhutteesjj at yahoo.com.au Date: Thu, 05 Nov 2009 22:23:40 -0800 (PST) Received: from 146.2.76.118 by 116.208.64.100; Fri, 06 Nov 2009 03:20:42 -0300 Message-ID:
[notmuch] Segfault searching for tags
On Fri, Nov 20, 2009 at 5:32 AM, Carl Worth wrote: > On Thu, 19 Nov 2009 16:45:43 +0100, Adrian Perez de Castro igalia.com> wrote: >> The thing is that in notmuch_message_get_in_reply_to(), line 288, a NULL >> instance of Xapian::TermIterator is dereferenced. In my particular case, >> the culpript is a cache file of Claws-Mail, as seen in the following GDB >> session: > > Not quite NULL, (nor is it quite dereferencing---this is nasty C++ > overloading), but yeah, the idea is the same. We need to protect all of > our "calls" to this overloaded operator to not call it when the iterator > is equal to the value returned by termlist_end (). > > On Thu, 19 Nov 2009 20:23:15 -0600, Jeffrey Ollie wrote: >> I straced some of the crashes, and the last file that was read before >> the crash was a malformed message. ?I've attached one of the messages. > > Thanks for the file. I never like to push code that I haven't tested, so > this was very helpful. > > Below is the patch that I just pushed which seems to do the trick. Ah, excellent! This does indeed seem to prevent the crash. Now I just need to figure out how to get all my mail out of GMail. -- Jeff Ollie
[notmuch] [PATCH] Missing final semi-colon in .desktop's Categories.
Found the same problem on my own, only to minutes later catch up on the backlog of patches in my email and find this! On Sat, Nov 21, 2009 at 3:13 AM, James Rowe wrote: > "Those keys which have several values should have a semicolon as the trailing > character." > ?-- http://standards.freedesktop.org/desktop-entry-spec/1.0/ar01s03.html > > Signed-off-by: James Rowe Reviewed-by: Jeffrey C. Ollie > --- > ?notmuch.desktop | ? ?2 +- > ?1 files changed, 1 insertions(+), 1 deletions(-) > > diff --git a/notmuch.desktop b/notmuch.desktop > index d29dff6..819cd1e 100644 > --- a/notmuch.desktop > +++ b/notmuch.desktop > @@ -4,4 +4,4 @@ Exec=emacs -f notmuch > ?Icon=emblem-mail > ?Terminal=false > ?Type=Application > -Categories=Network;Email > +Categories=Network;Email; > -- > 1.6.4.4 > > ___ > notmuch mailing list > notmuch at notmuchmail.org > http://notmuchmail.org/mailman/listinfo/notmuch > -- Jeff Ollie
[notmuch] [PATCH] Add SCons build files.
The SCons build files are not meant to require emacs. If I've messed something up and emacs is somehow required I would consider that a bug and would try to fix it. As far as availability, I'm sure that SCons is one yum or apt-get or away. The only people that would face more than a minor inconvenience are people that prefer to download tarballs and compile and install things themselves. Yes, I'm sure that make is widely available, but as notmuch gets used on a wider variety of systems some sort of configuration system will become necessary. If I can prevent another project from going down the autoconf/automake path I'll be happy. I started creating CMake build files but I don't know CMake well enough to come up with a working build. On 11/22/09, Bart Trojanowski wrote: > * Jeffrey C. Ollie [091122 08:47]: >> The SCons build files included here *should* do everything that the >> current Makefiles do, plus a little bit of configuration checking. To >> build/install: > > Wouldn't that have the unfortunate side effect of making notmuch > unusable w/o emacs and scons installed? > > GNU make has a much wider install base, and notmuch is still very useful > without emacs. > > -Bart > > -- > WebSig: http://www.jukie.net/~bart/sig/ > -- Sent from my mobile device Jeff Ollie
[notmuch] [PATCH] Add SCons build files.
On Sun, Nov 22, 2009 at 9:11 PM, Carl Worth wrote: > > But I'm afraid I really don't want to switch away from just using (GNU) > make for the actual compilation. > > I don't know anything about scons, but if you can use it to write a > python script that just does the configuration step, (outputting a > Makefile.config say), then that might be very interesting. Well, to me, part of the appeal of SCons is that it doesn't generate Makefiles.This article is a little old, but the first three sections are a pretty good rundown on the limitations of Make. http://www.scons.org/wiki/FromMakeToScons If you really want a tool that generates Makefiles, take a look at CMake. Unfortunately, I think CMake tries to do too much with the Makefiles and they become close to unreadable. In the attempt I made to come up with CMake build files, CMake generates at least 3800 lines of Makefiles (from a starting point of 30 lines of build files). > Some people > have recently told me that python would be a much more sane language for > doing configuration in than shell. Well, Python is a much more sane language for doing just about anything as far as I'm concerned. :) > I don't know if they're right or not, but I'm (somewhat) willing to have > multiple implementations of the configure script (since there's always > the option to just skip it and configure Makefile.config manually). But > I'm definitely not willing to have multiple build systems. I think that having multiple different configuration systems would be pretty awful, especially if people make changes to their favourite configuration system and hope that someone else fixes the others. > Meanwhile, the only advantage I know for automake is that once it's > setup, adding a new file to compile is as simple as adding one file to a > list in the Makefile.am. We've already got notmuch as easy as that with > just adding a file to a list in Makefile.local. Once SCons is set up, adding a new file is just a matter of adding the name of the file to a list as well. CMake is the same, and I'm sure that other systems are similar. > So I'd very much like to continue exploring what we can do with our own > configuration system, (in whatever language/language(s) make sense). It would seem to me that we would be re-inventing a lot of the work already done by other people. > Thanks for listening, Same here. -- Jeff Ollie
[notmuch] [PATCH] fix compiler warnings
On Sun, Nov 22, 2009 at 9:22 PM, Carl Worth wrote:
> On Sun, 22 Nov 2009 17:11:03 +0200, Dirk-Jan C. Binnema gmail.com> wrote:
>>
>> - ? ?write(2, msg, sizeof(msg)-1);
>> + ? ?if (write(2, msg, sizeof(msg)-1) < 0) {
>> + ? ? ? ? /* ignore...*/
>> + ? ?}
>
> I don't like the gratuitous conditional here. It clutters the code and
> make is less clear. If we're just trying to squelch a warning about an
> unused return value from a function, then I think I'd rather see:
>
> ? ? ? ?ssize_t ignored;
>
> ? ? ? ?ignored = write (2, msg, sizeof (msg) - 1);
Isn't the usual method for ignoring return values casting to void?
(void) write (2, msg, sizeof (msg) - 1);
--
Jeff Ollie
[notmuch] [PATCH -v4] notmuch.el: Add face support to search and show mode
On Tue, Nov 24, 2009 at 11:24 PM, Alexander Botero-Lowry wrote: > > This way of detecting the tags seems ok, but I think it would be nicer > if it could be done even more deterministically. :) One idea that be > neat is to have a --format=sexp for notmuch search, which exports sexps > (probably alists, but could be some other format) for the search results > that can just be eval'd and processed in a cleaner way (and would also > make for nicer APIs in emacs for querying notmuch itself). Actually I > really like the idea of a sexp output mode for show too, instead of the > markers *plots* Along the same lines, I was thinking that it would be nice to be able to get output formatted in JSON as well for possible use in a web front-end. Actually, it looks like Emacs has support for JSON as well[1]. Not sure what release it was introduced in, but I see that it's in 23.1. http://edward.oconnor.cx/2006/03/json.el -- Jeff Ollie
[notmuch] Using notmuch from Ruby, Python, et al.
On Wed, Nov 25, 2009 at 10:16 AM, Scott Robinson wrote: > > == Shared Library > > I have modified the build to output a shared library. It's a small change > relying on libtool. > > However, the archives indicate Carl dislikes libtool? Ugh, please, let's not go down the autoconf/automake/libtool route. > == JSON output > > I added a --output=(text|json) option to "search" and "show." Then, emit json > results appropriately. > > To accomplish this, I used the cJSON library sans parsing logic. > > However, I saw discussion about providing a --format option? Awesome, I was just looking at using json-glib but I haven't gotten very far yet, mainly because I've never used it before (nor done much of anything with JSON). I was going to call the option '--output-format=(standard|json)' but I'm not going to worry about the color of that bikeshed. IMHO, I think both approaches would be valuable, that way people writing front ends could choose the methods that worked best for them. Do you have patches that you are ready to share? -- Jeff Ollie
[notmuch] Using notmuch from Ruby, Python, et al.
On Wed, Nov 25, 2009 at 10:16 AM, Scott Robinson wrote: > > To accomplish this, I used the cJSON library sans parsing logic. I wasn't familiar with cJSON so I went over to take a look. I haven't really examined the code, but I really don't like the fact that you are expected to copy the cJSON code into your project. -- Jeff Ollie
[notmuch] Using notmuch from Ruby, Python, et al.
On Wed, Nov 25, 2009 at 11:40 AM, Scott Robinson wrote: > > As the json work is a bit more substantial, is it preferable to start a > [PATCH] > thread or link to a git repo? Probably a new [PATCH] thread so that the changes can be discussed. -- Jeff Ollie
[notmuch] Using notmuch from Ruby, Python, et al.
On Wed, Nov 25, 2009 at 11:58 AM, Scott Robinson wrote: > > I selected cJSON because it was simple for integration. Given how libsha1.c is > included in lib/, I figured adding a depedency wouldn't be very popular. Oh, I noticed libsha1.c as well, and it bugs the heck out of me. I just haven't gotten around to doing anything about it yet. > Would it be better to link with a previously existing library? IMHO yes. I was looking at json-glib because it seems to be the only C JSON API in Fedora, plus it's based upon glib. From what I've read it should be really easy to serialize a GMimeObject to a JSON structure with very few calls using json-glib. -- Jeff Ollie
[notmuch] Problems importing my mail...
I'm having some problems importing my mail. I've got quite a bit stored up, and some of them I'm sure are quite large. After several hours I get the following. Is notmuch running out of memory on me? $ ./notmuch new Found 328184 total files. Warning: Unexpected extra parts of multipart/signed. Indexing anyway. Note: Ignoring non-mail file: /home/jeff/mail/message/6/5/65c74c15a686187bb6bbf9958f494fc6b80068034a659a9ad44991b08c58f2d2 Note: Ignoring non-mail file: /home/jeff/mail/message/7/9/7902699be42c8a8e46fbbb4501726517e86b22c56a189f7625a6da49081b2451 Note: Ignoring non-mail file: /home/jeff/mail/message/8/0/802071f7fcd8b0b74a19e1ca64e5468184fee0c9171bacb77ae1fe1669c426ee A Xapian exception occurred creating message: Db block overwritten - are there multiple writers? Error: A Xapian exception occurred. Halting processing. Internal error: Message with document ID of 175013 has no thread ID. (lib/message.cc:353). [jeff at max1 notmuch]$ ./notmuch new Internal error: Message with document ID of 175013 has no thread ID. (lib/message.cc:353). -- Jeff Ollie
[notmuch] [PATCH] Use libgcrypt for hashing.
On Fri, Nov 27, 2009 at 9:31 PM, Mikhail Gusarov wrote: > Twas brillig at 21:28:03 27.11.2009 UTC-06 when jeff at ocjtech.us did gyre > and gimble: > > ?JCO> Instead of including a private implementation of the SHA1 hash > > xserver went this road, and now it has > --with-sha1=libc|libmd|libgcrypt|libcrypto|libsha1|CommonCrypto in > configure. I doubt that we'll get to that level of insanity. > ?JCO> This means less code of our own to maintain and > > As libsha1 maintainer I'm volunteering to maintain in-tree copy in > notmuch :) That's great that you're willing to take on the task, but as I do a lot of work for Fedora I tend to think about these things differently. It's not about a project here or there making private copies of some code, it's about tracking down *all* of the projects that have private copies of the code when something goes wrong, especially when there are security implications. The most famous example is Zlib. Based upon what Google turned up I see that you've been around long enough that you should have heard of the problems the computer world had when security flaws turned up in Zlib. There's even a project[1] that developed methods for identifying vulnerable code in binaries the problem was so pervasive. A more recent example is a cross-site ajax request vulnerability in the Prototype JavaScript framework[2]. A lot of projects copied that code into their repositories as well, and now someone has to track all of those down and get them fixed as well. When projects copy code into their repositories like this, it makes things a lot more difficult for someone else down the road. Both Fedora[3] and Debian[4] (and Ubuntu by extension), while not expressly forbidding the practice, *strongly* recommend against it. I'd really recommend reading the Fedora page that explains the policy as it has a great summation of the problem. [1] http://www.enyo.de/fw/security/zlib-fingerprint/ [2] http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2008-7220 [3] https://fedoraproject.org/wiki/Packaging:No_Bundled_Libraries [4] http://www.debian.org/doc/debian-policy/ch-source.html#s-embeddedfiles -- Jeff Ollie
[notmuch] [PATCH] Use libgcrypt for hashing.
On Fri, Nov 27, 2009 at 9:59 PM, Ingmar Vanhassel wrote: > > Most distributions have a rather strict policy to use system libraries > over internal copies. Fedora: https://fedoraproject.org/wiki/Packaging:No_Bundled_Libraries Debian: http://www.debian.org/doc/debian-policy/ch-source.html#s-embeddedfiles If there are other distributions out there that have similar policies I'd be interested in learning about them. -- Jeff Ollie
[notmuch] [PATCH] notmuch-new: Eliminate tallocs whilst construct filenames.
On Fri, Nov 27, 2009 at 11:41 PM, Carl Worth wrote: > > But yes, we need a test suite. I have zero experience, but Check[1] looks interesting. > Oh, and we'll also need to deal with remaining glib usage inside of > notmuch, (and inside of GMime as well), before we can do good testing > for memory-fault injection. Maybe what we'll end up with is a patch to > de-glib-ify GMime? I'm not sure. Rather than de-glib-ify'ing GMime wouldn't it be better to find a non glib-based library for MIME processing? I do most of my development in Python these days so I don't know of any examples. My googling turned up several C++ libraries or c-client and I wouldn't want to go down either of those paths. [1] http://check.sourceforge.net/ -- Jeff Ollie
[notmuch] Problems importing my mail...
On Sat, Nov 28, 2009 at 12:09 AM, Carl Worth wrote: > On Fri, 27 Nov 2009 19:09:56 -0600, Jeffrey Ollie wrote: >> >> $ ./notmuch new >> Found 328184 total files. > > That's certainly not the largest number of messages we've seen indexed > successfully by notmuch, (I think Keith has near 3 times that > number). [Maybe notmuch should be reporting the total size of the mail > store as well...] Heh, I'm not done downloading them all yet, but I doubt that I'll hit the 1M mark, maybe 500-600K. >> Warning: Unexpected extra parts of multipart/signed. Indexing anyway. > > Oh, that's a warning I put in place because I wasn't sure if it was > legitimate for a multipart/signed message to have more than two > parts. I'd actually be interested to know if the mail is correct, (and I > should just eliminate the warning), or if the mail is somehow malformed > and the warning is correct. No, I think it's legitimate to have multiple parts inside of a multipart/signed (just very rare). I've identified the message that caused the warning. I'm including it as an attachment, hopefully it won't get tagged as spam because it's a response to a spam report that I sent a while back. >> Note: Ignoring non-mail file: >> /home/jeff/mail/message/6/5/65c74c15a686187bb6bbf9958f494fc6b80068034a659a9ad44991b08c58f2d2 >> Note: Ignoring non-mail file: >> /home/jeff/mail/message/7/9/7902699be42c8a8e46fbbb4501726517e86b22c56a189f7625a6da49081b2451 >> Note: Ignoring non-mail file: >> /home/jeff/mail/message/8/0/802071f7fcd8b0b74a19e1ca64e5468184fee0c9171bacb77ae1fe1669c426ee > > Those you should check to see if they actually do look like mail > messages. Notmuch decides to ignore a file when it can't find any of the > following headers: Subject:, From:, not To:. Yes, all of those appear to not be complete mail messages, why they are in one of my IMAP servers remains to be seen. >> A Xapian exception occurred creating message: Db block overwritten - >> are there multiple writers? >> Error: A Xapian exception occurred. Halting processing. > > That's an error I've never seen before. We might want to talk to the > Xapian folks to see what that could be. There's really no way there can > be multiple writers here. So I don't know what the actual problem might > be. > >> Internal error: Message with document ID of 175013 has no thread ID. >> ?(lib/message.cc:353). >> [jeff at max1 notmuch]$ ./notmuch new >> Internal error: Message with document ID of 175013 has no thread ID. >> ?(lib/message.cc:353). > > Hmm... we could probably do better here. The fatal error you're getting > here is for an invariant that notmuch thinks is fairly important, (no > mail document should exist without a thread ID). Meanwhile, however when > adding a new message we do actually create a mail document in the > database, and only later resolve its thread ID and add that to the > database as well. A better solution would be to resolve the thread ID > before adding anything to the database so that this invariant would > never be violated. > > Some people have been proposing a "notmuch gc" command or so for > cleaning up problems like this. > > In the meantime, you could explore the current state of your database by > changing the code that's currently giving you an internal error to > instead return a fake thread ID. For example: > > ? ?if (i == message->doc.termlist_end () || id[0] != *prefix) > ? ? ? ?message->thread_id = talloc_strdup (message, > ""); > ? ?else > ? ? ? ?message->thread_id = talloc_strdup (message, id.c_str () + 1); Unfortunately I deleted the database and am in the process of recreating it with the verbose flag turned on. So far the problem has not occurred again. So if there's a real bug somewhere I'm wondering if there isn't a timing-related component to it. -- Jeff Ollie -- next part -- A non-text attachment was scrubbed... Name: multipart.zip Type: application/zip Size: 5446 bytes Desc: not available URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20091128/57e7b106/attachment-0001.zip>
[notmuch] [PATCH] Use libgcrypt for hashing.
On Sat, Nov 28, 2009 at 12:43 AM, Carl Worth wrote: > > Have you actually *looked* at the implementation of libsha1.c that we > have in notmuch? I can't say with 100% certainty that it's free of any > buffer overruns, but I can see that it's not doing any memory allocation > nor network communication. So there are entire classes of security > problems, (such as have afflicted libraries in your examples), that just > aren't present here. I've looked at the code, if only briefly. But you're wrong that the code doesn't do any "network communication" - we feed libsha1 hostile data every time we take the hash of a message. > And as for security compromises due to a bug in the cryptographic nature > of this function---well, notmuch isn't even *using* SHA-1 for any secure > purpose. >From a distributor's point of view, it doesn't matter what you use the code for, it only matters that it has the bug and someone has to spend the time to track down all of the copies of the code and replace the code with a fixed version. If the code is confined to one shared library it's trivial to update the shared library, if the code has been copied to N packages, it's at least N times the work to verify that all of those packages get updated. > The actual functionality that we need here is *so* small that I am > unwilling to introduce a required dependency on any library as large as > libcrypt. I mean, look at the actual sizes we're talking about If libgcrypt were some obscure library that wasn't already packaged up by your favorite distribution or took up hundreds of megabytes of RAM and/or disk you might have a point. But the fact is that it *doesn't matter* how big libgcrypt is because we essentially get it for free - I'd bet that libgcrypt is already installed on most people's systems. As a test I tried to remove libgcrypt from my laptop - if I actually had gone though with it I would have crippled my system because things like the X server and the package manager depend (albeit indirectly) on it. > Now, if somebody wanted to maintain libsha1 inside a distribution like > Debian, say, then I'd be happy to link against that version rather than > a locally compiled version. And like I said earlier, if people would > rather link against a large cyptographic library for this one tiny > function, then we could arrange that too, but I don't think that > justifies dropping this code from notmuch and introducing a hard > dependency. Given Debian's reputation for packaging the kitchen sink I'm surprised that it doesn't already. Fedora tends not to package libraries unless there is an application that is going to make use of it... -- Jeff Ollie
[notmuch] [PATCH v3] Add the beginnings of a test suite.
On Sun, Nov 29, 2009 at 8:08 AM, Jameson Graef Rollins wrote: > On Sat, Nov 28, 2009 at 03:33:14PM -0600, Jeffrey C. Ollie wrote: >> Run "make check" from the toplevel directory to build and run the >> checks. > > I think it's usually standard to call the tests with "make test", yes? > Is there a reason to call them with "make check" instead? There's no special reason to call it "make check". If that's the consensus I'm willing to change it. -- Jeff Ollie
