[notmuch] [PATCH v3] Add the beginnings of a test suite.

2009-11-29 Thread Jeffrey Ollie
On Sun, Nov 29, 2009 at 8:08 AM, Jameson Graef Rollins
 wrote:
> On Sat, Nov 28, 2009 at 03:33:14PM -0600, Jeffrey C. Ollie wrote:
>> Run "make check" from the toplevel directory to build and run the
>> checks.
>
> I think it's usually standard to call the tests with "make test", yes?
> Is there a reason to call them with "make check" instead?

There's no special reason to call it "make check".  If that's the
consensus I'm willing to change it.

-- 
Jeff Ollie


[notmuch] [PATCH] Use libgcrypt for hashing.

2009-11-28 Thread Jeffrey Ollie
On Sat, Nov 28, 2009 at 12:43 AM, Carl Worth  wrote:
>
> Have you actually *looked* at the implementation of libsha1.c that we
> have in notmuch? I can't say with 100% certainty that it's free of any
> buffer overruns, but I can see that it's not doing any memory allocation
> nor network communication. So there are entire classes of security
> problems, (such as have afflicted libraries in your examples), that just
> aren't present here.

I've looked at the code, if only briefly.  But you're wrong that the
code doesn't do any "network communication" - we feed libsha1 hostile
data every time we take the hash of a message.

> And as for security compromises due to a bug in the cryptographic nature
> of this function---well, notmuch isn't even *using* SHA-1 for any secure
> purpose.

>From a distributor's point of view, it doesn't matter what you use the
code for, it only matters that it has the bug and someone has to spend
the time to track down all of the copies of the code and replace the
code with a fixed version.  If the code is confined to one shared
library it's trivial to update the shared library, if the code has
been copied to N packages, it's at least N times the work to verify
that all of those packages get updated.

> The actual functionality that we need here is *so* small that I am
> unwilling to introduce a required dependency on any library as large as
> libcrypt. I mean, look at the actual sizes we're talking about

If libgcrypt were some obscure library that wasn't already packaged up
by your favorite distribution or took up hundreds of megabytes of RAM
and/or disk you might have a point.  But the fact is that it *doesn't
matter* how big libgcrypt is because we essentially get it for free -
I'd bet that libgcrypt is already installed on most people's systems.
As a test I tried to remove libgcrypt from my laptop - if I actually
had gone though with it I would have crippled my system because things
like the X server and the package manager depend (albeit indirectly)
on it.

> Now, if somebody wanted to maintain libsha1 inside a distribution like
> Debian, say, then I'd be happy to link against that version rather than
> a locally compiled version. And like I said earlier, if people would
> rather link against a large cyptographic library for this one tiny
> function, then we could arrange that too, but I don't think that
> justifies dropping this code from notmuch and introducing a hard
> dependency.

Given Debian's reputation for packaging the kitchen sink I'm surprised
that it doesn't already.  Fedora tends not to package libraries unless
there is an application that is going to make use of it...

-- 
Jeff Ollie


[notmuch] Problems importing my mail...

2009-11-28 Thread Jeffrey Ollie
On Sat, Nov 28, 2009 at 12:09 AM, Carl Worth  wrote:
> On Fri, 27 Nov 2009 19:09:56 -0600, Jeffrey Ollie  wrote:
>>
>> $ ./notmuch new
>> Found 328184 total files.
>
> That's certainly not the largest number of messages we've seen indexed
> successfully by notmuch, (I think Keith has near 3 times that
> number). [Maybe notmuch should be reporting the total size of the mail
> store as well...]

Heh, I'm not done downloading them all yet, but I doubt that I'll hit
the 1M mark, maybe 500-600K.

>> Warning: Unexpected extra parts of multipart/signed. Indexing anyway.
>
> Oh, that's a warning I put in place because I wasn't sure if it was
> legitimate for a multipart/signed message to have more than two
> parts. I'd actually be interested to know if the mail is correct, (and I
> should just eliminate the warning), or if the mail is somehow malformed
> and the warning is correct.

No, I think it's legitimate to have multiple parts inside of a
multipart/signed (just very rare).  I've identified the message that
caused the warning.  I'm including it as an attachment, hopefully it
won't get tagged as spam because it's a response to a spam report that
I sent a while back.

>> Note: Ignoring non-mail file:
>> /home/jeff/mail/message/6/5/65c74c15a686187bb6bbf9958f494fc6b80068034a659a9ad44991b08c58f2d2
>> Note: Ignoring non-mail file:
>> /home/jeff/mail/message/7/9/7902699be42c8a8e46fbbb4501726517e86b22c56a189f7625a6da49081b2451
>> Note: Ignoring non-mail file:
>> /home/jeff/mail/message/8/0/802071f7fcd8b0b74a19e1ca64e5468184fee0c9171bacb77ae1fe1669c426ee
>
> Those you should check to see if they actually do look like mail
> messages. Notmuch decides to ignore a file when it can't find any of the
> following headers: Subject:, From:, not To:.

Yes, all of those appear to not be complete mail messages, why they
are in one of my IMAP servers remains to be seen.

>> A Xapian exception occurred creating message: Db block overwritten -
>> are there multiple writers?
>> Error: A Xapian exception occurred. Halting processing.
>
> That's an error I've never seen before. We might want to talk to the
> Xapian folks to see what that could be. There's really no way there can
> be multiple writers here. So I don't know what the actual problem might
> be.
>
>> Internal error: Message with document ID of 175013 has no thread ID.
>> ?(lib/message.cc:353).
>> [jeff at max1 notmuch]$ ./notmuch new
>> Internal error: Message with document ID of 175013 has no thread ID.
>> ?(lib/message.cc:353).
>
> Hmm... we could probably do better here. The fatal error you're getting
> here is for an invariant that notmuch thinks is fairly important, (no
> mail document should exist without a thread ID). Meanwhile, however when
> adding a new message we do actually create a mail document in the
> database, and only later resolve its thread ID and add that to the
> database as well. A better solution would be to resolve the thread ID
> before adding anything to the database so that this invariant would
> never be violated.
>
> Some people have been proposing a "notmuch gc" command or so for
> cleaning up problems like this.
>
> In the meantime, you could explore the current state of your database by
> changing the code that's currently giving you an internal error to
> instead return a fake thread ID. For example:
>
> ? ?if (i == message->doc.termlist_end () || id[0] != *prefix)
> ? ? ? ?message->thread_id = talloc_strdup (message, 
> "");
> ? ?else
> ? ? ? ?message->thread_id = talloc_strdup (message, id.c_str () + 1);

Unfortunately I deleted the database and am in the process of
recreating it with the verbose flag turned on.  So far the problem has
not occurred again.  So if there's a real bug somewhere I'm wondering
if there isn't a timing-related component to it.

-- 
Jeff Ollie
-- next part --
A non-text attachment was scrubbed...
Name: multipart.zip
Type: application/zip
Size: 5446 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20091128/57e7b106/attachment-0001.zip>


[notmuch] [PATCH] notmuch-new: Eliminate tallocs whilst construct filenames.

2009-11-27 Thread Jeffrey Ollie
On Fri, Nov 27, 2009 at 11:41 PM, Carl Worth  wrote:
>
> But yes, we need a test suite.

I have zero experience, but Check[1] looks interesting.

> Oh, and we'll also need to deal with remaining glib usage inside of
> notmuch, (and inside of GMime as well), before we can do good testing
> for memory-fault injection. Maybe what we'll end up with is a patch to
> de-glib-ify GMime? I'm not sure.

Rather than de-glib-ify'ing GMime wouldn't it be better to find a non
glib-based library for MIME processing?  I do most of my development
in Python these days so I don't know of any examples.  My googling
turned up several C++ libraries or c-client and I wouldn't want to go
down either of those paths.

[1] http://check.sourceforge.net/

-- 
Jeff Ollie


[notmuch] [PATCH] Use libgcrypt for hashing.

2009-11-27 Thread Jeffrey Ollie
On Fri, Nov 27, 2009 at 9:59 PM, Ingmar Vanhassel  wrote:
>
> Most distributions have a rather strict policy to use system libraries
> over internal copies.

Fedora:

https://fedoraproject.org/wiki/Packaging:No_Bundled_Libraries

Debian:

http://www.debian.org/doc/debian-policy/ch-source.html#s-embeddedfiles

If there are other distributions out there that have similar policies
I'd be interested in learning about them.

-- 
Jeff Ollie


[notmuch] [PATCH] Use libgcrypt for hashing.

2009-11-27 Thread Jeffrey Ollie
On Fri, Nov 27, 2009 at 9:31 PM, Mikhail Gusarov
 wrote:
> Twas brillig at 21:28:03 27.11.2009 UTC-06 when jeff at ocjtech.us did gyre 
> and gimble:
>
> ?JCO> Instead of including a private implementation of the SHA1 hash
>
> xserver went this road, and now it has
> --with-sha1=libc|libmd|libgcrypt|libcrypto|libsha1|CommonCrypto in
> configure.

I doubt that we'll get to that level of insanity.

> ?JCO> This means less code of our own to maintain and
>
> As libsha1 maintainer I'm volunteering to maintain in-tree copy in
> notmuch :)

That's great that you're willing to take on the task, but as I do a
lot of work for Fedora I tend to think about these things differently.
 It's not about a project here or there making private copies of some
code, it's about tracking down *all* of the projects that have private
copies of the code when something goes wrong, especially when there
are security implications.

The most famous example is Zlib.  Based upon what Google turned up I
see that you've been around long enough that you should have heard of
the problems the computer world had when security flaws turned up in
Zlib.  There's even a project[1] that developed methods for
identifying vulnerable code in binaries the problem was so pervasive.

A more recent example is a cross-site ajax request vulnerability in
the Prototype JavaScript framework[2].  A lot of projects copied that
code into their repositories as well, and now someone has to track all
of those down and get them fixed as well.

When projects copy code into their repositories like this, it makes
things a lot more difficult for someone else down the road.  Both
Fedora[3] and Debian[4] (and Ubuntu by extension), while not expressly
forbidding the practice, *strongly* recommend against it.  I'd really
recommend reading the Fedora page that explains the policy as it has a
great summation of the problem.

[1] http://www.enyo.de/fw/security/zlib-fingerprint/
[2] http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2008-7220
[3] https://fedoraproject.org/wiki/Packaging:No_Bundled_Libraries
[4] http://www.debian.org/doc/debian-policy/ch-source.html#s-embeddedfiles

-- 
Jeff Ollie


[notmuch] Problems importing my mail...

2009-11-27 Thread Jeffrey Ollie
I'm having some problems importing my mail.  I've got quite a bit
stored up, and some of them I'm sure are quite large.  After several
hours I get the following.  Is notmuch running out of memory on me?

$ ./notmuch new
Found 328184 total files.
Warning: Unexpected extra parts of multipart/signed. Indexing anyway.
Note: Ignoring non-mail file:
/home/jeff/mail/message/6/5/65c74c15a686187bb6bbf9958f494fc6b80068034a659a9ad44991b08c58f2d2
Note: Ignoring non-mail file:
/home/jeff/mail/message/7/9/7902699be42c8a8e46fbbb4501726517e86b22c56a189f7625a6da49081b2451
Note: Ignoring non-mail file:
/home/jeff/mail/message/8/0/802071f7fcd8b0b74a19e1ca64e5468184fee0c9171bacb77ae1fe1669c426ee
A Xapian exception occurred creating message: Db block overwritten -
are there multiple writers?
Error: A Xapian exception occurred. Halting processing.
Internal error: Message with document ID of 175013 has no thread ID.
 (lib/message.cc:353).
[jeff at max1 notmuch]$ ./notmuch new
Internal error: Message with document ID of 175013 has no thread ID.
 (lib/message.cc:353).

-- 
Jeff Ollie


Re: [notmuch] [PATCH] Use libgcrypt for hashing.

2009-11-27 Thread Jeffrey Ollie
On Fri, Nov 27, 2009 at 9:31 PM, Mikhail Gusarov
dotted...@dottedmag.net wrote:
 Twas brillig at 21:28:03 27.11.2009 UTC-06 when j...@ocjtech.us did gyre and 
 gimble:

  JCO Instead of including a private implementation of the SHA1 hash

 xserver went this road, and now it has
 --with-sha1=libc|libmd|libgcrypt|libcrypto|libsha1|CommonCrypto in
 configure.

I doubt that we'll get to that level of insanity.

  JCO This means less code of our own to maintain and

 As libsha1 maintainer I'm volunteering to maintain in-tree copy in
 notmuch :)

That's great that you're willing to take on the task, but as I do a
lot of work for Fedora I tend to think about these things differently.
 It's not about a project here or there making private copies of some
code, it's about tracking down *all* of the projects that have private
copies of the code when something goes wrong, especially when there
are security implications.

The most famous example is Zlib.  Based upon what Google turned up I
see that you've been around long enough that you should have heard of
the problems the computer world had when security flaws turned up in
Zlib.  There's even a project[1] that developed methods for
identifying vulnerable code in binaries the problem was so pervasive.

A more recent example is a cross-site ajax request vulnerability in
the Prototype JavaScript framework[2].  A lot of projects copied that
code into their repositories as well, and now someone has to track all
of those down and get them fixed as well.

When projects copy code into their repositories like this, it makes
things a lot more difficult for someone else down the road.  Both
Fedora[3] and Debian[4] (and Ubuntu by extension), while not expressly
forbidding the practice, *strongly* recommend against it.  I'd really
recommend reading the Fedora page that explains the policy as it has a
great summation of the problem.

[1] http://www.enyo.de/fw/security/zlib-fingerprint/
[2] http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2008-7220
[3] https://fedoraproject.org/wiki/Packaging:No_Bundled_Libraries
[4] http://www.debian.org/doc/debian-policy/ch-source.html#s-embeddedfiles

-- 
Jeff Ollie
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] [PATCH] Use libgcrypt for hashing.

2009-11-27 Thread Jeffrey Ollie
On Fri, Nov 27, 2009 at 9:59 PM, Ingmar Vanhassel ing...@exherbo.org wrote:

 Most distributions have a rather strict policy to use system libraries
 over internal copies.

Fedora:

https://fedoraproject.org/wiki/Packaging:No_Bundled_Libraries

Debian:

http://www.debian.org/doc/debian-policy/ch-source.html#s-embeddedfiles

If there are other distributions out there that have similar policies
I'd be interested in learning about them.

-- 
Jeff Ollie
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[notmuch] Using notmuch from Ruby, Python, et al.

2009-11-25 Thread Jeffrey Ollie
On Wed, Nov 25, 2009 at 11:58 AM, Scott Robinson  wrote:
>
> I selected cJSON because it was simple for integration. Given how libsha1.c is
> included in lib/, I figured adding a depedency wouldn't be very popular.

Oh, I noticed libsha1.c as well, and it bugs the heck out of me.  I
just haven't gotten around to doing anything about it yet.

> Would it be better to link with a previously existing library?

IMHO yes.  I was looking at json-glib because it seems to be the only
C JSON API in Fedora, plus it's based upon glib.  From what I've read
it should be really easy to serialize a GMimeObject to a JSON
structure with very few calls using json-glib.

-- 
Jeff Ollie


[notmuch] Using notmuch from Ruby, Python, et al.

2009-11-25 Thread Jeffrey Ollie
On Wed, Nov 25, 2009 at 11:40 AM, Scott Robinson  wrote:
>
> As the json work is a bit more substantial, is it preferable to start a 
> [PATCH]
> thread or link to a git repo?

Probably a new [PATCH] thread so that the changes can be discussed.

-- 
Jeff Ollie


[notmuch] Using notmuch from Ruby, Python, et al.

2009-11-25 Thread Jeffrey Ollie
On Wed, Nov 25, 2009 at 10:16 AM, Scott Robinson  wrote:
>
> To accomplish this, I used the cJSON library sans parsing logic.

I wasn't familiar with cJSON so I went over to take a look.  I haven't
really examined the code, but I really don't like the fact that you
are expected to copy the cJSON code into your project.

-- 
Jeff Ollie


[notmuch] Using notmuch from Ruby, Python, et al.

2009-11-25 Thread Jeffrey Ollie
On Wed, Nov 25, 2009 at 10:16 AM, Scott Robinson  wrote:
>
> == Shared Library
>
> I have modified the build to output a shared library. It's a small change
> relying on libtool.
>
> However, the archives indicate Carl dislikes libtool?

Ugh, please, let's not go down the autoconf/automake/libtool route.

> == JSON output
>
> I added a --output=(text|json) option to "search" and "show." Then, emit json
> results appropriately.
>
> To accomplish this, I used the cJSON library sans parsing logic.
>
> However, I saw discussion about providing a --format option?

Awesome, I was just looking at using json-glib but I haven't gotten
very far yet, mainly because I've never used it before (nor done much
of anything with JSON). I was going to call the option
'--output-format=(standard|json)' but I'm not going to worry about the
color of that bikeshed.

IMHO, I think both approaches would be valuable, that way people
writing front ends could choose the methods that worked best for them.
 Do you have patches that you are ready to share?

-- 
Jeff Ollie


[notmuch] [PATCH -v4] notmuch.el: Add face support to search and show mode

2009-11-25 Thread Jeffrey Ollie
On Tue, Nov 24, 2009 at 11:24 PM, Alexander Botero-Lowry
 wrote:
>
> This way of detecting the tags seems ok, but I think it would be nicer
> if it could be done even more deterministically. :) One idea that be
> neat is to have a --format=sexp for notmuch search, which exports sexps
> (probably alists, but could be some other format) for the search results
> that can just be eval'd and processed in a cleaner way (and would also
> make for nicer APIs in emacs for querying notmuch itself). Actually I
> really like the idea of a sexp output mode for show too, instead of the
> markers *plots*

Along the same lines, I was thinking that it would be nice to be able
to get output formatted in JSON as well for possible use in a web
front-end.  Actually, it looks like Emacs has support for JSON as
well[1].  Not sure what release it was introduced in, but I see that
it's in 23.1.

http://edward.oconnor.cx/2006/03/json.el

-- 
Jeff Ollie


[notmuch] [PATCH] fix compiler warnings

2009-11-22 Thread Jeffrey Ollie
On Sun, Nov 22, 2009 at 9:22 PM, Carl Worth  wrote:
> On Sun, 22 Nov 2009 17:11:03 +0200, Dirk-Jan C. Binnema  gmail.com> wrote:
>>
>> - ? ?write(2, msg, sizeof(msg)-1);
>> + ? ?if (write(2, msg, sizeof(msg)-1) < 0) {
>> + ? ? ? ? /* ignore...*/
>> + ? ?}
>
> I don't like the gratuitous conditional here. It clutters the code and
> make is less clear. If we're just trying to squelch a warning about an
> unused return value from a function, then I think I'd rather see:
>
> ? ? ? ?ssize_t ignored;
>
> ? ? ? ?ignored = write (2, msg, sizeof (msg) - 1);

Isn't the usual method for ignoring return values casting to void?

(void) write (2, msg, sizeof (msg) - 1);

-- 
Jeff Ollie


[notmuch] [PATCH] Add SCons build files.

2009-11-22 Thread Jeffrey Ollie
On Sun, Nov 22, 2009 at 9:11 PM, Carl Worth  wrote:
>
> But I'm afraid I really don't want to switch away from just using (GNU)
> make for the actual compilation.
>
> I don't know anything about scons, but if you can use it to write a
> python script that just does the configuration step, (outputting a
> Makefile.config say), then that might be very interesting.

Well, to me, part of the appeal of SCons is that it doesn't generate
Makefiles.This article is a little old, but the first three
sections are a pretty good rundown on the limitations of Make.

http://www.scons.org/wiki/FromMakeToScons

If you really want a tool that generates Makefiles, take a look at
CMake.  Unfortunately, I think CMake tries to do too much with the
Makefiles and they become close to unreadable.  In the attempt I made
to come up with CMake build files, CMake generates at least 3800 lines
of Makefiles (from a starting point of 30 lines of build files).

> Some people
> have recently told me that python would be a much more sane language for
> doing configuration in than shell.

Well, Python is a much more sane language for doing just about
anything as far as I'm concerned. :)

> I don't know if they're right or not, but I'm (somewhat) willing to have
> multiple implementations of the configure script (since there's always
> the option to just skip it and configure Makefile.config manually). But
> I'm definitely not willing to have multiple build systems.

I think that having multiple different configuration systems would be
pretty awful, especially if people make changes to their favourite
configuration system and hope that someone else fixes the others.

> Meanwhile, the only advantage I know for automake is that once it's
> setup, adding a new file to compile is as simple as adding one file to a
> list in the Makefile.am. We've already got notmuch as easy as that with
> just adding a file to a list in Makefile.local.

Once SCons is set up, adding a new file is just a matter of adding the
name of the file to a list as well.  CMake is the same, and I'm sure
that other systems are similar.

> So I'd very much like to continue exploring what we can do with our own
> configuration system, (in whatever language/language(s) make sense).

It would seem to me that we would be re-inventing a lot of the work
already done by other people.

> Thanks for listening,

Same here.

-- 
Jeff Ollie


[notmuch] [PATCH] Add SCons build files.

2009-11-22 Thread Jeffrey Ollie
The SCons build files are not meant to require emacs.  If I've messed
something up and emacs is somehow required I would consider that a bug
and would try to fix it.

As far as availability, I'm sure that SCons is one yum or apt-get or
 away.   The only
people that would face more than a minor inconvenience are people that
prefer to download tarballs and compile and install things themselves.

Yes, I'm sure that make is widely available, but as notmuch gets used
on a wider variety of systems some sort of configuration system will
become necessary.  If I can prevent another project from going down
the autoconf/automake path I'll be happy.  I started creating CMake
build files but I don't know CMake well enough to come up with a
working build.

On 11/22/09, Bart Trojanowski  wrote:
> * Jeffrey C. Ollie  [091122 08:47]:
>> The SCons build files included here *should* do everything that the
>> current Makefiles do, plus a little bit of configuration checking.  To
>> build/install:
>
> Wouldn't that have the unfortunate side effect of making notmuch
> unusable w/o emacs and scons installed?
>
> GNU make has a much wider install base, and notmuch is still very useful
> without emacs.
>
> -Bart
>
> --
>   WebSig: http://www.jukie.net/~bart/sig/
>

-- 
Sent from my mobile device

Jeff Ollie


[notmuch] [PATCH] Missing final semi-colon in .desktop's Categories.

2009-11-22 Thread Jeffrey Ollie
Found the same problem on my own, only to minutes later catch up on
the backlog of patches in my email and find this!

On Sat, Nov 21, 2009 at 3:13 AM, James Rowe  wrote:
> "Those keys which have several values should have a semicolon as the trailing
> character."
> ?-- http://standards.freedesktop.org/desktop-entry-spec/1.0/ar01s03.html
>
> Signed-off-by: James Rowe 

Reviewed-by: Jeffrey C. Ollie 

> ---
> ?notmuch.desktop | ? ?2 +-
> ?1 files changed, 1 insertions(+), 1 deletions(-)
>
> diff --git a/notmuch.desktop b/notmuch.desktop
> index d29dff6..819cd1e 100644
> --- a/notmuch.desktop
> +++ b/notmuch.desktop
> @@ -4,4 +4,4 @@ Exec=emacs -f notmuch
> ?Icon=emblem-mail
> ?Terminal=false
> ?Type=Application
> -Categories=Network;Email
> +Categories=Network;Email;
> --
> 1.6.4.4
>
> ___
> notmuch mailing list
> notmuch at notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch
>



-- 
Jeff Ollie


[notmuch] Segfault searching for tags

2009-11-20 Thread Jeffrey Ollie
On Fri, Nov 20, 2009 at 5:32 AM, Carl Worth  wrote:
> On Thu, 19 Nov 2009 16:45:43 +0100, Adrian Perez de Castro  igalia.com> wrote:
>> The thing is that in notmuch_message_get_in_reply_to(), line 288, a NULL
>> instance of Xapian::TermIterator is dereferenced. In my particular case,
>> the culpript is a cache file of Claws-Mail, as seen in the following GDB
>> session:
>
> Not quite NULL, (nor is it quite dereferencing---this is nasty C++
> overloading), but yeah, the idea is the same. We need to protect all of
> our "calls" to this overloaded operator to not call it when the iterator
> is equal to the value returned by termlist_end ().
>
> On Thu, 19 Nov 2009 20:23:15 -0600, Jeffrey Ollie  wrote:
>> I straced some of the crashes, and the last file that was read before
>> the crash was a malformed message. ?I've attached one of the messages.
>
> Thanks for the file. I never like to push code that I haven't tested, so
> this was very helpful.
>
> Below is the patch that I just pushed which seems to do the trick.

Ah, excellent!  This does indeed seem to prevent the crash.  Now I
just need to figure out how to get all my mail out of GMail.

-- 
Jeff Ollie


[notmuch] Segfault searching for tags

2009-11-19 Thread Jeffrey Ollie
On Thu, Nov 19, 2009 at 9:45 AM, Adrian Perez de Castro
 wrote:
> On Wed, 18 Nov 2009 12:00:10 -0600, Jeffrey wrote:
>
>> Getting the following segfault with 306635c2 on Fedora 12. ?Seems to
>> be happening with any 'tag:' search that returns results. ?For
>> example, 'notmuch search tag:inbox' and 'notmuch search tag:unread'
>> segfault but 'notmuch search tag:nosuchtag', 'notmuch search
>> subject:logwatch' and 'notmuch search video' seem to work fine.
>>
>> Core was generated by `/usr/bin/notmuch search --sort=oldest-first 
>> tag:inbox'.
>> Program terminated with signal 11, Segmentation fault.
>> \#0 ?Xapian::TermIterator::operator* (this=)
>> ? ? at api/omtermlistiterator.cc:78
>> 78 ? ? ? ?RETURN(internal->get_termname());
>> Current language: ?auto
>> The current source language is "auto; currently c++".
>
> I have hit what I believe is exactly the same problem. In my case, some
> results are printed when I execute "notmuch search tag:inbox", and then
> the program crashes in the same exact place.
>
> The thing is that in notmuch_message_get_in_reply_to(), line 288, a NULL
> instance of Xapian::TermIterator is dereferenced. In my particular case,
> the culpript is a cache file of Claws-Mail, as seen in the following GDB
> session:
> [...]
> As you can see, there "filename" points to a Claws-Mail cache file, which
> is a binary file (I can provide a copy if needed). I suspect that this is
> related to the fact that the iterator ends up being NULL somehow.

I straced some of the crashes, and the last file that was read before
the crash was a malformed message.  I've attached one of the messages.
 I've been using offlineimap to sync my gmail mailbox to my laptop so
that I can use notmuch.  offlineimap isn't the most stable program,
but I'm not sure yet if offlineimap is causing the problem or if
that's the way the message is in gmail.

-- 
Jeff Ollie
-- next part --
Delivered-To: jeff at ollie.clive.ia.us
Received: by 10.90.86.18 with SMTP id j18cs228556agb;
Thu, 5 Nov 2009 22:23:50 -0800 (PST)
Received: by 10.90.16.38 with SMTP id 38mr7468290agp.112.1257488620374;
Thu, 05 Nov 2009 22:23:40 -0800 (PST)
Return-Path: 
Received: from 209.85.223.101 ([116.208.64.100])
by mx.google.com with SMTP id 41si9046203iwn.112.2009.11.05.22.23.38;
Thu, 05 Nov 2009 22:23:40 -0800 (PST)
Received-SPF: neutral (google.com: 116.208.64.100 is neither permitted nor 
denied by best guess record for domain of xhutteesjj at yahoo.com.au) 
client-ip=116.208.64.100;
Authentication-Results: mx.google.com; spf=neutral (google.com: 116.208.64.100 
is neither permitted nor denied by best guess record for domain of xhutteesjj 
at yahoo.com.au) smtp.mail=xhutteesjj at yahoo.com.au
Date: Thu, 05 Nov 2009 22:23:40 -0800 (PST)
Received: from 146.2.76.118 by 116.208.64.100; Fri, 06 Nov 2009 03:20:42 -0300
Message-ID: 

[notmuch] [PATCH 2/2] .gitignore: Add common editor droppings

2009-11-19 Thread Jeffrey Ollie
On Wed, Nov 18, 2009 at 11:39 PM, Karl Wiberg  wrote:
>
> Since these patterns are specific to a user rather than a project,
> shouldn't they go in the user's ignore file (~/.gitignore) rather than
> the project's?

If every user could be counted on to put them in their personal
.gitignore probably not, but I think that it's good practice to at
least ignore the stuff that emacs and vim use for temporary files and
backups.

-- 
Jeff Ollie


[notmuch] RPM spec file

2009-11-19 Thread Jeffrey Ollie
On Wed, Nov 18, 2009 at 7:03 PM, Carl Worth  wrote:
> On Wed, 18 Nov 2009 12:31:53 -0600, "Jeffrey C. Ollie"  
> wrote:
>> I prefere managing all of the software on my system with RPM, so I
>> came up with a quick spec file for building RPMS. ?Once notmuch
>> settles down and I continue using it I may submit notmuch to Fedora as
>> an official package. ?If you'd like your own RPMS build your own, I
>> don't plan on offering a repository of binary packages.
>
> This sounds great, Jeffrey.
>
> What makes the most sense for this file? Shall we just leave it in the
> central repository? Or put it in its own rpm branch? I'm open to your
> suggestions here.

I don't think that a separate Git branch makes sense, but maybe stuff
like this should be in a subdirectory like "packaging/fedora", since I
imagine that there will eventually be a need for "packaging/debian"
etc. as well.

-- 
Jeff Ollie


[notmuch] [PATCH] notmuch: Add tag archive when archiving the mails

2009-11-18 Thread Jeffrey Ollie
On Wed, Nov 18, 2009 at 1:07 PM, Keith Packard  wrote:
>
>> This enables us to search for archived mails later
>
> I think you can just use 'not tag:inbox' as 'archived' really just means
> 'doesn't have the inbox tag'.

That resulted in an error for me:

$ notmuch search not tag:inbox
A Xapian exception occurred: Syntax:  NOT 
A Xapian exception occurred: Syntax:  NOT 

-- 
Jeff Ollie


[notmuch] Segfault searching for tags

2009-11-18 Thread Jeffrey Ollie
Getting the following segfault with 306635c2 on Fedora 12.  Seems to
be happening with any 'tag:' search that returns results.  For
example, 'notmuch search tag:inbox' and 'notmuch search tag:unread'
segfault but 'notmuch search tag:nosuchtag', 'notmuch search
subject:logwatch' and 'notmuch search video' seem to work fine.

Core was generated by `/usr/bin/notmuch search --sort=oldest-first tag:inbox'.
Program terminated with signal 11, Segmentation fault.
\#0  Xapian::TermIterator::operator* (this=)
at api/omtermlistiterator.cc:78
78  RETURN(internal->get_termname());
Current language:  auto
The current source language is "auto; currently c++".

Thread 1 (Thread 15005):
\#0  Xapian::TermIterator::operator* (this=)
at api/omtermlistiterator.cc:78
No locals.
\#1  0x0040d213 in _notmuch_message_get_in_reply_to (message=0x1594f70)
at lib/message.cc:288
prefix = 0x415b77 "XREPLYTO"
prefix_len = 0
i = {internal = {dest = 0x0}}
in_reply_to = ""
\#2  0x0040f842 in _resolve_thread_relationships (thread=0x1595a00)
at lib/thread.cc:157
node = 0x1596130
message = 0x1594f70
parent = 0x7fff2cade9c8
prev = 0x1595cd0
in_reply_to = 
\#3  _notmuch_thread_create (thread=0x1595a00) at lib/thread.cc:285
thread = 0x1595a00
thread_id_query = 0x158beb0
matched_query = 
messages = 0x7fff2cade948
message = 
thread_id_query_string = 
matched_query_string = 
\#4  0x0040f3d0 in notmuch_query_search_threads (
query=, first=,
max_threads=) at lib/query.cc:217
threads = 0x158b5f0
thread = 0x6e0077
messages = 0x158b7c0
message = 0x158c580
thread_id = 0x158b890 "2065b08615b4cbbb22d9ee874bb84d3e"
seen = 0x15454a0
messages_seen = 0
threads_seen = 0
\#5  0x004089a1 in do_search_threads (ctx=0x1543140, query=
0x7fff2cade8a0, sort=NOTMUCH_SORT_OLDEST_FIRST,
first=, max_threads=)
at notmuch-search.c:40
thread = 
threads = 0x1551290
tags = 0x2
date = 
relative_date = 0x2 
\#6  0x00408ddd in notmuch_search_command (ctx=,
argc=1, argv=) at notmuch-search.c:156
config = 
query = 0x158b510
query_str = 
i = 1
first = 
max_threads = 
opt = 
end = 0x0
sort = 
\#7  0x0040636f in main (argc=4, argv=0x7fff2cadec98) at notmuch.c:400
local = 0x1543140
command = 


-- 
Jeff Ollie