[notmuch] [PATCH] Solaris doesn't have 'struct dirent::d_type'

2009-12-21 Thread Tomas Carnecky
On 12/20/09 7:02 PM, James Westby wrote:
> From: Tomas Carnecky
>
> Use stat(2) instead.
>
> Signed-off-by: Tomas Carnecky
> Signed-off-by: James Westby
> ---
>
>The original patch duplicated asprintf and stat calls, rearraging
>the code means we don't need to.
>
>I have a concern about the duplicated stats in is_maildir, but they
>are not so easy to save. I ran a quick timing test (3931 files), dropping
>caches before each set:
>
>  master:
>real  2m3.545s
>real  1m34.571s
>real  1m36.005s
>
>  original patch:
>real  2m18.114s
>real  1m34.843s
>real  1m36.317s
>
>  revised patch:
>real  2m5.890s
>real  1m36.387s
>real  1m36.453s
>
>This shoes there is little impact of the code, but given that it is
>around one percent we may want to make it conditional on platform
>and save the extra stat calls.

If performance regression is an issue, something like this could be used 
to keep the current code paths in linux and stat() on other platforms:

static bool
is_dir(const char *path, struct dirent *dirent)
{
#if defined(__sun__)
... sprintf, stat etc
#else
(void) path;
return dirent->d_type == DT_DIR;
#endif
}

tom



[notmuch] keeping a copy of sent mail locally

2009-12-21 Thread Alex Ghitza
On Sun, 20 Dec 2009 13:31:13 -0500, Jameson Graef Rollins  wrote:
> Hi, folks.  I've been following this thread a bit and I wanted to put
> in my argument that using Bcc is *NOT* the way to save local copies of
> sent mail.  I really don't think that we need to require that the mail
> actually get send out to the world wide net just to save a copy of it
> locally.  I think there must be better ways of doing this.

Agreed, and I think that directly saving to a file is the sane way to
go.  However, there might be other (more legitimate) uses for always
bcc-ing a certain address on all sent mail, and it might be good to have
an easy mechanism for this.  


Best,
Alex

-- 
Alex Ghitza -- Lecturer in Mathematics -- The University of Melbourne
-- Australia -- http://www.ms.unimelb.edu.au/~aghitza/


[notmuch] keeping a copy of sent mail locally

2009-12-21 Thread Adrien BUSTANY
On Mon, 21 Dec 2009 09:27:04 -0800, Carl Worth  wrote:
> On Sun, 20 Dec 2009 15:29:15 +1100, Alex Ghitza 
wrote:
>> It looks like we need a way to get the primary email address from the
>> config file.
> 
> Yes, we definitely need that.
> 
>> Actually, while we're at it, we can consider making this
>> more flexible and adding a new option to the config file (e.g. bcc=...)
>> which would take a semicolon-separated list of email addresses (in case
>> someone wants to always bcc an address other than the primary one).
> 
> And that would be fine too.
> 
>> Is there a nice clean way of getting the config variables in
notmuch.el?
>> Or should this go into a new file notmuch-compose.c?  Note that the
>> latter would imply having a new command like "notmuch compose".
> 
> I think we want a new C file to make it easy to ask for options out of
> the configuration file. The operation of "notmuch compose" seems simple
> enough to not be necessary, (though maybe it would make it easier to get
> standard behavior among different interfaces).
What about adding a way to get/set config variables from the notmuch
command
line ? Would that be bloat ?
That would even allow changing the internal format of the conf. file
without
breaking the clients...

[Snip]

Cheers

Adrien


[notmuch] [PATCH] Added regress option to tags iterator

2009-12-21 Thread Carl Worth
On Mon, 21 Dec 2009 17:23:55 -0800, Carl Worth  wrote:
>   New functionCorresponds to existing function (if any)
>   -
>   move_to_first   
>   has_nexthas_more
>   move_to_nextadvance
> 
>   move_to_last
>   has_previous
>   move_to_previous
> 
>   get get
> 
> The semantics of those all seem clear enough to me. They provide what's
> necessary for all three portions of a for loop, (in either direction),

Except that they don't. :-P

We don't want has_next and has_previous but something more like "has
current", (perhaps to pair with get_current?).

> The only downside is that the function names are a bit long in some
> cases, but I'm willing to live with that until someone comes up with
> better.

One option is to just drop the "move_ " prefix. Then everything will be
a two-word function. So the new proposal is:

to_first
has_current
to_next

to_last
has_current
to_previous

get_current

Better?

-Carl
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20091221/814adb6d/attachment.pgp>


[notmuch] [PATCH] Added regress option to tags iterator

2009-12-21 Thread Carl Worth
On Wed, 09 Dec 2009 12:08:43 -0800, Carl Worth  wrote:
> On Wed, 9 Dec 2009 14:24:46 +0100, Ruben Pollan  
> wrote:
> > Do you like to call them regress? Should I change that?
...
> > What about the functions notmuch_*_is_first? Is kind of reversed logic than
> > notmuch_*_has_more, the last are true when is not reach the limit but the
> > first ones are true when the limit is reached. But I think it make sense 
> > like
> > that.

It doesn't make sense in the case of trying to write a for loop that
iterates in the reverse order. The is_first semantic doesn't give you
what you want for the loop-control portion of the for loop.

> I'd like a more symmetric API here. Anyone have a favorite set of names
> for iterating a list in two directions?

In some recent coding I needed to implement a new iterator so I had the
chance to think about this some more. Here is what I came up with:

  New function  Corresponds to existing function (if any)
    -
  move_to_first 
  has_next  has_more
  move_to_next  advance

  move_to_last  
  has_previous  
  move_to_previous  

  get   get

The semantics of those all seem clear enough to me. They provide what's
necessary for all three portions of a for loop, (in either direction),
and everything pairs nicely.

The only downside is that the function names are a bit long in some
cases, but I'm willing to live with that until someone comes up with
better.

Thoughts?

-Carl

-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20091221/e3d08c4e/attachment.pgp>


[notmuch] keeping a copy of sent mail locally

2009-12-21 Thread Carl Worth
On Mon, 21 Dec 2009 20:39:01 +0100, Adrien BUSTANY  
wrote:
> On Mon, 21 Dec 2009 09:27:04 -0800, Carl Worth  wrote:
> > 
> > I think we want a new C file to make it easy to ask for options out of
> > the configuration file.
>
> What about adding a way to get/set config variables from the notmuch
> command line ? Would that be bloat ?

That's actually exactly what I meant above. I think we'll need this.

-Carl
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20091221/00b6828c/attachment.pgp>


[notmuch] keeping a copy of sent mail locally

2009-12-21 Thread Marten Veldthuis
On Mon, 21 Dec 2009 21:08:22 +1100, Alex Ghitza  wrote:
> 2. of course, filenames need to be unique.  Do we want/have to follow
> the maildir file naming conventions listed at
> http://cr.yp.to/proto/maildir.html
> or is it enough to use the Emacs lisp make-temp-file?

I'd very much prefer a real Maildir, because that would sync back
correctly with OfflineIMAP etc.

-- 
- Marten


[notmuch] [PATCH] Store documents for message-ids we haven't seen

2009-12-21 Thread Carl Worth
On Sun, 20 Dec 2009 20:27:32 +, James Westby <jw+debian at jameswestby.net> 
wrote:
>   One case that isn't handled is when we don't find the thread
>   id of the parent, but then find the message itself. I believe
>   this case shouldn't happen, but you never know.

It really shouldn't happen since we are holding a write lock on the
database, (so there's no possible race condition here with another
client delivering the parent message).

But since you almost can't help but detect the case, (just noticing a
NOTMUCH_STATUS_SUCCESS value from _create_for_message_id), please put an
INTERNAL_ERROR there rather than marching along with an incorrect thread
ID.

> + // We have yet to see the referenced message, generate a thread id
> + // for it if needed and store a dummy message for the parent. If we
> + // find the mail later we will replace the dummy.

Call me old-fashioned if you will, but I'd much rather have C style
multi-line comments (/* ... */) rather than these C++-style comments
with //.

> + if (private_status == 
> NOTMUCH_PRIVATE_STATUS_NO_DOCUMENT_FOUND) {
> + // expected

And I think this comment deserves a complete sentence before the
condition. Something like:

/* We expect this call to create a new document (return NO_DOCUMENT_FOUND) */

-Carl
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20091221/8c09d01a/attachment.pgp>


[notmuch] [PATCH] Reindex larger files that duplicate ids we have

2009-12-21 Thread Carl Worth
Hi James,

I just got to a point in my outstanding rework where I thought it would
make sense to pull this patch series in, (I'm adding support for storing
multiple filenames in a single mail document).

I took a closer look at this series, and I think it's still independent,
so I'll finish up what I'm doing and then add this series on top later.

But I can at least answer some of the questions you asked for now:

>   Does the re-indexing replace the old terms?

Before this patch, there's there's not yet any "re-indexing" in
notmuch. So we'll basically need to think about what we want to do here.

As this patch is written, (just calling into the existing _index_file
function), the re-indexing only adds new terms, (and doesn't delete
any). That's probably correct. We're using file size as an heuristic
that the larger file is a superset of the smaller file, but it doesn't
guarantee that the smaller file doesn't contain any unique terms. So I'd
be extremely hesitant to drop any terms here.

>   In the case
>   where you had a collision with different text this could
>   make a search return mails that don't contain that text.
>   I don't think it's a big issue though, even if that is the
>   case.

That's correct. As mentioned in a previous thread, this is likely only a
big issue in the face of deliberate message-ID spoofing or so. In that
thread we talked about some ideas for mitigating that. But I don't think
we need to solve that problem before applying this patch series.

-Carl
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20091221/81364919/attachment.pgp>


[notmuch] keeping a copy of sent mail locally

2009-12-21 Thread Jameson Graef Rollins
On Mon, Dec 21, 2009 at 08:57:40PM +1100, Alex Ghitza wrote:
> On Sun, 20 Dec 2009 13:31:13 -0500, Jameson Graef Rollins  finestructure.net> wrote:
> > Hi, folks.  I've been following this thread a bit and I wanted to put
> > in my argument that using Bcc is *NOT* the way to save local copies of
> > sent mail.  I really don't think that we need to require that the mail
> > actually get send out to the world wide net just to save a copy of it
> > locally.  I think there must be better ways of doing this.
> 
> Agreed, and I think that directly saving to a file is the sane way to
> go.  However, there might be other (more legitimate) uses for always
> bcc-ing a certain address on all sent mail, and it might be good to have
> an easy mechanism for this.  

I certainly have no problem with there being the ability to specify an
address that is always Bcc'd on sent mail.  I think that's a fine
idea.  I just think that saving local copies of sent mail directly is
an important thing to do, and we should figure out the best way to
support it correctly.  Every mailer that I know of has some way to
save local copies of sent mail, and I think most users will expect
such a feature.

An important thing to consider is that all of this locally saved sent
mail should also be immediately incorporated into the database so that
it can be immediately searched and displayed with it's relevant
threads.

jamie.
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20091221/0f72b37e/attachment-0001.pgp>


[notmuch] keeping a copy of sent mail locally

2009-12-21 Thread Carl Worth
On Sun, 20 Dec 2009 15:29:15 +1100, Alex Ghitza  wrote:
> It looks like we need a way to get the primary email address from the
> config file.

Yes, we definitely need that.

> Actually, while we're at it, we can consider making this
> more flexible and adding a new option to the config file (e.g. bcc=...)
> which would take a semicolon-separated list of email addresses (in case
> someone wants to always bcc an address other than the primary one).

And that would be fine too.

> Is there a nice clean way of getting the config variables in notmuch.el?
> Or should this go into a new file notmuch-compose.c?  Note that the
> latter would imply having a new command like "notmuch compose".

I think we want a new C file to make it easy to ask for options out of
the configuration file. The operation of "notmuch compose" seems simple
enough to not be necessary, (though maybe it would make it easier to get
standard behavior among different interfaces).

> It's not a big deal, but now the same thing must be set in two different
> places (.notmuch-config and .emacs),

Right. I definitely want to get away from that. I want people to put
stuff into .notmuch-config and then have our emacs code "just work".
Our emacs code could, for example, set variables like user-full-name and
user-mail-address from the notmuch configuration, (it wouldn't override
these if already set I think).

> Sorry about writing a lot of prose and no code!  I'd like to get a sense
> for people's reactions before trying to write a patch for this.

No problem at all. Thanks for the thoughts and ideas.

-Carl
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20091221/1ebf2a23/attachment.pgp>


Re: [notmuch] keeping a copy of sent mail locally

2009-12-21 Thread Alex Ghitza
On Sun, 20 Dec 2009 19:51:05 +0100, a...@koldfront.dk (Adam 
=?iso-8859-1?Q?Sj=F8gren?=) wrote:
 On Sun, 20 Dec 2009 13:31:13 -0500, Jameson wrote:
 
  There must be a way to tell emacs message-mode to save a copy of
  outgoing mail locally. Mutt does this with it's Fcc commands (ie. 
  file carbon copy). I think we should look for a solution like this.
 
 Gnus uses Gcc for this (see gnus-message-archive-group and
 gnus-message-archive-method); I think message.el also supports Fcc; eg. 
 see message-fcc-handler-function:
 

Adam, thanks for the info on the message-fcc-handler-function.  I'll try
to get something working based on this.  There are two questions of
design now:

1. where should the file be saved?  I'm thinking of a new setting in
$NOTMUCH_CONFIG, along the lines of sentmail_path=...
Should there be a default if this is not set, e.g. a subdirectory sent
of the directory given in the config variable path?

2. of course, filenames need to be unique.  Do we want/have to follow
the maildir file naming conventions listed at
http://cr.yp.to/proto/maildir.html
or is it enough to use the Emacs lisp make-temp-file?


Best,
Alex


-- 
Alex Ghitza -- Lecturer in Mathematics -- The University of Melbourne
-- Australia -- http://www.ms.unimelb.edu.au/~aghitza/
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] keeping a copy of sent mail locally

2009-12-21 Thread Marten Veldthuis
On Mon, 21 Dec 2009 21:08:22 +1100, Alex Ghitza aghi...@gmail.com wrote:
 2. of course, filenames need to be unique.  Do we want/have to follow
 the maildir file naming conventions listed at
 http://cr.yp.to/proto/maildir.html
 or is it enough to use the Emacs lisp make-temp-file?

I'd very much prefer a real Maildir, because that would sync back
correctly with OfflineIMAP etc.

-- 
- Marten
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] [PATCH] notmuch-query.el: new file to support access to the notmuch database.

2009-12-21 Thread Carl Worth
On Sun, 20 Dec 2009 16:31:21 -0400, da...@tethera.net wrote:
 I didn't know if you (Carl) want to do copyright assignment, or just
 the GPL license headers are enough. I don't mind pretty much any
 variation.

Please feel free to retain your own copyright. I certainly don't deserve
anything being assigned to me.

I like that you're doing a new file, separate from our current
notmuch.el. That file has already become extremely large and unwieldy
and needs to be broken up.

But to do that, I really want to get the emacs lisp files out of the
top-level directory, (and down into contrib/emacs, say). And it might be
nice to get some of the pending patches merged in before doing a big
restructuring here.

Hopefully I'll start making some real progress on the backlog soon...

-Carl


pgpGg5WCcr56W.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] keeping a copy of sent mail locally

2009-12-21 Thread Carl Worth
On Sun, 20 Dec 2009 15:29:15 +1100, Alex Ghitza aghi...@gmail.com wrote:
 It looks like we need a way to get the primary email address from the
 config file.

Yes, we definitely need that.

 Actually, while we're at it, we can consider making this
 more flexible and adding a new option to the config file (e.g. bcc=...)
 which would take a semicolon-separated list of email addresses (in case
 someone wants to always bcc an address other than the primary one).

And that would be fine too.

 Is there a nice clean way of getting the config variables in notmuch.el?
 Or should this go into a new file notmuch-compose.c?  Note that the
 latter would imply having a new command like notmuch compose.

I think we want a new C file to make it easy to ask for options out of
the configuration file. The operation of notmuch compose seems simple
enough to not be necessary, (though maybe it would make it easier to get
standard behavior among different interfaces).

 It's not a big deal, but now the same thing must be set in two different
 places (.notmuch-config and .emacs),

Right. I definitely want to get away from that. I want people to put
stuff into .notmuch-config and then have our emacs code just work.
Our emacs code could, for example, set variables like user-full-name and
user-mail-address from the notmuch configuration, (it wouldn't override
these if already set I think).

 Sorry about writing a lot of prose and no code!  I'd like to get a sense
 for people's reactions before trying to write a patch for this.

No problem at all. Thanks for the thoughts and ideas.

-Carl


pgpH7cc1xqwPT.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] [PATCH] notmuch-query.el: new file to support access to the notmuch database.

2009-12-21 Thread David Bremner
On Mon, 21 Dec 2009 09:21:40 -0800, Carl Worth cwo...@cworth.org wrote:
 I like that you're doing a new file, separate from our current
 notmuch.el. That file has already become extremely large and unwieldy
 and needs to be broken up.
 
 But to do that, I really want to get the emacs lisp files out of the
 top-level directory, (and down into contrib/emacs, say). And it might be
 nice to get some of the pending patches merged in before doing a big
 restructuring here.

OK, I'll no doubt have at least one more version of this patch before it
get merged in.  I already have a few minor changes, but I'm using it in 
production with the org-mode link stuff, so we'll see if that shakes
out any more issues.  I'll post an updated version if/when I see the
json output patches merged to master.  

David

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] [PATCH] Reindex larger files that duplicate ids we have

2009-12-21 Thread Carl Worth
Hi James,

I just got to a point in my outstanding rework where I thought it would
make sense to pull this patch series in, (I'm adding support for storing
multiple filenames in a single mail document).

I took a closer look at this series, and I think it's still independent,
so I'll finish up what I'm doing and then add this series on top later.

But I can at least answer some of the questions you asked for now:

   Does the re-indexing replace the old terms?

Before this patch, there's there's not yet any re-indexing in
notmuch. So we'll basically need to think about what we want to do here.

As this patch is written, (just calling into the existing _index_file
function), the re-indexing only adds new terms, (and doesn't delete
any). That's probably correct. We're using file size as an heuristic
that the larger file is a superset of the smaller file, but it doesn't
guarantee that the smaller file doesn't contain any unique terms. So I'd
be extremely hesitant to drop any terms here.

   In the case
   where you had a collision with different text this could
   make a search return mails that don't contain that text.
   I don't think it's a big issue though, even if that is the
   case.

That's correct. As mentioned in a previous thread, this is likely only a
big issue in the face of deliberate message-ID spoofing or so. In that
thread we talked about some ideas for mitigating that. But I don't think
we need to solve that problem before applying this patch series.

-Carl


pgpIsgTCssm3P.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] [PATCH] Store documents for message-ids we haven't seen

2009-12-21 Thread Carl Worth
On Sun, 20 Dec 2009 20:27:32 +, James Westby jw+deb...@jameswestby.net 
wrote:
   One case that isn't handled is when we don't find the thread
   id of the parent, but then find the message itself. I believe
   this case shouldn't happen, but you never know.

It really shouldn't happen since we are holding a write lock on the
database, (so there's no possible race condition here with another
client delivering the parent message).

But since you almost can't help but detect the case, (just noticing a
NOTMUCH_STATUS_SUCCESS value from _create_for_message_id), please put an
INTERNAL_ERROR there rather than marching along with an incorrect thread
ID.

 + // We have yet to see the referenced message, generate a thread id
 + // for it if needed and store a dummy message for the parent. If we
 + // find the mail later we will replace the dummy.

Call me old-fashioned if you will, but I'd much rather have C style
multi-line comments (/* ... */) rather than these C++-style comments
with //.

 + if (private_status == 
 NOTMUCH_PRIVATE_STATUS_NO_DOCUMENT_FOUND) {
 + // expected

And I think this comment deserves a complete sentence before the
condition. Something like:

/* We expect this call to create a new document (return NO_DOCUMENT_FOUND) */

-Carl


pgpC7E8uY0WI7.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch