Re: How to move from end of one thread to beginning of next by pressing "n"?

[Carl Worth]

On Mon, Apr 08 2024, Richard H. Stanton wrote:
> If I press RETURN to view a message, “n” and “p” move to the
> next/previous message *in the thread*, but motion with “n” stops when
> you get to the end of the thread. Is there a way to set things so that
> “n” moves from the end of the current thread to the beginning of the
> next? This would make going through my emails a lot more convenient,
> especially as most of my “threads” only have one message in them.

I don't think that exists as a current option.

You can use the space bar to advance to the next thread.

-Carl
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

Re: notmuch.el, needs without-restriction to properly save draft

[Carl Worth]

On Wed, Mar 13 2024, Marc Fargas wrote:
> Hi there,

Hi, Marc!

> I'm not sure if this is the appropiate mailing list for bugs regarding
> notmuch.el (Notmuch in Emacs). Also, I'm not subscribed so please keep
> me in CC!

Yes, this is a fine place to report things

> I recently noticed that when saving drafts in notmuch I would loose all
> the headers that are not normally visible on the compose
> buffer. References, In-Reply-To and the likes.

Thanks for the bug report, as well as the fix you provided. I haven't
run it, but it looks totally reasonable.

> Note that when resuming the draft later on all headers will be visible,
> still working out how to restore the narrow restriction there.

That seems worth chasing down, but shouldn't prevent the acceptance of
your fix here which seems like only progress in the right direction.

Could you put your fix together in the form of a git-appliable patch?
Such as by applying it to the notmuch source, running `git commit` and
then `git format-patch HEAD~` or similar.

Thanks again,

-Carl
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

Re: Getting read out of inbox

[Carl Worth]

On Mon, Nov 27 2023, Caleb Herbert wrote:
> Hi,

Hi Caleb,

> I have a stupid newbie question.

No problem at all.

> I just started using Notmuch, opened my email in Emacs, and was
> bombarded by a bunch of old mail.  My email was perfectly sorted in
> Icedove (Thunderbird) before I made the switch.  Everything was in the
> Archive/20xx folders.

When notmuch encounters mail for the first time it adds (by default
anyway) an "inbox" tag to each item.

In your case, where most of what you've just imported is not new, all
you need to do is remove that inbox tag from the mail that you've
already read.

Fortunately, notmuch makes that pretty easy since you can search for and
tag (or untag) messages matching search criteria of interest.

I'd suggest starting at the command line and doing some notmuch based
searches to see how things work and then switch from searching to
instead do the tag removal when you're happy with what things look like.

For example , you might start by just counting all messages:

notmuch count '*'

and then count messages matching some search criteria, such as, perhaps:

notmuch count folder:Archive/2022

If the counts look like what you are expecting, then you can untag
messages with "notmuch tag -inbox" and again, providing the same search
criteria such as:

notmuch tag -inbox folder:Archive/2022

Of course, your search criteria doesn't have to be based on maildir
directories like the above. You could also search for all messages prior
to some specific date with something like:

notmuch count date:..2023-06-01

and, of course, again:

notmuch tag -inbox date:..2023-06-01

See "man notmuch-search-terms" for many more examples of ways you can
search for messages.

I hope that gives you a good place to start. And I hope that notmuch
gives you a better experience once you're past this first hurdle.

-Carl
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

Re: macOS and go language bindings

[Carl Worth]

On Tue, Nov 21 2023, Peter Lawrence wrote:
> Thanks for the feedback, Carl!

No problem. I'm glad you were able to push even a little past what I was
able to do.

> I have a feeling now that this isn't just a macOS issue, but possibly a
> broader issue that would effect anyone using a newer version of Go.

Quite likely, yes!

> I am guessing that the go compiler (mine is at 1.20.8) doesn't like the
> source in notmuch.go. This may have been written a while ago?

Yes, definitely. The last non-trivial commit to contrib/go was in May
2015 (!).

> I'm definitely over my head in the Go world at this point.

That's OK. Let's look at the bright side. You also might be the most
experienced one here at the moment.

> I wonder if anyone proficient with Go might know how to get the source
> to build against a Go 1.20+ version?

It might be useful at this point to seek some porting advice from a
go-specific channel of one form or another.

If you're able to make some progress, we'll obviously gladly accept
patches that you develop to improve the Go binding within notmuch!

-Carl



signature.asc
Description: PGP signature
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

Re: macOS and go language bindings

[Carl Worth]

On Sun, Nov 19 2023, Peter Lawrence wrote:
> Hi.

Hi Peter!

> I'm trying to build an email client (aerc) with support for notmuch
> on macOS. aerc is written in go. I can't find an aerc package for brew
> which includes notmuch headers/library files.  Where can I get these
> language bindings and how do I "install" them, or where do I place them
> in the aerc source directory to get the notmuch library built and then
> accessible to aerc?

I don't have much direct experience with either macOS nor the Go
language bindings to notmuch, but I'll see how far I can get in
answering your questions.

The Go bindings for notmuch are in the contrib/go directory of the
source distribution which you can get from the latest source release


https://notmuchmail.org/releases/notmuch-0.38.tar.xz

Or via git:

git clone https://git.notmuchmail.org/git/notmuch

There's a brief README and a Makefile in the contrib/go directory, but
I'm not exactly clear on how things are supposed to work.

When I type "make" the first command it runs is "go install notmuch"
which fails as follows:

$ go install notmuch
go: 'go install' requires a version when current directory is not in a module
Try 'go install notmuch@latest' to install the latest version

The advice from the error message doesn't help much:

$ go install notmuch@latest
go: notmuch@latest: malformed module path "notmuch": missing dot in first path 
element

I'm guessing someone who knows even a tiny bit about go modules might be
able to explain what's needed here, so we can fix the Makefile to
actually work?

-Carl
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

Re: "notmuch compact" questions

[Carl Worth]

Hi Andy,

I'm really glad the compaction was so successful!

I think the Xapian folks deserve the big credit for this, more than any
code specific to notmuch, (other than user interface and safety
checking/backup, etc.).

I'm really glad you're finding notmuch helpful. And I'm glad you felt
comfortable reaching out for guidance.

Enjoy your new hard drive space!

-Carl

On Sun, Jun 18 2023, Andy Smith wrote:
> Hi Michael,
>
> On Sun, Jun 18, 2023 at 01:36:30PM +0200, Michael J Gruber wrote:
>> On my comparatively small db, gains are typically substantial
>> (postlist, position), even though I compact from time to time.
>
> Thanks for your help. The savings were indeed a lot more than I
> expected (and the compact didn't take anywhere near as long as I
> thought it would, either):
>
> $ sudo compsize ~/.notmuch
> Processed 7 files, 5159972 regular extents (5484004 refs), 2 inline.
> Type   Perc Disk Usage   Uncompressed Referenced  
> TOTAL   60%   51G  85G  56G   
> none   100%   18G  18G  16G   
> zstd49%   32G  66G  39G
> $ notmuch compact
> Compacting database...
> compacting table postlist
>  Reduced by 62% 8838736K (14138192K -> 5299456K)
> compacting table docdata
>  Reduced by 50% 1168K (2304K -> 1136K)
> compacting table termlist
>  Reduced by 51% 6816008K (13112728K -> 6296720K)
> compacting table position
>  Reduced by 54% 17223112K (31731384K -> 14508272K)
> compacting table spelling
>  doesn't exist
> compacting table synonym
>  doesn't exist
> Done.
> $ sudo compsize ~/.notmuch
> Processed 7 files, 186246 regular extents (187018 refs), 2 inline.
> Type   Perc Disk Usage   Uncompressed Referenced  
> TOTAL   56%   14G  25G  25G   
> none   100%  3.7G 3.7G 3.7G   
> zstd48%   10G  21G  21G
>
> I don't know why so much space should have been saved, though it may
> be something to do with the fact that I indexed everything and then
> decided to move a million or so messages around into a different
> folder structure.
>
> Thanks,
> Andy
>
> -- 
> https://bitfolk.com/ -- No-nonsense VPS hosting
> ___
> notmuch mailing list -- notmuch@notmuchmail.org
> To unsubscribe send an email to notmuch-le...@notmuchmail.org
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

Re: Selection bug

[Carl Worth]

On Tue, Apr 04 2023, Justus Winter wrote:
> My desktop now behaves exactly like my laptop, and it is really
> annoying.  Surely I'm not the only one with this problem.  Can no one
> else reproduce this?

I haven't tried using the reproduction test, but I did just refresh my
inbox and I saw the buggy behavior.

So there's a random anecdote which suggests it might not be that hard to
reproduce the bug, (perhaps doesn't require any special messages).

In my case, it's less annoying because I have more than a screenful of
messages in my inbox (for shame!) so the message with the green bar
isn't even visible when emacs' point is moved to the message that is
actually selected.

So if I hadn't been intentionally looking for this behavior I likely
would have never noticed.

-Carl
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

Re: Data loss

[Carl Worth]

On Tue, Apr 04 2023, Michael J. Gruber wrote:
>> It's really unfortunate if notmuch-mutt makes it that easy to throw away
>> your email.
>>
>> That sounds like a nasty bug that should be fixed in that program.
>
> To be fair to notmuch-mutt:
> ```
> --output-dir DIR
>
> Store search results as (symlink) messages under maildir DIR. Beware: DIR will
> be overwritten. (Default: F<~/.cache/notmuch/mutt/results/>)
> ```
> So the default is safe, and the warning is there. It would be a bug
> only if notmuch-mutt descended up and deleted a root dir or such.

OK. It's definitely acting as documented. So maybe "bug" was a little
harsh in my description.

It could still be made safer, and I think it would be a good idea to
change it.

For example, notmuch-mutt currently happily creates a directory that
doesn't already exists. That's safe and good.

What it could also do is create some placeholder file like
.NOTMUCH-MUTT-OUTPUT-DIR when it writes its output. And then it could
check for that file's existence before removing a directory.

-Carl
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

Re: Data loss

[Carl Worth]

Ouch.

It's really unfortunate if notmuch-mutt makes it that easy to throw away
your email.

That sounds like a nasty bug that should be fixed in that program.

As for recovering, I suppose there _is_ a fair amount of detail in your
notmuch index from all of the position-indexed terms, (as long as you
haven't run "notmuch new" again since the mail was deleted).

If you still have a large Xapian database in the notmuch database
directory, it would be theoretically possible to recover a lot of the
email content. But I don't know that anyone has ever written a recovery
tool to help with that process.

-Carl

On Mon, Apr 03 2023, Fulvio Pizzigoni wrote:
> On Mon, Apr 03, 2023 at 08:39:02PM +0200, Michael J Gruber wrote:
>> Am Mo., 3. Apr. 2023 um 20:17 Uhr schrieb Fulvio Pizzigoni
>> :
>> >
>> >
>> >
>> >
>> >
>> >
>> > Hi Carl e thanks for your prompt answer.
>> >
>> > As you suggested, I add notmuch@notmuchmail.org email as well.
>> >
>> > This is what I did:
>> > fulvio@linux:~$ notmuch setup
>> > Your full name [fulvio]:
>> > Your primary email address [my address]: return
>> > Additional email address [Press 'Enter' if none]: return
>> > Top-level directory of your email archive [/home/fulvio/.mutt]: return
>> > Tags to apply to all new messages (separated by spaces) [unread inbox]: 
>> > return
>> > Tags to exclude when searching messages (separated by spaces) [deleted 
>> > spam]: return
>> > fulvio@linux:~$
>> >
>> > After this my .mutt directory (~ 4 GB di mail-boxess) appears so:
>> > fulvio@linux:~$ ll .mutt
>> > totale 12
>> > drwxr-xr-x 2 fulvio fulvio 4096 18 feb 20.32 cur
>> > drwxr-xr-x 2 fulvio fulvio 4096 18 feb 20.32 new
>> > drwxr-xr-x 2 fulvio fulvio 4096 18 feb 20.32 tmp
>> > fulvio@linux:~$ du -h .mutt
>> > 4,0K.mutt/new
>> > 4,0K.mutt/tmp
>> > 4,0K.mutt/cur
>> > 16K .mutt
>> >
>> > That's all.
>> >
>> > Terrible!
>> >
>> > What happened?
>> >
>> > Thanks in advance.
>> >
>> > Fulvio
>
> Hi Michael.
>
>> 
>> First of all: Do you have a back-up? 
>
> G, no :-((
>
>> 
>> notmuch itself does not delete mails, as Carl pointed out. In addition
>> to notmuch you mentioned notmuch-mutt. Did you run that manually or
>> using some mutt config? 
>
> Yes I did this manually:
> fulvio@linux:~$ notmuch-mutt -o .mutt/ search 
>
>> It creates a maildir of symlinks with search
>> results. In order to do so, it deletes the maildir ... 
>
> I think that's the cause.
>
> ... And I think it's irreparable. Am I wrong?
>
> Thanks in advance.
>
> Fulvio
>
>> Usually this  sits in a cache dir, though.
>> 
>> Michael
>> 
>> > On Mon, Apr 03, 2023 at 09:27:22AM -0700, Carl Worth wrote:
>> > > Hi Fulvio,
>> > >
>> > > I've never used notmuch-mutt.
>> > >
>> > > But notmuch itself doesn't delete any mail. It's really paranoid that
>> > > way, (knowing how valuable mail is).
>> > >
>> > > I would suggest you write an email to the notmuch@notmuchmail.org
>> > > mailing list where you will be able to reach more people likely to have
>> > > experience with all of the software you were using.
>> > >
>> > > And if you could provide more details on the actual steps you used, that
>> > > would be useful. For example, you said "configuration process" and "at
>> > > the end". But what actual commands were you running on those steps, for
>> > > example?
>> > >
>> > > -Carl
>> > >
>> > > On Mon, Apr 03 2023, Fulvio Pizzigoni wrote:
>> > > > Hi.
>> > > >
>> > > > I have installed packages notmuch and notmuch-mutt.
>> > > >
>> > > > During configuration process I have indicated the directory used from
>> > > > Mutt for your mail-boxes.
>> > > >
>> > > > The configuration process have not indicated any allert.
>> > > >
>> > > > At the end the mail-boxes in that directory was removed; 3 new
>> > > > empty sub-directory were created: cur, new, tmp.
>> > > >
>> > > > Was the data irremediably lost?
>> > > >
>> > > > Thanks in advance.
>> > > >
>> > > > Fulvio Pizzigoni
>> > ___
>> > notmuch mailing list -- notmuch@notmuchmail.org
>> > To unsubscribe send an email to notmuch-le...@notmuchmail.org
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

Re: Help in Emacs config

[Carl Worth]

On Tue, Mar 07 2023, Atanas Janackovski wrote:
> I have justed strated using `notmuch' in Emacs (doom configuration),
> and I love it!

Great! I'm glad it's useful for you.

> 1. I have a number of accounts that I use. How do I configure my
> config to save draft messages based on the "From" header?

I'm not sure what you're looking for here. Do you want drafts saved in
different locations?

At some level, a draft message is just another message, so you can
search for drafts based on the "From" header just like any other
message.

> 2. From time to time, I want to view `html' content in my
> browser. Does anyone have any tips on how I would do this? I have
> tried the below but this did not seem to work:

This seems to work out of the box for me.

>From looking at the contents of my mailcap-mime-data variable, what
sticks out to me is that "text" and then "html" are at different levels
in the structure such as:

 ("text"
  ("html"
   (viewer . "/usr/bin/firefox '%s'")
   (type . "text/html")
   ("nametemplate" . "%s.html")
   (test . t)
   ("description" . "HTML Text")
   (source . user)))

Have fun with notmuch!

-Carl
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

Re: Thanks for notmuch-lore

[Carl Worth]

On Tue, Mar 22 2022, Kyle Meyer wrote:
> I may be missing something (I didn't know about notmuch-lore before
> seeing it mentioned here), but it looks like the initialization step of
> notmuch-lore's pre-new handles that already.  You just need to set
> `since` far enough back:

Hmm... I did see the "since" parameter and cranked it back.

It didn't seem to do what I wanted, but it's possible the bug is only
with multi-epoch archives, (I was trying to bring in LKML).

From poking at it, it looked like it did perform a "deepening" operation
using the "since" parameter after the initial clone, but then didn't use
anything older than the most-recent upstream commit for the range of
commits from which to get messages out.

But my examination of the code and behavior was very cursory, I admit.

> Also, just to list some other options in this space, l2md and impibe are
> mentioned at  as tools for
> converting public-inbox archives into maildir format.  (I haven't used
> either myself.)

Thanks! I clearly didn't look quite hard enough. I appreciate the
pointers.

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

Thanks for notmuch-lore

[Carl Worth]

On Tue, Feb 01 2022, Tobias Waldekranz wrote:
> I actually gave up on getting my mailinglists from my email provider,
> now I just download it directly from lore. I hacked together a script
> that will scrape a public-inbox repo and convert it to a Maildir:
>
> https://github.com/wkz/notmuch-lore

Thanks for sharing this, Tobias. I needed exactly this today, and was
happy to have found this.

It looks like you've coded something to efficiently do the work that's
needed periodically, (fetch new emails from the public-inbox git
repository, convert them to maildir files, and prune away git state
other than a pointer to what's been converted already).

What I'm missing is the piece to convert over the entire archive from
the past.

I can fetch it all easily enough with public-inbox-clone. Maybe what I
want could be captured in a tool named something like:

public-inbox-export --output=maildir

After which I'd be all bootstrapped and ready to use your notmuch-lore
pre-new hook.

> As you can tell from the name, it is tailored for plugging into notmuch,
> but the guts are pretty generic.

Indeed. And it looks like all the code I would need for the export I
described above is right there in your script. It's as simple as:

git rev-list | while read sha; do
$git show $sha:m > $maildir/new/$sha
done

So, next I should go put together a patch against public-inbox to add
that.

Thanks again,

-Carl

PS. I debated whether to CC lkml where the original message I was
replying to was from originally. I decided against it and almost just
emailed Tobias alone, but I really do want discussion like this to be
archived in public. So I CCed the notmuch mailing list at least.


signature.asc
Description: PGP signature
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

Confusing message from notmuch-maildir-fcc-with-notmuch-insert

[Carl Worth]

Hi folks,

I just setup notmuch from scratch for the first time in a very long
time, (as in, for the first time since I created notmuch originally).

And first, thanks to everyone who has kept notmuch going along so well!
It's delightful for me to see how much it keeps improving in the time I
have been hands off from the project. You're all wonderful!

When I first tried sending an email message from emacs with notmuch, it
apparently worked, but then the automatic fcc failed, (because I didn't
have a "sent" folder in my local mail store).

The error message I got was not entirely helpful (although it was trying
hard to be). It said:

  Insert failed: (r)etry, (c)reate folder, (i)gnore, or (e)dit the header?

I didn't clue into what "Insert" meant here. And since I was coming just
from having configured msmtp I thought maybe notmuch wasn't successful
in actually sending the email.

I hit (e) since that looked like something that would let me debug
further.

That took me back to the message where I saw "Fcc: sent" and I clued
into the fact that it was an FCC failure here.

At this point, a truly new user might have been stumped, but I started
code diving, (first with "M-x apropos fcc" since I wasn't sure if the
fcc bug I had hit was a notmuch thing or in some deeper layer).

I found notmuch-fcc-handler in the search results, and saw that it was
now defaulting to using "notmuch insert" by default, (which yes, is a
good thing, and better than the plain FCC handling I did originally that
didn't make recently-sent messages available to notmuch until the next
indexing operation).

So now I realized what "Insert" meant in "Insert failed" and I guessed
that the bug was from the fact that the "sent" directory didn't
exist.

So I ran "maildirmake ~/mail/sent" and retried sending my mail and
everything worked.

Then, I went back into the code to see why it wasn't clever enough to
make a directory itself. Here I found that
notmuch-maildir-fcc-with-notmuch-insert actually does have code to
create the missing directory, but only if told to with an optional
argument.

And the I looked at the code that emitted the error message I saw
originally and finally realized it was giving me the option to create
the missing directory if I had only understood what the message meant
and had pressed "c" to create it.

So, that's the long story of my little confusion this morning. I'm
trying to decide what would be a clean fix here.

Certainly if the code had just created the directory for me, everything
would have been fine. But I guess I can understand why the code doesn't
do that if we're trying to support a case where users are manually
typing random strings in for the Fcc header and might typo something?  I
don't know how much I _really_ care about that use case, (if notmuch
creates a directory based on a typo the mail will still be indexed and
perhaps some notmuch users don't care much about directory names inside
the mail store).

Otherwise, if the message had said:

  Failed to save sent message in local mail store. Directory
  /home/cworth/mail/sent does not exist. Create? (y/n)

Then I definitely would have understood the message and things would
have been much smoother for me.

Thoughts?

-Carl
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

Re: [PATCH] emacs: add global tag history

[Carl Worth]

On Wed, Jan 26 2022, inwit wrote:
>> Even that will not be perfect (since the messages matching the query
>> could change behind emacs back), but close enough for most interactive
>> use, maybe?
>
> I was thinking about saving the IDs of the messages affected by the 
> change, but I still don't know how would I go about that or if it's 
> sensible.

One could imagine a history that would enable a conventional undo stack
for a notmuch interface. The trick with making that usable would be the
need to refresh views to make what was undone evident, (and the fact
that some of the operations could be large/slow).

All of those issues kept me from pursuing the idea in early days of
coding up the emacs notmuch UI. But someone could certainly explore the
implementation further if desired.

-Carl


___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

Re: Dodo: a new graphical, hackable frontend for notmuch

[Carl Worth]

On Tue, Jan 04 2022, Aleks Kissinger wrote:
> We all like to unwind in different ways over the holidays. For me, I
> decided to write a new GUI frontend for notmuch based on PyQt.
>
> It's called Dodo: https://github.com/akissinger/dodo

Cool! I'll be excited to check this out.

> p.s. I added a link to Dodo on the notmuch site using the instructions
> (notmuchmail.org/wikiwriteaccess), but maybe didn't do it right or it
> takes some manual intervention to update the website.

This is on me. I learned over the holiday that the wiki updating is
currently broken, but I haven't yet taken the time to fix it. I hope to
do that soon.

Thanks for your patience!

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

Re: Statistics about senders with notmuch possible?

[Carl Worth]

On Mon, Dec 27 2021, signmeup wrote:
> I would like to create some kind of statistics to see from which sender
> I've got the most emails, having something like a TOP 10 list.
>
[ 4 more citation lines. Click/Enter to show. ]
> In the end of every year I delete all my emails, which are not marked as
> archive and this year I got over 4.000 and most of them are just
> notifications (like ebay, if someone buys something). But I really would
> like to figure out where else the most mails are coming from to avoid
> them in the next year.
>
> Is something like this possible with notmuch?

Yes, it's definitely possible. Being able to script up operations like
this that act on your email store has always been a primary design goal
of notmuch, so this isn't actually hard to do.

Given any sender address you can ask notmuch to count the emails sent
from that address. Such as:

notmuch count from:cwo...@cworth.org

And you can also use notmuch to generate a list of all sender email
addresses across all of your email, with:

notmuch address --output=sender --output=address '*'

So you can take the output from that second command and then run the
first command with email address in the output.

Doing that all combined, and the sorting the result might look something
like the following (assuming you are using bash as your shell):

for sender in $(notmuch address --output=sender --output=address
'*'); do count=$(notmuch count from:$sender); echo "$count $sender"; done |
sort -n

If you wanted to run this on a subset of your email, just change the '*'
to a search specification for the messages you want to match, (where '*'
will match all messages).

And of course, this is running a separate notmuch search for every
sender email address you have in all of your emails, (not only the top
10 like you really want), so this could take some time if you have a lot
of messages. I just tested with a small subset of my own email messages
(the 21k+ messages I have matching 'subject:notmuch') and the above
script took about 30 seconds on my laptop.

I hope that helps,

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

Re: Inconsistencies in handling command flags: `--flag=value` different than `--flag value`

[Carl Worth]

On Wed, Apr 29 2020, Jameson Graef Rollins wrote:
> I dare say there are few people that have muscle memory for the notmuch
> command line (especially for notmuch show), and fewer that aren't
> themselves developers...

Here, the concern isn't just "muscle memory" (forcing users to learn how
to retype commands at the command line) but also that users can have
developed scripts that call notmuch, and we would need to have an
extremely significant reason to break those.

And the notmuch CLI, from the beginning, was intentionally designed to
be comfortable for users to use directly (by typing at an interactive
shell), and for users to also incorporate into various scripts.

And even if they _are_ developers, they don't deserve to have their
environments broken either.

That said, it is unfortunate that some confusion has crept into the API.

I have to admit that I do not understand what this sentence of the
documentation means:

For the cases where it's not ambiguous (in particular excluding
boolean options), a space can also be used.

What are the possible cases where it could be ambiguous?

I'm personally opposed to adding any new option like --strict that
changes how command-line options are parsed.

I'm also opposed to renaming any of our existing command-line options.

It seems clear we could at least fix the wording of the above
documentation to make it clear that a space separator cannot be used for
Boolean options.

The only further change that could possibly make sense (to improve
consistency) would be to back out support for using a space separator
for _any_ options. But that introduces all the opportunities to break
users' environments, so it's probably a non-starter as well.

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: Weird tagging issue

[Carl Worth]

On Mon, Mar 30 2020, Brian May wrote:
> This applies to all 4 of those messages. Yes, they all look like they
> have the same Message-ID

4 different messages all with the same message ID definitely violates a
core assumption of notmuch (that each message as a unique message ID).

I think it would be reasonable to state that the behavior of notmuch is
officially "undefined" in that case.

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: email recipient not taken into account when replying?

[Carl Worth]

On Wed, Jan 08 2020, Alan Schmitt wrote:
> Is there another way to tell notmuch to use as From address the address
> the email was sent to, if it's one of my emails?

Yes. The user.other_email configuration entry should do that.

For example, in ~/.notmuch-config:

[user]
name=Alan Schmitt
primary_email=alan.schm...@polytechnique.org
other_email=alan.schm...@inria.fr,alan.schm...@irisa.fr,presid...@lemondedujeu.org"

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: script to move messages according to tags

[Carl Worth]

Yes,

Notmuch considers any email files with the same message-id header as
being the same message so won't show you multiple versions of the same
email, (unless you're specifically asking for it to with something like
--output=files).

In normal usage I never worry about the duplicates I get from mailing
lists, etc.

I hope you'll find notmuch very useful.

Have fun!

-Carl

On Wed, Dec 11 2019, Alan Schmitt wrote:
> On 2019-12-11 16:33, David Edmondson  writes:
>
>> How you decide to deal with multiple results is really up to you 
>> - in my
>> case I move/delete all of them (except that I don't actually 
>> delete
>> anything).
>
> This is probably the key: I'm used to file-based MUA that will 
> show me
> duplicates. If notmuch hides them from me, I could just ignore 
> that they
> exist (it's not like it's taking too much space anyway…)
>
> Best,
>
> Alan
> ___
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] Display extra headers for emacs-mua - db config option

[Carl Worth]

On Thu, Nov 21 2019, Johan Parin wrote:
> Johan Parin  writes:
>> Daniel Kahn Gillmor  writes:
>>>
>>> Is it that much worse to pass around the notmuch_database_t *?

My opinion is that we should not use the static notmuch_database_t
pointer. I think any desire to reach for that is, at best, papering over
some other problem that would be better fixed itself.

>> It has a lot of code impact, it really propagates to a lot of
>> places.
...
> Here is a taste (not fully tested, but seems to work).
...
>  static int
> -do_reply (notmuch_config_t *config,
> +do_reply (notmuch_database_t *notmuch,
> +   notmuch_config_t *config,

This passing around of the pair of notmuch_database_t* and a
notmuch_config_t* all over the place looks like an anti-pattern to me,
(and something that was just being hidden by the static
notmuch_database_t* that would be better to be fixed properly).

One option for reducing that pair to a single pointer would be to move
the configuration into the database, (as Daniel has been arguing for a
while).

I've argued against that in the past, (and obviously, I came up with the
current approach originally). But the historical situation has already
led to some unpleasant mixing of configuration state, (where some state
is in the configuration file and other state is in the database and it's
hard to predict which is where and which things can be controlled in the
configuration file).

And I think a patch like the above with the "notmuch, config" all over
the place would be another unpleasant result of the historical
convention.

So, I wouldn't be opposed to having the configuration move into the
database entirely. I would definitely like to see a tool that allows for
a dump/restore operation of the configuration state, (so users can still
edit configuration parameters with a text editor and with
fully-documented parameters). Bonus points if the users can also capture
their own commentary/explanation of values they are setting (as they can
do with the current configuration file).

And if things do move, existing configuration files should be updated
with a comment describing what that new dump/restore mechanism is and
that the current configuration file will no longer be used, (though, at
the transition point, obviously configuration parameters should be
copied from the configuration file into the database).

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: Workaround for Exchange-corrupted PGP/Mime structure

[Carl Worth]

On Wed, Nov 20 2019, Carl Worth wrote:
> I'll update my notmuch and give this a try.

Just a "git pull; make; make install" and my problem went away.

Thanks again for the fixes, Daniel.

-Carl



signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: Workaround for Exchange-corrupted PGP/Mime structure

[Carl Worth]

On Wed, Nov 20 2019, David Bremner wrote:
> There's some recent work by dkg in (currrently unreleased) commits on
> master. "git log -Smangled"

Thanks, David, and Daniel!

Those commit messages look very useful to me.

I'll update my notmuch and give this a try.

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Workaround for Exchange-corrupted PGP/Mime structure

[Carl Worth]

I recently received a GPG-encrypted message that notmuch was not able to
handle transparently as I'm used to. (That is, instead of decrypting the
message and verifying the signature inside, etc.—instead I was just
presented with an attachment that contained the encrypted message.)

After some poking around, it appears that the problem is that the
message was sent through Microsoft Exchange which corrupts PGP/mime
structure by changing the outer content-type from "multipart/encrypted"
to "multipart/mixed".

I've verified this bug by sending mail from notmuch through Exchange as
the MTA and it indeed does corrupt the mail in exactly this fashion.

It looks like this bug in Exchange is known[1] and there's evidence of
other MUAs working around the bug[2].

Have other notmuch users encountered this problem?

What would anyone think about a workaround where notmuch-emacs could
grub around inside a "multipart/mixed" structure looking for child parts
that look like they really should have been a part of a
"multipart/encrypted" container?

-Carl

[1] 
https://social.technet.microsoft.com/Forums/en-US/c5059734-5031-4d82-96eb-c74fe60b5e9e/quotthis-is-a-broken-pgpmime-message-from-msexchangequot

[2] https://gitlab.com/mikecardwell/gpgit/blob/master/gpgit.pl#L108
which also refers to Enigmail having a workaround for thi.


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [FEATURE] indexing arbitrary headers

[Carl Worth]

On Thu, May 30 2019, Daniel Kahn Gillmor wrote:
> I just wanted to point out that this specific request has now been added
> to notmuch, thanks to David Bremner!  It will hopefully be part of the
> forthcoming 0.29 release:

Woo-hoo! Thanks so much!!

This has been on my I-wish-I-would-have-done-that-from-the-beginning
list for notmuch for a very long time.

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: how to search for hyphenated words? (was: how to search for Morse code?)

[Carl Worth]

On Tue, Mar 12 2019, Gregor Zattler wrote:
> what I do not understand is that it dosn't matter if I search for
>
> org-notmuch
>
> or
>
> "org-notmuch"
>
> '"org-notmuch"'
>
> or even
>
> org ADJ/1 notmuch

Correct. All four of those forms are giving you phrase searches, (so a
term "org" followed immediately by a term "notmuch").

> a typical example of a matched message is the attached one.
> Somehow the search matches the address of this very mailing list
> in the body of the email (I assume).

No, I don't think you are seeing a match on the mailing-list address
itself, (which has "notmuch" two terms before "org").

> Therefore I wonder why notmuch matches 581 messages, not 16795
> messages or 77 messages.

David showed you one example from the message you copied:

> To: David Edmondson , notmuch@notmuchmail.org

And I showed one earlier in the thread.

In each case, the message includes "org" followed (after some amount of
punctuation and whitespace, perhaps including newlines) by "notmuch".

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: how to search for hyphenated words? (was: how to search for Morse code?)

[Carl Worth]

Hi Gregor,

The trick here is that when notmuch is indexing body text it feeds it
into a Xapian function that parses the text by finding "terms" in the
text. And this parser considers both punctuation and whitespace as
separators between terms.

So your messages are not being indexed in a way to let you distinguish
between "org notmuch" and "org-notmuch".

(Of note, the query parser applies the same parsing to your query---so
that even when you think you're typing an exact phrase like
"org-notmuch" that gets parsed into separate terms "org" and "notmuch"
for searching.)

> all these resulted in very many hits most or all of which do not
> contain the string "org-notmuch", one found email was e.g.
>
> id:20180904105723.15564-3-da...@tethera.net

That message does contain the following:

   +test_emacs '(notmuch-tree "id:000-real-r...@example.org")
   +   (notmuch-test-wait)

Where you will notice that there's a term "org" followed (after some
punctuation and whitespace separators) by a term "notmuch".

> How would one search for hyphenated words with notmuch?

You would need to arrange to have the indexer consider the hyphen as a
letter-like character to be made part of terms. Or be extra clever and
index something like "notmuch-test-wait" in multiple ways (such as a
single term "notmuch-test-wait" as well as three adjacent terms
"notmuch", "test", and "wait" as notmuch is doing currently).

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: travis build env switched to xenial

[Carl Worth]

On Wed, Mar 06 2019, David Bremner wrote:
> Those of you about to tell me how much better FreedomAvocadoCI [2] is
> than Travis are encouraged to set up instances and we can try it out. As
> long as Carl doesn't object, I would be interested in a setup where we
> run e.g. webhooks directly on git.notmuchmail.org to remove our
> dependence on github for CI.

I'm all in favor of removing a dependence on a third-party service here,
so would not object to adding some webhooks to our server.

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: Emacs and the missing 'g'

[Carl Worth]

On Tue, Feb 12 2019, Matt Armstrong wrote:
> I'm returning to notmuch after a hiatus.  It is still nice!

Glad to hear it! Welcome back.

> Most notmuch modes are "special" modes -- i.e. their text is generated
> by Emacs and does not reflect a file.  In these modes 'g' has
> traditionally been bound to revert the buffer [1].
...
> Any objection to also binding 'g'?

I have no opposition to this. I wasn't aware of the convention around
'g' when I first did the thing with '=', but I'd be happy to match that
convention to have notmuch feel more like a well-behaved citizen of the
emacs universe.

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: Release signatures

[Carl Worth]

On Sun, Feb 10 2019, David Bremner wrote:
> Adam Majer  writes:
>> The releases are signed in a funny way. The .asc file are not detached
>> signatures of the checksum, but actually contain it inside the .asc file.
>>
>> # gpg -v --verify notmuch-0.28.1.tar.gz.sha256.asc
>> ...
>> gpg: binary signature, digest algorithm SHA256, key algorithm rsa3072
>> gpg: WARNING: not a detached signature; file
>> 'notmuch-0.28.1.tar.gz.sha256' was NOT verified!
>>
>> A much better way of signing this would have been as a detached
>> signature of the tarball itself. Why sign a hash of a hash? ;)
>
> I'm not sure why Carl did it that way 10 years ago. Perhaps Carl
> remembers?  Offhand, I don't see any reason not to go with a more
> standard detached signature, other than it needs someone to do the
> relevant work.

If I did something non-standard here it certainly wasn't intentional.

I certainly would not oppose moving to a more standard (and obvious to
us) means of signing the releases.

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: question about deletion and counts

[Carl Worth]

On Sat, Nov 03 2018, Jeff Templon wrote:
> No new mail. Removed 577 messages. Detected 89 file renames.
>
> 577 + 89 = 666 ... my guess is that there were 577 messages and 89 files
> that represented duplicates of messages.

Yes. Among the messages you had tagged as deleted you had 577 unique
message IDs. In addition, you had another 89 files with message IDs that
were duplicates of one of the 577.

> But I didn't rename the files, I deleted them.  Should I worry?

Nope. Nothing to worry about here.

> Why is the message inaccurate?

Because notmuch has a more narrow view of what a "rename" is than you
do.

A file rename is a high-level operation that will be seen by notmuch as
multiple operations seen over the course of a single run of notmuch
new:

  1. A new file is added with a message ID that already exists in the
 database

  2. A file is removed with a message ID for which there are multiple
 files in the database

But notmuch doesn't detect whether both of these operations are seen in
a single pass in order to detect a rename. Instead, what it is doing is
counting every occurence of (2) above as a rename. Here's what the code
looks like (notmuch-new.c:remove_filename):

status = notmuch_database_remove_message (notmuch, path);
if (status == NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID) {
add_files_state->renamed_messages++;
if (add_files_state->synchronize_flags == true)
notmuch_message_maildir_flags_to_tags (message);
status = NOTMUCH_STATUS_SUCCESS;
} else if (status == NOTMUCH_STATUS_SUCCESS) {
add_files_state->removed_messages++;
}

So, whenever removing a filename, it will either get counted as a rename
(if there is still at least one other filename in the database with the
same message ID), or it will get counted as a removal (if this was the
last filename for message ID).

I suppose you could come up with some other name for what it is
counting, such as "removals of duplicate messages" instead of "rename",
but that's what's happening.

I hope that helps.

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: finding file by size

[Carl Worth]

On Tue, Nov 06 2018, Ralph Seichter wrote:
> I'm not sure about using Notmuch itself,

Right. Notmuch doesn't currently index (as far as I'm aware) anything
that would be useful for sorting by size.

> but this should work:
>
>   find /path/to/maildir -type f -size +50M | xargs rm

Hmm... I imagine that Mark would be more interested in viewing these
files to ensure they are what he thinks they are before deleting them.

So, capturing the results of that with a notmuch tag would be
a reasonable thing to do. The only trick there is that I don't see any
existing search term to find a message associated with a particular file
name, (we have "path:" and "folder:" to find messages in a specific
directory, but nothing I see for finding the message corresponding to a
specific file).

So, then we could extract the message-id from each file and do a search
based on that I guess?

Here's a (bash) command I just ran on my mail store of over a million
messages that tagged the 8 messages larger than 50MB. It took about a
minute to run (with a warm cache):

  for msg in $(
   for file in $(find . -type f -size +50M);
   do
 grep -i ^Message-Id $file | sed -e 's/^.*<\(.*\)>.*/\1/';
   done
 );
  do
notmuch tag +large id:$msg;
  done

With that, I'm able to go through the list from:

  notmuch search tag:large

to investigate whether these large emails are worth keeping.

So, that's obviously not extremely elegant, but it's at least possible.

-Carl



signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: tell me how to do this (mass tagging of messages)

[Carl Worth]

On Wed, Oct 17 2018, Jeff Templon wrote:
> Indeed, as long as I am satisfied that all the messages in the view are
> OK to delete / tag / whatever ... I usually look at such at view to make
> sure that eg. the SPAM filter is doing the right thing.

I have not found any case in the notmuch-emacs UI where this has been an
issue for me, personally. If you do find a use case that's problematic,
let us know and we could certainly expand the interface to accommodate
that in some way.

Meanwhile, at the command line this distinction is made very clear. Try
each of these commands, for example:

notmuch search some-search-pattern

notmuch search --output threads some-search-pattern

notmuch search --output messages some-search-pattern

The second and third options there are very appropriate for generating a
list of threads (or messages) that you could then iterate over and
perform operations on.

And of course, if you're wanting to do an operation such as tagging or
un-tagging, then you can do that at that command-line directly without
needing an external loop over the results:

notmuch tag +some-tag some-search-pattern

notmuch tag -some-tag some-search-pattern

Both of these operations will act on matched messages, (no threads are
involved).

So hopefully playing with the above operations makes it clear how
notmuch itself is operating.

And the notmuch-emacs stuff is a pretty simple layer on top of that, so
again, we could easily improve things as necessary.

> My opinion : the best option would be to have some flag / toggle
> variable / whatever that would cause a search results screen to display
> only matching messages and not the whole thread.

That would certainly be an option.

But again, what operation does notmuch-emacs allow you to do on the
thread that you are really only wanting to do on the matching messages?

For me, a frequent operation is to just _see_ the matching messages. And
for this particular operation, just pressing "Enter" on a thread does
exactly that. It brings up a view of the whole thread, but with only the
matching messages open (and non-matching messages collapsed).

-Carl




signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: tell me how to do this right (mail sent to lists)

[Carl Worth]

On Tue, Oct 09 2018, Jeff Templon wrote:
> So here’s the deal : I am tagging with afew, it has a special mailing
> list filter, which does the right thing - it tags the copy that was
> delivered from the mailing list with the list name.

I think you may be mistaken about the above details.

If the two messages have the same message ID, then as far as notmuch is
concerned that is just a single message, (that happens to be backed by
multiple files).

> However (and maybe this is just the Emacs interface?) when I view the
> message in notmuch-emacs, it only shows one of them, and this seems to
> be (always?  at least usually) the copy that is Fcc’d to my sent
> folder and tagged only with “sent” because it doesn’t have a list-id
> yet.

It is true that if a single message is backed by multiple files, the
notmuch-emacs interface will only display one of them. But even if you
changed which one it chooses (which would require some new code as far
as I know) shouldn't change what tags you are seeing.

> So I’m still looking for how to do this “right”.  I must not be the
> only one that sends mails to mailing lists?

You're certainly not the only one. Hopefully, what you really want
doesn't require the ability to maintain separate tags for these two
very-similar messages, (since notmuch cannot do that currently).

I guess the question I have is: What has given you the idea that afew is
tagging one message but not the other?

-Carl



signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: notmuch shows tag which is not on any email

[Carl Worth]

On Mon, Jun 04 2018, Gregor Zattler wrote:
> Dear notmuch developers,

Hi, Gregor!

> does notmuch remember tags even if at time of query there is no
> message tagged with the respective tag?:

No. At least, from when I originally invented the notmuch schema, tags
only existed on messages and there was no separate store of a list of
known tags anywhere.

If something in the schema has changed since, I would appreciate someone
correcting me.

So, I'm not sure what's going on in with your notmuch database.

I tried tagging a couple of messages with the same tags you are seeing
in case there was something in one of these tag names specifically that
was ticking a problem. But things look fine for me.

Here's the output I'm now getting on my notmuch store with the same
commands you gave, (and note that everything is working for me):

$ notmuch search --output=tags '*' | grep telegraph
EA%3Dtelegraph%40gmx%2Enet
EA=telegr...@gmx.net
$ notmuch count  -- is:EA%3Dtelegraph%40gmx%2Enet
1
$ notmuch count  -- is:/EA.*telegraph/
2
$ notmuch search --output=tags '*' | grep telegraph | head -n 1 | wc -c
27

Anyone ave a theory for what could explain what Gregor is seeing?

-Carl



signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: a temporary nmweb view of notmuch mailing list archive

[Carl Worth]

On Sun, Feb 04 2018, Daniel Kahn Gillmor wrote:
> I did a bit of experimenting with Brian Sniffen's proposed notmuch-web
> branch, and it's now (temporarily) publishing a view of the notmuch
> mailing list here:
>
> https://nmbug.notmuchmail.org/btsmail/

Thanks, Brian and Daniel!

When I first started on notmuch, I was hoping that one immediate benefit
would be that free-software projects could finally have working web
archives of their mailing lists, (without thread views breaking at
arbitrary boundaries such as months or years).

I guess the "immediate" aspect of my hope didn't play out so much, but I
am very excited to see this now. So, thanks again!

At first glance I saw "terms ... after ... before" and thought that the
after/before fields were for some sort of contextual context, but I
finally figured out what those were actually for. So maybe it would help
to add some text along the lines of "Limit to date range" or so.

And now that we have real date searching, it might make sense to use
that in the searching instead of the time stamp.

Another thing that might be nice for selecting dates ranges is the
ability to select something less precise than a particular day.

Searching for all the messages in a month with something like:

notmuch search date:2018-01

or even a year with:

notmuch search date:2018

is pretty darn convenient in notmuch, or even to search across two
months with:

notmuch search date:2018-01..2018-02

It would be nice to not lose that kind of convenience in a web
interface.

Anyway, thanks again. This is looking great, and I will look forward to
entirely removing the pipermail-based archives from the website as soon
as we can.

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: Fetching from the git repositories over https?

[Carl Worth]

On Sun, Jan 28 2018, Adam Plaice wrote:
> I apologise if I'm asking in the wrong place.

Not at all. This is the right place.

> Is it possible to clone/fetch from the notmuch git repositories
> (particularly https://git.notmuchmail.org/git/notmuch) over https
> rather than with the `git://' protocol?  (None of the likely
> alternatives seem to work.)

It wasn't possible when you asked, but I just configured this, and it
seems to work.

Specifically, I have tested that I can point my browser at:

https://git.notmuchmail.org/git/notmuch

to see the gitweb view of the git history on the web, and I can also use
that same URL for a git clone:

git clone https://git.notmuchmail.org/git/notmuch

and that works.

I also verified that I a "git push" from such a clone results in a 403
error as desired.

So give that a try, and anyone, let me know if you see anything that I
may have broken or setup incorrectly.

Also, I haven't yet updated any documentation to point to this new
mechanism, so that's something that could still be done.

> Thank you for your time and thank you for notmuch,

You're quite welcome. And thank you for your contribution!

-Carl
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] debian: drop export-dir from gbp.conf

[Carl Worth]

On Tue, Feb 06 2018, Daniel Kahn Gillmor wrote:
> No one knows why this invocation is here.  it does weird things during
> the build.  If a developer really wants this behavior, they can
> specify it in their own ~/.gbp.conf

I imagine I'm to blame for this one?

One of the things that has always bugged me about a bunch of that Debian
build utilities is that they spit out results into the parent directory
of the current working directory from where they are invoked.

I've always found that be really mystifying, (no well-behaved program
does such a thing).

I _think_ the export-dir directive here was something I found that made
git-buildpackage behave more sane in this regard.

But I'm not the one doing frequent package builds of notmuch these days,
so feel free to do what you think best.

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: Possible bug with named queries

[Carl Worth]

On Wed, Dec 13 2017, David Bremner wrote:
> STORED IN DATABASE, thanks to some doc cleanup by dkg. In general I'd
> recommend using using "notmuch config" for all configuration changes,
> then you don't have to keep track of which ones are stored in
> .notmuch-config.

Really? We're deprecating the ability to just edit the configuration
file by hand? That's unfortunate.

> There are technical reasons for certain items being in
> the database, and that's likely to continue.

I won't dispute the technical reasons, (I'm out of the loop on that).

But if running "notmuch config" to add a saved search stores some
necessary data in the database, couldn't the same code be invoked when
loading the configuration file and noticing a new saved search there in
the file?

We've gone out of our way to make the configuration file as useful as
possible, (such as by maintaining comments in the file even when using a
tool such as "notmuch config" to edit it). Let's please not throw out
all the benefits by just saying that manually editing the configuration
file is unreliable and people shouldn't do it.

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: notmuch.el: controlling what does and doesn't get expanded in searches

[Carl Worth]

On Thu, Aug 04 2016, Matt Armstrong wrote:
> Yes, I find the query semantics with respect to tags and threads a bit
> confusing at times.  This is not a problem specific to notmuch, as I
> find the same kinds of issues in GMail.  Usually the problem occurs at
> the semantic border between per-message tags and thread-based
> operations.
>
> I can now explain what I am seeing.

Hi Matt,

I haven't been very active on the notmuch mailing list in quite some
time, but I wanted to poke my head in quickly to welcome you and to
thank you for your contribution.

I really like detailed discussions like this about coming up with good
workflows, and trying to figure out how notmuch could better accommodate
those workflows. I think this is one of the most valuable aspects of
notmuch, (that it lets us ask these kinds of questions).

So, please, keep it up!

>  i) notmuch could have an "also expand tags" feature, where thread based
> results would auto expand matching tags.  I would set this to
> "unread".

This approach makes a lot of sense to me based on how notmuch.el works.

>  ii) notmuch could have an "expand query" feature, where thread based
>  results would use an entirely different query to decide, within a
>  thread, which messages to expand.  I would set this query to
>  "tag:unread".

This approach would necessarily be quite a bit more complex in the
implementation without much difference in the user-visible behavior. So
I don't think we would want to pursue this.

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: Push to wiki fails with "cannot run hooks/pre-receive" error

[Carl Worth]

On Wed, Mar 23 2016, J. Lewis Muir wrote:
> I just tried to push a minor fix to the wiki, but I get the following
> error:
...
> remote: error: cannot run hooks/pre-receive: No such file or directory

Oops. My mistake.

I recently converted all user-space packages on the server from 32-bit
to 64-bit, but was inadvertently over-zealous and removed the packages
necessary to still run a 32-bit binary.

Of course, there aren't any packaged 32-binaries left around, but this
pre-receive hook is a binary that ikiwiki compiles, so it is still
currently a 32-bit binary.

I just installed the support for running 32-bit binaries again, so
hopefully this will work now.

Let me know if you run into any further problems.

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: Notmuch -- Just an email system

[Carl Worth]

On Sat, Mar 12 2016, David Bremner wrote:
> I'm not carl, but you can find information about licensing of the wiki
> material here
>
>   https://notmuchmail.org/COPYING/
>
> We'd be happy to host a russian version of some of the pages on the
> wiki; see
>
>   https://notmuchmail.org/wikiwriteaccess/
>
> for how to add pages.

And I am Carl, and I approve of the above message!

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: Stolen email list?

[Carl Worth]

On Thu, Mar 03 2016, David Bremner wrote:
> Johnny Utahh  writes:
>
>> I recently received email sent to my notmuch-based email address that 
>> was clearly spam. I suspect that notmuch@notmuchmail.org email-address 
>> list may have been stolen. If you're not already aware.
>
> I'm not aware of any such event. I've CC'd Carl, who runs the lists.

I'm certainly not aware of anything either.

But I'd be happy to investigate anything I can, (if you have data to
point me to where to look).

In the meantime, if the list was leaked, are there any mitigating steps
you would recommend we perform at this point?

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: NOTMUCH_STATUS_LAST_STATUS problematic across additive library upgrades

[Carl Worth]

On Mon, Dec 07 2015, Daniel Kahn Gillmor wrote:
> I note that it's exposed in the go bindings -- maybe it shouldn't be
> exposed in any of the bindings that can avoid exposing it?

Quite likely. It's only there in C because there's no other way to know
the bounds for iterating over all possible values for a particular
enum. If other languages don't have this deficiency, then they shouldn't
need this silly thing.

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: elisp + company completion patches, v7

[Carl Worth]

On Sun, Oct 25 2015, Tomi Ollila wrote:
> I, if this does not sound nitpicking, would amend those 'This patch' and
> 'With this patch' texts in commit messages away by some rewording -- those
> pose as a bad example for someone reading the code -- and diminish the
> effect when I complain that again in future patched ;)

What's the motivation here?

It's often the case that a good commit message needs to describe the way
things were working prior to a commit, (describing the bug or missing
feature or what have you), and also needs to describe the change that is
affected by the commit itself.

So I actually like seeing commit messages that say things like:

Prior to this commit... (some bug existed...)

In this commit... (some code is reworked to fix the bug...)

But maybe I've missed the point of your message.

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] doc: update copyright year

[Carl Worth]

On Thu, Oct 15 2015, Tomi Ollila wrote:
> -copyright = u'2014, Carl Worth and many others'
> +copyright = u'2015, Carl Worth and many others'

I think the correct approach to copyright here is to expand the range of
years, (so "2014-2015") rather than increment it.

Presumably that means the starting date should be 2009, too.

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: correct way to search for only PDF attachments

[Carl Worth]

On Mon, Sep 28 2015, Xu Wang wrote:
> I would look to look for all emails from a colleague jongho. I tried:
>
> from:jongho attachment:pdf
>
> which seems to do as I wanted.

Good. That should work.

> To understand more, what does the following search for?
>
> from:jongho attachment:.*pdf

Uhm, probably only strange things. There are some mechanisms for getting
notmuch to emit some debugging information on what the final search
terms end up being, (but I don't recall if they still require
recompilation or not).

I'm not testing now, but I wouldn't be surprised if that ended up doing
something like searching for a phrase like "attachment pdf" anywhere
within a message. (The Xapian parser can be somewhat unpredictable when
you give it unexpected input.)

> Also, how does the first one above know that I want only PDF
> attachments and not an attachment called "pdformula.txt" ?

It doesn't know that you want only PDF attachments. The key part is that
the indexing is performed by breaking text up into individual terms, (at
punctuation boundaries usually). So a search specification like
"attachment:pdf" is searching for things that were indexed with the
"pdf" term within the attachment prefix. So that won't match a filename
like pdformula.txt, (which would be indexed as two terms, "pdformula"
and "txt"), but it would match pdf.ormula.txt, (which would be indexed
as three terms, "pdf", "ormula" and "txt").

The Xapian documentation can be examined if you want more details.

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Website and mailing list are hopefully back now

[Carl Worth]

On Fri, Apr 03 2015, Carl Worth wrote:
> The notmuchmail.org server experienced a catastrophic failure a few days
> ago. I gave up on getting access to that server anymore, and have
> instead brought things back up on a separate server that I had
> available, (and to which I have actual, physical access this time).

Oh, and by the way, when you use ssh to the new server from an account
that had connected to the old server, you'll likely get the scary
warning that the ssh host key has changed.

Here is the fingerprint of the RSA key that you should expect to get:

1f:64:19:21:50:8f:1a:72:bc:69:45:8c:25:c5:71:7a

With that fingerprint, you can assume that the key change isn't due to a
man-in-the-middle attack and follow the instructions on removing the
old key.

-Carl
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 818 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20150403/ae983f46/attachment.sig>

Website and mailing list are hopefully back now

[Carl Worth]

Hello all,

The notmuchmail.org server experienced a catastrophic failure a few days
ago. I gave up on getting access to that server anymore, and have
instead brought things back up on a separate server that I had
available, (and to which I have actual, physical access this time).

I restored the git repository and website from a nightly backup that was
taken late on March 30. (The server seems to have failed sometime
between then and when the nightly backup on March 31 was attempted.)

There are likely things that are not quite properly setup on the new
server, as well as data that I may not have copied from the backup.
If you encounter anything, please let me know and I'll see what I can
do.

Thanks for all your patience,

-Carl

PS. When the new server came up, the mail server wasn't properly
handling mailing-list traffic. So a few messages came to the server,
(and landed in the web archives), but didn't get sent out to subscribers
of the list. Obviously, the messages hopefully still went out to
individuals directly addressed by the mails.

Here are the list of message-ids for the mails that didn't go out. If
you sent any of these, you might want to check whether they are
important enough to resend to the list:

1428047832-14922-1-git-send-email-jani at nikula.org
1428047832-14922-1-git-send-email-jani at nikula.org
yq65pp7l9a6f.fsf at jinwoo-macbookair.roam.corp.google.com
1428047832-14922-1-git-send-email-jani at nikula.org
yq65pp7l9a6f.fsf at jinwoo-macbookair.roam.corp.google.com
1428047832-14922-1-git-send-email-jani at nikula.org
yq65pp7l9a6f.fsf at jinwoo-macbookair.roam.corp.google.com
1428047832-14922-1-git-send-email-jani at nikula.org
yq65pp7l9a6f.fsf at jinwoo-macbookair.roam.corp.google.com
87wq1s4yz5.fsf at maritornes.cs.unb.ca
1428047832-14922-1-git-send-email-jani at nikula.org
yq65pp7l9a6f.fsf at jinwoo-macbookair.roam.corp.google.com
87wq1s4yz5.fsf at maritornes.cs.unb.ca
87sichmoaa.fsf at hirscho.lama.univ-savoie.fr





















-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 818 bytes
Desc: not available
URL: 

Website and mailing list are hopefully back now

[Carl Worth]

Hello all,

The notmuchmail.org server experienced a catastrophic failure a few days
ago. I gave up on getting access to that server anymore, and have
instead brought things back up on a separate server that I had
available, (and to which I have actual, physical access this time).

I restored the git repository and website from a nightly backup that was
taken late on March 30. (The server seems to have failed sometime
between then and when the nightly backup on March 31 was attempted.)

There are likely things that are not quite properly setup on the new
server, as well as data that I may not have copied from the backup.
If you encounter anything, please let me know and I'll see what I can
do.

Thanks for all your patience,

-Carl

PS. When the new server came up, the mail server wasn't properly
handling mailing-list traffic. So a few messages came to the server,
(and landed in the web archives), but didn't get sent out to subscribers
of the list. Obviously, the messages hopefully still went out to
individuals directly addressed by the mails.

Here are the list of message-ids for the mails that didn't go out. If
you sent any of these, you might want to check whether they are
important enough to resend to the list:

1428047832-14922-1-git-send-email-j...@nikula.org
1428047832-14922-1-git-send-email-j...@nikula.org
yq65pp7l9a6f@jinwoo-macbookair.roam.corp.google.com
1428047832-14922-1-git-send-email-j...@nikula.org
yq65pp7l9a6f@jinwoo-macbookair.roam.corp.google.com
1428047832-14922-1-git-send-email-j...@nikula.org
yq65pp7l9a6f@jinwoo-macbookair.roam.corp.google.com
1428047832-14922-1-git-send-email-j...@nikula.org
yq65pp7l9a6f@jinwoo-macbookair.roam.corp.google.com
87wq1s4yz5@maritornes.cs.unb.ca
1428047832-14922-1-git-send-email-j...@nikula.org
yq65pp7l9a6f@jinwoo-macbookair.roam.corp.google.com
87wq1s4yz5@maritornes.cs.unb.ca
87sichmoaa@hirscho.lama.univ-savoie.fr























pgpfOqZrlixHH.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: Website and mailing list are hopefully back now

[Carl Worth]

On Fri, Apr 03 2015, Carl Worth wrote:
 The notmuchmail.org server experienced a catastrophic failure a few days
 ago. I gave up on getting access to that server anymore, and have
 instead brought things back up on a separate server that I had
 available, (and to which I have actual, physical access this time).

Oh, and by the way, when you use ssh to the new server from an account
that had connected to the old server, you'll likely get the scary
warning that the ssh host key has changed.

Here is the fingerprint of the RSA key that you should expect to get:

1f:64:19:21:50:8f:1a:72:bc:69:45:8c:25:c5:71:7a

With that fingerprint, you can assume that the key change isn't due to a
man-in-the-middle attack and follow the instructions on removing the
old key.

-Carl


pgpi1PXbXAk0R.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 07/11] lib: Implement ghost-based thread linking

[Carl Worth]

On Fri, Oct 03 2014, Austin Clements wrote:
 This updates the thread linking code to use ghost messages instead of
 user metadata to link messages into threads.

 In contrast with the old approach, this is actually correct.

Hi Austin,

I've read through your commit messages, (not the code), and I just
wanted to say a big thank you to you!

I'm really looking forward to the improved robustness in notmuch with
this support.

I really appreciate your diligence in working through a fairly large,
low-level change here.

-Carl


pgpavnlvD49KS.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] cli: Be more helpful when .notmuch-config does not exist

[Carl Worth]

Austin Clements amdra...@mit.edu writes:
 Since this condition is expected for new users, this patch adds
 specific handling for the file-not-found case to give a message that
 is friendly and actionable.
...
 + fprintf (stderr, Configuration file %s not found.\n
 +  Try running 'notmuch setup' to create a 
 configuration.\n,
 +  config-filename);

Thanks, Austin!

I've always wanted notmuch to provide as gentle an introduction as
possible, (since switching MUA and indexing all email[*] is already a
fairly invasive thing).

So I really appreciate improvements like this.

 +test_begin_subtest Notmuch new without a config suggests notmuch setup
 +output=$(notmuch --config=new-notmuch-config new 21)
 +test_expect_equal $output \
 +Configuration file new-notmuch-config not found.
 +Try running 'notmuch setup' to create a configuration.

And a test to boot. That really warms my heart.

-Carl

[*] A further possible improvement here would be some mechanism for
initially indexing only very recent email, (to let the user start
playing with notmuch right away), and then in some sort of ''offline''
fashion, completing the full indexing.


pgpf52pf6_z_w.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH v2] emacs: jump: sort-order bugfix

[Carl Worth]

Mark Walters markwalters1...@gmail.com writes:
 The symptom is that if you are in a search buffer and use notmuch jump
 to run a saved search which does not have an explicitly set sort order
 then the sort order of the saved-search is inherited from the current
 search buffer rather than being the default search order.

 Thanks to Jani for finding the bug.

Thanks, Mark. The description above is just what I was looking for.

 (Part of the reason I did not send more before is I had not checked
 what the exact outcome of the buggy code was: it was obvious what the
 code was intended to do, and that with the extra quote it
 would do what it was intended to do.)

Yes. I understand now how the bug-fix was more obvious than the bug.

But thanks for doing the extra investigation nonetheless.

-Carl


pgpUyACD1apZc.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] emacs: jump: sort-order bugfix

[Carl Worth]

Mark Walters markwalters1...@gmail.com writes:
 default-value needs its argument to be quoted.
 ---

Hi Mark,

I really appreciate you sending in a bug fix. Thanks!

I'd prefer to see the commit message describing the bug and the fix a
bit.

You do have some text here that would be a great addition to the commit
message:

 Slightly strangely default-value of 't or nil is 't or nil
 respectively so the code didn't give an error but just did the wrong
 thing.

But even beyond that. What is the wrong thing that the code did here?

Imagine someone (me!) trying to read and review the patch without ever
encountering the bug before. How can I test the bug both before and
after applying the patch?

I'd really like to see that information in the commit message.

Thanks,

-Carl


pgpkfzqsTF18i.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH v1] emacs: `notmuch-search-find-stable-query-region' should never return an empty query.

[Carl Worth]

Carl Worth cwo...@cworth.org writes:
 Mark Walters markwalters1...@gmail.com writes:
 Ok I now understand some of why it is a bug and it is a pretty gross
 mix: emacs does something odd, then notmuch-tag.c then xapian. It only
 happens in some cases:
...
 Anyway, I'm glad to see the notmuch emacs interface getting patched for
 this problem. We should ensure that the Xapian parsing bug is reported
 as well.

*And*, since Mark identified bugs in three different layers, we should
 fix that third layer as well.

It seems we could make notmuch-tag.c more robust by using native Xapian
interfaces to construct a query AND query data structure rather
than using string manipulation in an attempt to construct this same
query. That would prevent this ambiguity, and avoid any other cases of
emacs (or other users) running across a similar bug here.

-Carl

-- 
carl.d.wo...@intel.com


pgpqcsHyu7MQy.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH v1] emacs: `notmuch-search-find-stable-query-region' should never return an empty query.

[Carl Worth]

Mark Walters markwalters1...@gmail.com writes:
 Ok I now understand some of why it is a bug and it is a pretty gross
 mix: emacs does something odd, then notmuch-tag.c then xapian. It only
 happens in some cases:

 WARNING the following will archive some random messages: have a dump or
 backup or something!

Thank you!!

I'm quite sure that I've been bitten by this bug a couple of
times. (Actually, it's likely bitten me more than the couple of times I
happened to notice it.) When I noticed it, I had only a few messages in
my inbox, (had just been reading/archiving some), pressed '=' to
refresh, and several of them disappeared.

It was extremely disconcerting that notmuch would lose data like that on
me. I hadn't reported the bug because I hadn't found any reliable way to
replicate it, nor could I be sure that it wasn't something unique to my
setup, (such that I had actually archived those messages, but from some
other script or view, etc.).

But given your description, I'm confident that the data loss I watched
before my eyes fits that description.

 ( () ) and ( tag:inbox )

 for reasons which aren't clear to me this matches some but not all
 of the messages with tag:inbox. This is despite the fact that searching for
 ( () )  or () by itself yields no results.

You can use NOTMUCH_DEBUG_QUERY to get some insight into this.

So the query that works as expected:

$ NOTMUCH_DEBUG_QUERY=1 notmuch count (())
Query string is:
(())
Exclude query is:
Xapian::Query()
Final query is:
Xapian::Query()
0

And then the query that is misbehaving:

$ NOTMUCH_DEBUG_QUERY=1 notmuch count (()) and tag:inbox
Query string is:
(()) and tag:inbox
Exclude query is:
Xapian::Query()
Final query is:
Xapian::Query((Tmail AND (Zand:(pos=1) FILTER Kinbox)))
14

So you can see that the query is actually matching a term Zand, (that
is, matching on emails for which the word and has been indexed).

It's a funny bug. Note that using () and tag:inbox does not trigger
the same behavior:

$ notmuch count () and tag:inbox
A Xapian exception occurred: Syntax: expression AND expression
Query string was: () and tag:inbox
0

I would have much preferred getting that exception instead of the query
we were getting before.

Anyway, I'm glad to see the notmuch emacs interface getting patched for
this problem. We should ensure that the Xapian parsing bug is reported
as well.

-Carl


pgpDHzyKTi8qh.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 0/5] nmbug-status fixups from notmuch-to-html

[Carl Worth]

W. Trevor King wk...@tremily.us writes:
 I've cherry-picked the three patches I liked from Carl's master [1,2]
 and added two cleanup commits of my own.  I've left Carl's commit
 messages unchanged.  I'm fine squashing my fixups into Carl's
 originals (leaving Carl as the author) if that's ok with him.

Please feel free to do whatever you like with any of these.

Definitely update the commit messages to match the reality of what
you're committing. You can do that by directly editing the text, or
adding a clarifying paragraph after the fact.

And if you make any substantial changes, I certainly won't mind (or even
notice) if my authorship doesn't remain.

Thanks for ensuring the code doesn't get lost. I appreciate that.

-Carl


pgpsVqm6UK2YI.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] nmbug-status: Make the footer's hr gray

[Carl Worth]

W. Trevor King wk...@tremily.us writes:
 On Sun, May 11, 2014 at 12:08:10AM +0300, Tomi Ollila wrote:
 The rendered hr/ irritates me a bit, though. Maybe it could be
 colored light gray ?

 I can't reroll that one myself, unless Carl gives the ok.  But here's
 some light-gray styling as an additional patch.

To be totally clear. Yes, hack it up like crazy however you see fit!

-Carl


pgpM1HTI69LYl.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: folder and path completely broken in HEAD?

[Carl Worth]

dm-list-email-notm...@scs.stanford.edu writes:
 First, thanks for the response.  The responsiveness and friendliness of
 the notmuch mailing list goes a long way towards compensating for any
 missing features / customizability one might want.

I'm delighted to hear that the notmuch community maintains such a
helpful and friendly atmosphere.

 However, currently it seems strange that there are *two* different
 search terms (folder and path), and that neither one lets you search for
 a portion of your folder name.

For what it's worth, I totally agree. I'm guilty as far as sitting out
of the detailed design discussions, (I don't use any sort of
folder-based searching, so I don't care too much). I was aware of the
problems of the original folder: code I wrote, and I was happy to hear
that people were addressing those problems.

But it's terribly strange to me that notmuch now has two different
search terms that overlap so much in functionality. I know that I will
never trust myself to be able to distinguish/describe the folder:
vs. path: semantics without consulting the documentation. And that's
discouraging.

I think the original folder: shortcomings could have been addressed
without adding two terms, and also without losing some functionality,
(as shown in David's use case).

I would have liked to have seen some explicit syntax for anchoring the
beginning and end of the directory name, (which could have then been
re-used for anchoring subject: or even some future header: prefix).

 First, are there people out there who do not use a collection of maildir
 directories, with all mail in cur and new?

When I started notmuch, my entire mail archive was in non-maildir
directories. Since then, the bulk of my mail has been converted to
maildir (by accident more than anything). But I still do create
non-maildir directories for mail occasionally.

One use case for this is when someone forwards me an email (or a thread
of emails) to demonstrate a bug in notmuch. In a case like this, I will
do something like:

mkdir ~/mail/dm-folder-bug-2014-05

and then copy the mail files into that directory. I see no benefit nor
point to creating cur and new directories in a case like this.

 If not, why does notmuch try to find mail in non-mail-directories, and
 why do you need search terms differentiating new and cur?

I've always thought that the cur and new directories were somewhere
on the spectrum between pointless and annoying. The idea with the
original folder: indexing was to implicitly ignore these, (when both
existed).

It seems that the new folder: continues this idea, while the new
path: does not. Meanwhile, the new folder: anchors the search to the
beginning of the directory, while path: does not. And finally, path:
adds a magic syntax to do hierarchical matching while folder: does
not.

That's an odd hodgepodge of non-orthogonal distinction in functionality.

 Conversely, I find it particularly weird that there's no convenient
 way to say stop trying to index stuff that isn't in a maildir (cur or
 new).  You can do the inverse and blacklist files, but then I end up
 with stuff like this in my .notmuch-config:

Like I said, I started with an assumption of a hierarchy of directories
containing nothing but mail files, (that's exactly what I had at the
time).

 
 ignore=dovecot-keywords;dovecot-uidlist;dovecot-uidvalidity;dovecot.index;dovecot.index.cache;dovecot.index.log;dovecot.index.log.2;dovecot.index.search;dovecot.index.search.uids;maildirfolder;

I really do like explicitly describing the things you want to
ignore. One of the worst things that notmuch could do is to silently
fail to index legitimate emails. So I would not be in favor of features
that made that easier.

So the above looks fairly reasonable to me. Though it would also be
reasonable to allow for some wildcards there so that you could have
simply:

 ignore=dovecot*;maildirfolder;

 Second, does anyone out there have a collection with more than a few
 thousand maildirs?

I certainly don't. I have just short of one million messages, but only
841 directories in my mail store. And since that's counting .,
./tmp, ./cur, and ./new for every maildir, I probably only have
about 200 maildirs.

-Carl

-- 
carl.d.wo...@intel.com


pgpVzdGIUyAKY.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: folder and path completely broken in HEAD?

[Carl Worth]

Jani Nikula j...@nikula.org writes:
 The discussions about this were lengthy and tedious, and I was glad we
 eventually reached some consensus about what we wanted.

This much I do understand. And I apologize if I seem to be raising anew
issues that should finally be behind us.

 Meanwhile, the new folder: anchors the search to the beginning of
 the directory, while path: does not.

 Incorrect (or I don't understand you).
...
 That's an odd hodgepodge of non-orthogonal distinction in
 functionality.

 I'm sorry to hear you think that way. One is verbatim filesystem path,
 the other hides mail store folder implementation details as a
 convenience.

It may be that I simply misunderstood one aspect of path:. If it's
effectively ''anchored'' the same way as folder: then perhaps these
are more similar than I thought. In which case, I apologize for the
noise.

-Carl


pgpepWFsMp21N.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: Thanks for the nmbug-status program

[Carl Worth]

W. Trevor King wk...@tremily.us writes:
  I think that making nmbug and nmbug-status generally applicable for
  other projects is a good target.  I don't see any need to pull them
  out into a stand-alone project to do that though.
 
 For what it's worth, I don't disagree.

 Does that mean your patches are aimed upstream at the notmuch
 repository?

I didn't really intend them that way. What I meant was that I'd be happy
to switch to a script within the notmuch repository if it did what I
wanted, (which is basically just create an HTML file from a notmuch
query).

 (or their shell's approriate environment variable syntax), and I'd
 version the config file in the same repo as the associated tags.  I
 don't see the point of keeping the config file somewhere else, when
 you'll already need a repository to store the tags.

Ah, then you've missed something different about my usage. I don't have
any repository storing tags. My tags exist only in my notmuch
database. The tool I want here is a simpler one, (think, allow HTML
export from HTML). Heck, what I want could almost be nothing more than:

notmuch search --format=html

Other than that I want the other bits like multiple defined views with
their own queries and descriptions.

I see the part that has a repository holding tags as totally orthogonal
functionality. That orthogonal functionality is tag synchronization or
tag collaboration and is independent from HTML export to my way of
thinking.

So it makes sense to me that the HTML export just talks to the notmuch
database and doesn't need to assume anything else.

 * d456390 Add a comment describing the program

 I like the idea, but I'd prefer a docstring just after the leading
 comment.  Then we could reuse the docstring as the description for the
 ArgumentParser.

Sure. That's better.

 * 336411f Add notmuch search to the advertised queries.

 Folks can already copy-paste the terms into Emacs etc.  You only need
 the 'notmuch search' prefix if you're pasting into a shell, in which
 case you'll also want to escape shell-sensitive characters like
 parenthesis.  I'm fine without this one.

This is more about advertising notmuch to people not familiar with
it. It's more important for something like mesa stable queue than it
is for the notmuch bug list, of course.

 * 396869c Add support for a --query option (along with a default 
 configuration)

 I don't have an opinion about this one ;).  The default config doesn't
 look very useful (especially with it's dummy metadata settings), but
 I'd only be using the in-Git-config approach, and I don't mind if
 others want something like this.

This is about making this script self-documenting.

When I first tried nmbug-status I got just a stack trace[*], not even a
usage message. Even after running nmbug-status --help I wasn't any
closer to knowing how to run the thing. It wasn't obvious that --config
was the option I needed to get things to work, (from the stack trace,
clearly this script wants some git repository somewhere), and even if I
had know that --config was what I wanted, I had no idea how to construct
a config file.

I tried the documentation for nmbug which had a git clone command in
it, but even then I couldn't get nmbug-status to work, (I tried to
replicate this bug just now, but couldn't---it does seem to be working
after the documented git clone). But even then, all I had was a bare
repository and no clear indication that I needed a config file nor where
to find it.

By the time I had read the code and understood it well enough to know
where to find the config file, I also knew enough to fix the script to
be self-documenting as far as configuration. So that's what I did.

If you run notmuch-to-html with no arguments it will naturally walk
you through everything you need. It will give you functional sample
output before you've even constructed a configuration file. And then it
will give you the exact syntax for that example configuration file.

I can't see how these changes could be anything other than an
improvement.

-Carl

[*] Here it is, for reference:

(master)$ ./nmbug-status 
fatal: Not a git repository: '/home/cworth/.nmbug'
fatal: Not a git repository: '/home/cworth/.nmbug'
Traceback (most recent call last):
  File ./nmbug-status, line 254, in module
config = read_config(path=args.config)
  File ./nmbug-status, line 73, in read_config
return json.load(fp)
  File /usr/lib/python2.7/json/__init__.py, line 290, in load
**kw)
  File /usr/lib/python2.7/json/__init__.py, line 338, in loads
return _default_decoder.decode(s)
  File /usr/lib/python2.7/json/decoder.py, line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File /usr/lib/python2.7/json/decoder.py, line 384, in raw_decode
raise ValueError(No JSON object could be decoded)
ValueError: No JSON object could be decoded



pgpPcVgTDlY6t.pgp
Description: PGP signature
___
notmuch mailing list

Re: Thanks for the nmbug-status program

[Carl Worth]

Jani Nikula j...@nikula.org writes:
 On Wed, 23 Apr 2014, W. Trevor King wk...@tremily.us wrote:
 * f55ff7e Drop the --get-query option

 I'd like to keep the in-Git config, so I'm -1 here.

 FWIW I added the --get-query option as a convenience for accessing the
 queries from a nmbug companion shell script (which isn't in the notmuch
 repo though). Without it, one needs to duplicate the code digging the
 config from the nmbug repo and parsing the json (which includes the json
 schema). Doing that basically leads to a python snippet that is a
 subset of what nmbug-status already does.

OK. Color me convinced. I added the option back to the version of the
script that I'm hosting.

-Carl

-- 
carl.d.wo...@intel.com


pgpthKLlFrYQi.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: Thanks for the nmbug-status program

[Carl Worth]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

W. Trevor King wk...@tremily.us writes:
 So, “here's a goal to shoot for, and I'm happy with my fork.  If
 someone else wants to land patches like these upstream, I'm also happy
 dropping my fork.” ;). 

More just, Thanks for your code. I used it and happened to change it a
bit while using it. Take a look if you want.

 I think it would be easier to just mail your patches to the list and
 see what sticks.

Easier than what? I might be spending more time on emails now than I did
on any of my changes. And I'm not sure what we're talking about. :-)

I don't have any real plans for hacking on notmuch-to-html more (unless
I run into some problem). And if I do, I'll probably do just what I did
this time, (take a look the latest nmbug stuff and go from there).

But sure, next time I can try to send patches to the list first.

It's just that I never set out to make any changes this time around. I
was just trying to create/publish a web page, and some changes happened
as a side effect. Then I found myself unwilling to publish a page
without a pointer to the actual code that made it (so I had to publish
my changed version). Then I found myself unwilling to publish code
without history (so I had to publish a git repository). Then I found
myself unwilling to publish a git repository without at least some
guidance for a user who tried to run the code, (so I made a few more
changes beyond what I had strictly needed to make my page). So, yes,
some yak-shaving there, but git tells me it was only about one hour.

 That makes sense, but I'd prefer something like:
...
 Heh, that should probably be fixed ;)
...
 I'd prefer a clearer error message when we can't find a config file (I'm
 working up a patch now ;)

All sounds good to me. Thanks for everything you do. Have fun!

- -Carl

- -- 
carl.d.wo...@intel.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iQIcBAEBCgAGBQJTV/ruAAoJEGACM7qeVNxhwYkP+wTufLg1PvUCJMqHxFAMQCdM
gy81dXAaJvSSOWkp/A+fsE7i8JNZ96dIrPMHTKRp0dgpM/M89sXT1DHr4w7hNS67
sVmMawlsBElze19E0uTyWrcJuxk9A5QOaweCAWtLwrgacsZBfJ4mdZqWxXWdp8aw
ue8lELuVr3u66GCNYJqDWMgZNT0WPrGVz/3x3SShf+taWKlYHqnO0ESd+nDkU0WW
PxoGWxqdVptbZCLeJ8xuQN73KMiF7rkvuzYiFBqgWi2SZr/4heKP2VRa0G02KRjC
zgJVhKt7W3I/tV4qCOimaVCPL1sqRTNISh3sIJokW3IDLNzObK17/+sdYanxFEhI
RI6iM/hPy9iQ5wKSFa0M69FAk5+T0LQARYMxbzKogjXo55IanTK5gm4Mnh1uG86Z
205b5CzUL1Xetaw10luU9aIn1NAbE5VbNPIn0okNzpV8EkJOYZOFQXfxEjv5ntUQ
fin4n2XpBb4NTenQgSHCO/L8Hl2d6e6HmSSufRgbhBoR3OTmnv94sMhgOBnT/7AM
j5rJ6IHxHLQSvV5uPHUpSpY1xIasNnNgLnwZDYMtULxxhLQAQ4ikaKntDHLi8/+B
rbpIa5JFo4NqBuPs/MZDP4P7XCveLM8PWPJFwf1fh8Q6pVi2K0gbmlXZKq51DrcB
YPLzfFAr422ar7yjGVAl
=a4iT
-END PGP SIGNATURE-
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Thanks for the nmbug-status program

[Carl Worth]

Hi David,

I just wanted to give you a quick thank you for the nmbug-status
program. It's really handy to be able to publish decent-looking HTML
pages based on notmuch search results with this.

The script is quite general, (only a tiny portion was tied directly to
nmbug), so I renamed it notmuch-to-html, and hosted it here after a tiny
bit of generalization and a couple of minor user-interface changes:

http://git.cworth.org/git/notmuch-to-html

Feel free to re-adopt this as part of nmbug if you want or to suck it
back into notmuch/contrib or whatever.

Again, thanks!

-Carl

PS. For reference, here are the results I'm currently generating with
the script (for the Mesa project):

http://cworth.org/~cworth/mesa-stable-queue/

You'll see that I'm doing something very similar to what nmbug does,
(even with a compatible tag-naming scheme). In my case, I'm not (yet)
doing any collaborative maintenance of the patch queue, so I don't need
any of the tag-synchronization features that nmbug provides.

I would like to be able to do collaborative tag maintenance in the
future. And I think for that what I'd really like is a standalone tool
that allowed me to synchronize a set of tags (such as, anything with a
msq:: prefix). Of course, that's really close to what nmbug is. I'd
just like to see it be a bit more general, and walk the user more gently
through the setup.

(I'll confess that I tried to hack up nmbug a bit, but failed. There's
some mismatch between me and perl that is apparently not going to be
resolved.)

-- 
carl.d.wo...@intel.com


pgpQX1OhV5isz.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: Thanks for the nmbug-status program

[Carl Worth]

W. Trevor King wk...@tremily.us writes:
  http://git.cworth.org/git/notmuch-to-html

 Is there a clonable URL for this?

Yes. Sorry I didn't provide that, (and clearly I need to reconfigure my
git web pages to mention the URL):

git clone http://git.cworth.org/git/notmuch-to-html

 I think that making nmbug and nmbug-status generally applicable for
 other projects is a good target.  I don't see any need to pull them
 out into a stand-alone project to do that though.

For what it's worth, I don't disagree.

-- 
carl.d.wo...@intel.com


pgpseKquuCLLa.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: Thanks for the nmbug-status program

[Carl Worth]

W. Trevor King wk...@tremily.us writes:
   $ git clone http://git.cworth.org/git/notmuch-to-html

Sorry about that. What I had meant to say is:

  git clone git://git.cworth.org/git/notmuch-to-html

(This time I actually copy-pasted from a terminal where I tested that
the command actually works.)

And now, just in case you say you can talk http, but not git protocol,
I've made this work too:

  git clone http://http-git.cworth.org/git/notmuch-to-html

-Carl

PS. If someone knows magic so that git clients could clone from the same
URL that web browsers can use to get to the web pages, it would be nice
to have that set up.


pgpM0yuwJkWEf.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [RFC PATCH] Re: excessive thread fusing

[Carl Worth]

Austin Clements amdra...@mit.edu writes:
 But let me propose an idea I've been kicking around for a while: ghost
 message documents.  Rather than using user metadata for tracking these
 missing messages, use regular documents with the exact same terms we
 use now for message IDs and thread IDs, but with a Tghost term instead
 of a Tmail term to distinguish their type.

For what it's worth. I like this proposed fix very much. It's obviously
what I should have done in the first place.

 In effect, the database becomes truly monotonic.

Yes. All of these benefits are very compelling.

 [1] Curious?
 yes n | xapian-inspect postlist.DB | \
 awk '!/Key/ {next} /Key: \\x00\\xc0thread_id_/ {N++} /Key: \\x00\\xd0/ 
 {exit} END {print N}'

16725 for me.

-Carl



pgpuS8ZLLxizB.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [RFC PATCH] Re: excessive thread fusing

[Carl Worth]

Mark Walters markwalters1...@gmail.com writes:
 I have done dome debugging of this.

Thanks for looking closely, Mark!

 There is a patch below which fixes this test case but who knows what
 it breaks! Please DO NOT apply unless someone who knows this code says
 it's OK.

I wrote much of the original code being patched here, so hopefully I
understand it and can say something useful.

I agree that the patch should not be applied. I don't like to see one
piece of code not trusting another in the same code base. If the
parse_references() function doesn't deal well with a malformed header,
then we should fix it, not step around it.

Meanwhile, not treating all potential referenced message IDs
consistently could definitely make the notmuch algorithm more fragile
and sensitive to the order of message indexing, etc. So let's not do
that.

Instead, let's track down and fix the actual bug.

Thanks for the idea of using two-digit names for these messages. That
makes it much easier to inspect the relevant headers.

Below, I've grepped out the actual References and In-Reply-To headers
From the messages, and then simply substituted minimal, and
easy-to-understand values for the message IDs.

With these minimally modified headers, it's easy to manually inspect the
relationships and see that messages 17 and 18 belong in one thread, and
messages 32-52 belong in a separate thread.

It's also quite easy to see the potential source of the bug. The
In-Reply-To headers for messages 18, 32, and 52 all share a common
string (an email address) formatted to look like a message-id,
e.fr...@ucl.ac.uk. If notmuch looks at those headers, and treats
that string as a message-id, then all of theses messages will be
connected into a single thread.

And since that's the reported behavior, it seems likely that
e.fr...@ucl.ac.uk is the cause of this bug.

 I put some debug stuff in _notmuch_database_link_message_to_parents and
 I think that the problem comes from the call to parse_references on line
 1767 which adds the malformed in-reply-to header to the hash table, so
 this malformed line gets added as a potential parent. 

Am I correct that your debugging showed that e.fr...@ucl.ac.uk is
being added to the hash table?

My inspection of _parse_references() and parse_message_id() suggests
that that's exactly what notmuch is doing, (treating both of the
angle-bracketed portions (e.fr...@ucl.ac.uk as well as the actual
message-ID, ID17 or ID31 or ID39) as message IDs.

So it seems like we need a new _parse_in_reply_to() function to use in
place of _parse_references() and the new function will need a better
heuristic for dealing with the unpredictability of In-Reply-To.

The only real reason that we are trying to grab multiple message ID
values from an In-Reply-To header is that RFC 2822 explicitly allows
that, (to support a message simultaneously replying to multiple
messages). I don't believe that that's common, but we might as well
support it. At the same time, RFC 2822 also explicitly specifies that
the In-Reply-To header will consist of nothing but message IDs.

So perhaps the heuristic here could be to notice any characters outside
of angle brackets, (like Message in the headers below), and in that
case go to a strictly not RFC 2822 mode and look for exactly one
message ID. At that point, JWZ would recommend the first -bracketed
text found therein, but that would give precisely the wrong answer in
this particular case. Here the correct Message ID appears in the last
-bracketed text. I have not surveyed a large email corpus to determine
how often last -bracketed text would fail as a heuristic.

Another idea would be to trigger specifically on common forms. Judging
From the samples in this particular thread, it seems like a workable
heuristic would be:

If the In-Reply-To header begins with '':

Parse that initial portion as a message ID

Else if it ends with '':

Parse that final portion as a message ID

Else

Ignore this garbage-valued header.

That's probably the best and most reliably thing to do here.

Does anyone have any better ideas?

 As a clear example that I don't understand this code I don't know why
 this no longer causes a problem if message 17 gets added too.

I wanted to test my own knowledge of the code to see if I could explain
this. But I didn't precisely follow your explanation of the behavior you
saw. In cases (1) and (2) of your description, what order are you using
to add all messages or add all apart from 52?

Then, for cases (3) and (4), what is done before adding the messages
mentioned in these cases? Add all other messages? Again, in what order?

I haven't tracked through all the logic of the existing algorithm for
this case. But I don't like hearing that notmuch constructs different
threads for the same messages presented in different orders. This sounds
like a bug separate from what we've discussed above. 

-Carl

18:References: ID17

[PATCH] emacs: notmuch-hello: Add variable to control order or saved-search results

[Carl Worth]

This allows for the user to control the order that results are
returned from saved-searches, (and independently from the order of all
other searches which is controlled by notmuch-search-oldest-first).
---

This allows me to get back to the behavior that notmuch had long ago,
(the documentation for the changed behavior was updated in commit
c1a42652a173a4bb70ab72388e6ad150d19a2b06). I'm not sure when the
actual behavioral change was made.

A fanicer change might allow for a configurable sort order for each
saved search, but this more minimal change at least lets me use the
latest notmuch once again.

 emacs/notmuch-hello.el | 18 +-
 1 file changed, 17 insertions(+), 1 deletion(-)

diff --git a/emacs/notmuch-hello.el b/emacs/notmuch-hello.el
index 7b3d76b..a9f8230 100644
--- a/emacs/notmuch-hello.el
+++ b/emacs/notmuch-hello.el
@@ -59,6 +59,14 @@ alist to be used.
   :value notmuch-sort-saved-searches))
   :group 'notmuch-hello)
 
+(defcustom notmuch-saved-searches-oldest-first t
+  Show the oldest mail first for saved-search results.
+
+This variable defines the default sort order for displaying
+saved-search results.
+  :type 'boolean
+  :group 'notmuch-hello)
+
 (defvar notmuch-hello-indent 4
   How much to indent non-headers.)
 
@@ -327,6 +335,14 @@ diagonal.
  :notmuch-search-terms)
  notmuch-search-oldest-first))
 
+; Like notmuch-hello-widget-search, but with the
+; notmuch-saved-searches-oldest-first variable controlling the sort
+; order.
+(defun notmuch-hello-do-saved-search (widget rest ignore)
+  (notmuch-search (widget-get widget
+ :notmuch-search-terms)
+ notmuch-saved-searches-oldest-first))
+
 (defun notmuch-saved-search-count (search)
   (car (process-lines notmuch-command count search)))
 
@@ -460,7 +476,7 @@ Such a list can be computed with 
`notmuch-hello-query-counts'.
(widget-insert (format %8s 
   (notmuch-hello-nice-number msg-count)))
(widget-create 'push-button
-  :notify #'notmuch-hello-widget-search
+  :notify #'notmuch-hello-do-saved-search
   :notmuch-search-terms query
   name)
(setq column-indent
-- 
1.9.rc1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 0/5] lib: make folder: prefix literal

[Carl Worth]

Austin Clements amdra...@mit.edu writes:
 Agreed.  I believe this will also support MH, if I understand MH
 correctly (does anyone actually use MH?)

When I started notmuch, I had all of my mail in one-message-per-file in
various directories, (without these silly cur and new directories
that maildir uses).

At some point, I did a mass conversion of all of my directories to be
roughly:

~/mail//MM

So that I keep directories small by just delivering a month's worth of
mail to each directory. This conversion, (and the delivery agent I am
currently using, maildrop), happen to create the silly cur and new
directories.

So most of my mail still is in maildir format now.

But I do have a few messages in non-maildir directories. These have
generally come into being in cases such as someone providing me a
message to demonstrate a notmuch bug or use case. So in cases like this
I did things like:

mkdir ~/mail/bug-description
cp example-file ~/mail/bug-description

I also have a few directories created similarly when I've copied some
downloaded archives from a mailing list into my mail storage. (But often
I've used mb2md for those so the conversion has accidentally created
maildir directories).

I don't know if the non-maildir directories I have are strict mh
format, (did it have filenames with sequential numbers? I don't
recall).

But my intention with notmuch from the beginning was to support any
one-message-per-file layout without enforcing any particular naming of
directories or files. And I would like to see that preserved.

Since then, we have also supported various semantics when people do
encode information in directories and filenames, (such as ignoring
cur/new and interpreting maildir flags). This kind of thing does
seem good.

-Carl


pgpZXkuCQyczA.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 0/5] lib: make folder: prefix literal

[Carl Worth]

Jani Nikula j...@nikula.org writes:
 Unfortunately, I haven't had the time to experiment with this. But it
 bugs me that the probabilistic folder: prefix has stemming and it's case
 insensitive. It's possible to work around the stemming with the anchors
 you suggest or by quoting, but is there a way to have case sensitive
 probabilistic prefixes?

The stemming and case insensitivity just has to do with which terms are
shoved into the database, (you have to add extra terms to get these
features). If we're getting those features for folder now, (and I agree
that we don't want them), it's because we're calling some Xapian
convenience function along the lines of create a bunch of terms for
this chunk of text.

The fix for that is to do the simple thing and simply break the path at
each '/' and add a term for each component. Then these problems all go
away.

So fixes for this should not require switching from a probabilistic to a
Boolean prefix.

-Carl

-- 
carl.d.wo...@intel.com


pgpO4oiDDTjcx.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 0/5] lib: make folder: prefix literal

[Carl Worth]

Jani Nikula j...@nikula.org writes:
 Here's a thought. With boolean prefix folder:, we can devise a scheme
 where the folder: query defines what is to be matched.

I like the idea, but I tried to infer the rules from the examples, and I
failed. It looks like there are two new symbols, / and /. but I
couldn't decipher the exact semantics of each.

I think a proposal like this should not re-use the '/' symbol as we
already have that as a path divider. (See rsync for lots of user
confusion with a significant trailing '/').

I propose a similar, but slightly different approach, where we add two
additional symbols:

  '^'   Matches the beginning of a path

  '$'   Matches the end of a path

[Obviously, I chose these symbols from regular expressions. I would be
OK with alternate symbols, ('$' seems like it might be problematic in
the shell, but perhaps not too much if it's always at the end of a
phrase.)]

This way, one could search for:

  folder:fooWorks like folder: historically

  folder:^full/path$Works like Jani's proposal

  folder:^path/prefix   Satisfies Tomi's use case, (as well as anyone
who doesn't want to have to specify or
distinguish between /cur or /new.

Any extra '/' at the beginning or end of a search string, (such as
folder:^/full/path/$) would not change the semantics.

Further, I think we can implement this with less database bloat by
leaving folder as probabilistic and simply indexing two new terms to
indicate the beginning of the path and the end of the path.

Finally, we could also extend the scheme to other things like subject:
to allow for an exact subject search like:

subject:^lib: make folder: prefix literal$

It was with an eye toward something like this that I chose to make
folder: probabilistic in the first place. (I probably would have indexed
things appropriately in the first place as well, but at the time doing
the necessary query parsing for '^' and '$' seemed daunting).

-Carl

-- 
carl.d.wo...@intel.com


pgpGvAffMislw.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] debian: package ruby bindings

[Carl Worth]

Felipe Contreras felipe.contre...@gmail.com writes:
 My hypothetical laziness is not the reason. It was at the strong
 insistence of Carl that things are organized this way.

 I still don't see why we need to do that. The notmuch project
 shouldn't be bound to the debian project.

I'm not sure what your specific question is here.

My strong insistence was that the Debian packaging stuff, (like
debian/rules), be hosted in the main notmuch repository.

If that were separated to a separate repository (or a separate branch),
then nothing in the current situation would change. Your proposed
change, (which is to debian/rules if I read it correctly), would instead
be directed at that separate repository/branch.

And either way, David's currently the person to decide what lands there.

Is there anything I'm missing here?

-Carl


pgpbehbODblKH.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] bitmap:improve memory usage using CHAR_BITS and unsigned CHAR

[Carl Worth]

Robert Mast  writes:
> Re-submission for comment-reasons.

Very nicely done, Robert. Thanks.

-Carl (who's not quite merging patches again yet, but likely will be soon)

-- 
carl.d.worth at intel.com
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 

Reply all - issue

[Carl Worth]

Jameson Graef Rollins  writes:
> Just a thought: what if messages with a given tag (e.g. "new-thread")
> were always treated as the source of a new thread?

It's a good start. And an approach like that would have the advantage
that one could undo a thread-split by just removing the tag. (That's not
an explicit thread-join feature, but I don't think anyone has ever asked
for that.)

> A message with the given tag could just be (re)indexed with any
> In-Reply-To/References headers stripped before indexing.

It would require a little more than that. Imagine this thread:

  A: Subject: An original thread
  ??B: Subject: Thread hijacking is fun (tag:new-thread)
??C: Subject: Re: Thread hijacking is fun

In this case, message C is likely to have a References header that
mentions both A and B. So the thread stitching logic in notmuch will
want to merge threads A and B when indexing C. So special care will have
to be taken here as well, (not just when indexing B).

And that special care may not be cheap if it requires additional
database lookups for each unique thread ID encountered among references
of a message.

Though, I don't mean to dissuade anyone from thinking this through and
coding it up. The relevant code for the pieces I'm referring to starts
in _notmuch_database_link_message in lib/database.cc.

-Carl

-- 
carl.d.worth at intel.com
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 

RE: Reply all - issue

[Carl Worth]

Jameson Graef Rollins jroll...@finestructure.net writes:
 Just a thought: what if messages with a given tag (e.g. new-thread)
 were always treated as the source of a new thread?

It's a good start. And an approach like that would have the advantage
that one could undo a thread-split by just removing the tag. (That's not
an explicit thread-join feature, but I don't think anyone has ever asked
for that.)

 A message with the given tag could just be (re)indexed with any
 In-Reply-To/References headers stripped before indexing.

It would require a little more than that. Imagine this thread:

  A: Subject: An original thread
  └─B: Subject: Thread hijacking is fun (tag:new-thread)
└─C: Subject: Re: Thread hijacking is fun

In this case, message C is likely to have a References header that
mentions both A and B. So the thread stitching logic in notmuch will
want to merge threads A and B when indexing C. So special care will have
to be taken here as well, (not just when indexing B).

And that special care may not be cheap if it requires additional
database lookups for each unique thread ID encountered among references
of a message.

Though, I don't mean to dissuade anyone from thinking this through and
coding it up. The relevant code for the pieces I'm referring to starts
in _notmuch_database_link_message in lib/database.cc.

-Carl

-- 
carl.d.wo...@intel.com


pgptx_atmDvLv.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] convert bitmap to unsigned char

[Carl Worth]

Robert Mast  writes:
> ---

Hi Robert,

It looks like the git exercise is proving useful. Keep it up!

If you look through the git logs a bit you'll find that the overwhelming
convention is for a commit message to have a single-line summary
followed by a (potentially longer) expanded comment.

The convention I like best is for the single-line summary to
efficiently  describe everything about "what" the patch does. If it's
hard to fit this into a single line there's a good chance your patch
should be split up. With your commit message here, the one-line summary
is great, (and much better than in your first submission).

The expanded comment should describe the "why" of the change. And here,
your commit message doesn't have anything. So I'm left wondering why
your commit exists. Does this save memory? Does this fix some type error
or expected alignment somewhere? etc.

Please re-submit your patch with a little more explanation of the
motivation behind the patch.

-Carl

PS. I do know that you did have more in the commit message originally,
and Austin recommended removing "snide" issues, (doubts and
meta-questions about the patch---things that don't belong in the commit
history).

Your previous commit message was:

  Reading it in detail I thought it allocated way too much memory and
  didn't use the full size of the allocated unsigned ints for storing
  bits.

  Am I right, and is this the right way to patch code to notmuch?

I'm not actually looking at the code in context now, so I can't render a
correct commit message, but this attempted rewording should hopefully
give you the idea:

  Using char instead of int allows for simpler definitions of the
  DOCIDSET macros so the code is easier to understand.

  ---

  Am I reading that correctly? Or is there also some space saving
  happening here as well? Please re-submit a new patch with a commit
  message that makes things clear one way or the other.

Do you see how this "why" part of the commit message is ready to stand
alone in our commit history (assuming it's correct and
accepted). Meta-questions like "is this even a sane way of doing
things?" are great to ask, and very helpful for code reviewers, but best
placed after the "---" so that they don't become part of the commit
message.

Thanks for playing with notmuch!

-Carl

-- 
carl.d.worth at intel.com
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 

Re: [PATCH] convert bitmap to unsigned char

[Carl Worth]

Robert Mast beheer...@tekenbeetziekten.nl writes:
 ---

Hi Robert,

It looks like the git exercise is proving useful. Keep it up!

If you look through the git logs a bit you'll find that the overwhelming
convention is for a commit message to have a single-line summary
followed by a (potentially longer) expanded comment.

The convention I like best is for the single-line summary to
efficiently  describe everything about what the patch does. If it's
hard to fit this into a single line there's a good chance your patch
should be split up. With your commit message here, the one-line summary
is great, (and much better than in your first submission).

The expanded comment should describe the why of the change. And here,
your commit message doesn't have anything. So I'm left wondering why
your commit exists. Does this save memory? Does this fix some type error
or expected alignment somewhere? etc.

Please re-submit your patch with a little more explanation of the
motivation behind the patch.

-Carl

PS. I do know that you did have more in the commit message originally,
and Austin recommended removing snide issues, (doubts and
meta-questions about the patch---things that don't belong in the commit
history).

Your previous commit message was:

  Reading it in detail I thought it allocated way too much memory and
  didn't use the full size of the allocated unsigned ints for storing
  bits.

  Am I right, and is this the right way to patch code to notmuch?

I'm not actually looking at the code in context now, so I can't render a
correct commit message, but this attempted rewording should hopefully
give you the idea:

  Using char instead of int allows for simpler definitions of the
  DOCIDSET macros so the code is easier to understand.

  ---

  Am I reading that correctly? Or is there also some space saving
  happening here as well? Please re-submit a new patch with a commit
  message that makes things clear one way or the other.

Do you see how this why part of the commit message is ready to stand
alone in our commit history (assuming it's correct and
accepted). Meta-questions like is this even a sane way of doing
things? are great to ask, and very helpful for code reviewers, but best
placed after the --- so that they don't become part of the commit
message.

Thanks for playing with notmuch!

-Carl

-- 
carl.d.wo...@intel.com


pgp_QJvrmT3C4.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

It's good to be "back"!

[Carl Worth]

Elsewhere, in a non-hijacked thread, Carl Worth wrote:
> Is there any existing thread-breaking? There wasn't the last time I
> looked at the code closely, (but admittedly, that was a while ago).

Hi folks,

I've been away from any active involvement in the notmuch community for
far too long.

It turns out that even with an email system as sophisticated as notmuch,
it's possible to slip into bad habits that allow one to get behind with
email. So I got behind.

I also had some difficulty in accepting the fact that I wasn't able to
be as directly involved with notmuch as I was in the beginning. Early on
I would read every message sent to the list and try to ensure that every
question received at least some answer, (whether from me or for someone
else). But of course, I never succeeded at actually keeping up with
that, so my queues kept getting bigger and more intimidating, etc.

Anyway, it's been about 18 months since I've done anything real with
notmuch. I'm so delighted to see that the community that built up around
notmuch has been so independent, doing great things in my absence. I'm
excited to start looking close at the new features that I've not played
with (nmbug, date parsing, etc.), and maybe start implementing some of
those ancient features that I never got around to implementing in the
first place, (indexing of List-Id headers?).

So a huge thank you to all of you!

When I got over the psychological hurdles and decided to just archive
away the old, neglected mail queues, notmuch at least did make that
really easy. And I've now got an email workflow going really well for
me, that I think will be really useful. Notmuch allows the workflow
quite well, but doesn't guide me through it as smoothly as I would
like. More on that in a later message.

But for now, I should be reading the list much more regularly going
forward, and I'll be happy to have people CC me directly when they want
my particular attention for some specific reason.

And of course, all of the 12,000+ [*] messages I missed in the last
18 months[*] are still present in my archives. So new replies to old
messages might bring things to my attention. People can use that
intentionally if it will be useful.

I don't know how much code I'll be committing on a regular basis. But I
will definitely be "around" now much more.

Thanks again for all you've done over the last while. And thanks in
advance for what I know will be a warm welcome.

-Carl

[*] Hurrah for date searches! It's wonderful to have this feature!!
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20130129/3705c869/attachment.pgp>

Reply all - issue

[Carl Worth]

Robert Mast  writes:
> Your point on patch-breaking related to gmail and my proposal isn't
> completely clear to me, but I've probably addressed it well with my new
> approach.

The issue here is that many developers tend to develop a patch series
(perhaps with dozens of patches) as a single conceptual unit. When these
are emailed out, they are often sent as one thread with a new subject
for every patch. In particular, users of git and "git send-email" often
send patches this way. For what it's worth, it's my preferred way to
send and receive patches via email.

It's extremely useful for messages like this to be presented as a
single thread. This means that the dozens of messages don't clutter the
inbox, and it also allows for an operation to act on all of the messages
at once, (for example, notmuch provides "C-u |" which can apply all of
the received patches to a code repository in a single operation).

So, those of us accustomed to sending, receiving, reviewing, and
applying patches emailed in this way would be basically unable to use an
email program that split threads unconditionally on subject changes.

So it may be tricky to find a single behavior that would make everyone
happy. Perhaps a configuration option for splitting threads on subject
changes.

> I'll study the code for adding the option of unconditional (stripped)
> subject breaking on top of the existing thread-breaking.

Is there any existing thread-breaking? There wasn't the last time I
looked at the code closely, (but admittedly, that was a while ago).

-Carl

-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 

RE: Reply all - issue

[Carl Worth]

Robert Mast beheer...@tekenbeetziekten.nl writes:
 Your point on patch-breaking related to gmail and my proposal isn't
 completely clear to me, but I've probably addressed it well with my new
 approach.

The issue here is that many developers tend to develop a patch series
(perhaps with dozens of patches) as a single conceptual unit. When these
are emailed out, they are often sent as one thread with a new subject
for every patch. In particular, users of git and git send-email often
send patches this way. For what it's worth, it's my preferred way to
send and receive patches via email.

It's extremely useful for messages like this to be presented as a
single thread. This means that the dozens of messages don't clutter the
inbox, and it also allows for an operation to act on all of the messages
at once, (for example, notmuch provides C-u | which can apply all of
the received patches to a code repository in a single operation).

So, those of us accustomed to sending, receiving, reviewing, and
applying patches emailed in this way would be basically unable to use an
email program that split threads unconditionally on subject changes.

So it may be tricky to find a single behavior that would make everyone
happy. Perhaps a configuration option for splitting threads on subject
changes.

 I'll study the code for adding the option of unconditional (stripped)
 subject breaking on top of the existing thread-breaking.

Is there any existing thread-breaking? There wasn't the last time I
looked at the code closely, (but admittedly, that was a while ago).

-Carl



pgpsTAUNjRnwq.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

It's good to be back!

[Carl Worth]

Elsewhere, in a non-hijacked thread, Carl Worth wrote:
 Is there any existing thread-breaking? There wasn't the last time I
 looked at the code closely, (but admittedly, that was a while ago).

Hi folks,

I've been away from any active involvement in the notmuch community for
far too long.

It turns out that even with an email system as sophisticated as notmuch,
it's possible to slip into bad habits that allow one to get behind with
email. So I got behind.

I also had some difficulty in accepting the fact that I wasn't able to
be as directly involved with notmuch as I was in the beginning. Early on
I would read every message sent to the list and try to ensure that every
question received at least some answer, (whether from me or for someone
else). But of course, I never succeeded at actually keeping up with
that, so my queues kept getting bigger and more intimidating, etc.

Anyway, it's been about 18 months since I've done anything real with
notmuch. I'm so delighted to see that the community that built up around
notmuch has been so independent, doing great things in my absence. I'm
excited to start looking close at the new features that I've not played
with (nmbug, date parsing, etc.), and maybe start implementing some of
those ancient features that I never got around to implementing in the
first place, (indexing of List-Id headers?).

So a huge thank you to all of you!

When I got over the psychological hurdles and decided to just archive
away the old, neglected mail queues, notmuch at least did make that
really easy. And I've now got an email workflow going really well for
me, that I think will be really useful. Notmuch allows the workflow
quite well, but doesn't guide me through it as smoothly as I would
like. More on that in a later message.

But for now, I should be reading the list much more regularly going
forward, and I'll be happy to have people CC me directly when they want
my particular attention for some specific reason.

And of course, all of the 12,000+ [*] messages I missed in the last
18 months[*] are still present in my archives. So new replies to old
messages might bring things to my attention. People can use that
intentionally if it will be useful.

I don't know how much code I'll be committing on a regular basis. But I
will definitely be around now much more.

Thanks again for all you've done over the last while. And thanks in
advance for what I know will be a warm welcome.

-Carl

[*] Hurrah for date searches! It's wonderful to have this feature!!


pgpC4BQgkXm04.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] Restore original keybinding ('r' = reply-to-all)

[Carl Worth]

Philip Hands  writes:
> I find the change to the new (only reply to sender) behaviour serously
> irritating, because it seems I cannot train myself to hit R all the time
> (which is pretty much what I always want).

I'm in a similar camp. I tried (and failed) to adopt to the current
mode, (once I realized that the change had happened and that I had been
mis-replying).

I think part of the problem with training here is that if your mental
model is "I generally want to reply to all, and exceptionally want to
reply to only some" then the 'r == reply-to-sender' often does do what
you want. That is, when the CC list is empty, reply-to-sender is no
different than reply-to-all. So there's some mis-training that occurs,
('r' seems to do what I want). Worse, the results of the mis-training
are hidden from the user, (since if the user didn't pay attention to the
CC list before hitting 'r', the unintentional culling of recipients is
hidden within the composition buffer).

For me, personally, I tend to do a final examination of my message just
before sending. This is a quick scan to look for typos, make sure my
message is clear, etc. During part of this scan, it's also natural for
me to ensure that there isn't anyone in the recipient list that
shouldn't be receiving the message. If I do decide to remove some
recipients from my message, this is a natural time for me to do it, (or
perhaps earlier, during composition, when first writing something
sensitive).

So, for me, making a decision *before* writing anything doesn't fit my
mental model, (I'll often change the tone of my message while composing,
and in ways that the recipient list might change).

I also find reply-to-sender-only too limiting of an operation, (compared
to reply-to-all followed by header editing). For example, sometimes I
might want to drop some smaller subset of recipients from my reply.

My workflow is definitely influenced my the habits of coworkers in my
corporate environment. While there are many mailing-list addresses,
there are often large threads involving ad-hoc recipient lists. And as
conversations go, some groups of individuals needs to be added or
removed from the discussion. Header editing works for that, in a way
that reply-to-sender doesn't help.

> On the other hand, I'm perfectly capable of customising this, but have
> something of a fetish for at least trying to live with defaults for a
> period, so it's my own fault for putting up with it.

I'm obviously capable of making a customization as well, (and have done
so). The current mechanism[*] I'm using for this customization is
particularly clumsy, (it's not exposed in the customize buffer, it
requires the user to know the names of internal objects like
notmuch-show-mode-map and notmuch-search-reply-to-thread-sender), and it
requires the user to change 4 settings, (in two separate places), in
order to get a consistent experience, (so it would be easy to
accidentally make search-mode and show-mode behave differently).

There's clearly difference of opinion on what the defaults should be. So
at the very least, we should make it easier to customize this.

How about the following:

  With no customization in place, the first time the user hits 'r',
  notmuch prompts with something like:

Reply to all or sender only? [asAS?]:

  Hitting '?' would the provide more instructions:

'a': Reply to all recipients for this reply.
's': Reply to sender-only for this reply.
'A': Reply to all recipients now and in the future (no questions asked)
'S': Reply to sender-only now and in the future (no questions asked)

 Note: After setting a default behavior with 'A' or 'S' here, the
 alternate behavior can still be obtained by initiating a reply with
 'R' rather than 'r'.

That would satisfy me as being sufficiently easy-to-use and sufficiently
self-documenting.

It also as the advantage of letting us make a change now without
tripping up any users.

I think the implementation should function by setting a single
customize-based variable. But care should be taken such that the notmuch
help modes still correctly describe what 'r' and 'R' do depending on how
this variable is configured. My current approach of setting a preference
by changing the keybindings yields correct documentation "for
free". It's probably a little trickier to get that correct documentation
with a single variable controlling things, but I hope it's not too hard.

Anyone care to attempt an implementation of this?

-Carl

[*] Here's what I'm using now:

(define-key notmuch-show-mode-map "r" 'notmuch-show-reply)
(define-key notmuch-show-mode-map "R" 'notmuch-show-reply-sender)
(define-key notmuch-search-mode-map "r" 'notmuch-search-reply-to-thread)
(define-key notmuch-search-mode-map "R" 'notmuch-search-reply-to-thread-sender)
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 

Re: [PATCH] Restore original keybinding ('r' = reply-to-all)

[Carl Worth]

Philip Hands p...@hands.com writes:
 I find the change to the new (only reply to sender) behaviour serously
 irritating, because it seems I cannot train myself to hit R all the time
 (which is pretty much what I always want).

I'm in a similar camp. I tried (and failed) to adopt to the current
mode, (once I realized that the change had happened and that I had been
mis-replying).

I think part of the problem with training here is that if your mental
model is I generally want to reply to all, and exceptionally want to
reply to only some then the 'r == reply-to-sender' often does do what
you want. That is, when the CC list is empty, reply-to-sender is no
different than reply-to-all. So there's some mis-training that occurs,
('r' seems to do what I want). Worse, the results of the mis-training
are hidden from the user, (since if the user didn't pay attention to the
CC list before hitting 'r', the unintentional culling of recipients is
hidden within the composition buffer).

For me, personally, I tend to do a final examination of my message just
before sending. This is a quick scan to look for typos, make sure my
message is clear, etc. During part of this scan, it's also natural for
me to ensure that there isn't anyone in the recipient list that
shouldn't be receiving the message. If I do decide to remove some
recipients from my message, this is a natural time for me to do it, (or
perhaps earlier, during composition, when first writing something
sensitive).

So, for me, making a decision *before* writing anything doesn't fit my
mental model, (I'll often change the tone of my message while composing,
and in ways that the recipient list might change).

I also find reply-to-sender-only too limiting of an operation, (compared
to reply-to-all followed by header editing). For example, sometimes I
might want to drop some smaller subset of recipients from my reply.

My workflow is definitely influenced my the habits of coworkers in my
corporate environment. While there are many mailing-list addresses,
there are often large threads involving ad-hoc recipient lists. And as
conversations go, some groups of individuals needs to be added or
removed from the discussion. Header editing works for that, in a way
that reply-to-sender doesn't help.

 On the other hand, I'm perfectly capable of customising this, but have
 something of a fetish for at least trying to live with defaults for a
 period, so it's my own fault for putting up with it.

I'm obviously capable of making a customization as well, (and have done
so). The current mechanism[*] I'm using for this customization is
particularly clumsy, (it's not exposed in the customize buffer, it
requires the user to know the names of internal objects like
notmuch-show-mode-map and notmuch-search-reply-to-thread-sender), and it
requires the user to change 4 settings, (in two separate places), in
order to get a consistent experience, (so it would be easy to
accidentally make search-mode and show-mode behave differently).

There's clearly difference of opinion on what the defaults should be. So
at the very least, we should make it easier to customize this.

How about the following:

  With no customization in place, the first time the user hits 'r',
  notmuch prompts with something like:

Reply to all or sender only? [asAS?]:

  Hitting '?' would the provide more instructions:

'a': Reply to all recipients for this reply.
's': Reply to sender-only for this reply.
'A': Reply to all recipients now and in the future (no questions asked)
'S': Reply to sender-only now and in the future (no questions asked)

 Note: After setting a default behavior with 'A' or 'S' here, the
 alternate behavior can still be obtained by initiating a reply with
 'R' rather than 'r'.

That would satisfy me as being sufficiently easy-to-use and sufficiently
self-documenting.

It also as the advantage of letting us make a change now without
tripping up any users.

I think the implementation should function by setting a single
customize-based variable. But care should be taken such that the notmuch
help modes still correctly describe what 'r' and 'R' do depending on how
this variable is configured. My current approach of setting a preference
by changing the keybindings yields correct documentation for
free. It's probably a little trickier to get that correct documentation
with a single variable controlling things, but I hope it's not too hard.

Anyone care to attempt an implementation of this?

-Carl

[*] Here's what I'm using now:

(define-key notmuch-show-mode-map r 'notmuch-show-reply)
(define-key notmuch-show-mode-map R 'notmuch-show-reply-sender)
(define-key notmuch-search-mode-map r 'notmuch-search-reply-to-thread)
(define-key notmuch-search-mode-map R 'notmuch-search-reply-to-thread-sender)


pgp2bTFGturSn.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] Restore original keybinding ('r' = reply-to-all)

[Carl Worth]

Since the beginning of time, the emacs interface provided a keybinding
of 'r' to reply to a message, (and originally, all recipients).

Then, before release 0.12 the emacs interface acquired a new
reply-to-sender only feature. In commit
f02b475fa781bb5df3358c73213e7633a99f016e the new feature was put onto
the original keybinding, (and reply-to-all was moved to 'R').

This restores the original keybinding and uses the new keybinding for
the new feature.
---

The commit message above motivates this change from the point of view
of interface compatibility. That argument would have been stronger if
I had made it in January, (at the time of the change), rather than
now, (since there's now a new incompatiblity made by changing things
back).

That said, I still think the compatibility argument is sound.

I have other feelings as to which reply behavior should be treated
preferentially. But those are potentially more controversial, so I'll
leave the bikeshedding out of our commit history.

Thanks to everyone keeping notmuch alive and well during my extended
absence from maintainership. What a tremendous free-software success
story this all is. I'm so grateful to all of you.

-Carl

 emacs/notmuch-show.el |4 ++--
 emacs/notmuch.el  |4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
index 4349836..399a8bd 100644
--- a/emacs/notmuch-show.el
+++ b/emacs/notmuch-show.el
@@ -1145,8 +1145,8 @@ reset based on the original query."
(define-key map "s" 'notmuch-search)
(define-key map "m" 'notmuch-mua-new-mail)
(define-key map "f" 'notmuch-show-forward-message)
-   (define-key map "r" 'notmuch-show-reply-sender)
-   (define-key map "R" 'notmuch-show-reply)
+   (define-key map "r" 'notmuch-show-reply)
+   (define-key map "R" 'notmuch-show-reply-sender)
(define-key map "|" 'notmuch-show-pipe-message)
(define-key map "w" 'notmuch-show-save-attachments)
(define-key map "V" 'notmuch-show-view-raw-message)
diff --git a/emacs/notmuch.el b/emacs/notmuch.el
index c6236db..3a6b025 100644
--- a/emacs/notmuch.el
+++ b/emacs/notmuch.el
@@ -215,8 +215,8 @@ For a mouse binding, return nil."
 (define-key map ">" 'notmuch-search-last-thread)
 (define-key map "p" 'notmuch-search-previous-thread)
 (define-key map "n" 'notmuch-search-next-thread)
-(define-key map "r" 'notmuch-search-reply-to-thread-sender)
-(define-key map "R" 'notmuch-search-reply-to-thread)
+(define-key map "r" 'notmuch-search-reply-to-thread)
+(define-key map "R" 'notmuch-search-reply-to-thread-sender)
 (define-key map "m" 'notmuch-mua-new-mail)
 (define-key map "s" 'notmuch-search)
 (define-key map "o" 'notmuch-search-toggle-order)
-- 
1.7.10

[PATCH] Restore original keybinding ('r' = reply-to-all)

[Carl Worth]

Since the beginning of time, the emacs interface provided a keybinding
of 'r' to reply to a message, (and originally, all recipients).

Then, before release 0.12 the emacs interface acquired a new
reply-to-sender only feature. In commit
f02b475fa781bb5df3358c73213e7633a99f016e the new feature was put onto
the original keybinding, (and reply-to-all was moved to 'R').

This restores the original keybinding and uses the new keybinding for
the new feature.
---

The commit message above motivates this change from the point of view
of interface compatibility. That argument would have been stronger if
I had made it in January, (at the time of the change), rather than
now, (since there's now a new incompatiblity made by changing things
back).

That said, I still think the compatibility argument is sound.

I have other feelings as to which reply behavior should be treated
preferentially. But those are potentially more controversial, so I'll
leave the bikeshedding out of our commit history.

Thanks to everyone keeping notmuch alive and well during my extended
absence from maintainership. What a tremendous free-software success
story this all is. I'm so grateful to all of you.

-Carl

 emacs/notmuch-show.el |4 ++--
 emacs/notmuch.el  |4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
index 4349836..399a8bd 100644
--- a/emacs/notmuch-show.el
+++ b/emacs/notmuch-show.el
@@ -1145,8 +1145,8 @@ reset based on the original query.
(define-key map s 'notmuch-search)
(define-key map m 'notmuch-mua-new-mail)
(define-key map f 'notmuch-show-forward-message)
-   (define-key map r 'notmuch-show-reply-sender)
-   (define-key map R 'notmuch-show-reply)
+   (define-key map r 'notmuch-show-reply)
+   (define-key map R 'notmuch-show-reply-sender)
(define-key map | 'notmuch-show-pipe-message)
(define-key map w 'notmuch-show-save-attachments)
(define-key map V 'notmuch-show-view-raw-message)
diff --git a/emacs/notmuch.el b/emacs/notmuch.el
index c6236db..3a6b025 100644
--- a/emacs/notmuch.el
+++ b/emacs/notmuch.el
@@ -215,8 +215,8 @@ For a mouse binding, return nil.
 (define-key map  'notmuch-search-last-thread)
 (define-key map p 'notmuch-search-previous-thread)
 (define-key map n 'notmuch-search-next-thread)
-(define-key map r 'notmuch-search-reply-to-thread-sender)
-(define-key map R 'notmuch-search-reply-to-thread)
+(define-key map r 'notmuch-search-reply-to-thread)
+(define-key map R 'notmuch-search-reply-to-thread-sender)
 (define-key map m 'notmuch-mua-new-mail)
 (define-key map s 'notmuch-search)
 (define-key map o 'notmuch-search-toggle-order)
-- 
1.7.10

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

notmuch git's disk is full! ERROR pushing.

[Carl Worth]

On Sun, 08 Jan 2012 13:55:53 +0100, Sebastian Spaeth  
wrote:
> Actually trying to push the above 2 patches, I get 
> 
> Writing objects: 100% (8/8), 1.25 KiB, done.
> Total 8 (delta 5), reused 0 (delta 0)
> error: unable to create temporary sha1 filename : No space left on
> device

My apologies!

I've relieved the immediate problem for now. (And I'm hoping to soon
move notmuchmail.org to a different server with a disk that's a couple
of orders of magnitude larger.)

-Carl
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 

Re: notmuch git's disk is full! ERROR pushing.

[Carl Worth]

On Sun, 08 Jan 2012 13:55:53 +0100, Sebastian Spaeth sebast...@sspaeth.de 
wrote:
 Actually trying to push the above 2 patches, I get 
 
 Writing objects: 100% (8/8), 1.25 KiB, done.
 Total 8 (delta 5), reused 0 (delta 0)
 error: unable to create temporary sha1 filename : No space left on
 device

My apologies!

I've relieved the immediate problem for now. (And I'm hoping to soon
move notmuchmail.org to a different server with a disk that's a couple
of orders of magnitude larger.)

-Carl


pgpiSNTzVnsTM.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Rethinking *_destroy()

[Carl Worth]

On Mon, 19 Sep 2011 16:22:39 -0400, Ben Gamari  
wrote:
> As many might have noticed, there was recently a bit of a discussion on
> this list concerning the state of memory management in libnotmuch,
> especially regarding some classes of garbage collectors.
> 
> To summarize (someone correct me if I get something wrong),

Thanks for the summary, Ben. As many might have noticed, I'm somewhat
behind on my reading of the notmuch mailing list right now. So I
appreciate you bringing this issue to my attention.

> Overall, this means that languages with cyclical garbage collectors
> (Python, Haskell, and I'm sure others) can not bind libnotmuch
> correctly.

That certainly sound unappealing.

> I can see at least two ways of doing this,
> 
>   1) Acknowledging that we use talloc and allowing users to use
>  talloc_ref and talloc_unlink directly

I like this option, myself. I think talloc has been a wonderful boon to
my programming. So I don't have a problem with the notmuch API
documentation committing the implementation to using talloc. I'm also
quite glad to let the notmuch documentation advertise talloc to any
readers.

>   2) Wrapping talloc by adding a *_ref() and *_unref() to each object

That looks like a lot of extra API, but with no substantial benefit. (We
would get the freedom to switch to some other implementation of talloc,
but, I don't think we need that.)

> Additionally, we need to decide to what extent we want to break the
> libnotmuch API. While strictly speaking we could keep *_destroy() around
> without breaking existing code, this will mean we will have two ways of
> freeing an object. Perhaps a scheduled deprecation in a release or two
> is in order here?

Actually, I would prefer to leave *_destroy around, (as long as it's
doing nothing other than tall_free (or talloc_unlink? [*]) which does
appear to be the case for all existing functions).

To me, it doesn't really look like two ways of freeing an object.
Anyone who calls notmuch_query_create, (for example), should call
notmuch_query_destroy and those two calls pair nicely.

Then, *some* callers will want to take advantage of talloc. Those users
can add calls to talloc_reference and talloc_unlink (which pair nicely
themselves).

What I don't want is the unnatural pairing of something like
"notmuch_query_create" with "talloc_unlink". That just seems uselessly
harder to learn and remember.

> Any and all feedback is desired.

Thanks for pointing out the issue. Let me know if I've missed
anything. And, happy binding!

-Carl

[*] These destroy() functions are all currently calling talloc_free()
which won't work if the user has any remaining parents still present due
to calls to talloc_reference(). Perhaps we can just document that all
talloc_reference()/talloc_unlink() pairs must be complete before the
caller calls _destroy()?

-- 
carl.d.worth at intel.com
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 

Re: Rethinking *_destroy()

[Carl Worth]

On Mon, 19 Sep 2011 16:22:39 -0400, Ben Gamari bgamari.f...@gmail.com wrote:
 As many might have noticed, there was recently a bit of a discussion on
 this list concerning the state of memory management in libnotmuch,
 especially regarding some classes of garbage collectors.
 
 To summarize (someone correct me if I get something wrong),

Thanks for the summary, Ben. As many might have noticed, I'm somewhat
behind on my reading of the notmuch mailing list right now. So I
appreciate you bringing this issue to my attention.

 Overall, this means that languages with cyclical garbage collectors
 (Python, Haskell, and I'm sure others) can not bind libnotmuch
 correctly.

That certainly sound unappealing.

 I can see at least two ways of doing this,
 
   1) Acknowledging that we use talloc and allowing users to use
  talloc_ref and talloc_unlink directly

I like this option, myself. I think talloc has been a wonderful boon to
my programming. So I don't have a problem with the notmuch API
documentation committing the implementation to using talloc. I'm also
quite glad to let the notmuch documentation advertise talloc to any
readers.

   2) Wrapping talloc by adding a *_ref() and *_unref() to each object

That looks like a lot of extra API, but with no substantial benefit. (We
would get the freedom to switch to some other implementation of talloc,
but, I don't think we need that.)

 Additionally, we need to decide to what extent we want to break the
 libnotmuch API. While strictly speaking we could keep *_destroy() around
 without breaking existing code, this will mean we will have two ways of
 freeing an object. Perhaps a scheduled deprecation in a release or two
 is in order here?

Actually, I would prefer to leave *_destroy around, (as long as it's
doing nothing other than tall_free (or talloc_unlink? [*]) which does
appear to be the case for all existing functions).

To me, it doesn't really look like two ways of freeing an object.
Anyone who calls notmuch_query_create, (for example), should call
notmuch_query_destroy and those two calls pair nicely.

Then, *some* callers will want to take advantage of talloc. Those users
can add calls to talloc_reference and talloc_unlink (which pair nicely
themselves).

What I don't want is the unnatural pairing of something like
notmuch_query_create with talloc_unlink. That just seems uselessly
harder to learn and remember.

 Any and all feedback is desired.

Thanks for pointing out the issue. Let me know if I've missed
anything. And, happy binding!

-Carl

[*] These destroy() functions are all currently calling talloc_free()
which won't work if the user has any remaining parents still present due
to calls to talloc_reference(). Perhaps we can just document that all
talloc_reference()/talloc_unlink() pairs must be complete before the
caller calls _destroy()?

-- 
carl.d.wo...@intel.com


pgpZlK4NrApVm.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Encodings

[Carl Worth]

On Mon, 11 Jul 2011 16:04:17 +0200, Sebastian Spaeth  
wrote:
> The answer is that things are very implicit. notmuch.h speaks of
> strings but never mentions encodings

Much of this was intentional on my part.

For example, I intentionally avoided restrictions on what could be
stored as a tag in the database, (other than the terminating character
implied by "string" of course).

> So, can be document what encoding we are expected to pass in the various
> APIs

Yes, let's clarify documentation wherever we need to.

> For some of the stuff we read directly from the files, eg
> arbitrary headers, we can probably be least sure

The headers should be decoded to utf-8, (via
g_mime_utils_header_decode_text), before being stored in the database.

> but are e.g. the returned tags always utf-8?

No. The tag data is returned exactly as the user presented it.

> I would love to make the python bindings use unicode() instances in
> cases where we can be sure to actually receive utf-8 encoded strings.
> 
> Encodings make my brain hurt. Unfortunately one cannot simply ignore
> them.

I think a lot of the pain here is due to some bad design decisions in
python itself. Of course, my saying that doesn't make things any easier
for you.

But do tell me what more we can do to clarify behavior or documentation.

-Carl

-- 
carl.d.worth at intel.com
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 

Re: Encodings

[Carl Worth]

On Mon, 11 Jul 2011 16:04:17 +0200, Sebastian Spaeth sebast...@sspaeth.de 
wrote:
 The answer is that things are very implicit. notmuch.h speaks of
 strings but never mentions encodings

Much of this was intentional on my part.

For example, I intentionally avoided restrictions on what could be
stored as a tag in the database, (other than the terminating character
implied by string of course).

 So, can be document what encoding we are expected to pass in the various
 APIs

Yes, let's clarify documentation wherever we need to.

 For some of the stuff we read directly from the files, eg
 arbitrary headers, we can probably be least sure

The headers should be decoded to utf-8, (via
g_mime_utils_header_decode_text), before being stored in the database.

 but are e.g. the returned tags always utf-8?

No. The tag data is returned exactly as the user presented it.

 I would love to make the python bindings use unicode() instances in
 cases where we can be sure to actually receive utf-8 encoded strings.
 
 Encodings make my brain hurt. Unfortunately one cannot simply ignore
 them.

I think a lot of the pain here is due to some bad design decisions in
python itself. Of course, my saying that doesn't make things any easier
for you.

But do tell me what more we can do to clarify behavior or documentation.

-Carl

-- 
carl.d.wo...@intel.com


pgpa9xhIIcXO4.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] NEWS: Add notes for (imminent) notmuch 0.6 release

[Carl Worth]

By skimming through "git log 0.5..origin/release" late at
night. Hopefully everything here is accurate.
---

I've already pushed this out to master.

David, obviously this is intended to be cherry-picked over to the
release branch.

And everyone, please review what I've written and provide any
clarfications or corections necessary. Let me know if you think I've
included more detail than necessary or if I've missed anything
important.

Thanks,

-Carl


 NEWS |  344 ++
 1 files changed, 344 insertions(+), 0 deletions(-)

diff --git a/NEWS b/NEWS
index dae7832..d55453b 100644
--- a/NEWS
+++ b/NEWS
@@ -1,3 +1,347 @@
+Notmuch 0.6 (2011-07-XX)
+===
+New, general features
+-
+Folder-based searching
+
+  Notmuch queries can now include a search term to match the
+  directories in which mail files are stored (within the mail
+  storage). The syntax is as follows:
+
+   folder:
+
+  For example, one might use things such as:
+
+   folder:spam
+   folder:2011/06
+
+  or anything else that matches directories within your mail storage.
+
+  This feature is particularly useful for users of delivery-agent
+  software (such as procmail or maildrop) that is filtering mail and
+  delivering it to particular folders, or users of systems such as
+  Gmail that use filesystem directories to indicate message tags.
+
+  NOTE: Only messages that are newly indexed with this version of
+  notmuch will be searchable with folder: terms. In order to enable
+  this feature for all mail, the entire notmuch index will need to be
+  rebuilt as follows:
+
+   notmuch dump > notmuch.dump
+   # Backup, then remove notmuch database ($MAIL/.notmuch)
+   notmuch new
+   notmuch restore notmuch.dump
+
+New, automatic tags: "signed" and "encrypted"
+
+  These tags will automatically be applied to messages containing
+  multipart/signed and multipart/encrypted parts.
+
+  NOTE: Only messages that are newly indexed with this version of
+  notmuch will receive these tags. In order to enable this feature for
+  all mail, the entire notmuch index will need to be rebuilt (see
+  above).
+
+New command-line features
+-
+Add new "notmuch show --verify" option for signature verification
+
+  This option instruct notmuch to verify the signature of
+  PGP/MIME-signed parts.
+
+Add new "notmuch show --decrypt" and "notmuch reply --decrypt" options
+
+  This option instructs notmuch to decrypt PGP/MIME-encrypted parts.
+
+Proper nesting of multipart parts in "notmuch show" output
+
+  MIME parts are now display with proper nesting to reflect original
+  MIME hierarchy of a message. This allows clients to correctly
+  analyze the MIME structure, (such as, for example, determining to
+  which parts a signature part applies).
+
+Add new "notmuch show --part" option
+
+  This is a replacement for the older "notmuch part" command, (which
+  is now deprecated?it should still work as always, but is no longer
+  documented). Putting part output under "notmuch show" allows for all
+  of the "notmuch show" options to be applied when extracting a single
+  part, (such as --format=json for extracting a message part with JSON
+  formatting).
+
+Deprecate "notmuch search-tags", (in favor of "notmuch search --output=tags *")
+
+  The "notmuch search-tags" sub-command has been redundant since the
+  addition of the --output=tags option to "notmuch search". We now
+  make that more clear by deprecating "notmuch search-tags", (dropping
+  it from the documentation). We do continue to support the old syntax
+  by translating it internally to the new call.
+
+Performance improvements
+
+Faster searches (by doing fewer serches to construct threads)
+
+  Whenever a user asks for search results as threads, notmuch first
+  performs a search for messages matching the query, then performs
+  additional searches to find other messages in the resulting threads.
+
+  Removing inefficiences and redundancies in these secondary searches
+  results in a measured speedups of 1.5x for a typical search.
+
+Faster searches (by doing fewer passes to gather message data)
+
+  Optimizing Xapian data access patterns (using a single pass to get
+  all message-document data rather than a ps for each data type)
+  results in a measured speedup of 1.7x for a typical search.
+
+  The benefits of this optimization combine with the preceding
+  optimization. With both in place, Austin Clements measured a speedup
+  of 2.5x for a search of all messages in his inbox (was 4.5s, now
+  1.8s). Thanks, Austin!
+
+Faster initial indexing
+
+  More efficient indexing of new messages results in a measured
+  speedup of 1.4x for the initial indexing of 3 GB of mail (1h 14m
+  rather than 1h 46m). Thanks to Austin Clements and Michal Sojka.
+
+Make "notmuch new" faster for unchanged directories
+
+  Optimizing to not do any further 

[PATCH] emacs: Fix to unconditionally display subject changes in collapsed thread view

[Carl Worth]

On Fri,  1 Jul 2011 02:01:08 -0700, Carl Worth  wrote:
> Rather than fixing the name of the variable and changing its default
> value, here we remove the condition entirely, such that the feature is
> enabled unconditionally.

I've just pushed this change out.

I emailed it here just to be able to comment on it.

David, I implemented this fix while writing up the NEWS[*] notes for the
0.6 release (and noticing that this feature which was implemented wasn't
actually usable as documented so I couldn't really add a NEWS item for
it).

If you want to sneak it into 0.6 I think it's safe, (it even hits one
existing test case---though an additional test case to show an actual
subject change appearing would be even better).

It's your call of course, and if the change doesn't go, then that's fine
too. In that case, I'll just not comment on this feature in NEWS until
our next release.

-Carl

[*] Yes, I'm working away at this?almost done now...
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20110701/8bb02d6b/attachment-0001.pgp>

[PATCH] emacs: Fix to unconditionally display subject changes in collapsed thread view

[Carl Worth]

The feature to show subject changes in the collapsed thread view was
originally added (8ab433607) with an option
(notmuch-show-always-show-subject) to display the subject
for all messages, even when there was no change.

The subsequent commit (4f04d273) changed the sense of the test (or to
and) and the name of the controlling variable
(notmuch-show-elide-same-subject).

But this commit is broken in a few ways:

  1. The original definition of notmuch-show-always-show-subject was
 left around

 But the variable isn't actually used in the code at all, so it
 just adds clutter and confusion to the customization interface.

  2. The name and description of the controlling variable doesn't
 match the implementation

 The name suggests that setting the variable to t will cause
 repeated subjects to be elided, (suggesting that when it is nil
 all subjects will be shown).

 However, when the variable is nil, no subjects are shown. So a
 correct name for the variable in this sense would be
 notmuch-show-subject-changes.

Showing subject changes is a useful feature, and should be on by
default. (We don't want to bury generally useful features behind
customizations that users have to find).

Rather than fixing the name of the variable and changing its default
value, here we remove the condition entirely, such that the feature is
enabled unconditionally.

So both the currently-used variable and the stale definition of the
formerly-used are removed.

Also, the one relevant test-suite result is updated, (showing the
intial subject of a collapsed thread, and no subject display for later
messages that do not change the subject).
---
 emacs/notmuch-show.el  |   16 ++--
 .../notmuch-show-thread-with-hidden-messages   |1 +
 2 files changed, 3 insertions(+), 14 deletions(-)

diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
index 6685717..f96743b 100644
--- a/emacs/notmuch-show.el
+++ b/emacs/notmuch-show.el
@@ -65,17 +65,6 @@ any given message."
   :group 'notmuch
   :type 'boolean)

-(defcustom notmuch-show-elide-same-subject nil
-  "Do not show the subject of a collapsed message if it is the
-same as that of the previous message."
-  :group 'notmuch
-  :type 'boolean)
-
-(defcustom notmuch-show-always-show-subject t
-  "Should a collapsed message show the `Subject:' line?"
-  :group 'notmuch
-  :type 'boolean)
-
 (defvar notmuch-show-markup-headers-hook '(notmuch-show-colour-headers)
   "A list of functions called to decorate the headers listed in
 `notmuch-message-headers'.")
@@ -727,9 +716,8 @@ current buffer, if possible."
   ;; If the subject of this message is the same as that of the
   ;; previous message, don't display it when this message is
   ;; collapsed.
-  (when (and notmuch-show-elide-same-subject
-(not (string= notmuch-show-previous-subject
-  bare-subject)))
+  (when (not (string= notmuch-show-previous-subject
+ bare-subject))
(forward-line 1))
   (setq headers-start (point-marker)))
 (setq headers-end (point-marker))
diff --git 
a/test/emacs.expected-output/notmuch-show-thread-with-hidden-messages 
b/test/emacs.expected-output/notmuch-show-thread-with-hidden-messages
index 5df6606..8a0660f 100644
--- a/test/emacs.expected-output/notmuch-show-thread-with-hidden-messages
+++ b/test/emacs.expected-output/notmuch-show-thread-with-hidden-messages
@@ -1,3 +1,4 @@
 Jan Janak  (2009-11-17) (inbox unread)
+Subject: [notmuch] What a great idea!
  Jan Janak  (2009-11-17) (inbox)
  Carl Worth  (2009-11-18) (inbox unread)
-- 
1.7.5.4

[PATCH] emacs: Fix to unconditionally display subject changes in collapsed thread view

[Carl Worth]

The feature to show subject changes in the collapsed thread view was
originally added (8ab433607) with an option
(notmuch-show-always-show-subject) to display the subject
for all messages, even when there was no change.

The subsequent commit (4f04d273) changed the sense of the test (or to
and) and the name of the controlling variable
(notmuch-show-elide-same-subject).

But this commit is broken in a few ways:

  1. The original definition of notmuch-show-always-show-subject was
 left around

 But the variable isn't actually used in the code at all, so it
 just adds clutter and confusion to the customization interface.

  2. The name and description of the controlling variable doesn't
 match the implementation

 The name suggests that setting the variable to t will cause
 repeated subjects to be elided, (suggesting that when it is nil
 all subjects will be shown).

 However, when the variable is nil, no subjects are shown. So a
 correct name for the variable in this sense would be
 notmuch-show-subject-changes.

Showing subject changes is a useful feature, and should be on by
default. (We don't want to bury generally useful features behind
customizations that users have to find).

Rather than fixing the name of the variable and changing its default
value, here we remove the condition entirely, such that the feature is
enabled unconditionally.

So both the currently-used variable and the stale definition of the
formerly-used are removed.

Also, the one relevant test-suite result is updated, (showing the
intial subject of a collapsed thread, and no subject display for later
messages that do not change the subject).
---
 emacs/notmuch-show.el  |   16 ++--
 .../notmuch-show-thread-with-hidden-messages   |1 +
 2 files changed, 3 insertions(+), 14 deletions(-)

diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
index 6685717..f96743b 100644
--- a/emacs/notmuch-show.el
+++ b/emacs/notmuch-show.el
@@ -65,17 +65,6 @@ any given message.
   :group 'notmuch
   :type 'boolean)
 
-(defcustom notmuch-show-elide-same-subject nil
-  Do not show the subject of a collapsed message if it is the
-same as that of the previous message.
-  :group 'notmuch
-  :type 'boolean)
-
-(defcustom notmuch-show-always-show-subject t
-  Should a collapsed message show the `Subject:' line?
-  :group 'notmuch
-  :type 'boolean)
-
 (defvar notmuch-show-markup-headers-hook '(notmuch-show-colour-headers)
   A list of functions called to decorate the headers listed in
 `notmuch-message-headers'.)
@@ -727,9 +716,8 @@ current buffer, if possible.
   ;; If the subject of this message is the same as that of the
   ;; previous message, don't display it when this message is
   ;; collapsed.
-  (when (and notmuch-show-elide-same-subject
-(not (string= notmuch-show-previous-subject
-  bare-subject)))
+  (when (not (string= notmuch-show-previous-subject
+ bare-subject))
(forward-line 1))
   (setq headers-start (point-marker)))
 (setq headers-end (point-marker))
diff --git 
a/test/emacs.expected-output/notmuch-show-thread-with-hidden-messages 
b/test/emacs.expected-output/notmuch-show-thread-with-hidden-messages
index 5df6606..8a0660f 100644
--- a/test/emacs.expected-output/notmuch-show-thread-with-hidden-messages
+++ b/test/emacs.expected-output/notmuch-show-thread-with-hidden-messages
@@ -1,3 +1,4 @@
 Jan Janak j...@ryngle.com (2009-11-17) (inbox unread)
+Subject: [notmuch] What a great idea!
  Jan Janak j...@ryngle.com (2009-11-17) (inbox)
  Carl Worth cwo...@cworth.org (2009-11-18) (inbox unread)
-- 
1.7.5.4

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] emacs: Fix to unconditionally display subject changes in collapsed thread view

[Carl Worth]

On Fri,  1 Jul 2011 02:01:08 -0700, Carl Worth cwo...@cworth.org wrote:
 Rather than fixing the name of the variable and changing its default
 value, here we remove the condition entirely, such that the feature is
 enabled unconditionally.

I've just pushed this change out.

I emailed it here just to be able to comment on it.

David, I implemented this fix while writing up the NEWS[*] notes for the
0.6 release (and noticing that this feature which was implemented wasn't
actually usable as documented so I couldn't really add a NEWS item for
it).

If you want to sneak it into 0.6 I think it's safe, (it even hits one
existing test case---though an additional test case to show an actual
subject change appearing would be even better).

It's your call of course, and if the change doesn't go, then that's fine
too. In that case, I'll just not comment on this feature in NEWS until
our next release.

-Carl

[*] Yes, I'm working away at this—almost done now...


pgpXiFxJ4yJoe.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] NEWS: Add notes for (imminent) notmuch 0.6 release

[Carl Worth]

By skimming through git log 0.5..origin/release late at
night. Hopefully everything here is accurate.
---

I've already pushed this out to master.

David, obviously this is intended to be cherry-picked over to the
release branch.

And everyone, please review what I've written and provide any
clarfications or corections necessary. Let me know if you think I've
included more detail than necessary or if I've missed anything
important.

Thanks,

-Carl


 NEWS |  344 ++
 1 files changed, 344 insertions(+), 0 deletions(-)

diff --git a/NEWS b/NEWS
index dae7832..d55453b 100644
--- a/NEWS
+++ b/NEWS
@@ -1,3 +1,347 @@
+Notmuch 0.6 (2011-07-XX)
+===
+New, general features
+-
+Folder-based searching
+
+  Notmuch queries can now include a search term to match the
+  directories in which mail files are stored (within the mail
+  storage). The syntax is as follows:
+
+   folder:path
+
+  For example, one might use things such as:
+
+   folder:spam
+   folder:2011/06
+
+  or anything else that matches directories within your mail storage.
+
+  This feature is particularly useful for users of delivery-agent
+  software (such as procmail or maildrop) that is filtering mail and
+  delivering it to particular folders, or users of systems such as
+  Gmail that use filesystem directories to indicate message tags.
+
+  NOTE: Only messages that are newly indexed with this version of
+  notmuch will be searchable with folder: terms. In order to enable
+  this feature for all mail, the entire notmuch index will need to be
+  rebuilt as follows:
+
+   notmuch dump  notmuch.dump
+   # Backup, then remove notmuch database ($MAIL/.notmuch)
+   notmuch new
+   notmuch restore notmuch.dump
+
+New, automatic tags: signed and encrypted
+
+  These tags will automatically be applied to messages containing
+  multipart/signed and multipart/encrypted parts.
+
+  NOTE: Only messages that are newly indexed with this version of
+  notmuch will receive these tags. In order to enable this feature for
+  all mail, the entire notmuch index will need to be rebuilt (see
+  above).
+
+New command-line features
+-
+Add new notmuch show --verify option for signature verification
+
+  This option instruct notmuch to verify the signature of
+  PGP/MIME-signed parts.
+
+Add new notmuch show --decrypt and notmuch reply --decrypt options
+
+  This option instructs notmuch to decrypt PGP/MIME-encrypted parts.
+
+Proper nesting of multipart parts in notmuch show output
+
+  MIME parts are now display with proper nesting to reflect original
+  MIME hierarchy of a message. This allows clients to correctly
+  analyze the MIME structure, (such as, for example, determining to
+  which parts a signature part applies).
+
+Add new notmuch show --part option
+
+  This is a replacement for the older notmuch part command, (which
+  is now deprecated—it should still work as always, but is no longer
+  documented). Putting part output under notmuch show allows for all
+  of the notmuch show options to be applied when extracting a single
+  part, (such as --format=json for extracting a message part with JSON
+  formatting).
+
+Deprecate notmuch search-tags, (in favor of notmuch search --output=tags *)
+
+  The notmuch search-tags sub-command has been redundant since the
+  addition of the --output=tags option to notmuch search. We now
+  make that more clear by deprecating notmuch search-tags, (dropping
+  it from the documentation). We do continue to support the old syntax
+  by translating it internally to the new call.
+
+Performance improvements
+
+Faster searches (by doing fewer serches to construct threads)
+
+  Whenever a user asks for search results as threads, notmuch first
+  performs a search for messages matching the query, then performs
+  additional searches to find other messages in the resulting threads.
+
+  Removing inefficiences and redundancies in these secondary searches
+  results in a measured speedups of 1.5x for a typical search.
+
+Faster searches (by doing fewer passes to gather message data)
+
+  Optimizing Xapian data access patterns (using a single pass to get
+  all message-document data rather than a ps for each data type)
+  results in a measured speedup of 1.7x for a typical search.
+
+  The benefits of this optimization combine with the preceding
+  optimization. With both in place, Austin Clements measured a speedup
+  of 2.5x for a search of all messages in his inbox (was 4.5s, now
+  1.8s). Thanks, Austin!
+
+Faster initial indexing
+
+  More efficient indexing of new messages results in a measured
+  speedup of 1.4x for the initial indexing of 3 GB of mail (1h 14m
+  rather than 1h 46m). Thanks to Austin Clements and Michal Sojka.
+
+Make notmuch new faster for unchanged directories
+
+  Optimizing to not do any further examinations of sub-directories
+  

Re: [PATCH] remove prefixes from `--output={threads, messages}' results

[Carl Worth]

On Thu, 30 Jun 2011 10:19:49 +0200, Pieter Praet pie...@praet.org wrote:
 Alter `do_search_threads()' and `do_search_messages()'
 to not prepend each result with `thread:' respectively `id:'.

My one concern here is that I've sometimes had a message-id without the
id prefix and run a command like this:

notmuch search 1309421989-22410-1-git-send-email-pie...@praet.org

And I've gotten confused when I've received no output, (didn't I receive
that mail? what happened?).

But I think this is a separate bug where the right fix is to make any
search terms with no prefix search through *all* prefixed terms
generated from email content. This would allow us to also avoid indexing
some content twice, (currently we store subject, from, and to both with
a prefix and without a prefix).

-Carl

-- 
carl.d.wo...@intel.com


pgpidP7oSWd6S.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch