Re: find threads where I and Jian participated but not Dave

[Matt Armstrong]

Gaute Hope  writes:

> Gaute Hope writes on juni 22, 2017 8:08:
>> Daniel Kahn Gillmor writes on juni 21, 2017 23:30:
>>> On Wed 2017-06-21 13:04:53 -0700, Matt Armstrong wrote:
 For what it is worth, I've found this idea from Daniel intriguing and
 pretty useful in practice:

   "show me threads in which i've participated, where there are some
messages flagged with 'inbox'"

 I implement it like this in my post-new hook:

 # All messages in threads in which I participate get tag:participated
 notmuch search --output=threads from:marmstrong | \
   sed -e 's,^,+participated -- ,' | \
   notmuch tag --batch
>>> 
>>> cool, thx for the suggestion.
>>> 
>>> the "notmuch search" part of the pipeline alone takes ~19s (wall time,
>>> and actual CPU time) for me though :/  It returns 30504 threads!  how
>>> many threads do you get?
>> 
>> Is there any reason why you do not filter on a tag 'new' as well?
>> 
>>  notmuch search --output=threads from:marmstrong and tag:new | \
>>sed -e 's,^,+participated -- ,' | \
>>notmuch tag --batch
>> 
>
> Nevermind, I get it - it might be possible to add a temporary tag 
> new-tag to the whole thread first and not just new messages. That might 
> be faster. As long as all sent messages get the new tag as well.

Gaute, I took this as a challenge and came up with what I think is an
equivalent but more efficient approach.  The disadvantage is that it is
much more complex.  The advantage is that it runs in under 0.2 seconds
to process a day's worth of my "new" mail.

I now have this in my notmuch post-hook.  I believe I could change the
"tag:new OR date:today" query to just "tag:new".  The "OR date:today"
helped during interactive development.

# All threads in which I participate get tag:participated
#  1) Find all threads with a message tagged new
# (finding all 'today' messages helps during testing,
# but isn't necessary)
#  2) Run through "xargs -s 2048 echo" to to group threads
# lines of about 2K in size.
#  3) For each line (2) produces, narrow the threads to
# those containing a message from me.
#  4) For each such thread, tag every message with +participated.
notmuch search --output=threads tag:new OR date:today | \
  xargs -s 2048 echo | \
  xargs -I '{}' notmuch search \
  --output=threads from:marmstrong AND \( '{}' \) | \
  sed -e 's,^,+participated -- ,' | \
  notmuch tag --batch


The basic idea is that each run of the notmuch post-hook will
incorporate relatively little mail, so the number of unique threads will
be relatively small.  So, we just list them all by thread ID.

Then for each thread with new messages, we figure out which threads have
a message from:marmstrong (it need not be the new message).

We then tag all messages in each of those threads with +participated.

You said "it might be possible to add a temporary tag new-tag to the
whole thread first and not just new messages." -- Yes, and that is
implicitly what I am doing, except that each such thread is instead
tracked in an ephemeral way through the xargs based shell pipeline.

I did try an approach of explicitly labeling all messages in "new"
threads, temporarily, but that was slower.

You said "As long as all sent messages get the new tag as well." --
true, and I'm not sure about that.  My primary use for this is to
discover new activity from others *after* I've participated in a thread,
so I don't much care if a thread that is "participated in" is not tagged
that way until some mail from somebody else arrives.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: find threads where I and Jian participated but not Dave

[Matt Armstrong]

Daniel Kahn Gillmor  writes:

> On Wed 2017-06-21 13:04:53 -0700, Matt Armstrong wrote:
>> For what it is worth, I've found this idea from Daniel intriguing and
>> pretty useful in practice:
>>
>>   "show me threads in which i've participated, where there are some
>>messages flagged with 'inbox'"
>>
>> I implement it like this in my post-new hook:
>>
>> # All messages in threads in which I participate get tag:participated
>> notmuch search --output=threads from:marmstrong | \
>>   sed -e 's,^,+participated -- ,' | \
>>   notmuch tag --batch
>
> cool, thx for the suggestion.
>
> the "notmuch search" part of the pipeline alone takes ~19s (wall time,
> and actual CPU time) for me though :/  It returns 30504 threads!  how
> many threads do you get?

The query returns 6600 threads.  I'm getting 2 seconds wall clock time.


> you're effectively re-tagging every single message in every participated
> thread every time you run "notmuch new", right?

Yeah, the "batch script" that the above search+sed creates and pipes
into "notmuch tag --batch" is 265K, but it only takes 0.5 seconds to
execute.  My understanding is that "notmuch tag" is smart enough to do
no work if the tag is already present on a message, so the only changes
happening in the database are actually for new stuff.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: find threads where I and Jian participated but not Dave

[Gaute Hope]

Gaute Hope writes on juni 22, 2017 8:08:

Daniel Kahn Gillmor writes on juni 21, 2017 23:30:

On Wed 2017-06-21 13:04:53 -0700, Matt Armstrong wrote:

For what it is worth, I've found this idea from Daniel intriguing and
pretty useful in practice:

  "show me threads in which i've participated, where there are some
   messages flagged with 'inbox'"

I implement it like this in my post-new hook:

# All messages in threads in which I participate get tag:participated
notmuch search --output=threads from:marmstrong | \
  sed -e 's,^,+participated -- ,' | \
  notmuch tag --batch


cool, thx for the suggestion.

the "notmuch search" part of the pipeline alone takes ~19s (wall time,
and actual CPU time) for me though :/  It returns 30504 threads!  how
many threads do you get?


Is there any reason why you do not filter on a tag 'new' as well?

 notmuch search --output=threads from:marmstrong and tag:new | \
   sed -e 's,^,+participated -- ,' | \
   notmuch tag --batch



Nevermind, I get it - it might be possible to add a temporary tag 
new-tag to the whole thread first and not just new messages. That might 
be faster. As long as all sent messages get the new tag as well.




pgpIkGDZ3Jptl.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: find threads where I and Jian participated but not Dave

[Gaute Hope]

Daniel Kahn Gillmor writes on juni 21, 2017 23:30:

On Wed 2017-06-21 13:04:53 -0700, Matt Armstrong wrote:

For what it is worth, I've found this idea from Daniel intriguing and
pretty useful in practice:

  "show me threads in which i've participated, where there are some
   messages flagged with 'inbox'"

I implement it like this in my post-new hook:

# All messages in threads in which I participate get tag:participated
notmuch search --output=threads from:marmstrong | \
  sed -e 's,^,+participated -- ,' | \
  notmuch tag --batch


cool, thx for the suggestion.

the "notmuch search" part of the pipeline alone takes ~19s (wall time,
and actual CPU time) for me though :/  It returns 30504 threads!  how
many threads do you get?


Is there any reason why you do not filter on a tag 'new' as well?

notmuch search --output=threads from:marmstrong and tag:new | \
  sed -e 's,^,+participated -- ,' | \
  notmuch tag --batch


Regards, Gaute



pgpUmguMAhigd.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch