Re: bug#50460: 27.1; defcustom for mail-user-agent is too strict?

2021-09-08 Thread Gregor Zattler
Hi David, Lars,
* Lars Ingebrigtsen  [2021-09-08; 08:56]:
> David Bremner  writes:
>
>> The defcustom for mail-user-agent looks like
>>
>> :type '(radio (function-item :tag "Message package"
>> :format "%t\n"
>> message-user-agent)
>> ;; [...] snip
>>  (function-item :tag "Message with full Gnus features"
>> :format "%t\n"
>> gnus-user-agent)
>>  (function :tag "Other"))
>>
>> This means that a symbol without a function definition cannot be set
>> (using customize) as mail-user-agent. This seem inconsistent with the
>> docstring of define-mail-user-agent,
>
> Yup.  Now fixed in Emacs 28.

thanks for the fast bug report and fixing.  I confirm it is
now possible to customize `mail-user-agent' to
`notmuch-user-agent' via `customize-variable'.


Ciao; Gregor
--
 -... --- .-. . -.. ..--.. ...-.-
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [bug] Invalid function: notmuch-user-agent (was: notmuch release 0.33 now available)

2021-09-06 Thread Gregor Zattler
Hi David,
* David Bremner  [2021-09-06; 10:00]:
> Gregor Zattler  writes:
>> I therefore tried to change `mail-user-agent' via the customization
>> interface (clicking the radio button "Other:" and providing
>> "notmuch-user-agent" as value) but then I got an error
>> message:
>>
>> custom-variable-mark-to-save: Saving mail-user-agent: Invalid function: 
>> notmuch-user-agent
>>
>
> This seems like a bug in the defcustom for mail-user-agent (i.e. a bug
> in emacs).
> I think you will find that
>
>(setq mail-user-agent 'notmuch-user-agent)

thanks, I will use this.

> works. My reading of the documentation (and either my reading or the
> docs could be wrong) is that the symbol for a mail-user-agent need not
> be a function.  Quoting the docstring for define-mail-user-agent
>
> SYMBOL can be any Lisp symbol.  Its function definition and/or
> value as a variable do not matter for this usage; we use only certain
> properties on its property list, to encode the rest of the arguments.
>
> Other user agents, e.g. 'gnus-user-agent, 'message-user-agent, and
> 'sendmail-user-agent do not seem to have function definition. On the
> other hand, it is a bit surprising that no-one noticed a problem with
> "Other" mail-user-agent values before now.

Actually this is a bit above my lisp knowledge.  Do you
think your explanation suffices for a bug report against emacs?

Ciao; Gregor
--
 -... --- .-. . -.. ..--.. ...-.-
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


[bug] Invalid function: notmuch-user-agent (was: notmuch release 0.33 now available)

2021-09-06 Thread Gregor Zattler
Hi David, notmuch developers,
* David Bremner  [2021-09-03; 13:29]:
> Emacs
> -
>
> `notmuch` no longer sets `mail-user-agent` on load. To restore the
> previous behaviour of using notmuch to send mail by default, customize
> `mail-user-agent` to `notmuch-user-agent`.

While reading this news, I realized, that `mail-user-agent'
is set to "message-user-agent" in my emacs instance.

I therefore tried to change `mail-user-agent' via the customization
interface (clicking the radio button "Other:" and providing
"notmuch-user-agent" as value) but then I got an error
message:

custom-variable-mark-to-save: Saving mail-user-agent: Invalid function: 
notmuch-user-agent

This also happened from "emacs -Q"  and when calling
notmuch-emacs-mua --hello
from the command line.  Even then, `mail-user-agent' is set to
"message-user-agent".

This happens with:  notmuch version 0.33+14~g005c620   on

GNU Emacs 28.0.50 (build 1, x86_64-pc-linux-gnu, X toolkit, cairo version 
1.16.0) of 2021-09-05


What to do?



Thanks for notmuch and your attention, Gregor
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


make: *** No rule to make target 'notmuch-client-init.o', needed by 'notmuch'. Stop.

2021-05-15 Thread Gregor Zattler
Dear notmuch developers, after a fresh git clone of the
notmuch sources and a make produces an error:

grfz@no:/tmp$ git clone git://notmuchmail.org/git/notmuch
[...]
grfz@no:/tmp$ cd notmuch
grfz@no:/tmp/notmuch$ make
[...]
make: *** No rule to make target 'notmuch-client-init.o', needed by 'notmuch'.  
Stop.



2 (master=) grfz@no:/tmp/notmuch$ rgrep notmuch-client-init
Makefile.local: notmuch-client-init.c   \
notmuch-client.h:/* notmuch-client-init.c */
0 (master=) grfz@no:/tmp/notmuch$ find | grep notmuch-client-init.c
1 (master=) grfz@no:/tmp/notmuch$


I have no idea what's happening here.


Ciao; Gregor
--
 -... --- .-. . -.. ..--.. ...-.-
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


bug: chokes on long directory names (was: Re: out of memory on idle machine)

2021-03-17 Thread Gregor Zattler
Hi David, Olly, notmuch and xapian developers,
* David Bremner  [11. Feb. 2021]:
> David Bremner  writes:
> As a kind of desperation move, you could try bisecting your mailstore,
> to see how small of a set of messages you can duplicate the problem
> with.

this I did, somehow.  I found the culprit: It's a maildir
with one single mail in it.  The name of the maildir is
exceptionally long [because generated from a List-Id:
-Header] and the mail arrived at the very day, my notmuch
database corrupted.  This maildir alone provokes that every
next notmuch new will rescan all (?) files.

Then I tried to only index this maildir, it showed the same
strange re-indexing but even when running notmuch new for a
while in a loop (>1000 times), the database showed no
corruption.

When instead I shorten the name of the maildir to three
characters with the very same email file in it, nothing
happens, it indexes the file once and not again.

Then I prolonged the name of the file instead of the
directory and even with the longest possible filename (or
path?)

/home/grfz/Mail/nuk/new/1607641473.31514_2.no1607641473.31514_2.no1607641473.31514_2.no1607641473.31514_2.no1607641473.31514_2.no1607641473.31514_2.no1607641473.31514_2.no1607641473.31514_2.no1607641473.31514_2.no1607641473.31514_2.no1607641473.31514_2.no16076414734160.14_2.no

notmuch has no problem indexing this and not to reindex it
in the next run.


So notmuch or xapian (I don't know) chokes on extreme long
directory names.  I consider this to be a bug.



My scripts create this long names from List-Id and some
such.  The one which triggered the problems is from an online
shop:

u+mq6tamjqhe3cm2j5giydembrgiytamrtga2deojogexdsmzygm4egnbuifatcnrsgazdejjugbzgkylmfvxw43djnzsxg2dpoaxgizjgna6ton3bg4zdsobsgmytczlcme3dentehaydmnjxmy4doyrwha4tgobgoi6xizlmmvtxeylqnastimdhnv4c43tfoqthipldovzxi33nmvzhgllxmvwgg33...@real-onlineshop.de/

Since, as I tested, this can be reproduced with the simplest
of email in a maildir with an extremly long name, I do not
attach the maildir in question.  But if anyone wants it I
can send it.



I then had a look at other long directory names and there is
another one which also triggers the problem, it also has
only one email in it and arrived on 12th of January:

u+mq6wcodfgmygcjtjhuzdamrrgaytemjrhe2dqmbqfyys4mbxgazugnbsie3doobsgfcdmobfgqygg5ltorxw2zlsomxgo2lunrqweltdn5wsm2b5mu3tkmddhbrdoyrwgvsgeobymi2dszbtg4zdamztmm4dsmzvgjssm4r5orswyzlhojqxa2bfgqygo3lyfzxgk5bgoq6xa4tjozqw...@customers.gitlab.com


Since I removed both on my laptop, notmuch new works again,
yeah!  Now I will have a look on my .procmailrc.

Thanks for your attention, thanks for notmuch and for xapian,
Grgeor

--
 -... --- .-. . -.. ..--.. ...-.-
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: out of memory on idle machine

2021-02-21 Thread Gregor Zattler
Hi Olly, David, xapian and notmuch developers,
* Olly Betts  [12. Feb. 2021]:
> On Thu, Feb 11, 2021 at 06:53:27AM -0400, David Bremner wrote:
>> At this point I don't really have any good ideas, so I'm waiting for
>> results from the 1.4.18 trial.
>
> I've uploaded a backport, but it's the first backport of xapian-core to
> buster so it'll need manual approval.  Hopefully that'll happen over the
> weekend, but it could take longer.

I installed version 1.4.18 and the errors are the same.  I
will take Davids advice and try to bisect my mail store.
This will take some time.  I'll report back.


Ciao, Gregor
--
 -... --- .-. . -.. ..--.. ...-.-
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: out of memory on idle machine

2021-02-13 Thread Gregor Zattler
Hi Olly, notmuch and xapian developers,

sorry for late answer, I had problems with the test system:
* Olly Betts  [09. Feb. 2021]:
> On Wed, Feb 03, 2021 at 07:59:43AM -0400, David Bremner wrote:
>> Gregor Zattler  writes:
>>> A Xapian exception occurred finding message: Db block overwritten - are 
>>> there multiple writers?.
>>
>> I have included the Xapian list in copy in case that message rings a
>> bell.
>
> There was a bug fixed in 1.4.7 which incorrectly resulted in this error
> message, but it seems from the quoted text you're using 1.4.11.

yes.

>> I guess you know there are not multiple writers in your setup.
>
> There's a lock file locked by fcntl() which protects against multiple
> writers, so someone/something would have need to have deleted that
> behind Xapian's back, or else a bug somewhere in the locking code stack.

There is no other writer.  The system is used only for this
test atm, the emails are on a dedicated data partition.

> (Aside from that bug, probably the most common case here over time has
> been that someone deleted the lock file thinking it's "stale", but it's
> not the mere presence of the file that means the lock is held.  It's
> not at all frequent, but perhaps we should adjust this message to better
> reflect that.)
>
> Have you tried xapian-check on this database?

not this time.  Yust wait.  I have different databases from
different runs of notmuch new, "xapian-3" being identical
with "xapian-3"


grfz@mic:~/Mail/.notmuch$ for name in xapian  xapian-3 xapian-2 xapian-1 ; do 
echo "= $name"; xapian-check $name ; done > /tmp/xapian-checks 2>&1

results in:

= xapian
docdata:
blocksize=8K items=577 firstunused=10 revision=4 levels=1 root=6
B-tree checked okay
docdata table structure checked OK

termlist:
blocksize=8K items=68006 firstunused=20302 revision=4 levels=2 root=687
xapian-check: DatabaseError: Block 17959: Used block also in freelist
= xapian-3
docdata:
blocksize=8K items=577 firstunused=10 revision=4 levels=1 root=6
B-tree checked okay
docdata table structure checked OK

termlist:
blocksize=8K items=68006 firstunused=20302 revision=4 levels=2 root=687
xapian-check: DatabaseError: Block 17959: Used block also in freelist
= xapian-2
docdata:
blocksize=8K items=577 firstunused=10 revision=3 levels=1 root=6
B-tree checked okay
docdata table structure checked OK

termlist:
blocksize=8K items=68006 firstunused=20302 revision=3 levels=2 root=8651
B-tree checked okay
termlist table structure checked OK

postlist:
blocksize=8K items=1971726 firstunused=25177 revision=3 levels=2 root=6651
B-tree checked okay
postlist table structure checked OK

position:
blocksize=8K items=9589984 firstunused=52204 revision=3 levels=2 root=18102
B-tree checked okay
position table structure checked OK

spelling:
Lazily created, and not yet used.

synonym:
Lazily created, and not yet used.

No errors found
= xapian-1
docdata:
blocksize=8K items=371 firstunused=4 revision=2 levels=1 root=2
B-tree checked okay
docdata table structure checked OK

termlist:
blocksize=8K items=32426 firstunused=8650 revision=2 levels=2 root=687
B-tree checked okay
termlist table structure checked OK

postlist:
blocksize=8K items=562665 firstunused=6650 revision=2 levels=2 root=380
B-tree checked okay
postlist table structure checked OK

position:
blocksize=8K items=4359147 firstunused=18099 revision=2 levels=2 root=377
B-tree checked okay
position table structure checked OK

spelling:
Lazily created, and not yet used.

synonym:
Lazily created, and not yet used.

No errors found



There are no problems reported on earlier databases although
they resulted in reindexing of almost all of the emails and
in case of the corrupted xapian database not all file are
checked.


This is not fixable:
grfz@mic:~/Mail/.notmuch$ cp -a xapian xapian-checked
grfz@mic:~/Mail/.notmuch$ xapian-check xapian-checked F
docdata:
B-tree checked okay
docdata table structure checked OK

termlist:
xapian-check: DatabaseError: Block 17959: Used block also in freelist

grfz@mic:~/Mail/.notmuch$ xapian-check xapian-checked
docdata:
blocksize=8K items=577 firstunused=10 revision=4 levels=1 root=6
B-tree checked okay
docdata table structure checked OK

termlist:
blocksize=8K items=68006 firstunused=20302 revision=4 levels=2 root=687
xapian-check: DatabaseError: Block 17959: Used block also in freelist


>> Olly Betts mentioned in a different thread that he will build a version
>> of xapian 1.4.18 for buster backports, so trying with that is probably a
>> good step when it is available.
>
> Yes - 1.4.18 packages are now in Debian testing, so hopefully I can get
> this done soon.

OK, I'll wait for that.

>> % xapian-delve -1 -A XDIRECTORY ~/Mail/.notmuch/xapian | sort -u > delve.txt
>
> FWIW, the output should be sorted and unique al

Re: out of memory on idle machine

2021-02-07 Thread Gregor Zattler
Hi David, notmuch and xapian developers,
* David Bremner  [03. Feb. 2021]:
> Gregor Zattler  writes:
>>
>> Installed notmuch-dbgsym (0.28.4-1) and gdb.
>>
>> grfz@mic:/etc$ gdb --args notmuch new
>> [...]
>> (gdb) b notmuch-new.c:420
>> Breakpoint 1 at 0x10601: file notmuch-new.c, line 421.
>> (gdb) run
>> Starting program: /usr/bin/notmuch new
>> [Thread debugging using libthread_db enabled]
>> Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
>> add_file: A Xapian exception occurred
>> A Xapian exception occurred finding message: Db block overwritten - are 
>> there multiple writers?.
>> Processed 24 total files in almost no time.
>> Added 23 new messages to the database.
>> Note: A fatal error was encountered: A Xapian exception occurred
>> [Inferior 1 (process 22756) exited with code 01]
>> (gdb)
>>
>> This time it's no OOM it's a xapian exeption again.
>>
>>
>
> I have included the Xapian list in copy in case that message rings a
> bell. I guess you know there are not multiple writers in your setup.

Absolutely.  This is an otherwise unused minimal debian
buster system, the emails are a static copy from from my
production system (laptop) some days ago on a mounted
filesystem.  There are no other writes to this file system
besides notmuch new.

When I do subsequent notmuch new, most of the files are
reindexed in the second and third run, in the fourth run
there is a OOM although the system has 16GB RAM, 16GB swap.
The fourth notmuch new invocation throws an exeption:

grfz@mic:~/Mail/.notmuch$ nice ionice -c 3 notmuch new
Welcome to a new version of notmuch! Your database will now be upgraded.
This process is safe to interrupt.
Backing up tags to /home/grfz/Mail/.notmuch/dump-20210202T075743.gz...
Your notmuch database has now been upgraded.
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607947606.8134_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607940473.9509_1.no:2,S
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607969276.21046_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607987211.1395_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607979988.4942_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607972847.4857_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607943993.24776_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607976389.23296_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607983586.19063_1.no:2,
Note: Ignoring non-mail file: /home/grfz/Mail/drafts.mbox
Note: Ignoring non-mail file: /home/grfz/Mail/postponed.mbox
Processed 1183682 total files in 11h 33m 56s (28 files/sec.).
Added 1091038 new messages to the database.
grfz@mic:~/Mail/.notmuch$ cp -a xapian xapian-1
grfz@mic:~/Mail/.notmuch$ nice ionice -c 3 notmuch new
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607947606.8134_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607940473.9509_1.no:2,S
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607969276.21046_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607987211.1395_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607979988.4942_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607972847.4857_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607943993.24776_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607976389.23296_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607983586.19063_1.no:2,
Processed 1169095 total files in 13h 13m 16s (24 files/sec.).
Added 1077686 new messages to the database.
grfz@mic:~/Mail/.notmuch$ cp -a xapian xapian-2
grfz@mic:~/Mail/.notmuch$ nice ionice -c 3 notmuch new
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607947606.8134_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607940473.9509_1.no:2,S
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607969276.21046_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607987211.1395_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607979988.4942_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607972847.4857_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607943993.24776_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607976389.23296_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607983586.19063_1.no:2,
Processed 1151900 total files in 12h 55m 51s (24 files/sec.).
Added 1050106 new messages to the database.
grfz@mic:~/Mail/.notmuch$ cp -a xapian xapian-3
grfz@mic:~/Mail/.notmuch$ nice ionice -c 3 notmuch new
add_fi

Re: out of memory on idle machine

2021-01-31 Thread Gregor Zattler
Hi David, notmuch developers,
* Gregor Zattler  [31. Jan. 2021]:
> I'll redo idea #2 on my laptop and will report it's results.

Now on the laptop:

grfz@no:~/Mail/.notmuch$ nice ionice -c 3 notmuch new
Welcome to a new version of notmuch! Your database will now be upgraded.
This process is safe to interrupt.
Backing up tags to /home/grfz/Mail/.notmuch/dump-20210131T080546.gz...
Your notmuch database has now been upgraded.
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607940473.9509_1.no:2,S
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607943993.24776_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607947606.8134_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1611742380.14576_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607969276.21046_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607972847.4857_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607976389.23296_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607979988.4942_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607983586.19063_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607987211.1395_1.no:2,
Note: Ignoring non-mail file: /home/grfz/Mail/drafts.mbox
Processed 1184742 total files in 3h 33m 35s (92 files/sec.).
Added 1115147 new messages to the database.

0 (master *) grfz@no:~/Mail/.notmuch$ stat --format "%Y"  ~/Mail/inbox/cur
1612091679
0 (master *) grfz@no:~/Mail/.notmuch$ stat --format "%y"  ~/Mail/inbox/cur
2021-01-31 12:14:39.771049424 +0100
0 (master *) grfz@no:~/Mail/.notmuch$
0 (master *) grfz@no:~/Mail/.notmuch$ quest -bdir:XDIRECTORY -d 
~/Mail/.notmuch/xapian/ dir:inbox/cur
bash: quest: command not found
127 (master *) grfz@no:~/Mail/.notmuch$ quest -bdir:XDIRECTORY -d 
~/Mail/.notmuch/xapian/ dir:inbox/cur
Parsed Query: Query(0 * XDIRECTORYinbox/cur)
MSet:
1114128: [0]
inbox/cur

0 (master *) grfz@no:~/Mail/.notmuch$ xapian-delve -r 1114128 -VS0 
~/Mail/.notmuch/xapian
Value 0 for record #1114128: 1.61208e+09
Term List for record #1114128: XDDIRENTRY1114127:cur XDIRECTORYinbox/cur


So I think that's OK on my laptop, after the first notmuch
new.  Now I do another one.

0 (master *) grfz@no:~/Mail/.notmuch$ nice ionice -c 3 notmuch new
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607940473.9509_1.no:2,S
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607943993.24776_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607947606.8134_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607969276.21046_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607972847.4857_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607976389.23296_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607979988.4942_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607983586.19063_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607987211.1395_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1611742380.14576_1.no:2,
Note: Ignoring non-mail file: /home/grfz/Mail/drafts.mbox
Processed 121712 total files in 45m 4s (45 files/sec.).
Added 85345 new messages to the database. Removed 3 messages.
0 (master *) grfz@no:~/Mail/.notmuch$

The Problem remains, but at a different scale: In between
these two notmuch new runs there were only a few hours,
there's no way I received 121712 or 85345 emails in this time
frame.

0 (master *) grfz@no:~/Mail/.notmuch$ nice ionice -c 3 notmuch new
Note: Ignoring non-mail file: /home/grfz/Mail/spam/new/1612101986.719_1.no
Note: Ignoring non-mail file: /home/grfz/Mail/spam/new/1612105584.10821_1.no
Note: Ignoring non-mail file: /home/grfz/Mail/spam/new/1612109187.21153_1.no
Note: Ignoring non-mail file: /home/grfz/Mail/spam/new/1612112775.31326_1.no
Note: Ignoring non-mail file: /home/grfz/Mail/spam/new/1612116375.9385_1.no
Note: Ignoring non-mail file: /home/grfz/Mail/spam/new/1612120297.21133_1.no
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607940473.9509_1.no:2,S
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607943993.24776_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607947606.8134_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607969276.21046_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607972847.4857_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607976389.23296_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607979988.4942_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607983586.19063_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607987211.1395_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1611742380.14576_1.no:2,
terminat

Re: out of memory on idle machine

2021-01-31 Thread Gregor Zattler
Hi David, notmuch developers,

thanks a lot: your second idea this time showed a database
corruption instead of an OOM.

The first idea showed, that there is data missing in the
database:

* David Bremner  [30. Jan. 2021]:
Since idea #1 involves removing the xapian database I first
tried idea #2:
> Idea #2
> ---
>
> Try to figure out if some specific file is causing the OOM.
>
> Run notmuch-new in gdb
>
> There is a check for NOTMUCH_STATUS_OUT_OF_MEMORY around line 419/420 of
> notmuch-new.c. If I understand correctly, that is where things are
> failing. The following is untested; you will need the package
> notmuch-dbgsym installed [1]
>
> % gdb --args notmuch new
> (gdb) b notmuch-new.c:420
> (gdb) run
> (gdb) p filename
> [1]: https://wiki.debian.org/AutomaticDebugPackages

Installed notmuch-dbgsym (0.28.4-1) and gdb.

grfz@mic:/etc$ gdb --args notmuch new
[...]
(gdb) b notmuch-new.c:420
Breakpoint 1 at 0x10601: file notmuch-new.c, line 421.
(gdb) run
Starting program: /usr/bin/notmuch new
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
add_file: A Xapian exception occurred
A Xapian exception occurred finding message: Db block overwritten - are there 
multiple writers?.
Processed 24 total files in almost no time.
Added 23 new messages to the database.
Note: A fatal error was encountered: A Xapian exception occurred
[Inferior 1 (process 22756) exited with code 01]
(gdb)

This time it's no OOM it's a xapian exeption again.



Now for idea #1:

> Idea #1
> ---
>
> There are several mysteries here, but maybe we should begin at the
> beginning. Something is wrong if notmuch scans your entire mail tree the
> second time you run notmuch new.
>
> Notmuch checks the mtime of directories against the time stored in the
> database. As a sanitity check, maybe you can do that for one of your
> directories with many messages. This needs "quest" and "xapian-delve",
> from the package xapian-tools.
>
> Unfortunately this should probably be done after the first notmuch
> new. I have another idea to try (below) in the state after several news where
> you are getting OOM.
>
> I'll use real paths for my system; you'll need to update them.
>
> This gives a time in seconds
>
> % stat --format "%Y" ~/Maildir/tethera/cur
> 1612008734
>
> Now let us find the database document for that directory
>
> % quest -bdir:XDIRECTORY -d ~/Maildir/.notmuch/xapian/ dir:tethera/cur
>
> Parsed Query: Query(0 * XDIRECTORYtethera/cur)
> Exactly 1 matches
> MSet:
> 431067: [0]
> tethera/cur
>
> Grabbing the record number from the output of quest:
>
> % xapian-delve -r 431067 -VS0 ~/Maildir/.notmuch/xapian
>
> Value 0 for record #431067: 1.61201e+09
> Term List for record #431067: XDDIRENTRY387045:cur XDIRECTORYtethera/cur
>
> You can see the value matches the mtime up to 6 decimal places.

grfz@mic:~/Mail/.notmuch$ mv xapian xapian-corrupted
grfz@mic:~/Mail/.notmuch$ notmuch new
Welcome to a new version of notmuch! Your database will now be upgraded.
This process is safe to interrupt.
Backing up tags to /home/grfz/Mail/.notmuch/dump-20210130T170349.gz...
Your notmuch database has now been upgraded.
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607947606.8134_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607940473.9509_1.no:2,S
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607969276.21046_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607987211.1395_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607979988.4942_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607972847.4857_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607943993.24776_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607976389.23296_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607983586.19063_1.no:2,
Note: Ignoring non-mail file: /home/grfz/Mail/drafts.mbox
Note: Ignoring non-mail file: /home/grfz/Mail/postponed.mbox
Processed 1183682 total files in 13h 38m 31s (24 files/sec.).
Added 1091038 new messages to the database.

I then installed xapian-tools amd64 1.4.11-1.

grfz@mic:~/Mail/.notmuch$ stat --format "%Y"  ~/Mail/inbox/cur
1611646289

grfz@mic:~/Mail/.notmuch$ quest -bdir:XDIRECTORY -d ~/Mail/.notmuch/xapian/ 
dir:inbox/cur
Parsed Query: Query(0 * XDIRECTORYinbox/cur)
MSet:

That's it, there is data missing in the database.



I have no clue why this is the case.  Just in case here is

grfz@mic:~$ dpkg -l notmuch libnotmuch5 libc6 libglib2.0-0 libgmime-3.0-0 
libtalloc2 zlib1g libxapian30
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version  Architecture Description

out of memory on idle machine (was: Re: consistent database corruption with notmuch new)

2021-01-30 Thread Gregor Zattler
Hi notmuch developers,,
* Gregor Zattler  [14. Dez. 2020]:
> notmuch new still corrupts the database, the second notmuch new
> invocation finds emails the first did not find.

I'm still searching for the reason notmuch chokes on my mails.

I assembled a HP MicroServer, installed basic debian buster and
notmuch from the debian buster repo, rsynced my mail to a
separate file system symlinked to the same location as on my
laptop.

There are now
grfz@mic:~/Mail$ find -type f | wc -l
1209419
files on this file system.  no other process touches this
file system, actually the machine is otherwise ilde.

I did notmuch new several times in a row:

grfz@mic:~/Mail/.notmuch$ rm -rf xapian
grfz@mic:~/Mail/.notmuch$ notmuch new
Welcome to a new version of notmuch! Your database will now be upgraded.
This process is safe to interrupt.
Backing up tags to /home/grfz/Mail/.notmuch/dump-20210127T114210.gz...
Your notmuch database has now been upgraded.
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607947606.8134_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607940473.9509_1.no:2,S
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607969276.21046_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607987211.1395_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607979988.4942_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607972847.4857_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607943993.24776_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607976389.23296_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607983586.19063_1.no:2,
Note: Ignoring non-mail file: /home/grfz/Mail/drafts.mbox
Note: Ignoring non-mail file: /home/grfz/Mail/postponed.mbox
Processed 1183682 total files in 16h 43m 27s (19 files/sec.).
Added 1091038 new messages to the database.
grfz@mic:~/Mail/.notmuch$ notmuch new
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607947606.8134_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607940473.9509_1.no:2,S
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607969276.21046_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607987211.1395_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607979988.4942_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607972847.4857_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607943993.24776_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607976389.23296_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607983586.19063_1.no:2,
Processed 1169095 total files in 16h 52m 48s (19 files/sec.).
Added 1077686 new messages to the database.
grfz@mic:~/Mail/.notmuch$ notmuch new
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607947606.8134_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607940473.9509_1.no:2,S
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607969276.21046_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607987211.1395_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607979988.4942_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607972847.4857_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607943993.24776_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607976389.23296_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607983586.19063_1.no:2,
Processed 1151900 total files in 16h 59m 36s (18 files/sec.).
Added 1050106 new messages to the database.
grfz@mic:~/Mail/.notmuch$ notmuch new
add_file: Out of memory files/sec.).
Processed 205 total files in 6s (29 files/sec.).
Added 193 new messages to the database.
Note: A fatal error was encountered: Out of memory
grfz@mic:~/Mail/.notmuch$


notmuch new processes not all files, later invocations
re-add most of the files and the fourth notmuch new
invocation runs out of memory after a few seconds.

This machine has

grfz@mic:~/Mail$ free
totalusedfree  shared  buff/cache   available
Mem:   16394760  72 229340454641387908415831448
Swap:  15622140   1510415607036

enough memory.  It's ECC Memory.


Any ideas on what is the cause of this?  Even if we ignore the
out-of-memory condition: Why does notmuch re-add a million
already added files?


Ciao; Gregor














































































+ my-fetchmail.services stop
+ /home/grfz/bin/notmuch-new.screen-locked--low-load
+ sed -i -e 's/^#exit 0$/exit 0/' 
/home/grfz/bin/notmuch-new.screen-locked--low-load
+ sleep 77
+ my-fetchmail.services stop
/home/grfz/bin/my-notmuch-renew: line 4: my-fetchmail.services: command not 
found
+ /home/grfz/bin/notmuch-new.screen-locked--low-load

Re: What do people use for calendar invites?

2021-01-22 Thread Gregor Zattler
Hi David, notmuch users and developers,
* David Mazieres  [20. Jan. 2021]:
>   * What's a good way to generate text/calendar attachments from within
> notmuch (especially the emacs interface, which I use)?

there was at least a WIP-Patch by Mark Walters in
order to enable  notmuch -emacs notmuch-show to show and reply
to calendar invites in

id:1468618318-19476-1-git-send-email-markwalters1...@gmail.com

sadly it AFAIK nothing happened with this patch.


Ciao, Gregor
--
 -... --- .-. . -.. ..--.. ...-.-
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: consistent database corruption with notmuch new

2020-12-14 Thread Gregor Zattler
Hi notmuch developers,
* Gregor Zattler  [13. Dez. 2020]:
> * Gregor Zattler  [13. Dez. 2020]:
>> I now do a notmuch new with libxapian30 version 1.4.17-1
>> and will report back in a few hours.
>
> The result is only slightly different from version 1.4.11:

actually now I realized, that notmuch was not linked against
libxapian v1.4.17-1.

Now I build notmuch from master and linked it with
libxapian 1.4.17-1.  The result is almost the same, though:
0 (master *) grfz@no:~/Mail/.notmuch$ nice ionice -c3 notmuch new --full-scan ; 
nice ionice -c3 notmuch new --full-scan ; nice ionice -c3 notmuch new 
--full-scan
Welcome to a new version of notmuch! Your database will now be upgraded.
This process is safe to interrupt.
Backing up tags to /home/grfz/Mail/.notmuch/dump-20201214T124836.gz...
Your notmuch database has now been upgraded.
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607940473.9509_1.no:2,S
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607943993.24776_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607947606.8134_1.no:2,
Note: Ignoring non-mail file: /home/grfz/Mail/drafts.mbox
Note: Ignoring non-mail file: /home/grfz/Mail/postponed.mbox
Processed 1176599 total files in 3h 31m 23s (92 files/sec.).
Added 1102787 new messages to the database.
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607940473.9509_1.no:2,S
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607943993.24776_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607947606.8134_1.no:2,
Note: Ignoring non-mail file: /home/grfz/Mail/drafts.mbox
Note: Ignoring non-mail file: /home/grfz/Mail/postponed.mbox
Processed 125008 total files in 44m 59s (46 files/sec.).
Added 83898 new messages to the database.
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607940473.9509_1.no:2,S
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607943993.24776_1.no:2,
Note: Ignoring non-mail file: 
/home/grfz/Mail/spam-old/cur/1607947606.8134_1.no:2,
add_file: A Xapian exception occurred).
A Xapian exception occurred at lib/message.cc:1182: Too few chunks of 
compressed data
Processed 122480 total files in 42m 1s (48 files/sec.).
Added 83907 new messages to the database.
Note: A fatal error was encountered: A Xapian exception occurred
1 (master *) grfz@no:~/Mail/.notmuch$

notmuch new still corrupts the database, the second notmuch new
invocation finds emails the first did not find.


Ciao, Gregor
--
 -... --- .-. . -.. ..--.. ...-.-
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: consistent database corruption with notmuch new

2020-12-14 Thread Gregor Zattler
Hi David, notmuch developers,
* David Edmondson  [14. Dez. 2020]:
> Which filesystem are you using for the mail store?
>
> I had similar looking problems when using ZFS and never got to the
> bottom of it.

this is a ext4 file system, which gets checked
every time I boot my laptop.

I also run memtest 8.4 (but since it's the gratis
version only for 4 passes) without errors.


Ciao, Gregor
--
 -... --- .-. . -.. ..--.. ...-.-
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: consistent database corruption with notmuch new

2020-12-13 Thread Gregor Zattler
Hi David, notmuch developers,
* Gregor Zattler  [13. Dez. 2020]:
> I now got the bullseye version of xapian-core source
> package, build it on my buster machine (all 4 tests
> passed, expected tests failed).
>
> xapian-check F version 1.4.17-1 did not succed in fixing
> the last corrupted database
>
> I now do a notmuch new with libxapian30 version 1.4.17-1
> and will report back in a few hours.

The result is only slightly different from version 1.4.11:

0 (master *) grfz@no:~/Mail/.notmuch$ nice ionice -c3 notmuch new ; nice ionice 
-c3 notmuch new ; nice ionice -c3 notmuch new   


   Welcome to a new 
version of notmuch! Your database will now be upgraded.
This process is safe to interrupt.
Backing up tags to /home/grfz/Mail/.notmuch/dump-20201213T144839.gz...
Your notmuch database has now been upgraded.
Note: Ignoring non-mail file: /home/grfz/Mail/drafts.mbox
Note: Ignoring non-mail file: /home/grfz/Mail/postponed.mbox
Processed 1175849 total files in 3h 1m 40s (107 files/sec.).
Added 1102076 new messages to the database.
Note: Ignoring non-mail file: /home/grfz/Mail/drafts.mbox
Note: Ignoring non-mail file: /home/grfz/Mail/postponed.mbox
Processed 143329 total files in 1h 21m 23s (29 files/sec.).
Added 83756 new messages to the database. Removed 18389 messages. Detected 96 
file renames.
Processed 124614 total files in 39m 23s (52 files/sec.).
Added 83712 new messages to the database.
0 (master *) grfz@no:~/Mail/.notmuch$ nice ionice -c3 notmuch new ; nice ionice 
-c3 notmuch new ; nice ionice -c3 notmuch new
add_file: A Xapian exception occurred
A Xapian exception occurred at lib/message.cc:1182: No termlist for document 
1114342
Processed 697 total files in 14s (46 files/sec.).
Added 299 new messages to the database.
Note: A fatal error was encountered: A Xapian exception occurred
add_file: A Xapian exception occurred
A Xapian exception occurred at lib/message.cc:1182: No termlist for document 
1114342
Processed 697 total files in 2s (250 files/sec.).
Added 299 new messages to the database.
Note: A fatal error was encountered: A Xapian exception occurred
add_file: A Xapian exception occurred
A Xapian exception occurred at lib/message.cc:1182: No termlist for document 
1114342
Processed 697 total files in 2s (252 files/sec.).
Added 299 new messages to the database.
Note: A fatal error was encountered: A Xapian exception occurred
1 (master *) grfz@no:~/Mail/.notmuch$


This time email werde delivered while notmuch
new was running but there were certainly not
83756 new messages or 18389 messages removed
ones.

Since I now for the frist time in years read
email in mutt emails were moved from new/ to cur/
and renamed at the same time, but there were
neither such many new ones nore were thousands
removed.

Next I will test with notmuch from git master,
but this will have to wait a day or two.


Ciao, Gregor
--
 -... --- .-. . -.. ..--.. ...-.-
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: consistent database corruption with notmuch new

2020-12-13 Thread Gregor Zattler
Hi David, notmuch developers,
* Gregor Zattler  [13. Dez. 2020]:
> * David Bremner  [13. Dez. 2020]:
>> What version of Xapian are you linking with? Are you using the 1.4.11-1
>> in Debian buster?
>
> yes.

I now got the bullseye version of xapian-core source
package, build it on my buster machine (all 4 tests
passed, expected tests failed).

xapian-check F version 1.4.17-1 did not succed in fixing
the last corrupted database

I now do a notmuch new with libxapian30 version 1.4.17-1
and will report back in a few hours.

Ciao, Gregor
--
 -... --- .-. . -.. ..--.. ...-.-
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: consistent database corruption with notmuch new

2020-12-13 Thread Gregor Zattler
Hi David,
* David Bremner  [13. Dez. 2020]:
> Gregor Zattler  writes:
>
>>
>> This is a Thinkpad x240, 8GB RAM on debian buster but with
>> notmuch compiled from source (0.31.2+28~gadfded9).
>>
>> I think the SSD is OK (no hints to SSD failures in log
>> files, smart tests OK, I even filled the free rest of
>> the SSD with randomm data and after that compared it
>> to the source with no difference.
>
> What version of Xapian are you linking with? Are you using the 1.4.11-1
> in Debian buster?


yes.

Ciao, Gregor
--
 -... --- .-. . -.. ..--.. ...-.-
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


consistent database corruption with notmuch new

2020-12-13 Thread Gregor Zattler
Dear notmuch developers, I need help because notmuch
new on my configured notmuch system without
index/database consistently produces a corrupted
database.

On Friday my notmuch database got corrupted.
xapian-check xapian F did not fix the database.

I moved the xapian directory to xapian-corrupted and
did a notmuch new.  This consistently yields a
corrupted database after some hours (no other notmuch
running, no mail delivery, no cron jobs etc):

1 (master *) grfz@no:~/Mail/.notmuch$ rm -rf xapian/
0 (master *) grfz@no:~/Mail/.notmuch$ time nice ionice -c3 notmuch new ; 
time nice ionice -c3 notmuch new ; time nice ionice -c3 notmuch new ; zcat 
notmuch.dump.2020-12-11.1607641607 | notmuch tag --batch ; notmuch tag 
--input=notmuch-doit-batch-flow-via-procmail-since-11 ; notmuch tag 
--input=notmuch-doit-batch-flow-via-procmail ; notmuch tag 
--input=notmuch-doit-batch-flow-via-procmail.with-failure ; time nice ionice 
-c3 notmuch new ; time nice ionice -c3 notmuch new ; time nice ionice -c3 
notmuch new
Welcome to a new version of notmuch! Your database will now be upgraded.
This process is safe to interrupt.
Backing up tags to /home/grfz/Mail/.notmuch/dump-20201212T212701.gz...
Your notmuch database has now been upgraded.
Note: Ignoring non-mail file: /home/grfz/Mail/drafts.mbox
Note: Ignoring non-mail file: /home/grfz/Mail/postponed.mbox
Processed 1175458 total files in 2h 47m 16s (117 files/sec.).
Added 1101746 new messages to the database.

real167m16,589s
user79m23,083s
sys 6m19,064s
Processed 124685 total files in 29m 21s (70 files/sec.).
Added 83646 new messages to the database.

real29m25,353s
user10m24,737s
sys 2m14,977s
add_file: A Xapian exception occurred).
A Xapian exception occurred at lib/message.cc:1182: Expected block 325393 
to be level 0, not 1
Processed 122555 total files in 30m 46s (66 files/sec.).
Added 83636 new messages to the database.
Note: A fatal error was encountered: A Xapian exception occurred

real30m49,774s
user9m44,716s
sys 2m3,859s
adding tag frommeMessage-ID: (null)
Status: A Xapian exception occurred
add_file: A Xapian exception occurred).
A Xapian exception occurred at lib/message.cc:1182: Too few chunks of 
compressed data
Processed 122555 total files in 30m 38s (66 files/sec.).
Added 83636 new messages to the database.
Note: A fatal error was encountered: A Xapian exception occurred

real30m42,427s
user10m22,614s
sys 2m2,737s
add_file: A Xapian exception occurred).
A Xapian exception occurred at lib/message.cc:1182: Too few chunks of 
compressed data
Processed 122555 total files in 31m 39s (64 files/sec.).
Added 83636 new messages to the database.
Note: A fatal error was encountered: A Xapian exception occurred

real31m43,529s
user10m33,959s
sys 2m7,378s
add_file: A Xapian exception occurred).
A Xapian exception occurred at lib/message.cc:1182: Too few chunks of 
compressed data
Processed 122555 total files in 30m 44s (66 files/sec.).
Added 83636 new messages to the database.
Note: A fatal error was encountered: A Xapian exception occurred

real30m48,443s
user10m14,171s
sys 2m8,209s


While the first notmuch new run seems OK, there at least
is the question why the second one adds another 83646
new messages.

This happened several times in a row.

This is a Thinkpad x240, 8GB RAM on debian buster but with
notmuch compiled from source (0.31.2+28~gadfded9).

I think the SSD is OK (no hints to SSD failures in log
files, smart tests OK, I even filled the free rest of
the SSD with randomm data and after that compared it
to the source with no difference.


Any idea what to do in order to get a running notmuch
mail system going?  If it's interesting to nomuch
development, what happens here, I'm happy to so some
digging if someone gives me instructions.  I'm not a
developer and have no clue how to debug this.

Ciao, Gregor
--
 -... --- .-. . -.. ..--.. ...-.-
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: waiting tag

2020-05-04 Thread Gregor Zattler
Hi Keegan,
* Keegan Carruthers-Smith  [2020-05-03; 22:37]:
> Chris Tennant  writes:
>> I would like to have emails that I am waiting on a response for to have a
>> special tag that is removed once I get a reply.  There are other products
>> that do this, but I would much rather do it all from NotMuchMail.

I have the same need.

> You can add a "post-new" hook which does this. Assuming you are use
> "waiting" as the special tag and notmuch's default "new" tag this could work
> for you:
>
>   notmuch tag -waiting -- tag:waiting and 'thread:{tag:new}'

but this removes the waiting tag if there is some response
to some message in the thread, not necessary to the message
tagged waiting, isn't it?

Ciao; Gregor
--
 -... --- .-. . -.. ..--.. ...-.-

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


one specific Mail not found by notmuch

2020-04-05 Thread Gregor Zattler
Dear notmuch developers,

please find attached an redacted copy of an actual email in my
Maildir which notmuch does not find, even not with

path:inbox/**

as search term (as opposed to other mail in inbox/cur/ which
shows up fine).

I can see and read this very mail with mutt and mairix find's it.

Actually this very email is filed in three maildirs (inbox, sent,
outbox) which are all indexed by notmuch and mairix finds all of
them.

I did a test with a modified .notmuch-config, provided a database
path /tmp/Mail, did mkdir -p /tmp/Mail/inbox{cur,new,tmp}, copied
this file into /tmp/Mail/inbox/cur, did a
NOTMUCH_CONFIG=/tmp/.notmuch-config notmuch new, and then notmuch
find this very email just fine.

I'm astonished.  Otherwise notmuch works great.  Any ideas why
this mail does not show up in my inbox or how to debug this?

Thanks for your attention, Gregor

--- Begin Message ---
X X, XX  X X XX XXX Häschentüte XX XX
Tür.   XXX XXX XX X XXX XX täten, könnte
XXX XXX XXX XX.

,
-- 
XX


--- End Message ---
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: [wish] notmuch-emacs: allow operation on marked messages in notmuch-tree-mode

2020-01-04 Thread Gregor Zattler
Hi Gennady, notmuch developers,

this reply from Gennady was directed only to me but not to the
notmuch mailing list:
* Gennady Uraltsev  [2020-01-04; 06:52]:
> Dear notmuch developers,
>
> I recently set up notmuch on my machine and it is great. However it is true 
> that this functionality would be really great. Is there, by chance, another 
> suggested approach. Currently I tag with the tag "selected", filter on 
> "selected" and then apply tags using "*" and then, possibly remove the 
> "selected" tag. However this procedure is a bit cumbersome.
>
> Once I train my elisp-fu I could code commands to do this more easily but 
> maybe supporting it natively is best.
>
> Cheers,
>
> Gennady
>
> On Fri, Jan 03 2020, Gregor Zattler wrote:
>
>> Dear notmuch developers,
>>
>> it would be helpful if it was possible to mark individual
>> messages in notmuch-tree-mode and later apply a command to all
>> marked messages (UI as in dired).
>>
>> I'm used to this for managing emails in mutt.

Ciao; Gregor
--
 -... --- .-. . -.. ..--.. ...-.-

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[wish] notmuch-emacs: allow operation on marked messages in notmuch-tree-mode

2020-01-03 Thread Gregor Zattler
Dear notmuch developers,

it would be helpful if it was possible to mark individual
messages in notmuch-tree-mode and later apply a command to all
marked messages (UI as in dired).

I'm used to this for managing emails in mutt.


Ciao; Gregor
--
 -... --- .-. . -.. ..--.. ...-.-

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: possibly outdated info on notmuch website(?)

2019-12-04 Thread Gregor Zattler
Hi David,
* David Bremner  [2019-12-03; 19:19]:
> Yes, the list should probably be archived or something, it's not active
> these days. A note to that effect would make sense.

perhaps this secondary mailn list should be closed/archived
before the wiki is changed.

>
> See
>
> https://notmuchmail.org/wikiwriteaccess/


How about:

~/src/notmuch-wiki$ git diff
diff --git a/index.mdwn b/index.mdwn
index 0f0477b..d164204 100644
--- a/index.mdwn
+++ b/index.mdwn
@@ -126,12 +126,11 @@ read them as a
 download an [mbox file](https://notmuchmail.org/archives/notmuch.mbox)
 of the entire mailing-list.

-### Another mailing list
+There used to be a temporary mailing list at
+[freelists.org](https://www.freelists.org/list/notmuch)
+while the primary mailing list did not work well.  You might find mails in 
their
+archives which are not on notmuchmail.org.

-Currently the mailing list at notmuchmail.org is not working much, so
-we have a temporary (?) list at
-[freelists.org](https://www.freelists.org/list/notmuch). Feel free to
-subscribe if you like, and to CC both lists with problem reports.

 The `mb2md` utility can be used to convert the archives to maildir
 format which is convenient for reading the archives within notmuch


???

Ciao; Gregor
--
 -... --- .-. . -.. ..--.. ...-.-

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: [BUG] notmuch-emacs: spoils sendmail -f with emacs 27 commit 3a59cc84069376802ba8fd731b524d78db58262c and later

2019-12-04 Thread Gregor Zattler
Hi Stefan, David,
* Stefan Monnier  [2019-12-03; 18:10]:
>>> +(const :tag "Obey `sendmail-envelope-from'"
>> `sendmail-envelope-from' here should be `mail-envelope-from'.
>
> Duh!  Thanks,

works with my configuration if this email hits the notmuch
mailing list.

Thanks for this patch, Gregor
--
 -... --- .-. . -.. ..--.. ...-.-

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


possibly outdated info on notmuch website(?)

2019-12-03 Thread Gregor Zattler
Dear notmuch developers,

https://notmuchmail.org/#index8h2

states among other things:

Another mailing list

Currently the mailing list at notmuchmail.org is not working
much, so we have a temporary (?) list at freelists.org. Feel
free to subscribe if you like, and to CC both lists with
problem reports.

since the most current archive on freelists.org ist for 04-2019 I
assume this is info is dated and should be deleted.


Ciao; Gregor
--
 -... --- .-. . -.. ..--.. ...-.-

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: [BUG] notmuch-emacs: spoils sendmail -f with emacs 27 commit 3a59cc84069376802ba8fd731b524d78db58262c and later

2019-12-03 Thread Gregor Zattler
Hi David,
* David Edmondson  [2019-12-03; 18:43]:
> On Sunday, 2019-12-01 at 18:10:59 +01, Gregor Zattler wrote:
>> I use notmuch-emacs configured with
>> (setq mail-specify-envelope-from t)
>> (setq mail-envelope-from 'header)
>> (setq send-mail-function 'sendmail-send-it)
>>
>> this should/used to invoke sendmail (in my case exim4) with the email
>> address given in the From: header of the message buffer as
>> argument to sendmails -f option.
>
> Wow, this was, err, “interesting”.
>
> Comments below.
>
>> Since emacs 27 commit 3a59cc84069376802ba8fd731b524d78db58262c
>>
>>  3a59cc84069376802ba8fd731b524d78db58262c
>>  Author: Stefan Monnier 
>>  AuthorDate: Tue Jul 30 16:37:01 2019 -0400
>>  Commit: Stefan Monnier 
>>  CommitDate: Tue Jul 30 16:37:01 2019 -0400
>>
>>  Parent: add146f09f * lisp/bindings.el 
>> (mode-line-defining-kbd-macro): New defvar.
>>  Contained:  master
>>  Follows:emacs-26.1 (6691)
>>
>>  * lisp/gnus/message.el: Reduce redundancy with send-mail-function
>>
>>  (message-send-mail-function) : Remove `local-library` tests
>>  for libs distributed with Emacs.
>>  (message-use-send-mail-function): New function.
>>  (message-default-send-mail-function): Default to it, and remove cases
>>  already handled by it.
>>  (message--default-send-mail-function): New function.
>>  (message-send-mail-function) : Use it as new default.
>>  (message-sendmail-f-is-evil): Obey mail-specify-envelope-from if 
>> available.
>>  (message-check, message-with-reply-buffer): Use `declare`.
>>  (message-smtpmail-send-it): smtpmail accepts mail-header-separator,
>>  so simplify and declare obsolete.
>>  (message-send-mail-with-mailclient): Declare obsolete.
>>  (message-check-news-body-syntax): Don't presume that the checksum is
>>  a fixnum.
>>
>>
>> this actually invokes sendmail -f with an address derived from the
>> EMAIL environment variable.
>>
>>
>> It took me some time to understand, that there is no problem with
>> compose-mail/message-send-and-exit which are bound to ^X m and ^C
>> ^C respectively within a barely configured emacs which I used for
>> testing.
>> If instead notmuch is loaded compose-mail/message-send-and-exit
>> and notmuch-mua-new-mail/notmuch-mua-send behave different.
>>
>> With this command (with test.el as attached):
>>
>> ~/src/emacs/src/emacs -Q -nw -l /tmp/test.el
>>
>> compose-mail/message-send-and-exit rightly show
>> -f exam...@example.org in the file /tmp/result (and then there is
>> an error because the notmuch commands are not known).
>>
>> While this command
>>
>> ~/src/emacs/src/emacs -Q -nw -L ~/src/notmuch/emacs/ --eval "(require 
>> 'notmuch)" -f notmuch -l /tmp/test.el
>>
>> wrongly show some other value for -f, depending on your EMAIL
>> environment variable, for both compose-mail/message-send-and-exit
>> and notmuch-mua-new-mail/notmuch-mua-send.

>> (find-file "/tmp/sent")
>> (insert "empty")
>> (kill-region (point-min) (point-max))
>> (write-file "/tmp/sent")
>> (kill-buffer)
>>
>> (find-file "/tmp/result")
>> (kill-region (point-min) (point-max))
>> (write-file "/tmp/result")
>> (kill-buffer)
>>
>> (find-file "/tmp/show-sendmail-args.sh")
>> (kill-region (point-min) (point-max))
>> (insert "#!/bin/sh
>> echo $* >> /tmp/result")
>> (write-file "/tmp/show-sendmail-args.sh")
>> (chmod "/tmp/show-sendmail-args.sh" 448)
>>
>> (setq sendmail-program "/tmp/show-sendmail-args.sh")
>> (setq mail-specify-envelope-from t)
>> (setq mail-envelope-from 'header)
>> (setq send-mail-function 'sendmail-send-it)
>>
>> (compose-mail "Echo Mail Server " "test"
>>'((From: . "exam...@example.org")))
>
> This example, and the one below, are broken - you can't use “From:”
> here, it has to be “From”. Changing that doesn't fix it, though.

OK, thanks.

>> (message-send-and-exit)
>>
>> (notmuch-mua-mail "Echo Mail Server " "test"
>>'((From: . "exam...@example.org")))
>> (notmuch-mua-send-and-exit)
>>
>> (save-buffers-kill-terminal)
>
> As best I can determine, this relates to the order in which things are
> loaded.
>
> If you load mess

[BUG] notmuch-emacs: spoils sendmail -f with emacs 27 commit 3a59cc84069376802ba8fd731b524d78db58262c and later

2019-12-01 Thread Gregor Zattler
[@Stefan: FYI]
Dear notmuch developers,

I use notmuch-emacs configured with
(setq mail-specify-envelope-from t)
(setq mail-envelope-from 'header)
(setq send-mail-function 'sendmail-send-it)

this should/used to invoke sendmail (in my case exim4) with the email
address given in the From: header of the message buffer as
argument to sendmails -f option.

Since emacs 27 commit 3a59cc84069376802ba8fd731b524d78db58262c

 3a59cc84069376802ba8fd731b524d78db58262c
 Author: Stefan Monnier 
 AuthorDate: Tue Jul 30 16:37:01 2019 -0400
 Commit: Stefan Monnier 
 CommitDate: Tue Jul 30 16:37:01 2019 -0400

 Parent: add146f09f * lisp/bindings.el (mode-line-defining-kbd-macro): 
New defvar.
 Contained:  master
 Follows:emacs-26.1 (6691)

 * lisp/gnus/message.el: Reduce redundancy with send-mail-function

 (message-send-mail-function) : Remove `local-library` tests
 for libs distributed with Emacs.
 (message-use-send-mail-function): New function.
 (message-default-send-mail-function): Default to it, and remove cases
 already handled by it.
 (message--default-send-mail-function): New function.
 (message-send-mail-function) : Use it as new default.
 (message-sendmail-f-is-evil): Obey mail-specify-envelope-from if available.
 (message-check, message-with-reply-buffer): Use `declare`.
 (message-smtpmail-send-it): smtpmail accepts mail-header-separator,
 so simplify and declare obsolete.
 (message-send-mail-with-mailclient): Declare obsolete.
 (message-check-news-body-syntax): Don't presume that the checksum is
 a fixnum.


this actually invokes sendmail -f with an address derived from the
EMAIL environment variable.


It took me some time to understand, that there is no problem with
compose-mail/message-send-and-exit which are bound to ^X m and ^C
^C respectively within a barely configured emacs which I used for
testing.
If instead notmuch is loaded compose-mail/message-send-and-exit
and notmuch-mua-new-mail/notmuch-mua-send behave different.

With this command (with test.el as attached):

~/src/emacs/src/emacs -Q -nw -l /tmp/test.el

compose-mail/message-send-and-exit rightly show
-f exam...@example.org in the file /tmp/result (and then there is
an error because the notmuch commands are not known).

While this command

~/src/emacs/src/emacs -Q -nw -L ~/src/notmuch/emacs/ --eval "(require 
'notmuch)" -f notmuch -l /tmp/test.el

wrongly show some other value for -f, depending on your EMAIL
environment variable, for both compose-mail/message-send-and-exit
and notmuch-mua-new-mail/notmuch-mua-send.


I read the code but I have no idea why and how to debug this.

Ciao; Gregor
--
 -... --- .-. . -.. ..--.. ...-.-



test.el
Description: application/emacs-lisp
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


wish: notmuch-emacs: handle RFC822 attachments as email (allow for replying)

2019-11-13 Thread Gregor Zattler
Dear notmuch developers,

notmuch-emacs shows rfc822 attachments as emails which is nice,
but with point in such attachment it is not possible to act on it
as an email as for instance reply to it.

It would be nice if replying and forwarding such
email-as-attachment would be possible.


As a mailing list owner I get emails from mailman if mails are
for some reason to be approved before distribution.  They contain
two rfc822 Attachments: The original email which is to be
approved and a second email to which I am could reply in order to
delete or approve said message.  This is an incredibly useful
interface to mailman since it does not involve a context switch.
You read the mailinglist and act on emails, that's it.  Sadly
with notmoch-show this is not possible (as for instance is with
mutt).

Thanks for your attention, Gregor
--
 -... --- .-. . -.. ..--.. ...-.-

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


notmuch-emacs: WISH: command notmuch-show-message-up with key bindung u

2019-08-25 Thread Gregor Zattler
Dear notmuch developers, when reading a long going discussion
which is deeply threaded, it often would come in handy if it was
possible to jump up to the parent message of the message on is
reading, unfolding this message in notmuch-show if necessary.
This command notmuch-show-message-up or
notmuch-show-parent-message could be bound to "u" since this key
is not used in notmuch-show-mode-map.



Thanks for your attention, Gregor
--
 -... --- .-. . -.. ..--.. ...-.-

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[bug?][notmuch-emacs] X11: renders Headers with face attributes

2019-03-21 Thread Gregor Zattler
Dear notmuch developers,

today I got a spam mail which's Subject: and From: header were
displayed as bold (with the exception of German umlauts) in
notmuch-emacs in a graphical frame (X11), they were displayed as
a string of hollow rectangles in a frame on a urxvt terminal (see
attached screen shots).  I see there are bold and italic
mathematical bold letters in Unicode.  The bold ones are used
here.

This are the headers in question:

Subject:=?UTF-8?B?IPCdl5jwnZe78J2YgfCdl7nwnZeu8J2YgPCdmIHwnZiC8J2Xu/Cdl7Qg8J2Xs8O88J2XvyDwnZeU8J2Xu/Cdl7TwnZey8J2XtcO28J2Xv/Cdl7bwnZe08J2XsiDwnZe28J2XuiDwnZen8J2Xv/Cdl67wnZiC8J2XsvCdl7/wnZez8J2XrvCdl7nwnZe5IA==?=
From:=?UTF-8?B?IPCdl6bwnZiB8J2XsvCdl7/wnZev8J2Xsi3wnZep8J2XvPCdl7/wnZiA8J2XvPCdl7/wnZe08J2XsiA=?=<2m...@0ytix511n5cb63oezbgj69wpm.com>

Is this a bug?  Should untrusted senders be allowed to influence
how headers are displayed?  I don't think so.  For me this case
should be handled as a homograph attack
(https://en.wikipedia.org/wiki/IDN_homograph_attack).

I do know there is character folding for searches but emacs would
need to use the reverse for displaying characters and only
characters which deviate in a way that is recognized as a font
attribute.

Thanks for your attention, Gregor
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: how to search for hyphenated words? (was: how to search for Morse code?)

2019-03-12 Thread Gregor Zattler
Hi David,
* David Bremner  [2019-03-12; 07:41]:
> Gregor Zattler  writes:
>
>
>> From: root@len.workgroup (Cron Daemon)
>> Subject: Cron  ~/bin/mailwiederdurchschleusen
>> To: root@localhost
>> Date: Fri, 29 Dec 2017 17:00:09 +0100
>>
>> Date: Thu, 28 Dec 2017 21:04:52 -0500
>> From: Maxim Cournoyer 
>> To: help-gnu-em...@gnu.org
>> Subject: Re: Gnus and emails sent by me
>> --
>> Date: Thu, 28 Dec 2017 22:00:56 -0400
>> From: David Bremner 
>> To: David Edmondson , notmuch@notmuchmail.org
>> Subject: Re: Xapian exception leading to database corruption
>> --
>
> The line
>
> To: David Edmondson , notmuch@notmuchmail.org
>
> contains the phrase "org notmuch". You can see this easier by stripping
> all the punctuation.


Thanks, now I see (the light :-)

Ciao; Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: how to search for hyphenated words? (was: how to search for Morse code?)

2019-03-12 Thread Gregor Zattler
Hi David, Matt, Carl, notmuch developers,
* David Bremner  [2019-03-11; 22:13]:
> Matt Armstrong  writes:
>> Carl Worth  writes:
>>> The trick here is that when notmuch is indexing body text it feeds it
>>> into a Xapian function that parses the text by finding "terms" in the
>>> text. And this parser considers both punctuation and whitespace as
>>> separators between terms.
>>
>> I notice that Xapian supports something called "phrase searches",
>> documented as:
>>
>>   "A phrase surrounded with double quotes ("") matches documents
>>   containing that exact phrase. Hyphenated words are also treated as
>>   phrases, as are cases such as filenames and email addresses
>>   (e.g. /etc/passwd or presid...@whitehouse.gov)."
>>
>> I assume that this particular Xapian feature is unavailable in notmuch?
>> If so, I wonder if enabling has ever been considered?
>
> It is enabled, and documented in notmuch-search-terms(7). Unfortunately
> I don't think it's related to the original request. The mention of
> hyphenated words is about the input to the query parser, not the
> (necessarily) the retrieved text.

what I do not understand is that it dosn't matter if I search for

org-notmuch

or

"org-notmuch"

'"org-notmuch"'

or even

org ADJ/1 notmuch

$ notmuch count --output=messages '"org-notmuch"'
581
$ notmuch count --output=messages 'org-notmuch'
581
$ notmuch count --output=messages org-notmuch
581
$ notmuch count --output=messages org ADJ/1 notmuch
581

a typical example of a matched message is the attached one.
Somehow the search matches the address of this very mailing list
in the body of the email (I assume).


But obviously there are much more emails with this address in
them:

$ notmuch count --output=messages 'notmuch@notmuchmail.org'
27396
$ notmuch count --output=messages '"notmuch@notmuchmail.org"'
27396

Or with a naive search (no decoding of possible base64 encoded
parts) there are

$ find /home/grfz/Mail/~ml/emacs-orgm...@gnu.org 
/home/grfz/Mail/~ml/notmuch@notmuchmail.org* -type f -print0 | xargs -0r grep 
-l -- 'notmuch@notmuchmail.org' | xargs -I sh -c "cat  | sed -e '1,/^$/ 
d' | grep -c notmuch@notmuchmail.org " | egrep -c "1|2|3|4|5|6|7|8|9"
16795

emails with the address at least once in the body.


Therefore I wonder why notmuch matches 581 messages.



A naive search for org-notmuch on the files (no decoding of
possible base64 encoded parts) only shows 79 files (77 unique
emails):

mkdir -vp /tmp/test/{cur,new,tmp}

$ find /home/grfz/Mail/~ml/emacs-orgm...@gnu.org 
/home/grfz/Mail/~ml/notmuch@notmuchmail.org* -type f -print0 | xargs -0r grep 
-l -- 'org-notmuch' | xargs ln -vs --target-directory=/tmp/kolp/cur/ | wc -l
79


Therefore I wonder why notmuch matches 581 messages, not 16795
messages or 77 messages.


Somehow these numbers do not fit!?


Ciao; Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-
--- Begin Message ---
Date: Thu, 28 Dec 2017 21:04:52 -0500
From: Maxim Cournoyer 
To: help-gnu-em...@gnu.org
Subject: Re: Gnus and emails sent by me
--
Date: Thu, 28 Dec 2017 22:00:56 -0400
From: David Bremner 
To: David Edmondson , notmuch@notmuchmail.org
Subject: Re: Xapian exception leading to database corruption
--

--- End Message ---
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: how to search for hyphenated words? (was: how to search for Morse code?)

2019-03-11 Thread Gregor Zattler
Hi David, notmuch developers,
* David Bremner  [2019-03-10; 20:22]:
> Gregor Zattler  writes:
>> How would one search for hyphenated words with notmuch?
>>
>
> In special cases, explained in notmuch-search-terms(7), one can use
> regexp searches, which are slower, but don't drop punctuation.

thanks, this works for the subject: field, which helps a lot.

Regexes do not work on the body of messages and I assume they
will not work with the upcoming "body:" field?


Thanks for your attention, Gregor


___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


how to search for hyphenated words? (was: how to search for Morse code?)

2019-03-08 Thread Gregor Zattler
Hello,
* Gregor Zattler  [2018-07-23; 14:20]:
> today I searched for emails containing
>
> -... --- .-. . -.. ..--.. ...-.-

today I searched for emails containing "org-notmuch" (which
supports org links to notmuch searches), e.g. with

notmuch search org-notmuch
notmuch search -- org-notmuch
notmuch search -- "org-notmuch"
notmuch search -- '"org-notmuch"'
notmuch search -- '+"org-notmuch"'
notmuch search -- org ADJ/1 notmuch

all these resulted in very many hits most or all of which do not
contain the string "org-notmuch", one found email was e.g.

id:20180904105723.15564-3-da...@tethera.net


How would one search for hyphenated words with notmuch?

Ciao; Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Reply to content of "List-Post" header?

2019-03-02 Thread Gregor Zattler
Hi Ralph,
* Ralph Seichter  [2019-03-02; 17:34]:
> * Brian Sniffen:
>
>> You can set message-subscribed-addresses or its cousins; then
>> message-to-list-only will work for you.
>
> Ah, that's what was missing, thank you.

indeed, I forgot: my procmmail scripts recognize mailing list
headers and in the end there is a list of all mailing lists I
ever got mails from and this list is referenced in
message-subscribed-address-file.  So I pretend beeing subscribed to
all these lists which is not true but helps.

Ciao; Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Reply to content of "List-Post" header?

2019-03-02 Thread Gregor Zattler
Hi Ralph,
* Ralph Seichter  [2019-03-01; 23:26]:
> Either my search-fu is lacking today, or there is really not much
> information to be found about this:
>
> Using Notmuch with Emacs, the keys "r" and "R" are bound to "reply to
> sender" and "reply to all", respectively. Neither is what I require when
> it comes to mailing lists. Is there any way to "reply to list", i.e. to
> only the address specified in the "List-Post" header?

I do "R" for reply-to-all and then C-c C-l for
message-to-list-only which reduces the To: and Cc: to the mailing
list address.  I don't know how this works, though.  I assume it
evaluates List- headers.


Ciao; Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: [WIP PATCH] emacs: show: reply to calendar parts

2019-01-14 Thread Gregor Zattler
Hi Mark,
* Mark Walters  [2016-07-15; 22:31]:
> This allows the user to reply to calendar parts. To reply go to the
> text/calendar part and use ". c", then select accept, decline, or
> tentative. Notmuch gives a reply buffer ready to send as the response.
> For the moment this must be sent manually (C-c C-c as normal).
> ---
>
> This is a first pass at allowing the user to reply to calendar
> invites. Emacs (at least emacs 24) has gnus-icalendar which contains
> all the things necessary for gnus to reply to calendar parts. This
> wraps some of the functions for gnus-icalendar so they work from notmuch.
>
> This version does not send the reply automatically, so it is possible
> to review the reply before sending.
>
> It appears to work with google calendar and outlook invites, but is
> not heavily tested. Please do not use for "important" things!
>
> If people find it works then I will think about how to polish
> it. Feedback of any sort very gratefully received!

I sent me an calendar invitation via the Open Exchange web
interface, was able to reply within notmuch, Open Exchange
recognised the reply as containing a date.  Open Exchange showed
a button to accept changes to the calendar, when I click on this
button a notification appears, stating that the appointment was
actualised.

So I think your patch mostly works.

But sadly in Open Exchenge the calendar the status of the
appointment stays the same: There is no hint, that the invitation
was replied to, neither "accept" nor "decline" appear there: The
person who created the appointment is shown with a checked
checkbox in color green, such a sign does not show up for the
person who was invited.
I cannot say who is to blame for this, since I don't know the
first thing about this.

I could provide the accept/decline mails if needed.



Thanks for your patch, I wished for such functionality in
notmuch.


Ciao; Gregor 


___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Mail to self

2018-10-28 Thread Gregor Zattler
Hi Leo, notmuch users and developers,

I too filter emails into different folders.  And I used to filter
some emails in several folders in order to find them in different
contexts. 

* Leo Gaspard  [2018-10-28; 16:31]:
> I can confirm this is my issue. My thinking is that if any file does not
> have the ,S maildir flag, then I would prefer the mail to be marked as
> unread, as a read mail that is spuriously marked as unread is a small
> inconvenience, while an unread mail that is spuriously marked as read
> can have huge consequences.

For me, when I filtered emails in several folders and read them
with mutt (1) it was the other way around: I wished for the
possibility to mark them as read in all folders if read in one.

I think your reasoning is also sound but only iff tags would
belong to files, but they belong to messages.

> Actually, I just noticed while writing this mail that I had setup a
> sieve filter for duplicate email that put them in a "Duplicate" folder
> and automatically marks them as "read", so that at the same time they
> wouldn't bother me in the thunderbird interface but I could still check
> dovecot's duplicate detection didn't have false positives.
>
> This behaviour of notmuch thus made me miss 9 emails in the 2 weeks I've
> been using it, the oldest being 12 days ago, without any warning. Well,
> now I know about it and can try to change my setup (except I can't
> really touch this sieve filter as other people rely on it), but...

But why would you tag a message (as opposed to a file) as
"unread" because it's a duplicate?

If it's somehow important to you to know that a certain email
your read via notmuch is a duplicate, why not tag it as
"duplicate" instead?

I suppose you do this filtering in case you read those emails
with a file/folders based client.  But would you actually "read"
emails in a duplicate folder?  I too have a duplicate folder and
it was useful some times in order to investigate problems with my
processing of incoming emails, but I do not read them in order to
understand the meaning of some message a human sent me.

Since I also filter emails in folders my searches heavily relay
on the path the message is in with the "path:" search prefix.
These is true for all mailing lists, for instance.

For the specific case of the duplicates folder: It's simply not
in the same maildir notmuch indexes and therefore there is no
issue with them for me.  My procmail scripts first do a backup of
an incoming email, later they filter duplicates in a duplicates
folder and when an email actually is delivered in the maildir
hierarchy notmuch indexes, then it is also backed up in a third
backup folder for post-processed emails.  These three special
folders are directories under my ~/.procmail directory and are
not indexed by notmuch.


HTH, Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: tell me how to do this (mass tagging of messages)

2018-10-17 Thread Gregor Zattler
Hi Jeff,
* Jeff Templon  [2018-10-17; 10:06]:
> and what comes up is something like this ...
>
>   2013-07-27 [1/1]   ASN Bank SURF***SPAM***6.8 ASN Bank: 
> Belangrijke update mededeling (lists lists/grid spam)
>   2013-07-27 [1/1]   Overdracht van geld  SURF***SPAM***5.7 Uw account is 
> geactiveerd. (spam)
>   2013-07-26 [1/1]   Full NameSURF***SPAM***6.2 Need (spam)
>   2013-07-16 [1/14]  Carlos Zorraquino... SURF***SPAM***6.2 Re: CHEP2013 
> Proceedings: invitation to submit a full paper for your contribution (spam)
>   2013-07-15 [1/1]   Rabo BankSURF***SPAM***5.8 2013 Rabo Bank 
> Algemene voorwaarden en informatie (spam)
>
> I take a look and decide I want to delete a bunch of messages, but take
> a look at the fourth in the list - this is one where the SURF spam
> filter has decided that one of the messages in the thread is spam, but
> the entire thread has shown up here.  If I take bulk action on such a
> buffer, I've seen before that most commands work on the threads and not
> the messages themselves, like the 'k' command.
>
> So how do I make sure that I'm operating on the messages themselves
> (only one of the 14 messages is marked 'spam') and not the threads?

AFAIK you navigate to this fourth line, hit ENTER, navigate to
the message tagged spam (probably it's the one unfolded) and use
"k" and presumably "D" (or however you configured
notmuch-tagging-keys) to mark this one as to be deleted.

> Thanks.  BTW I like notmuch pretty much.

me too

> ps while I'm at it - inline display of attachments or in any case firing
> up an external viewer?  I only know about hitting return on the
> attachment link and having it saved to a file.

With point on the attachment, hit ".v" in order to see the
attachment in your preferred application.  Hit ".o" instead in
order to choose the right application from a list.

Ciao; Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: bug: wrong order of messages in notmuch-show

2018-09-08 Thread Gregor Zattler
Hi David, notmuch developers,
* David Bremner  [2018-09-08; 07:35]:
> Gregor Zattler  writes:
>> notmuch sometimes shows emails of one thread in wrong order.  Now
>> I found an example in a public mailing list so it is possible to
>> share this example.  It is a small thread from the notmuch
>> mailing list, you might find it by searching for 
>> id:87zhzccrwg@tethera.net
>>
>> While this email is the last written and sent in this thread it
>> is shown as first email in the notmuch show output, as shown in
>> this "screenshot" of the corresponding notmuch-emacs buffer:
>
> As far as I know the problems discussed in this message should be fixed
> in commit 87934c432c4bee9df09f268a3f05933c59c2caf1, so I'm marking it
> fixed.

Yes, for me these problems are fixed now, as the ones described in
id:87fu05p2iu.fsf@len.workgroup

Thanks, it's much easier now for me to follow long email threads.


Ciao; Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH] WIP: sort top level messages in thread

2018-08-27 Thread Gregor Zattler
Hi David,
* David Bremner  [2018-08-26; 22:53]:
> this needs a test, and  memory de-allocation of the replaced
> lists. Currently it creates a fair amount of garbage.
> ---

> Sorry for the lack of reply, some vacation intervened.

Thanks, hope it was relaxing.

> Can you test
> this patch? I think it fixes the second thread also.

The patch applied cleanly on top of the others (which I use
regularly with no problems) and fixed the threading order problem
in the two test cases I send you.  Other RT-emails are also
ordered nicely now.

Thanks for this patch, I'll also use it regularly.

Gregor

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Really nice way to search for email

2018-08-13 Thread Gregor Zattler
Hi Domingo,
* Domingo Gómez  [2018-08-11; 11:14]:
>- How can I sort the messages in a thread from the newest to
>oldest?

I would not know of a way to achieve this.

>- Some emails are answers to emails that I sent and contain things like: On
>09/08/18 13:13,  wrote: Is it possible to not show it by
>default? and is it possible to set a key to show it again?

How about adding 'AND NOT ""' (without the single
quotes)?  Notmuch-emacs shows not-matching messages folded 
or at least you can configure notmuch-emacs to do so.  Show the
folded message with RETURN or all of them with ALT+RETURN.

>- Also in a thread, using notmuch-show, is there any key to move to the
>next message of the thread?

That would be "n"?

Ciao; Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Threading patches v2

2018-08-01 Thread Gregor Zattler
Hi David,
* David Bremner  [2018-07-31; 06:45]:
> This series obsoletes the patch (series) 
>
>  id:20180727083720.19909-1-da...@tethera.net
>  id:20180723082259.32440-2-da...@tethera.net
>  id:20180720233746.2844-1-da...@tethera.net
>
> It also includes a proposed fix for the problem reported by Gregor in
>
>id:87y3ducffj.fsf@len.workgroup

I applied the  patch set and it solves the problem for the first
thread shown in id:87fu05p2iu.fsf@len.workgroup
For me this fixes the more important issue of the two.


It does not fix the ordering in the second example:

 [support.xxx-x-x.de #33575] AutoReply: FYI: 
  xxx   xxx
   1  via RT  
(Tue, 26 Jun 2018 14:44:24 +0200) (inbox)
   -->   Subject: [support.xxx-x-x.de #33575] Resolved: 
FYI:   xxx  xxxxxxxx xxx
   2 Gregor Zattler  (Tue, 19 Jun 2018 
18:26:26 +0200) (inbox)
 Subject: FYI:   xxx   xxx
   2a   via RT  
(Tue, 19 Jun 2018 18:26:36 +0200) (inbox)
   Subject: [support.xxx-x-x.de #33575] 
AutoReply: FYI:   xxx   xxx


I have to admit, this is not one thread if one accounts for
threads only with respect to References -headers. 

But even if it's two threads we have two level 1 emails (1, 2) and
I would expect the level 1 emails to be sorted according to
date, the whole tree of sibling emails to every level 1 email
behind and indented relative to it's level 1 email:

2
  2a
1

This is, what mutt does.



Thanks for your efforts, it's much easier now to follow email
discussions.

Ciao; Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: bug: wrong order of messages in notmuch-show

2018-07-27 Thread Gregor Zattler
Hi David, dear notmuch developers,
* Gregor Zattler  [2018-07-01; 23:27]:
> This happens often to me with emails from a ticket system which I
> cannot disclose.  In this kind of technical threads order is
> necessary to understand current status of affairs.

I finally isolated two such threads, anonymized them and indexed
them.  tar'ed and compressed the maildir is small enough to attach,
therefore I attached it.


With the first thread (Ticket 33712) notmuch shows the "-->"
marked email last in thread, instead (as according to date sent)
before position marked "+++":

  [support.xxx-x-x.de #33712] 
xxx-x-server xxxx
  Gregor Zattler  (Mon, 09 Jul 2018 
10:03:54 +0200) (inbox)
  Subject: xxx-x-server 
via RT  (Mon, 
09 Jul 2018 10:04:03 +0200) (inbox replied)
Subject: [support.xxx-x-x.de #33712] AutoReply: 
xxx-x-server xxxx
  Gregor Zattler  (Mon, 09 Jul 2018 
10:24:26 +0200) (inbox replied)
Gregor Zattler  (Tue, 10 Jul 2018 
12:18:39 +0200) (inbox)
Subject: Re: [support.xxx-x-x.de #33712] 
xxx-x-server xxxx
+++ Gregor Zattler  (Tue, 24 Jul 2018 
14:03:22 +0200) (inbox)
Subject: Re: [support.xxx-x-x.de #33712] 
AutoReply: xxx-x-server 
--> via RT  
(Mon, 09 Jul 2018 10:37:43 +0200) (inbox)
Subject: [support.xxx-x-x.de #33712] 
xxx-x-server 
  
  

With the second thread the email marked "-->" should be at the
end, IMHO.  I don'nt know how this Resolved-Message is produced,
perhaps it is not a reply to the last message in thread:

  
  [support.xxx-x-x.de #33575] AutoReply: FYI:   
xxx   xxx
   via RT  (Tue, 
26 Jun 2018 14:44:24 +0200) (inbox)
-->   Subject: [support.xxx-x-x.de #33575] Resolved: 
FYI: xxxx  xxx   xxx
  Gregor Zattler  (Tue, 19 Jun 2018 
18:26:26 +0200) (inbox)
  Subject: FYI:   xxx   xxx
 via RT  (Tue, 
19 Jun 2018 18:26:36 +0200) (inbox)
Subject: [support.xxx-x-x.de #33575] AutoReply: 
FYI:   xxx   xxx





Thanks again for your attention.


Ciao; Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-


test-email-order-in-threads.tar.xz
Description: application/xz
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


how to search for Morse code?

2018-07-23 Thread Gregor Zattler
Hello,

today I searched for emails containing

-... --- .-. . -.. ..--.. ...-.-

tried with

notmuch search "-... --- .-. . -.. ..--.. ...-.-"

and

notmuch search '-... --- .-. . -.. ..--.. ...-.-'

and even

notmuch search '"-... --- .-. . -.. ..--.. ...-.-"'

and also with double dashes in front of the search term:

notmuch search -- "-... --- .-. . -.. ..--.. ...-.-"


All these searches produce

notmuch search: A Xapian exception occurred
A Xapian exception occurred parsing query: Unknown range operation
Query string was: "-... --- .-. . -.. ..--.. ...-.-"


Is it possible to search for emails containing my supposedly
funny signature?


Obviously this is not much of a problem for me, but perhaps I hit
a hidden bug?


Ciao; Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: v1: fix for threading of replies to ghost messages

2018-07-23 Thread Gregor Zattler
Hi David,
* David Bremner  [2018-07-23; 16:41]:
> Gregor Zattler  writes:
>> I applied the patches and looked with notmuch-emacs for threads
>> which before would probably show wrong ordering of messages.  The
>> half a dozen threads were all shown in the correct order with
>> respect to parents and children, but I found from those not
>> disclosable RT ticket mails (I mentioned in above referenced
>> thread):
>
> Can you also try the series at
>
> id:20180723082259.32440-1-da...@tethera.net?
>
> I'm not sure if it's related, but it does fix replies for a different
> bug tracker. You will have to reindex the messages in question
> (e.g. using notmuch reindex).

I applied this patch set on top of the other one, copied
the relevant threads to a test-maildir, changed path in
.notmuch-config, thus building a new index: Same order of
messages in notmuch-emacs-show as with the first patch set only.

I asked for permission to disclose two small threads, have to
wait for answer, will report back


Thanks again; Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: v1: fix for threading of replies to ghost messages

2018-07-21 Thread Gregor Zattler
Hi David,
* David Bremner  [2018-07-21; 08:37]:
> See the thread at id:87efgmmysi.fsf@len.workgroup for discussion.
>
> This series finally bites the bullet and uses the References header
> for threading.
>
> Please test this series, there are lots of tricky cases with threading.

I applied the patches and looked with notmuch-emacs for threads
which before would probably show wrong ordering of messages.  The
half a dozen threads were all shown in the correct order with
respect to parents and children, but I found from those not
disclosable RT ticket mails (I mentioned in above referenced
thread):

- Threads where the "Resolved" message was shown first but it's
  References: header contains only a fake Message-Id: (which is
  strange: why does notmuch show this resolved-mail in this
  thread?).  


- one thread where two children of one message were shown with
  correct (meaning: same) indentation but the older one before
  (over) the newer one.


I'll ask if I may disclose this threads if this it would be
helpful for diagnosis.


Thanks for your attention and work regarding my problem report
with notmuch.

Ciao; Gregor 

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Bug: fatal error with notmuch new, second run starts indexing all over again

2018-07-08 Thread Gregor Zattler
Hi David, Leonard,
* David Bremner  [2018-07-07; 23:07]:
> Gregor Zattler  writes:
>> * Leonard Lausen  [2018-07-01; 15:36]:
>>> Also it seems that the midsize corpus
>>> https://giku.de/reduced-sample-midsized.xapian-exeption.tar.xz is not
>>> available anymore.
>>
>> I still have this archive on my local hard drive.  If you (or
>> anyone else) are interested in this, I will upload it again.
>>
> Please do. I seem to have deleted my copy.

Done.  It's at
https://giku.de/reduced-sample-midsized.xapian-exeption.tar.xz
again.

Thanks for looking into this, Gregor


___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Bug: fatal error with notmuch new, second run starts indexing all over again

2018-07-03 Thread Gregor Zattler
Hi Leonard, nomuch developers,
* Leonard Lausen  [2018-07-01; 15:36]:
> I also experience this error trying to import the emacs-orgmode mailing
> list archive. Has there been any progress on fixing the issue? What
> steps should be taken to help?
>
> Also it seems that the midsize corpus
> https://giku.de/reduced-sample-midsized.xapian-exeption.tar.xz is not
> available anymore.

I still have this archive on my local hard drive.  If you (or
anyone else) are interested in this, I will upload it again.

Ciao; Gregor 

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: bug: wrong order of messages in notmuch-show

2018-07-02 Thread Gregor Zattler
Hi David,
* David Bremner  [2018-07-01; 21:23]:
> Gregor Zattler  writes:
> The thread is not ordered that way here. Can you run the "draw-thread"
> tool in id:20180410014539.24717-1-da...@tethera.net and send the output
> (either dot or PDF) as an attachement to the list?

sure, I produced two sets of files:

the ones with "for-real" in their names are done with my working
notmuch installation, the ones with "test-order-notmuch" in their
names are done with ta maildir which only contains the relevant
messages.

> Yes, I'll probably need the tar file to reproduce it. If it's small
> (<50k), send it to the list, otherwise let me know how I can pick it up.


This tar file also contains a tar file with the xapian database for these 8
messages etc.:
https://giku.de/4iczfdzgnedth8ni--notmuch-show-order-problem.tar.7z

Thanks for looking into this.  Ciao; Gregor 

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


bug: wrong order of messages in notmuch-show

2018-07-01 Thread Gregor Zattler
Dear notmuch developers,

notmuch sometimes shows emails of one thread in wrong order.  Now
I found an example in a public mailing list so it is possible to
share this example.  It is a small thread from the notmuch
mailing list, you might find it by searching for id:87zhzccrwg@tethera.net

While this email is the last written and sent in this thread it
is shown as first email in the notmuch show output, as shown in
this "screenshot" of the corresponding notmuch-emacs buffer:

  File Edit Options Buffers Tools Help
  'notmuch search thread:<>' lists multiple threads
-->   David Bremner  (Yest. 15:42) (inbox new unread)
  Subject: Re: 'notmuch search thread:<>' lists multiple threads
  Naveen N. Rao  (April 06) (inbox new 
unread)
   Naveen N. Rao  (April 06) (inbox new 
unread)
   David Bremner  (April 08) (inbox new unread)
David Bremner  (April 09) (inbox new unread)
 David Bremner  (April 10) (inbox new unread)
 Subject: [PATCH] devel: add new tool to draw thread structure
 Naveen N. Rao  (April 18) (inbox new 
unread)
 Subject: Re: 'notmuch search thread:<>' lists multiple threads
  David Bremner  (April 22) (inbox new unread)


This happens often to me with emails from a ticket system which I
cannot disclose.  In this kind of technical threads order is
necessary to understand current status of affairs.

I did a test with notmuch 0.27+7~gfd3c936 and .notmuch-config
pointing to a test maildir with this 8 messages as they arrived
at my email account.  I saved a tar archive of this in case this
is important to debug the problem.

Thanks, for your attention, Gregor

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


wish: notmuch-emacs: "Good signature by:" shows only email address, should show fingerprint also

2018-06-22 Thread Gregor Zattler
Dear notmuch developers,

it would be great if notmuch-emacs (optionally?) showed the fingerprint of the 
key an email was signed with, in order to aide the user in selecting the right 
key to encrypt a reply to.

My correspondents email showed successful signature verification but when I 
want to reply I do not know which of several keys with the respective user id I 
should select for encryption.

It was also helpful if the key selection interface showed more info regarding 
the keys, especially date of creation and expiration , but I assume this is 
outside your area of development!?

Ciao; Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: notmuch shows tag which is not on any email

2018-06-04 Thread Gregor Zattler
Hi Jani, Carl,
* Jani Nikula  [2018-06-04; 21:36]:
> On Mon, 04 Jun 2018, Gregor Zattler  wrote:
>> $ notmuch search --output=tags '*' | grep telegraph
>> EA%3Dtelegraph%40gmx%2Enet
>> EA=telegr...@gmx.net
>>
>>
>> $ notmuch count  -- is:EA%3Dtelegraph%40gmx%2Enet
>> 0
[...]
> Try all of the above with --exclude=false parameter and see if it makes
> a difference. If it does, each of the messages tagged with
> EA%3Dtelegraph%40gmx%2Enet is probably also tagged with one of the tags
> in 'notmuch config get search.exclude_tags'.

You hit it spot on.

[...]
>> $ notmuch tag +EA=telegr...@gmx.net  -- is:/EA.*telegraph/
>
> Try that with
>
> $ notmuch tag +EA=telegr...@gmx.net -EA%3Dtelegraph%40gmx%2Enet -- 
> is:/EA.*telegraph/
>
> to remedy the situation.


Thanks for your help, now the spurious tag is gone.

Ciao; Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


notmuch shows tag which is not on any email

2018-06-04 Thread Gregor Zattler
Dear notmuch developers,

does notmuch remember tags even if at time of query there is no
message tagged with the respective tag?:

$ notmuch search --output=tags '*' | grep telegraph
EA%3Dtelegraph%40gmx%2Enet
EA=telegr...@gmx.net


$ notmuch count  -- is:EA%3Dtelegraph%40gmx%2Enet
0


$ notmuch count  -- is:/EA.*telegraph/
4498


There is no invisible character or some such:
$ notmuch search --output=tags '*' | grep telegraph | head -n 1 | wc -c
27


I might have done something wrong while experimenting with this
tags and  'EA%3Dtelegraph%40gmx%2Enet' may be a leftover.  I
would like to remove it but since no message matches it's not
possible to remove it from messages and therefore the tag remains:

$ notmuch tag +EA=telegr...@gmx.net  -- is:/EA.*telegraph/
$ notmuch search --output=tags '*' | grep telegraph
EA%3Dtelegraph%40gmx%2Enet
EA=telegr...@gmx.net


I tried to utilise emacs to remove the tag, but notmuch-emacs
does not show 'EA%3Dtelegraph%40gmx%2Enet' in notmuch-hello's
'All tags' section.



Why is said tag in the tags listing, how to get rid of it?


Thanks, Gregor

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Database corruption after clean rebuild

2018-04-29 Thread Gregor Zattler
Hi David,
* David Bremner <da...@tethera.net> [2018-04-29; 11:27]:
> Gregor Zattler <telegr...@gmx.net> writes:
>> I also had this database corruption, I waited for the fix to land
>> in notmuch 0.26.2, build it, moved the xapian directory away, did
>> a notmuch new and restored the tags from a dump.  But the problem
>> remains:
>>
>> ~$ xapian-check ~/Mail/.notmuch/xapian
>> docdata:
>> blocksize=8K items=10841 firstunused=75 revision=82 levels=1 root=2
>> B-tree checked okay
>> docdata table structure checked OK
>>
>> termlist:
>> blocksize=8K items=1893162 firstunused=368983 revision=82 levels=3 
>> root=177608
>> xapian-check: DatabaseError: 1 unused block(s) missing from the free list, 
>> first is 0
>
> This is a known bug in Xapian, fixed in xapian master. The message will
> go away if you run notmuch-compact, or you can just ignore it.

OK, thanks.  Gregor

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Database corruption after clean rebuild

2018-04-29 Thread Gregor Zattler
Hi notmuch developers,

I also had this database corruption, I waited for the fix to land
in notmuch 0.26.2, build it, moved the xapian directory away, did
a notmuch new and restored the tags from a dump.  But the problem
remains:

~$ xapian-check ~/Mail/.notmuch/xapian
docdata:
blocksize=8K items=10841 firstunused=75 revision=82 levels=1 root=2
B-tree checked okay
docdata table structure checked OK

termlist:
blocksize=8K items=1893162 firstunused=368983 revision=82 levels=3 root=177608
xapian-check: DatabaseError: 1 unused block(s) missing from the free list, 
first is 0


this is very similar to the old database which I had moved away:
~$ xapian-check ~/Mail/.notmuch/xapian-2018-04-29-00-22/
docdata:
blocksize=8K items=10863 firstunused=78 revision=59623 levels=1 root=2
B-tree checked okay
docdata table structure checked OK

termlist:
blocksize=8K items=1894648 firstunused=360821 revision=59623 levels=3 
root=360580
xapian-check: DatabaseError: 1 unused block(s) missing from the free list, 
first is 0


Now I did notmuch compact and the database check says there are
no errors.


This seems to me as if the fix had not helped or there is another problem.


$ notmuch --version
notmuch 0.26.2+26~g9e158fb
~$ xapian-compact --version
xapian-compact - xapian-core 1.4.3

Thanks for developing notmuch, Gregor


* Javier Garcia  [2018-04-07; 17:09]:
> Unfortunately I can't share my emails without the approval of other
> parties. The minimum subsets that trigger the error are in the range of
> 1000-5000 mails, so asking each and everyone of them is out of my reach.
> I tried to replicate the problem using just spam folders without success.
>
> The following is a solid workaround I've stumbled upon. Afew no longer
> complains and database corruption is gone.
>
> $ notmuch compact
> $ xapian-check ~/.mail/.notmuch/xapian
>    
>    No errors found
>
> I built xapian-core 1.50 but I can't compile notmuch 0.26.1 against it.
> I will wait and test again in a few weeks.
>
> If you are interested in my setup, the error happens with this minimal
> configuration.
>
> #~/.config/afew/config
> [Filter.1]
> query = 'folder:"//(INBOX|Inbox|inbox)$/" AND (NOT tag:inbox)'
> tags = +inbox;-new
> message = Messages in INBOX folder are tagged as inbox
>
> [Filter.2]
> query = '(NOT folder:"//(INBOX|Inbox|inbox)$/") AND (tag:inbox)'
> tags = -inbox
> message = Messages not in INBOX folder cannot be inbox
>
> #~/.notmuch-config
> [database]
> path=/home-path/.mail
> [new]
> tags=new
>
> On 07/04/18 12:51, David Bremner wrote:
>> Javier Garcia  writes:
>>
>>> I've applied the path to notmuch 0.26.1 without success.
>>>
>>> $ rm -rf ~/.mail/.notmuch
>>> $ LD_LIBRARY_PATH=/hidden-path/notmuch-0.26.1/lib/:$LD_LIBRARY_PATH
>>> ./notmuch new
>>>    Found 20065 total files (that's not much mail).
>>>    Processed 20065 total files in 58s (341 files/sec.).
>>>    Added 19605 new messages to the database.
>>>
>>> $ xapian-check .mail/.notmuch/xapian/
>>>    docdata:
>>>    blocksize=8K items=63 firstunused=1 revision=2 levels=0 root=0
>>>    B-tree checked okay
>>>    docdata table structure checked OK
>>>    termlist:
>>>    blocksize=8K items=43520 firstunused=8293 revision=2 levels=2 root=748
>>>    xapian-check: DatabaseError: 1 unused block(s) missing from the free
>>> list, first is 0
>> OK, so probably not related to reference loops (although that patch is
>> not very well tested).  It's not clear how notmuch is triggering it, but
>> this looks like the same bug in Xapian that olly fixed recently [1].
>>
>> A possible next step is to try building xapian master, and linking
>> notmuch against that.
>>
>> Maybe Patrick or Justus (in copy) has some idea why you're only seeing
>> problems in afew.
>>
>> Another debugging direction is to try to duplicate your problem with
>> some subset of mail that you're willing to share (bisection is the usual
>> strategy).
>>
>> [1] https://notmuchmail.org/pipermail/notmuch/2018/026369.html
>
> ___
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch
Ciao; Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Bug? notmuch-emacs: "All tags" in notmuch-hello does not show all tags

2018-04-21 Thread Gregor Zattler
Hi Tomi, notmuch-emacs developers,
* Tomi Ollila  [2018-04-21; 20:38]:
[...]
> That doesn't mean there could not be a bug there, just
> that I cannot reproduce it...

Thanks for your investigation.

I now prepared a minimal example, please extract the archive in
/tmp.  It contains a quite minmalistic .notmuch-config, a Maildir
with one email and a test script which runs notmuch new, notmuch
tag, starts emacs with no configuration and hopefully shows
notmuch-hello.[1] Please klick on "show", you'll see only one
tag: "spam", klick on "spam" you'll see this one email shown with
its four (!) tags: "inbox", "new" and "certaintag" are not shown
in "All tags".

At least this is what I see with emacs25 and emacs27 and current
notmuch.

Do you see the same?


Thanks for investing time in this issue.  Regards, gregor

[1] The script tries to guess your notmuch/emacs source path as
load-path for emacs.  If this fails please edit the emacs
invocation according to your local setup


minimal-example.tar
Description: Unix tar archive
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Bug? notmuch-emacs: "All tags" in notmuch-hello does not show all tags

2018-04-20 Thread Gregor Zattler
Dear notmuch-emacs developers,

I use a certain tag which until now is only used with regards to
one single message which is also tagged "spam".  This certain tag
is among the output of notmuch search --output=tags '*'

The part of notmuch-hello which is supposed to show "All tags"
does not show this certain tag.  But it shows the certain tag with
count = 1 if I remove the tag "spam" from this message.  The same
happens if this message is tagged "deleted" instead of "spam".


My ~/.notmuch-config contains:

[search]
exclude_tags=deleted;spam;


The relevant section of my customzation.el contains:

 '(notmuch-hello-sections
   '(notmuch-hello-insert-search notmuch-hello-insert-saved-searches 
notmuch-hello-insert-recent-searches
 (notmuch-hello-insert-tags-section "All tags" 
:initially-hidden t nil nil)))


I can show the message in question with a search for
(is:spam OR NOT is:spam OR is:deleted OR NOT is:deleted) AND is:certaintag
notmuch-emacs then shows the single message and its tags.  Among
those is the certain tag.


I expected the certain tag to be shown in the notmuch-hello
section "All tags" and consider it a bug that this is not the case.

My guess is that somewhere in the code notmuch count is called
without --exclude=false.


Ciao; Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


wish: notmuch-emacs: wash From: / To: and Subject: in notmuch search / show

2018-03-02 Thread Gregor Zattler
Dear notmuch-emacs developers,

it would be nice if there was a facility to wash (= shorten)
the displayed To:, From:  and Subject headers at least for
the display of notmuch search results.

>From the users perspective this could be done via a list of
regular expressions with corresponding short forms.

I assume this would be best achieved in a frontend, not in notmuch
the command line utility.


Example use case:
Some of my emails are from a request tracker and have
extraordinary long From: and Subject: suffixes which add up to 49
characters.  The only way to see the relevant parts of both From:
and Subject: in this cases is to use notmuch-emacs in a single,
maximized window.  I would like to radically shorten this
boilerplate.


Ciao; Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


wish: notmuch-emacs: provide completion in notmuch-search for path: as for tag:

2018-02-07 Thread Gregor Zattler
Dear notmuch developers,

when used interactively notmuch-search provides completion for
tag: (and is:) but not for e.g. path: (and folder:) prefixes.

For obvious reasons it would be very convenient if directory/file name
completion were available especially for users who still filter
their emails in different folders.

Other candidates for completion are mimetype: id: and perhaps attachment:.


Thanks for developing notmuch and your attention, Gregor

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Notmuch Emacs: tab completion for tags in Fcc:

2017-12-23 Thread Gregor Zattler
Hi Alex,
* Alex Abdo  [2017-12-19; 23:10]:
> All,
>
> One small suggestion for Notmuch Emacs: it would be nice if there were 
> tab completion for tags (and, if possible, folders) in the Fcc header 
> when composing messages in Emacs.
>
> There is already tab completion for tags in a number of other places, 
> but for some reason, there isn’t tab completion in the Fcc header.

I also agree this would be a very helpful feature.  I suggested
tagging with the same "k" key combos as in notmuch-show in
id:87shgtgiux.fsf@len

For the time being I use abbrevs as workaround:

(notmuch-message-mode-abbrev-table)

"kkt"  3"-todo"
"kkw"  3"-waiting"
"kt"   6"+todo"
"kw"   5"+waiting"

Ciao; Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


notmuch does not find some mails which actually are part of a thread

2017-10-28 Thread Gregor Zattler
Hi notmuch developers,

notmuch does not find some messages searched for with their
Message-Id, while they are part of threads which can be shown
with searches for Message-Ids of other mails in the very thread:

$ notmuch show --entire-thread=false  --format=raw --exclude=false -- 
id:003601c7767f$99ed7a30$380a@Audrones
Error: search term did not match precisely one message (matched 0 messages).

An email with this Message-Id exists in the mail store and it
happens to be the first message in the following thread:

$ notmuch show  --entire-thread=true  --format=text --exclude=false --  
id:69a0875ba022c84aba2dc975490de6810aa...@dissens-server.adk.dissens.de |grep 
^.message
message{ id:003601c7767f$99ed7a30$380a@Audrones depth:0 match:0 excluded:0 
filename:/home/grfz/Mail/dissens_e.V./cur/1175862594.14633_1751.pit:2,S
message}
message{ 
id:69a0875ba022c84aba2dc975490de6810aa...@dissens-server.adk.dissens.de depth:0 
match:1 excluded:0 
filename:/home/grfz/Mail/dissens_e.V./cur/1175863469.14633_3866.pit:2,S
message}
message{ id:004f01c776c2$a5587800$ef00a8c0@frauenservice39 depth:0 match:0 
excluded:0 
filename:/home/grfz/Mail/dissens_e.V./cur/1175862594.14633_1776.pit:2,S
message}
message{ 
id:69a0875ba022c84aba2dc975490de6810aa...@dissens-server.adk.dissens.de depth:0 
match:0 excluded:0 
filename:/home/grfz/Mail/dissens_e.V./cur/1175863469.14633_3881.pit:2,S
message}

there are 4 messages in the thread, the 4 respective file names
are correct.

The mails are special insofar as message 1 is essentially the
same as message 2 which seems to be a bounced and somewhat
mangled version of message 1, the same is true for messages 3 and
4 respective.

But nonetheless these are legitimate mails, but messages 1 and 3
cannot be found with their respective Message-Ids.


Any ideas why this happens?

Ciao; Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


wish: notmuch-emacs: tag unsent email with same key bindings as in notmuch show

2017-08-15 Thread Gregor Zattler
Hi notmuch developers, it's super fast to use "k" + key binding
to tag mails in notmuch-emacs show, it's great that one may tag
unsent emails via the Fcc: header, but even there is C-c C-f C-w
in message mode it's error prone and especially slow to edit tags
without UI support.

It would be great if in notmuch-emacs the same tagging bindings
worked for unsent mails (obviously not with "k" but another key
combo as prefix key ["C-c k" is not bound in message mode at
least in my configuration]) as in notmuch show.

Thanks for your attention, Gregor

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Wish: notmuch-emacs: allow for reply to attached message/rfc822 message

2017-08-12 Thread Gregor Zattler
Dear notmuch developers,

it would be great if it was possible in notmuch-emacs to reply to
the very mime part of a message, the cursor is in.

Use case: Mailman sends emails for approval to list owners where
the last mime parts are a rfc822 attachment to which said list
owner may answer in order to delete the pending message from the
mailman queue (approval is more difficult).

In mutt (1) for instance it is possible to show the mime
structure of an email, show only a specific one and -- in case
its message/rfc822 -- reply to this.  This is very convenient.

[This being emacs I then tried to open mailmans web interface
from within emacs but in the end an error occurred an I had to
use firefox.]

Thanks for your attention, Gregor

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Bug: fatal error with notmuch new, second run starts indexing all over again

2017-07-14 Thread Gregor Zattler
Hi David,
* David Bremner <da...@tethera.net> [14. Jul. 2017]:
> Gregor Zattler <telegr...@gmx.net> writes:
>> I redid the test on this mid-sized corpus and I got an
>> exeption (again):
> [snip]
>> Error: A Xapian exception occurred. Halting processing.
>> Processed 27611 total files in 21m 2s (21 files/sec.).
>> Added 27481 new messages to the database.
>> Note: A fatal error was encountered: A Xapian exception occurred
>> [Inferior 1 (process 6752) exited with code 01]
>> (gdb) 
> 
> I can finally replicate the crash. I do crash in a different place
> (after processing 25184 files), so it suggests a certain non-determinism
> or dependence on the host configuration.

I would assume the first difference is in the file system:
Perhaps the files are accessed in a different order!?

> Anyway, now that I have a test case, I'll try and figure out what the
> underlying Xapian exeception is.


Thanks.

Ciao, Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Bug: fatal error with notmuch new, second run starts indexing all over again

2017-07-14 Thread Gregor Zattler
Hi David,
* David Bremner <da...@tethera.net> [13. Jul. 2017]:
>> * David Bremner <da...@tethera.net> [13. Jul. 2017]:
>>> Gregor Zattler <telegr...@gmx.net> writes:
>>>> I then downloaded parts of the archive, merged and sliced it and
>>>> now I have a sample of 25001 total files (that's not much mail),
>>>> on which notmuch new produces a xapian exeption:
[...]
>>> For me this corpus does not produce an exception (although it does have
>>> one thread with 5067 messages).

>>> I'm running Debian testing, so there could be slightly different
>>> versions of xapian and gmime. What versions of libxapian-dev and
>>> libgmime-2.6-dev have you built notmuch against?
[...]
> I tried on a debian stretch with the same library versions, and I still
> can't duplicate the crash.

This is strange.  I rerun the self compiled notmuch new on this
corpus and it did not throw an exeption, nore did the debian
stretch binary.  I have no idea why this is different with the
first run.

On my way reducing the corpus I had an intermediate corpus of
~43000 files on which notmuch new produced a xapian exeption.  I
kept this corpus but did not include the test results in my email
because the later, smaller corpus seemed more interesting.

I redid the test on this mid-sized corpus and I got an
exeption (again):

0 (master *) grfz@len:~$ gdb --args /home/grfz/src/notmuch/notmuch new
GNU gdb (Debian 7.12-6) 7.12.0.20161007-git
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/grfz/src/notmuch/notmuch...done.
(gdb) b _notmuch_database_log
Breakpoint 1 at 0x1f6e0: file lib/database.cc, line 426.
(gdb) run
Starting program: /home/grfz/src/notmuch/notmuch new
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Found 43255 total files (that's not much mail).

Breakpoint 1, _notmuch_database_log (notmuch=0x557d2820, 
format=0x5558d258 "A Xapian exception occurred adding message: %s.\n") at 
lib/database.cc:426
426 {
(gdb) bt
#0  _notmuch_database_log (notmuch=0x557d2820, format=0x5558d258 "A 
Xapian exception occurred adding message: %s.\n") at lib/database.cc:426
#1  0x55577f58 in notmuch_database_add_message 
(notmuch=notmuch@entry=0x557d2820, filename=filename@entry=0x56fa4870 
"/tmp/reduced-sample.xapian-exeption/new/1499897795.R10126604875560291557.len",

  message_ret=message_ret@entry=0x7fffd2e8) at 
lib/database.cc:2597
#2  0x5556802f in add_file (state=0x7fffd560, 
filename=0x56fa4870 
"/tmp/reduced-sample.xapian-exeption/new/1499897795.R10126604875560291557.len", 
notmuch=0x557d2820) at notmuch-new.c:264
#3  add_files (notmuch=notmuch@entry=0x557d2820, 
path=path@entry=0x557d26e0 "/tmp/reduced-sample.xapian-exeption/new", 
state=state@entry=0x7fffd560) at notmuch-new.c:599
#4  0x55567b44 in add_files (notmuch=0x557d2820, 
path=path@entry=0x557cf730 "/tmp/reduced-sample.xapian-exeption", 
state=state@entry=0x7fffd560) at notmuch-new.c:483
#5  0x555689ed in notmuch_new_command (config=0x557ce1d0, 
argc=, argv=) at notmuch-new.c:1099
#6  0x55561a27 in main (argc=, argv=0x7fffda88) at 
notmuch.c:456
(gdb) continue
Continuing.
Error: A Xapian exception occurred. Halting processing.
Processed 27611 total files in 21m 2s (21 files/sec.).
Added 27481 new messages to the database.
Note: A fatal error was encountered: A Xapian exception occurred
[Inferior 1 (process 6752) exited with code 01]
(gdb) 



I redid the test with this medium sized corpus with the notmuch
binary from debian stretch, the xapian exeption hits after the
same number of scanned/added files:

0 (master *) grfz@len:~$ gdb --args /usr/bin/notmuch new
GNU gdb (Debian 7.12-6) 7.12.0.20161007-git
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to c

Re: all intermediate drafts appear in threads (v0.24.2, Emacs 25.2)

2017-07-13 Thread Gregor Zattler
Hi Sanjoy,
* Sanjoy Mahajan  [13. Jul. 2017]:
> I save an outgoing email as I write it and often (as if it were a
> regular document) by using the usual Emacs key sequence C-x C-s.  With
> notmuch v0.24, notmuch saves a proper draft each time I use C-x C-s.
> The result is a million drafts, which may be good.
> 
> However, it means that the thread view (in notmuch-show mode) includes
> the zillion drafts.  
[... 38 Zeilen gelöscht ...]

> Did I miss something in the configuration?  Or should I be using the
> drafts differently?

Same here.  Once in a while I do a

notmuch search --format=text0 --output=files is:deleted | xargs -0r rm

, which deletes all emails marked as "deleted".  After the next
notmuch new they do not show up in the threads any more.  This
could be automated by a cron job or so, but it didn't bother me
enough to do so.

Ciao, Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Bug: fatal error with notmuch new, second run starts indexing all over again

2017-07-13 Thread Gregor Zattler
Hi David,
* David Bremner <da...@tethera.net> [13. Jul. 2017]:
> Gregor Zattler <telegr...@gmx.net> writes:
>> I then downloaded parts of the archive, merged and sliced it and
>> now I have a sample of 25001 total files (that's not much mail),
>> on which notmuch new produces a xapian exeption:
>>
> 
>> As a tar.xz it weighs 28 MB.  I could provide this for download
>> if someone is interested.
> 
> For me this corpus does not produce an exception (although it does have
> one thread with 5067 messages).

I cannot view this ATM because I cannot index these emails.  When
viewd with mutt there are no such gargantuan threads.

> I'm running Debian testing, so there could be slightly different
> versions of xapian and gmime. What versions of libxapian-dev and
> libgmime-2.6-dev have you built notmuch against?

Mhm.  I'm running debian stretch.  The versions are:

0 grfz@len:/tmp$ apt-cache policy libxapian-dev libgmime-2.6-dev
libxapian-dev:
Installed: 1.4.3-2
Candidate: 1.4.3-2
Version table:
*** 1.4.3-2 500
500 http://deb.debian.org/debian stretch/main amd64 Packages
100 /var/lib/dpkg/status
libgmime-2.6-dev:
Installed: 2.6.22+dfsg2-1
Candidate: 2.6.22+dfsg2-1
Version table:
*** 2.6.22+dfsg2-1 500
500 http://deb.debian.org/debian stretch/main amd64 Packages
100 /var/lib/dpkg/status


BTW:


0 grfz@len:/tmp$ apt-cache policy gcc
gcc:
Installed: 4:6.3.0-4
Candidate: 4:6.3.0-4
Version table:
*** 4:6.3.0-4 500
500 http://deb.debian.org/debian stretch/main amd64 Packages
100 /var/lib/dpkg/status
0 grfz@len:/tmp$ gcc --version
gcc (Debian 6.3.0-18) 6.3.0 20170516





HTH, Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Bug: fatal error with notmuch new, second run starts indexing all over again

2017-07-13 Thread Gregor Zattler
Hi David, notmuch developers,
* David Bremner <da...@tethera.net> [13. Jul. 2017]:
>> * David Bremner <da...@tethera.net> [09. Jul. 2017]:
>>> Gregor Zattler <telegr...@gmx.net> writes:
>>> I was wondering if the exception could result from overflowing some
>>> internal limit due to threads many thousands of messages long. I will be
>>> hard to know until I can replicate the exception. Perhaps a more
>>> complete org-mode list archive would do it.
>>
>> Quite possible, when I do 
>> notmuch show  --entire-thread=true --format=mbox 
>> path:Mail/~ml/emacs-orgm...@gnu.org/**   date 64 bit 32 > /tmp/emo.mbox
>> and open the resulting mbox with mutt, it shows 10199 messages.
> 
> Are those literal search terms "date 64 bit 32"?

yes.

>> As a tar.xz it weighs 28 MB.  I could provide this for download
>> if someone is interested.

I'll do so in a private email.  


Ciao, Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Bug: fatal error with notmuch new, second run starts indexing all over again

2017-07-13 Thread Gregor Zattler
Hi David, notmuch developers,
* David Bremner <da...@tethera.net> [09. Jul. 2017]:
> Gregor Zattler <telegr...@gmx.net> writes:
>> Short version: Some of the messages on this mailinglist had very
>> weired References: headers mot probably causing notmuch to
>> misbehave while threading the messages.  But then there was no
>> xapian exeption involved.
>>
> 
> Right. I looks like there are indeed many message-ids in the resulting
> database that look like valid email addresses. So that problem persists,
> despite an attempted fix discussed in that thread.
> 
> I was wondering if the exception could result from overflowing some
> internal limit due to threads many thousands of messages long. I will be
> hard to know until I can replicate the exception. Perhaps a more
> complete org-mode list archive would do it.

Quite possible, when I do 
notmuch show  --entire-thread=true --format=mbox 
path:Mail/~ml/emacs-orgm...@gnu.org/**   date 64 bit 32 > /tmp/emo.mbox
and open the resulting mbox with mutt, it shows 10199 messages.

There is fun to have regarding message ids:

0 grfz@len:/tmp$ grep -c "^Message-ID:[[:space:]]" eom.mbox 
8758
0 grfz@len:/tmp$ grep -ci "^Message-ID:[[:space:]]" eom.mbox 
10174

I then split the mbox with mutt in a maildir folder.  This were
10199 individual files.  I then extracted the message ids via
formail from this files and piped this through sort -u.  These
were 10105 message ids.

The maildir with the emacs-orgm...@gnu.org emails contained 114563
individual files.  I removed every file somewhere containing one
of those message ids.  Now there were only 102164 individual files
in the maildir.

And after that I indexed this cleaned up maildir.  If somehow
this message threading problem affects the indexing it should now
index the files without xapian exeption:


This is with newest notmuch:
0 grfz@len:/tmp$ /home/grfz/src/notmuch/notmuch --version
notmuch 0.24.2+112~g37d1fa5


0 grfz@len:/tmp$ gdb --args /home/grfz/src/notmuch/notmuch new
GNU gdb (Debian 7.12-6) 7.12.0.20161007-git
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/grfz/src/notmuch/notmuch...done.
(gdb) b _notmuch_database_log
Breakpoint 1 at 0x1f6e0: file lib/database.cc, line 426.
(gdb) run
Starting program: /home/grfz/src/notmuch/notmuch new
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Found 102164 total files (that's not much mail).
Processed 102164 total files in 5m 58s (284 files/sec.).
Added 102151 new messages to the database.
[Inferior 1 (process 26339) exited normally]
(gdb) 


While indexing the original emacs-org mode mailing list maildir
with it's 114563 files with the very same binary results in a
xapian exeption:


0 grfz@len:/tmp$ gdb --args /home/grfz/src/notmuch/notmuch new
GNU gdb (Debian 7.12-6) 7.12.0.20161007-git
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/grfz/src/notmuch/notmuch...done.
(gdb) b _notmuch_database_log
Breakpoint 1 at 0x1f6e0: file lib/database.cc, line 426.
(gdb) run
Starting program: /home/grfz/src/notmuch/notmuch new
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Found 114563 total files (that's not much mail).

Breakpoint 1, _notmuch_database_log (notmuch=0x557d2780, 
format=0x5558d258 "A Xapian exception occurred

Re: Bug: fatal error with notmuch new, second run starts indexing all over again

2017-07-09 Thread Gregor Zattler
Hi David,
* David Bremner <da...@tethera.net> [09. Jul. 2017]:
> Gregor Zattler <telegr...@gmx.net> writes:
>> Since its an email from an open mailing list I attach it to
>> this very email.
>>
> 
> It might or might not be related, but I noticed that those two messages
> get merged for me into a thread 1734 messages long of org-mode messages
> (I have some old org messages, but I think I not the two you mention).
> 
> I'm not sure yet if it's a bug in notmuch, or something strange about
> mail from the org-mode list.

This *might* be related with some other problem I reported
regarding false threading of messages, as discussed in the thread
containing amongst others id:874nvcekjk@qmul.ac.uk

Short version: Some of the messages on this mailinglist had very
weired References: headers mot probably causing notmuch to
misbehave while threading the messages.  But then there was no
xapian exeption involved.


Otherwise I did an extensive memory test: no prblem and an
smartctl -t long and -t offline test: no problems.  Remains to do
a bisecting but I have no free time atm.  Will report back later.

Ciao, Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Bug: fatal error with notmuch new, second run starts indexing all over again

2017-07-06 Thread Gregor Zattler
Hi David, notmuch developers,
* David Bremner  [03. Jul. 2017]:
> Try setting a break in _notmuch_database_log (b _notmuch_database_log)
> and run "bt" at that break point. It might (or might not) be worth
> continuing after the first breakpoint to inspect other errors.

I did so with newest notmuch:

0 (master *) grfz@len:~/tmp$ notmuch --version
notmuch 0.24.2+84~g1ec6344
0 (master *) grfz@len:~/tmp$

0 (master=) grfz@len:~/src/notmuch$ gdb --args /home/grfz/src/notmuch/notmuch 
new
GNU gdb (Debian 7.12-6) 7.12.0.20161007-git
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
.
Find the GDB manual and other documentation resources online at:
.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/grfz/src/notmuch/notmuch...done.
(gdb) b _notmuch_database_log
Breakpoint 1 at 0x1f6e0: file lib/database.cc, line 426.
(gdb) run
Starting program: /home/grfz/src/notmuch/notmuch new
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Found 941498 total files (that's not much mail).
q
Breakpoint 1, _notmuch_database_log (notmuch=0x55825dd0, 
format=0x5558d258 "A Xapian exception occurred adding message: %s.\n") at 
lib/database.cc:426
426 {
(gdb) bt
#0  _notmuch_database_log (notmuch=0x55825dd0, format=0x5558d258 "A 
Xapian exception occurred adding message: %s.\n") at lib/database.cc:426
#1  0x55577f58 in notmuch_database_add_message 
(notmuch=notmuch@entry=0x55825dd0, filename=filename@entry=0x7b204400 
"/home/grfz/Mail/~ml/emacs-orgm...@gnu.org/cur/1488666048.R6065357166258349555.len:2,",
message_ret=message_ret@entry=0x7fffd048) at lib/database.cc:2597
#2  0x5556802f in add_file (state=0x7fffd540, 
filename=0x7b204400 
"/home/grfz/Mail/~ml/emacs-orgm...@gnu.org/cur/1488666048.R6065357166258349555.len:2,",
 notmuch=0x55825dd0) at notmuch-new.c:264
#3  add_files (notmuch=notmuch@entry=0x55825dd0, 
path=path@entry=0x74a15b80 "/home/grfz/Mail/~ml/emacs-orgm...@gnu.org/cur", 
state=state@entry=0x7fffd540) at notmuch-new.c:599
#4  0x55567b44 in add_files (notmuch=notmuch@entry=0x55825dd0, 
path=path@entry=0x8520f1b0 "/home/grfz/Mail/~ml/emacs-orgm...@gnu.org", 
state=state@entry=0x7fffd540) at notmuch-new.c:483
#5  0x55567b44 in add_files (notmuch=notmuch@entry=0x55825dd0, 
path=path@entry=0x638993b0 "/home/grfz/Mail/~ml", 
state=state@entry=0x7fffd540) at notmuch-new.c:483
#6  0x55567b44 in add_files (notmuch=0x55825dd0, 
path=path@entry=0x557cf710 "/home/grfz/Mail", 
state=state@entry=0x7fffd540) at notmuch-new.c:483
#7  0x555689ed in notmuch_new_command (config=0x557ce1d0, 
argc=, argv=) at notmuch-new.c:1099
#8  0x55561a27 in main (argc=, argv=0x7fffda68) at 
notmuch.c:456
(gdb) continue
Continuing.
Error: A Xapian exception occurred. Halting processing.
Processed 660118 total files in 8h 59m 43s (20 files/sec.).
Added 584749 new messages to the database.
Note: A fatal error was encountered: A Xapian exception occurred
[Inferior 1 (process 14593) exited with code 01]
(gdb) continue
The program is not being run.
(gdb)



Since the gdb output shows a filename I looked at its contents:
There is a second "Content-Length:" -Header which does not make
sense but otherwise should not be a problem.  

Since its an email from an open mailing list I attach it to
this very email.

I then configured notmuch to scan the whole mail folder
/home/grfz/Mail/~ml/emacs-orgm...@gnu.org: 

0 (master=) grfz@len:~/src/notmuch$ gdb --args /home/grfz/src/notmuch/notmuch 
new
GNU gdb (Debian 7.12-6) 7.12.0.20161007-git
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
.
Find the GDB manual and other documentation resources online at:
.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/grfz/src/notmuch/notmuch...done.
(gdb) 

Re: Bug: fatal error with notmuch new, second run starts indexing all over again

2017-07-03 Thread Gregor Zattler
Hi David, notmuch developers,
* David Bremner <da...@tethera.net> [03. Jul. 2017]:
> Gregor Zattler <telegr...@gmx.net> writes:
>> notmuch new fails while indexing my emails and when executed a
>> second time it obviously starts over again.  This is on a
>> up-to-date debian stetch with notmuch from git:
>> Any ideas how to proceed in order to get a working notmuch setup?

> 1. Check disk space

Sorry, forgot to mention: there is enough free space on RAM (8
GB) and SSD (40 G free).

> 2. run xapian-fsck

xapian-check does not find any errors, only spelling and synonym
are "Lazily created, and not yet used."

> 3. run "notmuch new" under gdb to try to get a backtrace

I have no clue about gdb, there was no backtrace:

0 (master) grfz@len:~$ gdb --args notmuch new
GNU gdb (Debian 7.12-6) 7.12.0.20161007-git
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show
copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from notmuch...done.
(gdb) run
Starting program: /usr/local/bin/notmuch new
[Thread debugging using libthread_db enabled]
Using host libthread_db library
"/lib/x86_64-linux-gnu/libthread_db.so.1".
Error: A Xapian exception occurred. Halting processing.
Processed 662357 total files in 1h 52m 51s (97 files/sec.).
Added 587023 new messages to the database.
Note: A fatal error was encountered: A Xapian exception occurred
[Inferior 1 (process 4509) exited with code 01]
(gdb) bt
No stack.
(gdb) 


> 4. use "git bisect" to try to find what commit introduced this problem

I'll do so in the next few days.

Thanks, Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Bug: fatal error with notmuch new, second run starts indexing all over again

2017-07-03 Thread Gregor Zattler
Dear notmuch developers,

notmuch new fails while indexing my emails and when executed a
second time it obviously starts over again.  This is on a
up-to-date debian stetch with notmuch from git:

0 (master) grfz@len:~/Mail$ notmuch new
Found 940740 total files (that's not much mail).
Error: A Xapian exception occurred. Halting processing.
Processed 659979 total files in 1h 43m 20s (106 files/sec.).
Added 584654 new messages to the database.
Note: A fatal error was encountered: A Xapian exception occurred
1 (master) grfz@len:~/Mail$ notmuch new
Error: A Xapian exception occurred. Halting processing.
Processed 660023 total files in 1h 44m 45s (105 files/sec.).
Added 584698 new messages to the database.
Note: A fatal error was encountered: A Xapian exception occurred
1 (master) grfz@len:~/Mail$ notmuch --version
notmuch 0.24.2+100~gab02265

The resutling xapian databases are not as large as I would
expect (an older xapian database from another config is 11 G
big): 

0 (master) grfz@len:~/Mail$ du -hs .
28G .
0 (master) grfz@len:~/Mail$ du -hs .notmuch
7,1G.notmuch

It's now not possible to search with notmuch:
0 (master) grfz@len:~/Mail$ notmuch search date:year..
0 (master) grfz@len:~/Mail$

Any ideas how to proceed in order to get a working notmuch setup?


Ciao, Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


how to configure notmuch-emacs to automatically show application/pgp partsFcc: sent -unread

2017-04-18 Thread Gregor Zattler
Dear notmuch users and developers, could one please enlighten me,
how to configure notmuch-emacs in order to automatically show
application/pgp mime parts in emails as in e.g. cryptograhically
signed debian security advisories (as attached).

Thanks, Gregor

--- Begin Message ---


pgp7inH7iQtVg.pgp
Description: PGP message
--- End Message ---
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: OOM killer hits emacs on thread with 439 messages

2017-04-01 Thread Gregor Zattler
Hi David,

actually, this was a false alarm:  I didn't re-test before posting:
Actually it works with emacs -Q and with my full configuration.

Sorry for the noise, Gregor


* David Bremner <da...@tethera.net> [2017-04-01; 15:52]:
> Gregor Zattler <telegr...@gmx.net> writes:
>
>> Hi, this thread from emacs-devel:
>>
>>  Yest. 10:40 [4/309] Eli Zaretskii, Ken Raeburn, 
>> npost...@users.sourceforge.net, Paul Eggert, Stefan Monnier, Andreas Schwab, 
>> Richard Stallman, Lars Ingebrigtsen, Daniel Colascione, Philipp Stephani, 
>> Lars Brinkhoff, Perry E. Metzger, Noam Postavsky, Fabrice Popineau, Jérémie 
>> Courrèges-Anglas, Simon Leinen, Alp Aker, Ken Brown, Phil Sainty, Yuri Khan, 
>> Clément Pit--Claudel, Robert Pluim, Phillip Lord, Alan Mackenzie   
>> Skipping unexec via a big .elc file (attachment inbox signed unread)
>>
>> brought notmuch-emacs to a grinding halt on a system which 
>> otherwise was used far below memory capapilities (8G RAM, 4.5GB
>> swap, AMD64, emacs compiled from source with 64 bit), later the
>> OMM killer got it.
>>
>
> No, that's definitely a bug. Hard to say if it's a bug in emacs or in
> notmuch-emacs. Can you make an mbox out of the thread (e.g. with notmuch
> show --format=mbox thread:whatever) and either put it somewhere or mail
> it to me (if it's below 2M). I'd do it myself but it lokos like I'd need
> to download at least 4 months of archives.
>
> It's always good to specify what version of emacs and what version of
> notmuch you encounter problems with.
>
> d
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


OOM killer hits emacs on thread with 439 messages

2017-04-01 Thread Gregor Zattler

Hi, this thread from emacs-devel:

 Yest. 10:40 [4/309] Eli Zaretskii, Ken Raeburn, 
npost...@users.sourceforge.net, Paul Eggert, Stefan Monnier, Andreas Schwab, 
Richard Stallman, Lars Ingebrigtsen, Daniel Colascione, Philipp Stephani, Lars 
Brinkhoff, Perry E. Metzger, Noam Postavsky, Fabrice Popineau, Jérémie 
Courrèges-Anglas, Simon Leinen, Alp Aker, Ken Brown, Phil Sainty, Yuri Khan, 
Clément Pit--Claudel, Robert Pluim, Phillip Lord, Alan Mackenzie   
Skipping unexec via a big .elc file (attachment inbox signed unread)

brought notmuch-emacs to a grinding halt on a system which 
otherwise was used far below memory capapilities (8G RAM, 4.5GB
swap, AMD64, emacs compiled from source with 64 bit), later the
OMM killer got it.


Is this to be expected?

Thanks, Gregor
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


search query "replytoid:"

2015-06-15 Thread Gregor Zattler
Hi Suvayu, notmuch users and developers,
* Suvayu Ali  [14. Jun. 2015]:
> Try the attached script.  It accepts notmuch queries by message id.
> E.g. to get the responses to your OP, you can do:
> 
>   $ ./nm-ack id:CAJhTkNhYew6H-bptACTew3gN3DLWg6agTYu8hAkdwFS=z4VFWg at 
> mail.gmail.com
>   id:877fr79upd.fsf at maritornes.cs.unb.ca id:20150613205552.GC17381 at 
> chitra.no-ip.org
> 
> The first one is David's response, the second one is mine.

This is a nice script.  The very first Message(-Id:) I tried was
replied to by a message with this In-Reply-To: header:

In-Reply-To: <7e093509.51e.14ddb300091.Coremail.chxp_moon at 163.com> 
(windy’s
   message of „Wed, 10 Jun 2015 09:57:45 +0800 (CST)“)

Since the script extracts Message-Id:s from In-Reply-To; headers via

> function strip_mid() {
> sed -e 's/[<> ]//g'
> }

the Message-Id:s did not match.  I therefore changed this to

function strip_mid() {
egrep  -a -o "<[^[:space:]<>]+@[^@[:space:]<>]+>"|sed -e 's/[<> ]//g'
}

this did the trick.


Thanx, Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-


Re: search query replytoid:blah

2015-06-14 Thread Gregor Zattler
Hi Suvayu, notmuch users and developers,
* Suvayu Ali fatkasuvayu+li...@gmail.com [14. Jun. 2015]:
 Try the attached script.  It accepts notmuch queries by message id.
 E.g. to get the responses to your OP, you can do:
 
   $ ./nm-ack 
 id:CAJhTkNhYew6H-bptACTew3gN3DLWg6agTYu8hAkdwFS=z4v...@mail.gmail.com
   id:877fr79upd@maritornes.cs.unb.ca 
 id:20150613205552.gc17...@chitra.no-ip.org
 
 The first one is David's response, the second one is mine.

This is a nice script.  The very first Message(-Id:) I tried was
replied to by a message with this In-Reply-To: header:

In-Reply-To: 7e093509.51e.14ddb300091.coremail.chxp_m...@163.com 
(windy’s
   message of „Wed, 10 Jun 2015 09:57:45 +0800 (CST)“)

Since the script extracts Message-Id:s from In-Reply-To; headers via

 function strip_mid() {
 sed -e 's/[ ]//g'
 }

the Message-Id:s did not match.  I therefore changed this to

function strip_mid() {
egrep  -a -o [^[:space:]]+@[^@[:space:]]+|sed -e 's/[ ]//g'
}

this did the trick.


Thanx, Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


notmuch should search on date received

2015-05-05 Thread Gregor Zattler
Hi David,
* David Bremner  [04. May. 2015]:
> Gregor Zattler  writes:
>> I wished notmuch could search on (the top most) Received: -header
>> since not all emails have correct Date: -headers.
>>
>> Rationale: Show all emails since yesterday…  This is immensely
>> important for my work flow.
> 
> These are two somewhat different requests. Being able to search on the
> Received headers would involve a usual kind of term search, but not date
> searching.

This way one could at least search for years but it would not be
convenient. 

> This might be problematic as a default because of the volume
> of received headers.  In order to use it for date search, it would need
> to be parsed for data information and either override the Date:-header
> or provide a second prefix like received-date.

This is what I wish for.  There are several Received: -headers,
the top most contains the actual moment when the email arrived on
the system.  I would like to search for date ranges of arrival.


> Or maybe a single
> message could have multiple date: prefixed terms, I'm not sure.

This I don’t understand.


Ciao, Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-


Re: notmuch should search on date received

2015-05-05 Thread Gregor Zattler
Hi David,
* David Bremner da...@tethera.net [04. May. 2015]:
 Gregor Zattler telegr...@gmx.net writes:
 I wished notmuch could search on (the top most) Received: -header
 since not all emails have correct Date: -headers.

 Rationale: Show all emails since yesterday…  This is immensely
 important for my work flow.
 
 These are two somewhat different requests. Being able to search on the
 Received headers would involve a usual kind of term search, but not date
 searching.

This way one could at least search for years but it would not be
convenient. 

 This might be problematic as a default because of the volume
 of received headers.  In order to use it for date search, it would need
 to be parsed for data information and either override the Date:-header
 or provide a second prefix like received-date.

This is what I wish for.  There are several Received: -headers,
the top most contains the actual moment when the email arrived on
the system.  I would like to search for date ranges of arrival.


 Or maybe a single
 message could have multiple date: prefixed terms, I'm not sure.

This I don’t understand.


Ciao, Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Thanx (was: Re: notmuch should show Message-Id: -headers)

2015-05-03 Thread Gregor Zattler
Hi Tomi,
* Tomi Ollila  [03. May. 2015]:
> On Sun, May 03 2015, Gregor Zattler  wrote:
>> it would be helpful if it was possible to configure notmuch
>> (Emacs interface) to show the Message-Id: -header in notmuch
>> show.
> 
> I once desired this feature too (and tried to set the visible headers (or
> whatnot)... but since then I've managed myself with 'c' 'i'. I.e. keybinding
> 
> c i runs the command notmuch-show-stash-message-id

Thanks.  That’s helpful.


Ciao, Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-


[bug] [wish] notmuch should show Message-Id: -headers

2015-05-03 Thread Gregor Zattler
Dear notmuch developers,

it would be helpful if it was possible to configure notmuch
(Emacs interface) to show the Message-Id: -header in notmuch
show.

Thanks for your attention.
Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-


[bug] [wish] notmuch should search on date received

2015-05-03 Thread Gregor Zattler
Dear notmuch developers,

I wished notmuch could search on (the top most) Received: -header
since not all emails have correct Date: -headers.

Rationale: Show all emails since yesterday…  This is immensely
important for my work flow.

Ciao, Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-


[bug] [wish] notmuch should show Message-Id: -headers

2015-05-03 Thread Gregor Zattler
Dear notmuch developers,

it would be helpful if it was possible to configure notmuch
(Emacs interface) to show the Message-Id: -header in notmuch
show.

Thanks for your attention.
Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[bug] [wish] notmuch should search on date received

2015-05-03 Thread Gregor Zattler
Dear notmuch developers,

I wished notmuch could search on (the top most) Received: -header
since not all emails have correct Date: -headers.

Rationale: Show all emails since yesterday…  This is immensely
important for my work flow.

Ciao, Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


NEWS: add news for 0.18.1

2014-06-22 Thread Gregor Zattler
Hi David, notmuch developers,
* David Bremner  [21. Jun. 2014]:
> This is my summary of the commits from 0.18.

[...]
> +Fix for phrase indexing
> +
> +  There were several bugs involving overlapping "phrases" accross
> +  header and mime part boundaries.

Perhaps this should be amended with the following excerpt from a
commit message:

"This fix only affects newly indexed messages.  Messages that are
already indexed won't benefit from this fix without re-indexing, but
the fix won't make things any worse for existing messages."


Ciao; Gregor


Re: NEWS: add news for 0.18.1

2014-06-22 Thread Gregor Zattler
Hi David, notmuch developers,
* David Bremner da...@tethera.net [21. Jun. 2014]:
 This is my summary of the commits from 0.18.

[...]
 +Fix for phrase indexing
 +
 +  There were several bugs involving overlapping phrases accross
 +  header and mime part boundaries.

Perhaps this should be amended with the following excerpt from a
commit message:

This fix only affects newly indexed messages.  Messages that are
already indexed won't benefit from this fix without re-indexing, but
the fix won't make things any worse for existing messages.


Ciao; Gregor
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Bug?: notmuch-search-show-thread shows several threads; only one containing matching messages

2014-01-27 Thread Gregor Zattler
Hi David,
* David Bremner  [26. Jan. 2014]:
> Gregor Zattler  writes:
> 
>> Today I produced another mbox with the very same command but with
>> a now larger email corpus freshly indexed with a fresh notmuch.
>> The mbox contains (according to mutt) 507 messages in 34 threads.
>> One of them is the thread I searched for.
>>
>> I grepped for the 7 subjects within the 34 subjects and only 5
>> showed up.
> 
> I don't know what you mean here. Grepped where? in the raw messages?

With mutt I had a view at the collapsed 7 respective 34 threads.
One then sees the very first E-Mails of a thread and among other
information their subjects.
Via editing I produced two lists with subjects and then searched
each of the 7 in the list with the 34.

>> If somebody want's to dig into this: I can provide the two
>> mboxes. 
>>
>> Disclaimer: Many of the emails which arrived before the problem
>> report are not the exact same than then, because since the I
>> mangled them with a script.  This should have not changed the
>> threading but I cannot be 100% sure.  But if it's important for
>> further investigation I'm probably able to reproduce the status
>> quo of the email corpus then from my backups.
> 
> If it's currently not working then I guess your current corpus should be
> fine. It would probably help to restate what exactly is wrong. There was
> a lot of discussion, and the concrete problem I saw identified (in
> id:874nvcekjk.fsf at qmul.ac.uk ) was that certain malformed In-reply-to
> headers were causing unrelated threads to merge.

Yes.  I understood the commit message of the commit you
referenced in the email I answered to, that now notmuch uses
 Reference: headers to do the threading.  I had a quick view at
the References header in the mbox file and none looked
suspicious.

Ciao, Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-


Bug: notmuch new chokes on dangling symlinks and quits

2014-01-27 Thread Gregor Zattler
Hi David,
* David Bremner  [26. Jan. 2014]:
> Gregor Zattler  writes:
>> I consider this to be a bug.  Instead notmuch should simply
>> ignore the symlink.
>>
> 
> Since there is a test for specifically this behaviour, I'd have to say
> it's a design decision you don't agree with, not a bug ;).

May I ask why this is so?

> In any case, if there is just a few broken symlinks, and you want to
> ignore them, you can add them the ignore= line in .notmuch-config

Thanks, I deleted some, and ignored others.  Insofar my problem is
solved. 

Ciao, Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-


Bug?: notmuch-search-show-thread shows several threads; only one containing matching messages

2014-01-26 Thread Gregor Zattler
Hi David,
* David Bremner  [24. Jan. 2014]:
> Mark Walters  writes:
>> I have looked at this and I think this is not notmuch's fault: I think
>> it is a mua doing strange things:
>>
>> One of the mails has an in-reply-to header which looks like
>>
>> In-reply-to: Message from Carsten Dominik  of  
>>   "Tue, 15 Mar 2011 12:18:51 BST."
>> <17242340-A14F-495A-B144-20C96D52B620 at gmail.com>
>>
>> and I think notmuch is taking the carsten.dominik at gmail.com as message
>> id.
>>
> 
> Can someone test if this is fixed by cf8aaafbad68 (i.e. does the problem
> persist in git master or 0.17)?

The problem is *not* fixed.  

I was the one who reported this problem two years ago.  I did the
same notmuch search again.  Since I did no know if it's important
with which version the emails were indexed I did a full index
with notmuch 0.17+40~gecbb29e.

I still have the mbox produced with notmuch show two years ago.
Viewed with mutt (1) I see 206 messages in 7 threads (number of
lines after collapse-all) (notmuch emacs show showed three
threads then).  One of the threads is the one I searched for.

Today I produced another mbox with the very same command but with
a now larger email corpus freshly indexed with a fresh notmuch.
The mbox contains (according to mutt) 507 messages in 34 threads.
One of them is the thread I searched for.

I grepped for the 7 subjects within the 34 subjects and only 5
showed up.

Only 17 of the 507 messages arrived since the problem report two
years ago.

If somebody want's to dig into this: I can provide the two
mboxes. 

Disclaimer: Many of the emails which arrived before the problem
report are not the exact same than then, because since the I
mangled them with a script.  This should have not changed the
threading but I cannot be 100% sure.  But if it's important for
further investigation I'm probably able to reproduce the status
quo of the email corpus then from my backups.

Thanks for your persistence. 

Ciao, Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-


Bug: notmuch new chokes on dangling symlinks and quits

2014-01-26 Thread Gregor Zattler
Dear notmuch developers,

notmuch new quits operation if it hit's a dangling symlink:


Error reading file 
/home/grfz/notmuch2/Mail/findex/cur/123456789.684055.mairix:2,: No such file or 
directory
Note: A fatal error was encountered: Something went wrong trying
to read or write a file Processed 85 total files in 1s (47
files/sec.)


I consider this to be a bug.  Instead notmuch should simply
ignore the symlink.

This is especially problematic while indexing a huge amount of
mail the first time and notmuch hits the dangling symlink at an
early stage of indexing.


Ciao, Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-


Bug: notmuch new chokes on dangling symlinks and quits

2014-01-26 Thread Gregor Zattler
Dear notmuch developers,

notmuch new quits operation if it hit's a dangling symlink:


Error reading file 
/home/grfz/notmuch2/Mail/findex/cur/123456789.684055.mairix:2,: No such file or 
directory
Note: A fatal error was encountered: Something went wrong trying
to read or write a file Processed 85 total files in 1s (47
files/sec.)


I consider this to be a bug.  Instead notmuch should simply
ignore the symlink.

This is especially problematic while indexing a huge amount of
mail the first time and notmuch hits the dangling symlink at an
early stage of indexing.


Ciao, Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: Bug?: notmuch-search-show-thread shows several threads; only one containing matching messages

2014-01-26 Thread Gregor Zattler
Hi David,
* David Bremner da...@tethera.net [24. Jan. 2014]:
 Mark Walters markwalters1...@gmail.com writes:
 I have looked at this and I think this is not notmuch's fault: I think
 it is a mua doing strange things:

 One of the mails has an in-reply-to header which looks like

 In-reply-to: Message from Carsten Dominik carsten.domi...@gmail.com of 
Tue, 15 Mar 2011 12:18:51 BST.
 17242340-a14f-495a-b144-20c96d52b...@gmail.com

 and I think notmuch is taking the carsten.domi...@gmail.com as message
 id.

 
 Can someone test if this is fixed by cf8aaafbad68 (i.e. does the problem
 persist in git master or 0.17)?

The problem is *not* fixed.  

I was the one who reported this problem two years ago.  I did the
same notmuch search again.  Since I did no know if it's important
with which version the emails were indexed I did a full index
with notmuch 0.17+40~gecbb29e.

I still have the mbox produced with notmuch show two years ago.
Viewed with mutt (1) I see 206 messages in 7 threads (number of
lines after collapse-all) (notmuch emacs show showed three
threads then).  One of the threads is the one I searched for.

Today I produced another mbox with the very same command but with
a now larger email corpus freshly indexed with a fresh notmuch.
The mbox contains (according to mutt) 507 messages in 34 threads.
One of them is the thread I searched for.

I grepped for the 7 subjects within the 34 subjects and only 5
showed up.

Only 17 of the 507 messages arrived since the problem report two
years ago.

If somebody want's to dig into this: I can provide the two
mboxes. 

Disclaimer: Many of the emails which arrived before the problem
report are not the exact same than then, because since the I
mangled them with a script.  This should have not changed the
threading but I cannot be 100% sure.  But if it's important for
further investigation I'm probably able to reproduce the status
quo of the email corpus then from my backups.

Thanks for your persistence. 

Ciao, Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: Bug: notmuch new chokes on dangling symlinks and quits

2014-01-26 Thread Gregor Zattler
Hi David,
* David Bremner da...@tethera.net [26. Jan. 2014]:
 Gregor Zattler telegr...@gmx.net writes:
 I consider this to be a bug.  Instead notmuch should simply
 ignore the symlink.

 
 Since there is a test for specifically this behaviour, I'd have to say
 it's a design decision you don't agree with, not a bug ;).

May I ask why this is so?

 In any case, if there is just a few broken symlinks, and you want to
 ignore them, you can add them the ignore= line in .notmuch-config

Thanks, I deleted some, and ignored others.  Insofar my problem is
solved. 

Ciao, Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: Bug?: notmuch-search-show-thread shows several threads; only one containing matching messages

2014-01-26 Thread Gregor Zattler
Hi David,
* David Bremner da...@tethera.net [26. Jan. 2014]:
 Gregor Zattler telegr...@gmx.net writes:
 
 Today I produced another mbox with the very same command but with
 a now larger email corpus freshly indexed with a fresh notmuch.
 The mbox contains (according to mutt) 507 messages in 34 threads.
 One of them is the thread I searched for.

 I grepped for the 7 subjects within the 34 subjects and only 5
 showed up.
 
 I don't know what you mean here. Grepped where? in the raw messages?

With mutt I had a view at the collapsed 7 respective 34 threads.
One then sees the very first E-Mails of a thread and among other
information their subjects.
Via editing I produced two lists with subjects and then searched
each of the 7 in the list with the 34.

 If somebody want's to dig into this: I can provide the two
 mboxes. 

 Disclaimer: Many of the emails which arrived before the problem
 report are not the exact same than then, because since the I
 mangled them with a script.  This should have not changed the
 threading but I cannot be 100% sure.  But if it's important for
 further investigation I'm probably able to reproduce the status
 quo of the email corpus then from my backups.
 
 If it's currently not working then I guess your current corpus should be
 fine. It would probably help to restate what exactly is wrong. There was
 a lot of discussion, and the concrete problem I saw identified (in
 id:874nvcekjk@qmul.ac.uk ) was that certain malformed In-reply-to
 headers were causing unrelated threads to merge.

Yes.  I understood the commit message of the commit you
referenced in the email I answered to, that now notmuch uses
 Reference: headers to do the threading.  I had a quick view at
the References header in the mbox file and none looked
suspicious.

Ciao, Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[Patch v3 2/3] man: partial conversion to pod.

2014-01-18 Thread Gregor Zattler
Hi David, notmuch developers,
* David Bremner  [17. Jan. 2014]:
> A more radical proposal would be to skip generating info and assuming
> everybody can browse html in emacs. That assumption is supposed to
> become less ludicrous in emacs24.4 with the inclusion of "eww".

While html-rendering is somewhat more dynamic (I wished I knew how
to build emacs info documentation with less width) and at least
in graphical environments even provides pictures, it's easier to
navigate info documentation (going "up", notion of TOC,).

I prefer info docs and do read info manuals more closely than any
documentation in any other format.

Ciao; Gregor


Re: [Patch v3 2/3] man: partial conversion to pod.

2014-01-17 Thread Gregor Zattler
Hi David, notmuch developers,
* David Bremner da...@tethera.net [17. Jan. 2014]:
 A more radical proposal would be to skip generating info and assuming
 everybody can browse html in emacs. That assumption is supposed to
 become less ludicrous in emacs24.4 with the inclusion of eww.

While html-rendering is somewhat more dynamic (I wished I knew how
to build emacs info documentation with less width) and at least
in graphical environments even provides pictures, it's easier to
navigate info documentation (going up, notion of TOC,).

I prefer info docs and do read info manuals more closely than any
documentation in any other format.

Ciao; Gregor
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Emacs: how to remove "unread" tag while reading emails

2013-09-18 Thread Gregor Zattler
Dear notmuchers,

I had difficulties to reliably remove the "unread" tag from
messages.  Mostly I page through threads with the space bar and
all is well.  But when the beginning of the thread is already
collapsed and I "jump" in the middle of a message pressing space
bar does not remove the unread tag.  It's only removed when
*entering* the message via space bar from the previous message.
So the last press on space bar in the previous message jumps to
the next message and at the same time removes its unread tag.

This seems strange to me.  I would say the unread tag should be
removed when leaving the message with the last press on space
bar, indicating that one really paged trough the whole message
instead of only seeing the very beginning of it.

What?s the rationale to this behaviour?  Am I missing something?  

Thanks for your attention, gregor


Emacs: how to remove unread tag while reading emails

2013-09-18 Thread Gregor Zattler
Dear notmuchers,

I had difficulties to reliably remove the unread tag from
messages.  Mostly I page through threads with the space bar and
all is well.  But when the beginning of the thread is already
collapsed and I jump in the middle of a message pressing space
bar does not remove the unread tag.  It's only removed when
*entering* the message via space bar from the previous message.
So the last press on space bar in the previous message jumps to
the next message and at the same time removes its unread tag.

This seems strange to me.  I would say the unread tag should be
removed when leaving the message with the last press on space
bar, indicating that one really paged trough the whole message
instead of only seeing the very beginning of it.

What’s the rationale to this behaviour?  Am I missing something?  

Thanks for your attention, gregor
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


  1   2   >