Re: [PATCH] notmuch-mutt: check that the search cache Maildir is not a real Maildir WAS: Re: Data loss

2023-05-27 Thread Paul Wise
On Sat, 2023-05-27 at 21:10 -0300, David Bremner wrote:

> If possible, please use git-send-email for future series. 

Will do, I forgot about the --cover-letter option.

> This seems to have introduced a warning, and I think there's a bug.
> 
> % notmuch mutt search tag:inbox 
> Argument "new" isn't numeric in array or hash lookup at /usr/bin/notmuch-mutt 
> line 70.
> Argument "cur" isn't numeric in array or hash lookup at /usr/bin/notmuch-mutt 
> line 70.
> Argument "tmp" isn't numeric in array or hash lookup at /usr/bin/notmuch-mutt 
> line 70.
> 
> If I remember correctly you need to use {} for hash references in Perl.

Woops! Thanks, fixed in the v2 patch series.

> It might clearer to use the return value from (a possibly renamed
> check_search_cache (then the -d can be eliminated / folded into the new
> function).

When the checks fail, the Maildir creation and Maildir symlink
additions shouldn't happen either, otherwise Maildirs with real data
may get mail files overwritten with symlinks back to themselves. So
there needs to be a die() or return before the creation and the search
anyway and so checks may as well just die() when the first check fails.

I experimented with returning anyway and it makes things more complex:

File::Find cannot abort the search except via die() and so the calling
code would need to use eval in order to catch the die() and return it.

Threading return values all the way back up to search_action adds lots
of error checks to every function along the way.

The search_action interactive mode doesn't do more than one search
anyway so it would still just need to die() when there is an error.

If the code were to grow a multi-command interactive prompt, that could
use eval to catch and print the die() errors. Such a prompt would need
to do that anyway, because the non-search actions call die() already.

So I think that throwing exceptions via die() is the simpler method.

Since checking that the Maildir is suitable is really a separate action
to clearing the previous search results, I've moved the check function
from the empty function to the search function in the v2 patch series.

-- 
bye,
pabs

https://bonedaddy.net/pabs3/


signature.asc
Description: This is a digitally signed message part
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH] notmuch-mutt: check that the search cache Maildir is not a real Maildir WAS: Re: Data loss

2023-05-27 Thread David Bremner
Paul Wise  writes:

> I think a better option is to detect if the search cache Maildir is
> actually a Maildir and doesn't contain any data that could be lost.
>
> I have attached three patches I created to improve notmuch-mutt safety.

If possible, please use git-send-email for future series. 

> +my %required = map { $_ => 1 } @required;
> +foreach my $d (@contents) {
> +-l "$maildir/$d" and die_dir( $maildir, "contains symlink $d");
> +-d "$maildir/$d" or die_dir( $maildir, "contains non-directory $d");
> +exists($required[$d]) or die_dir( $maildir, "contains directory $d");
> +}
> +}

This seems to have introduced a warning, and I think there's a bug.

% notmuch mutt search tag:inbox 
Argument "new" isn't numeric in array or hash lookup at /usr/bin/notmuch-mutt 
line 70.
Argument "cur" isn't numeric in array or hash lookup at /usr/bin/notmuch-mutt 
line 70.
Argument "tmp" isn't numeric in array or hash lookup at /usr/bin/notmuch-mutt 
line 70.

If I remember correctly you need to use {} for hash references in Perl.

>  
>  # create an empty search cache maildir (if missing) or empty existing one
>  sub empty_search_cache_maildir($) {
>  my ($maildir) = (@_);
> +check_search_cache_maildir($maildir);
>  rmtree($maildir) if (-d $maildir);
>  my $folder = new Mail::Box::Maildir(folder => $maildir,
>   create => 1);

It might clearer to use the return value from (a possibly renamed
check_search_cache (then the -d can be eliminated / folded into the new
function).
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


[PATCH] notmuch-mutt: check that the search cache Maildir is not a real Maildir WAS: Re: Data loss

2023-05-23 Thread Paul Wise
David Bremner wrote:

> Carl Worth  writes:
> 
> > What it could also do is create some placeholder file like
> > .NOTMUCH-MUTT-OUTPUT-DIR when it writes its output. And then it could
> > check for that file's existence before removing a directory.
> 
> Sounds reasonable to me. Would someone like to make a patch for
> notmuch-mutt?

This has the problem that the migration from existing search cache
Maildirs to the new setup needs users to do manual work and that they
or something else could later modify the migrated Maildir by adding
real mail data or subfolders containing real data.

I think a better option is to detect if the search cache Maildir is
actually a Maildir and doesn't contain any data that could be lost.

I have attached three patches I created to improve notmuch-mutt safety.

-- 
bye,
pabs

https://bonedaddy.net/pabs3/
From 22dad230dc368ff09cef2fd1eddd10ba89d41d16 Mon Sep 17 00:00:00 2001
From: Paul Wise 
Date: Wed, 24 May 2023 08:26:38 +0800
Subject: [PATCH 1/3] notmuch-mutt: clarify the empty Maildir function operates
 on search caches

Rename the function and adjust the documentation comment.
---
 contrib/notmuch-mutt/notmuch-mutt | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/contrib/notmuch-mutt/notmuch-mutt b/contrib/notmuch-mutt/notmuch-mutt
index d1e2c084..eef1dea5 100755
--- a/contrib/notmuch-mutt/notmuch-mutt
+++ b/contrib/notmuch-mutt/notmuch-mutt
@@ -27,8 +27,8 @@ $xdg_cache_dir = $ENV{XDG_CACHE_HOME} if $ENV{XDG_CACHE_HOME};
 my $cache_dir = "$xdg_cache_dir/notmuch/mutt";
 
 
-# create an empty maildir (if missing) or empty an existing maildir"
-sub empty_maildir($) {
+# create an empty search cache maildir (if missing) or empty existing one
+sub empty_search_cache_maildir($) {
 my ($maildir) = (@_);
 rmtree($maildir) if (-d $maildir);
 my $folder = new Mail::Box::Maildir(folder => $maildir,
@@ -46,7 +46,7 @@ sub search($$$) {
 push @args, "--duplicate=1" if $remove_dups;
 push @args, $query;
 
-empty_maildir($maildir);
+empty_search_cache_maildir($maildir);
 open my $pipe, '-|', @args or die "Running @args failed: $!\n";
 while (<$pipe>) {
 	chomp;
@@ -121,7 +121,7 @@ sub thread_action($$@) {
 
 my $mid = get_message_id();
 if (! defined $mid) {
-	empty_maildir($results_dir);
+	empty_search_cache_maildir($results_dir);
 	die "notmuch-mutt: cannot find Message-Id, abort.\n";
 }
 my $search_cmd = 'notmuch search --output=threads ' . shell_quote("id:$mid");
-- 
2.40.1

From 4cb6a83e634b4c2dbbaa1099811cf84d65627029 Mon Sep 17 00:00:00 2001
From: Paul Wise 
Date: Wed, 24 May 2023 08:24:02 +0800
Subject: [PATCH 2/3] notmuch-mutt: do not clear search cache Maildir when
 nothing is found

The previous results might be useful to the user but
an empty directory definitely isn't useful.
---
 contrib/notmuch-mutt/notmuch-mutt | 1 -
 1 file changed, 1 deletion(-)

diff --git a/contrib/notmuch-mutt/notmuch-mutt b/contrib/notmuch-mutt/notmuch-mutt
index eef1dea5..9badc59c 100755
--- a/contrib/notmuch-mutt/notmuch-mutt
+++ b/contrib/notmuch-mutt/notmuch-mutt
@@ -121,7 +121,6 @@ sub thread_action($$@) {
 
 my $mid = get_message_id();
 if (! defined $mid) {
-	empty_search_cache_maildir($results_dir);
 	die "notmuch-mutt: cannot find Message-Id, abort.\n";
 }
 my $search_cmd = 'notmuch search --output=threads ' . shell_quote("id:$mid");
-- 
2.40.1

From 46036801970d43c4aee684a46ff0b0e0649e7cfd Mon Sep 17 00:00:00 2001
From: Paul Wise 
Date: Wed, 24 May 2023 06:22:01 +0800
Subject: [PATCH 3/3] notmuch-mutt: check that the search cache Maildir is not
 a real Maildir

This prevents data loss when users configure the search cache Maildir to be a
real Maildir containing their real mail data, since the search cache Maildir
is expected to contain only symlinks to the real mail data.

Prevents: 
---
 contrib/notmuch-mutt/notmuch-mutt | 46 +++
 1 file changed, 46 insertions(+)

diff --git a/contrib/notmuch-mutt/notmuch-mutt b/contrib/notmuch-mutt/notmuch-mutt
index 9badc59c..2569d055 100755
--- a/contrib/notmuch-mutt/notmuch-mutt
+++ b/contrib/notmuch-mutt/notmuch-mutt
@@ -13,6 +13,7 @@ use warnings;
 
 use File::Path;
 use File::Basename;
+use File::Find;
 use Getopt::Long qw(:config no_getopt_compat);
 use Mail::Header;
 use Mail::Box::Maildir;
@@ -26,10 +27,55 @@ my $xdg_cache_dir = "$ENV{HOME}/.cache";
 $xdg_cache_dir = $ENV{XDG_CACHE_HOME} if $ENV{XDG_CACHE_HOME};
 my $cache_dir = "$xdg_cache_dir/notmuch/mutt";
 
+sub die_dir($$) {
+my ($maildir, $error) = @_;
+die "notmuch-mutt: search cache maildir $maildir $error\n".
+"Please ensure that the notmuch-mutt search cache Maildir\n".
+"contains no subfolders or real mail data, only symlinks to mail\n";
+}
+
+sub die_subdir($$$) {
+my ($maildir, $subdir, $error) = @_;
+die_dir($maildir, "subdir $subdir $error");
+}
+
+# check that the search cache maildir is that and not a real maildir
+# otherwise