[Patch v2 16/17] notmuch-{dump, restore}.1: document new format options

[Jani Nikula]

On Sat, 24 Nov 2012, david at tethera.net wrote:
> From: David Bremner 
>
> More or less arbitrarily, notmuch-dump.1 gets the more detailed
> description of the format.
> ---
>  man/man1/notmuch-dump.1|   58 +++
>  man/man1/notmuch-restore.1 |   59 
> +++-
>  2 files changed, 111 insertions(+), 6 deletions(-)
>
> diff --git a/man/man1/notmuch-dump.1 b/man/man1/notmuch-dump.1
> index 230deec..9f59905 100644
> --- a/man/man1/notmuch-dump.1
> +++ b/man/man1/notmuch-dump.1
> @@ -5,6 +5,7 @@ notmuch-dump \- creates a plain-text dump of the tags of each 
> message
>  .SH SYNOPSIS
>  
>  .B "notmuch dump"
> +.RB  [ "\-\-format=(sup|batch-tag)"  "] [--]"
>  .RI "[ --output=<" filename "> ] [--]"
>  .RI "[ <" search-term ">...]"
>  
> @@ -19,6 +20,63 @@ recreated from the messages themselves.  The output of 
> notmuch dump is
>  therefore the only critical thing to backup (and much more friendly to
>  incremental backup than the native database files.)
>  
> +.TP 4
> +.B \-\-format=(sup|batch-tag)
> +
> +Notmuch restore supports two plain text dump formats, both with one 
> message-id
> +per line, followed by a list of tags.

"followed by tags" is not entirely accurate for batch-tag.

> +
> +.RS 4
> +.TP 4
> +.B sup
> +
> +The
> +.B sup
> +dump file format is specifically chosen to be
> +compatible with the format of files produced by sup-dump.
> +So if you've previously been using sup for mail, then the
> +.B "notmuch restore"
> +command provides you a way to import all of your tags (or labels as
> +sup calls them).
> +Each line has the following form

Should we deprecate the sup format for new dumps at the same time? Issue
a warning message on dumping too (unless --format is explicitly
specified), telling about the new batch-tag format. I think we should
eventually make batch-tag the default.

> +
> +.RS 4
> +.RI < message-id >
> +.B (
> +.RI < tag "> ..."
> +.B )
> +
> +with zero or more tags are separated by spaces. Note that (malformed)
> +message-ids may contain arbitrary non-null characters. Note also
> +that tags with spaces will not be correctly restored with this format.
> +
> +.RE
> +
> +.RE
> +.RS 4
> +.TP 4
> +.B batch-tag
> +
> +The
> +.B batch-tag
> +dump format is intended to more robust against malformed message-ids
> +and tags containing whitespace or non-\fBascii\fR(7) characters.
> +Each line has the form
> +
> +.RS 4
> +.RI "+<" "encoded-tag" "> " "" "+<" "encoded-tag" "> ... -- " "" " <" 
> encoded-message-id >

Mention that the id: prefix is present in dump and required in restore.

BR,
Jani.

> +
> +where encoded means that every byte not matching the regex
> +.B [A-Za-z0-9+-_@=.:,]
> +is replace by
> +.B %nn
> +where nn is the two digit hex encoding.
> +The astute reader will notice this is a special case of the batch input
> +format for \fBnotmuch-tag\fR(1).
> +
> +.RE
> +
> +
>  With no search terms, a dump of all messages in the database will be
>  generated.  A "--" argument instructs notmuch that the
>  remaining arguments are search terms.
> diff --git a/man/man1/notmuch-restore.1 b/man/man1/notmuch-restore.1
> index 2fa8733..3860829 100644
> --- a/man/man1/notmuch-restore.1
> +++ b/man/man1/notmuch-restore.1
> @@ -6,6 +6,7 @@ notmuch-restore \- restores the tags from the given file (see 
> notmuch dump)
>  
>  .B "notmuch restore"
>  .RB [ "--accumulate" ]
> +.RB [ "--format=(auto|batch-tag|sup)" ]
>  .RI "[ --input=<" filename "> ]"
>  
>  .SH DESCRIPTION
> @@ -15,19 +16,51 @@ Restores the tags from the given file (see
>  
>  The input is read from the given filename, if any, or from stdin.
>  
> -Note: The dump file format is specifically chosen to be
> +
> +Supported options for
> +.B restore
> +include
> +.RS 4
> +.TP 4
> +.B \-\-accumulate
> +
> +The union of the existing and new tags is applied, instead of
> +replacing each message's tags as they are read in from the dump file.
> +
> +.RE
> +.RS 4
> +.TP 4
> +.B \-\-format=(sup|batch-tag|auto)
> +
> +Notmuch restore supports two plain text dump formats, with one message-id
> +per line, and a list of tags.
> +For details of the actual formats, see \fBnotmuch-dump\fR(1).
> +
> +.RS 4
> +.TP 4
> +.B sup
> +
> +The
> +.B sup
> +dump file format is specifically chosen to be
>  compatible with the format of files produced by sup-dump.
>  So if you've previously been using sup for mail, then the
>  .B "notmuch restore"
>  command provides you a way to import all of your tags (or labels as
>  sup calls them).
>  
> -The --accumulate switch causes the union of the existing and new tags to be
> -applied, instead of replacing each message's tags as they are read in from 
> the
> -dump file.
> +.RE
> +.RS 4
> +.TP 4
> +.B batch-tag
>  
> -See \fBnotmuch-search-terms\fR(7)
> -for details of the supported syntax for .
> +The
> +.B batch-tag
> +dump format is intended to more robust against malformed message-ids
> +and tags containing whitespace or 

[Patch v2 16/17] notmuch-{dump, restore}.1: document new format options

[da...@tethera.net]

From: David Bremner 

More or less arbitrarily, notmuch-dump.1 gets the more detailed
description of the format.
---
 man/man1/notmuch-dump.1|   58 +++
 man/man1/notmuch-restore.1 |   59 +++-
 2 files changed, 111 insertions(+), 6 deletions(-)

diff --git a/man/man1/notmuch-dump.1 b/man/man1/notmuch-dump.1
index 230deec..9f59905 100644
--- a/man/man1/notmuch-dump.1
+++ b/man/man1/notmuch-dump.1
@@ -5,6 +5,7 @@ notmuch-dump \- creates a plain-text dump of the tags of each 
message
 .SH SYNOPSIS

 .B "notmuch dump"
+.RB  [ "\-\-format=(sup|batch-tag)"  "] [--]"
 .RI "[ --output=<" filename "> ] [--]"
 .RI "[ <" search-term ">...]"

@@ -19,6 +20,63 @@ recreated from the messages themselves.  The output of 
notmuch dump is
 therefore the only critical thing to backup (and much more friendly to
 incremental backup than the native database files.)

+.TP 4
+.B \-\-format=(sup|batch-tag)
+
+Notmuch restore supports two plain text dump formats, both with one message-id
+per line, followed by a list of tags.
+
+.RS 4
+.TP 4
+.B sup
+
+The
+.B sup
+dump file format is specifically chosen to be
+compatible with the format of files produced by sup-dump.
+So if you've previously been using sup for mail, then the
+.B "notmuch restore"
+command provides you a way to import all of your tags (or labels as
+sup calls them).
+Each line has the following form
+
+.RS 4
+.RI < message-id >
+.B (
+.RI < tag "> ..."
+.B )
+
+with zero or more tags are separated by spaces. Note that (malformed)
+message-ids may contain arbitrary non-null characters. Note also
+that tags with spaces will not be correctly restored with this format.
+
+.RE
+
+.RE
+.RS 4
+.TP 4
+.B batch-tag
+
+The
+.B batch-tag
+dump format is intended to more robust against malformed message-ids
+and tags containing whitespace or non-\fBascii\fR(7) characters.
+Each line has the form
+
+.RS 4
+.RI "+<" "encoded-tag" "> " "" "+<" "encoded-tag" "> ... -- " "" " <" 
encoded-message-id >
+
+where encoded means that every byte not matching the regex
+.B [A-Za-z0-9+-_@=.:,]
+is replace by
+.B %nn
+where nn is the two digit hex encoding.
+The astute reader will notice this is a special case of the batch input
+format for \fBnotmuch-tag\fR(1).
+
+.RE
+
+
 With no search terms, a dump of all messages in the database will be
 generated.  A "--" argument instructs notmuch that the
 remaining arguments are search terms.
diff --git a/man/man1/notmuch-restore.1 b/man/man1/notmuch-restore.1
index 2fa8733..3860829 100644
--- a/man/man1/notmuch-restore.1
+++ b/man/man1/notmuch-restore.1
@@ -6,6 +6,7 @@ notmuch-restore \- restores the tags from the given file (see 
notmuch dump)

 .B "notmuch restore"
 .RB [ "--accumulate" ]
+.RB [ "--format=(auto|batch-tag|sup)" ]
 .RI "[ --input=<" filename "> ]"

 .SH DESCRIPTION
@@ -15,19 +16,51 @@ Restores the tags from the given file (see

 The input is read from the given filename, if any, or from stdin.

-Note: The dump file format is specifically chosen to be
+
+Supported options for
+.B restore
+include
+.RS 4
+.TP 4
+.B \-\-accumulate
+
+The union of the existing and new tags is applied, instead of
+replacing each message's tags as they are read in from the dump file.
+
+.RE
+.RS 4
+.TP 4
+.B \-\-format=(sup|batch-tag|auto)
+
+Notmuch restore supports two plain text dump formats, with one message-id
+per line, and a list of tags.
+For details of the actual formats, see \fBnotmuch-dump\fR(1).
+
+.RS 4
+.TP 4
+.B sup
+
+The
+.B sup
+dump file format is specifically chosen to be
 compatible with the format of files produced by sup-dump.
 So if you've previously been using sup for mail, then the
 .B "notmuch restore"
 command provides you a way to import all of your tags (or labels as
 sup calls them).

-The --accumulate switch causes the union of the existing and new tags to be
-applied, instead of replacing each message's tags as they are read in from the
-dump file.
+.RE
+.RS 4
+.TP 4
+.B batch-tag

-See \fBnotmuch-search-terms\fR(7)
-for details of the supported syntax for .
+The
+.B batch-tag
+dump format is intended to more robust against malformed message-ids
+and tags containing whitespace or non-\fBascii\fR(7) characters.  This
+format hex-escapes all characters those outside of a small character
+set, intended to be suitable for e.g. pathnames in most UNIX-like
+systems.

 .B "notmuch restore"
 updates the maildir flags according to tag changes if the
@@ -36,6 +69,20 @@ configuration option is enabled. See \fBnotmuch-config\fR(1) 
for
 details.

 .RE
+
+.RS 4
+.TP 4
+.B auto
+
+This option (the default) tries to guess the format from the
+input. For correctly formed input in either supported format, this
+heuristic, based the fact that batch-tag format contains no parentheses,
+should be accurate.
+
+.RE
+
+.RE
+
 .SH SEE ALSO

 \fBnotmuch\fR(1), \fBnotmuch-config\fR(1), \fBnotmuch-count\fR(1),
-- 
1.7.10.4

[Patch v2 16/17] notmuch-{dump, restore}.1: document new format options

[david]

From: David Bremner brem...@debian.org

More or less arbitrarily, notmuch-dump.1 gets the more detailed
description of the format.
---
 man/man1/notmuch-dump.1|   58 +++
 man/man1/notmuch-restore.1 |   59 +++-
 2 files changed, 111 insertions(+), 6 deletions(-)

diff --git a/man/man1/notmuch-dump.1 b/man/man1/notmuch-dump.1
index 230deec..9f59905 100644
--- a/man/man1/notmuch-dump.1
+++ b/man/man1/notmuch-dump.1
@@ -5,6 +5,7 @@ notmuch-dump \- creates a plain-text dump of the tags of each 
message
 .SH SYNOPSIS
 
 .B notmuch dump
+.RB  [ \-\-format=(sup|batch-tag)  ] [--]
 .RI [ --output= filename  ] [--]
 .RI [  search-term ...]
 
@@ -19,6 +20,63 @@ recreated from the messages themselves.  The output of 
notmuch dump is
 therefore the only critical thing to backup (and much more friendly to
 incremental backup than the native database files.)
 
+.TP 4
+.B \-\-format=(sup|batch-tag)
+
+Notmuch restore supports two plain text dump formats, both with one message-id
+per line, followed by a list of tags.
+
+.RS 4
+.TP 4
+.B sup
+
+The
+.B sup
+dump file format is specifically chosen to be
+compatible with the format of files produced by sup-dump.
+So if you've previously been using sup for mail, then the
+.B notmuch restore
+command provides you a way to import all of your tags (or labels as
+sup calls them).
+Each line has the following form
+
+.RS 4
+.RI  message-id 
+.B (
+.RI  tag  ...
+.B )
+
+with zero or more tags are separated by spaces. Note that (malformed)
+message-ids may contain arbitrary non-null characters. Note also
+that tags with spaces will not be correctly restored with this format.
+
+.RE
+
+.RE
+.RS 4
+.TP 4
+.B batch-tag
+
+The
+.B batch-tag
+dump format is intended to more robust against malformed message-ids
+and tags containing whitespace or non-\fBascii\fR(7) characters.
+Each line has the form
+
+.RS 4
+.RI + encoded-tag+ encoded-tag  ... -- 
encoded-message-id 
+
+where encoded means that every byte not matching the regex
+.B [A-Za-z0-9+-_@=.:,]
+is replace by
+.B %nn
+where nn is the two digit hex encoding.
+The astute reader will notice this is a special case of the batch input
+format for \fBnotmuch-tag\fR(1).
+
+.RE
+
+
 With no search terms, a dump of all messages in the database will be
 generated.  A -- argument instructs notmuch that the
 remaining arguments are search terms.
diff --git a/man/man1/notmuch-restore.1 b/man/man1/notmuch-restore.1
index 2fa8733..3860829 100644
--- a/man/man1/notmuch-restore.1
+++ b/man/man1/notmuch-restore.1
@@ -6,6 +6,7 @@ notmuch-restore \- restores the tags from the given file (see 
notmuch dump)
 
 .B notmuch restore
 .RB [ --accumulate ]
+.RB [ --format=(auto|batch-tag|sup) ]
 .RI [ --input= filename  ]
 
 .SH DESCRIPTION
@@ -15,19 +16,51 @@ Restores the tags from the given file (see
 
 The input is read from the given filename, if any, or from stdin.
 
-Note: The dump file format is specifically chosen to be
+
+Supported options for
+.B restore
+include
+.RS 4
+.TP 4
+.B \-\-accumulate
+
+The union of the existing and new tags is applied, instead of
+replacing each message's tags as they are read in from the dump file.
+
+.RE
+.RS 4
+.TP 4
+.B \-\-format=(sup|batch-tag|auto)
+
+Notmuch restore supports two plain text dump formats, with one message-id
+per line, and a list of tags.
+For details of the actual formats, see \fBnotmuch-dump\fR(1).
+
+.RS 4
+.TP 4
+.B sup
+
+The
+.B sup
+dump file format is specifically chosen to be
 compatible with the format of files produced by sup-dump.
 So if you've previously been using sup for mail, then the
 .B notmuch restore
 command provides you a way to import all of your tags (or labels as
 sup calls them).
 
-The --accumulate switch causes the union of the existing and new tags to be
-applied, instead of replacing each message's tags as they are read in from the
-dump file.
+.RE
+.RS 4
+.TP 4
+.B batch-tag
 
-See \fBnotmuch-search-terms\fR(7)
-for details of the supported syntax for search-terms.
+The
+.B batch-tag
+dump format is intended to more robust against malformed message-ids
+and tags containing whitespace or non-\fBascii\fR(7) characters.  This
+format hex-escapes all characters those outside of a small character
+set, intended to be suitable for e.g. pathnames in most UNIX-like
+systems.
 
 .B notmuch restore
 updates the maildir flags according to tag changes if the
@@ -36,6 +69,20 @@ configuration option is enabled. See \fBnotmuch-config\fR(1) 
for
 details.
 
 .RE
+
+.RS 4
+.TP 4
+.B auto
+
+This option (the default) tries to guess the format from the
+input. For correctly formed input in either supported format, this
+heuristic, based the fact that batch-tag format contains no parentheses,
+should be accurate.
+
+.RE
+
+.RE
+
 .SH SEE ALSO
 
 \fBnotmuch\fR(1), \fBnotmuch-config\fR(1), \fBnotmuch-count\fR(1),
-- 
1.7.10.4

___
notmuch mailing list