Re: [PATCH 3 of 3] obsolete: explode if metadata contains invalid UTF-8 sequence (API)

2019-01-16 Thread Augie Fackler
On Mon, Jul 16, 2018 at 07:49:11PM +0900, Yuya Nishihara wrote:
> # HG changeset patch
> # User Yuya Nishihara 
> # Date 1531647137 -32400
> #  Sun Jul 15 18:32:17 2018 +0900
> # Node ID d113293cbe05e5026bf0359ace3790ea2f3d2f95
> # Parent  f34a53a6a9a0df0458669d5ce6a55a4af85e9cb6
> obsolete: explode if metadata contains invalid UTF-8 sequence (API)

queued, thanks
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


[PATCH 3 of 3] obsolete: explode if metadata contains invalid UTF-8 sequence (API)

2018-07-16 Thread Yuya Nishihara
# HG changeset patch
# User Yuya Nishihara 
# Date 1531647137 -32400
#  Sun Jul 15 18:32:17 2018 +0900
# Node ID d113293cbe05e5026bf0359ace3790ea2f3d2f95
# Parent  f34a53a6a9a0df0458669d5ce6a55a4af85e9cb6
obsolete: explode if metadata contains invalid UTF-8 sequence (API)

The current metadata API can be a source of bugs since it forces callers to
process encoding conversion by themselves. So let's make it reject bad data
as a last ditch. I assume there's no metadata field which is supposed to store
arbitrary BLOB like transplant_source.

diff --git a/mercurial/obsolete.py b/mercurial/obsolete.py
--- a/mercurial/obsolete.py
+++ b/mercurial/obsolete.py
@@ -80,6 +80,7 @@ from . import (
 obsutil,
 phases,
 policy,
+pycompat,
 util,
 )
 from .utils import dateutil
@@ -600,6 +601,16 @@ class obsstore(object):
 raise ValueError(_('in-marker cycle with %s') % node.hex(prec))
 
 metadata = tuple(sorted(metadata.iteritems()))
+for k, v in metadata:
+try:
+# might be better to reject non-ASCII keys
+k.decode('utf-8')
+v.decode('utf-8')
+except UnicodeDecodeError:
+raise error.ProgrammingError(
+'obsstore metadata must be valid UTF-8 sequence '
+'(key = %r, value = %r)'
+% (pycompat.bytestr(k), pycompat.bytestr(v)))
 
 marker = (bytes(prec), tuple(succs), int(flag), metadata, date, 
parents)
 return bool(self.add(transaction, [marker]))
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel