Gabe Black has submitted this change. (
https://gem5-review.googlesource.com/c/public/gem5/+/33575 )
Change subject: util: Explicitly decode/encode in utf-8.
......................................................................
util: Explicitly decode/encode in utf-8.
The default encoding for python 2 is ascii which can't handle some
characters in, for instance, people's names which have accented letters.
This change explicitly selects the utf-8 encoding which pacifies python
and is mostly equivalent except in these rare cases.
In python 3, the default encoding is utf-8 to begin with, and it's no
longer possible to change it. In this case, explicitly selecting the
encoding is redundant but harmless.
When we support only python 3, then this change can be reverted.
Thanks to Lakin Smith for proposing a related solution and pointing out
some information that led to this one.
Change-Id: I99bd59063c77edd712954ffe90d7de320ade49ea
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/33575
Maintainer: Gabe Black <[email protected]>
Tested-by: kokoro <[email protected]>
Reviewed-by: Lakin Smith <[email protected]>
Reviewed-by: Jason Lowe-Power <[email protected]>
---
M util/git-pre-commit.py
M util/style/repo.py
M util/style/verifiers.py
3 files changed, 6 insertions(+), 6 deletions(-)
Approvals:
Jason Lowe-Power: Looks good to me, approved
Lakin Smith: Looks good to me, approved
Gabe Black: Looks good to me, approved
kokoro: Regressions pass
diff --git a/util/git-pre-commit.py b/util/git-pre-commit.py
index b6d124a..7681b87 100755
--- a/util/git-pre-commit.py
+++ b/util/git-pre-commit.py
@@ -79,7 +79,7 @@
# Show they appropriate object and dump it to a file
status = git.file_from_index(fname)
f = TemporaryFile()
- f.write(status.encode())
+ f.write(status.encode('utf-8'))
verifiers = [ v(ui, opts, base=repo_base) for v in all_verifiers ]
for v in verifiers:
diff --git a/util/style/repo.py b/util/style/repo.py
index f66c16b..b5b4256 100644
--- a/util/style/repo.py
+++ b/util/style/repo.py
@@ -186,7 +186,7 @@
if filter:
cmd += [ "--diff-filter=%s" % filter ]
cmd += [ self.head_revision(), "--" ] + files
- status = subprocess.check_output(cmd).decode().rstrip("\n")
+ status = subprocess.check_output(cmd).decode('utf-8').rstrip("\n")
if status:
return [ f.split("\t") for f in status.split("\n") ]
@@ -195,12 +195,12 @@
def file_from_index(self, name):
return subprocess.check_output(
- [ self.git, "show", ":%s" % (name, ) ]).decode()
+ [ self.git, "show", ":%s" % (name, ) ]).decode('utf-8')
def file_from_head(self, name):
return subprocess.check_output(
[ self.git, "show", "%s:%s" % (self.head_revision(), name) ]) \
- .decode()
+ .decode('utf-8')
def detect_repo(path="."):
"""Auto-detect the revision control system used for a source code
diff --git a/util/style/verifiers.py b/util/style/verifiers.py
index 85f31ce..681efac 100644
--- a/util/style/verifiers.py
+++ b/util/style/verifiers.py
@@ -239,7 +239,7 @@
for num,line in enumerate(fobj):
if num not in regions:
continue
- s_line = line.decode().rstrip('\n')
+ s_line = line.decode('utf-8').rstrip('\n')
if not self.check_line(s_line, language=lang):
if not silent:
self.ui.write("invalid %s in %s:%d\n" % \
@@ -351,7 +351,7 @@
close = True
norm_fname = self.normalize_filename(filename)
- old = [ l.decode().rstrip('\n') for l in fobj ]
+ old = [ l.decode('utf-8').rstrip('\n') for l in fobj ]
if close:
fobj.close()
--
To view, visit https://gem5-review.googlesource.com/c/public/gem5/+/33575
To unsubscribe, or for help writing mail filters, visit
https://gem5-review.googlesource.com/settings
Gerrit-Project: public/gem5
Gerrit-Branch: develop
Gerrit-Change-Id: I99bd59063c77edd712954ffe90d7de320ade49ea
Gerrit-Change-Number: 33575
Gerrit-PatchSet: 2
Gerrit-Owner: Gabe Black <[email protected]>
Gerrit-Reviewer: Gabe Black <[email protected]>
Gerrit-Reviewer: Jason Lowe-Power <[email protected]>
Gerrit-Reviewer: Jason Lowe-Power <[email protected]>
Gerrit-Reviewer: Lakin Smith <[email protected]>
Gerrit-Reviewer: kokoro <[email protected]>
Gerrit-MessageType: merged
_______________________________________________
gem5-dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
