Hello community,
here is the log from the commit of package python-astor for openSUSE:Factory
checked in at 2019-06-03 18:56:38
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Comparing /work/SRC/openSUSE:Factory/python-astor (Old)
and /work/SRC/openSUSE:Factory/.python-astor.new.5148 (New)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "python-astor"
Mon Jun 3 18:56:38 2019 rev:2 rq:707079 version:0.8
Changes:
--------
--- /work/SRC/openSUSE:Factory/python-astor/python-astor.changes
2018-11-12 09:41:47.529157517 +0100
+++ /work/SRC/openSUSE:Factory/.python-astor.new.5148/python-astor.changes
2019-06-03 18:56:40.876399680 +0200
@@ -1,0 +2,23 @@
+Mon Jun 3 08:18:07 UTC 2019 - pgaj...@suse.com
+
+- version update to 0.8
+ * Support ``ast.Constant`` nodes being emitted by Python 3.8 (and initially
+ created in Python 3.6).
+ (Reported and fixed by Chris Rink in `Issue 120`_ and `PR 121`_.)
+ * Support Python 3.8's assignment expressions.
+ (Reported and fixed by Kodi Arfer in `Issue 126`_ and `PR 134`_.)
+ * Support Python 3.8's f-string debugging syntax.
+ (Reported and fixed by Batuhan Taskaya in `Issue 138`_ and `PR 139`_.)
+ * :func:`astor.to_source` now has a *source_generator_class* parameter to
+ customize source code generation.
+ (Reported and fixed by matham in `Issue 113`_ and `PR 114`_.)
+ * The :class:`~SourceGenerator` class can now be imported from the
+ :mod:`astor` package directly. Previously, the ``astor.code_gen``
+ submodule was needed to be imported.
+ * Support Python 3.8's positional only arguments. See :pep:`570` for
+ more details.
+ (Reported and fixed by Batuhan Taskaya in `Issue 142`_ and `PR 143`_.)
+ * bug fixes, see changelog.rst
+- run the testsuite
+
+-------------------------------------------------------------------
Old:
----
astor-0.7.1.tar.gz
New:
----
astor-0.8.tar.gz
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Other differences:
------------------
++++++ python-astor.spec ++++++
--- /var/tmp/diff_new_pack.8ULWju/_old 2019-06-03 18:56:41.852399318 +0200
+++ /var/tmp/diff_new_pack.8ULWju/_new 2019-06-03 18:56:41.856399317 +0200
@@ -1,7 +1,7 @@
#
# spec file for package python-astor
#
-# Copyright (c) 2018 SUSE LINUX GmbH, Nuernberg, Germany.
+# Copyright (c) 2019 SUSE LINUX GmbH, Nuernberg, Germany.
#
# All modifications and additions to the file contributed by third parties
# remain the property of their copyright owners, unless otherwise agreed
@@ -19,7 +19,7 @@
%{?!python_module:%define python_module() python-%{**} python3-%{**}}
Name: python-astor
-Version: 0.7.1
+Version: 0.8
Release: 0
Summary: Read/rewrite/write Python ASTs
License: BSD-3-Clause
@@ -27,6 +27,10 @@
Url: https://github.com/berkerpeksag/astor
Source:
https://github.com/berkerpeksag/astor/archive/%{version}.tar.gz#/astor-%{version}.tar.gz
BuildRequires: %{python_module setuptools}
+# SECTION test requirements
+BuildRequires: %{python_module nose}
+BuildRequires: python2-unittest2
+# /SECTION
BuildRequires: fdupes
BuildRequires: python-rpm-macros
BuildArch: noarch
@@ -70,8 +74,12 @@
%python_expand %fdupes %{buildroot}%{$python_sitelib}
# fix executeable bits
%python_expand chmod 755 %{buildroot}%{$python_sitelib}/astor/rtrip.py
+
+%check
+%python_expand nosetests-%{$python_bin_suffix} -v
+
%files %{python_files}
-%doc AUTHORS README.rst
+%doc AUTHORS README.rst docs/*.rst
%license LICENSE
%{python_sitelib}/*
++++++ astor-0.7.1.tar.gz -> astor-0.8.tar.gz ++++++
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/astor-0.7.1/.coveragerc new/astor-0.8/.coveragerc
--- old/astor-0.7.1/.coveragerc 1970-01-01 01:00:00.000000000 +0100
+++ new/astor-0.8/.coveragerc 2019-05-19 18:51:56.000000000 +0200
@@ -0,0 +1,8 @@
+[run]
+branch = True
+omit =
+ # omit the codegen because of deprecation
+ astor/codegen.py
+ .tox/*
+[report]
+show_missing = True
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/astor-0.7.1/.gitignore new/astor-0.8/.gitignore
--- old/astor-0.7.1/.gitignore 2018-07-06 10:03:51.000000000 +0200
+++ new/astor-0.8/.gitignore 2019-05-19 18:51:56.000000000 +0200
@@ -43,3 +43,5 @@
# Sphinx documentation
docs/_build/
+
+all_expr_*.py
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/astor-0.7.1/.travis.yml new/astor-0.8/.travis.yml
--- old/astor-0.7.1/.travis.yml 2018-07-06 10:03:51.000000000 +0200
+++ new/astor-0.8/.travis.yml 2019-05-19 18:51:56.000000000 +0200
@@ -7,6 +7,16 @@
- 3.6
- pypy
- pypy3.5
+matrix:
+ include:
+ # See https://github.com/travis-ci/travis-ci/issues/9069
+ # for more information.
+ - python: 3.7
+ sudo: required
+ dist: xenial
+ - python: 3.8-dev
+ sudo: required
+ dist: xenial
cache: pip
install:
- pip install tox-travis
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/astor-0.7.1/AUTHORS new/astor-0.8/AUTHORS
--- old/astor-0.7.1/AUTHORS 2018-07-06 10:03:51.000000000 +0200
+++ new/astor-0.8/AUTHORS 2019-05-19 18:51:56.000000000 +0200
@@ -14,3 +14,6 @@
* Lenny Truong <leonardtru...@protonmail.com>
* Radomír Bosák <radomir.bo...@gmail.com>
* Kodi Arfer <g...@arfer.net>
+* Felix Yan <felixonm...@archlinux.org>
+* Chris Rink <chrisrin...@gmail.com>
+* Batuhan Taskaya <batuhanosmantask...@gmail.com>
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/astor-0.7.1/Makefile new/astor-0.8/Makefile
--- old/astor-0.7.1/Makefile 2018-07-06 10:03:51.000000000 +0200
+++ new/astor-0.8/Makefile 2019-05-19 18:51:56.000000000 +0200
@@ -5,19 +5,13 @@
python setup.py sdist bdist_wheel
twine upload dist/*
-register:
- python setup.py sdist register -r pypi
-
# Test it via `pip install -i https://test.pypi.org/simple/ <project_name>`
test-release:
- python setup.py sdist bdist_wheel upload -r test
-
-test-register:
- python setup.py sdist register -r test
+ twine upload -r test dist/*
clean:
find . -name "*.pyc" -exec rm {} \;
rm -rf *.egg-info
rm -rf build/ dist/ __pycache__/
-.PHONY: clean register release
+.PHONY: clean release
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/astor-0.7.1/astor/__init__.py
new/astor-0.8/astor/__init__.py
--- old/astor-0.7.1/astor/__init__.py 2018-07-06 10:03:51.000000000 +0200
+++ new/astor-0.8/astor/__init__.py 2019-05-19 18:51:56.000000000 +0200
@@ -11,7 +11,7 @@
import warnings
-from .code_gen import to_source # NOQA
+from .code_gen import SourceGenerator, to_source # NOQA
from .node_util import iter_node, strip_tree, dump_tree # NOQA
from .node_util import ExplicitNodeVisitor # NOQA
from .file_util import CodeToAst, code_to_ast # NOQA
@@ -19,7 +19,7 @@
from .op_util import symbol_data # NOQA
from .tree_walk import TreeWalk # NOQA
-__version__ = '0.7.1'
+__version__ = '0.8.0'
parse_file = code_to_ast.parse_file
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/astor-0.7.1/astor/code_gen.py
new/astor-0.8/astor/code_gen.py
--- old/astor-0.7.1/astor/code_gen.py 2018-07-06 10:03:51.000000000 +0200
+++ new/astor-0.8/astor/code_gen.py 2019-05-19 18:51:56.000000000 +0200
@@ -23,12 +23,13 @@
from .op_util import get_op_symbol, get_op_precedence, Precedence
from .node_util import ExplicitNodeVisitor
-from .string_repr import pretty_string, string_triplequote_repr
+from .string_repr import pretty_string
from .source_repr import pretty_source
def to_source(node, indent_with=' ' * 4, add_line_information=False,
- pretty_string=pretty_string, pretty_source=pretty_source):
+ pretty_string=pretty_string, pretty_source=pretty_source,
+ source_generator_class=None):
"""This function can convert a node tree back into python sourcecode.
This is useful for debugging purposes, especially if you're dealing with
custom asts not generated by python itself.
@@ -46,9 +47,18 @@
of the nodes are added to the output. This can be used to spot wrong line
number information of statement nodes.
+ `source_generator_class` defaults to `SourceGenerator`, and specifies the
+ class that will be instantiated and used to generate the source code.
+
"""
- generator = SourceGenerator(indent_with, add_line_information,
- pretty_string)
+ if source_generator_class is None:
+ source_generator_class = SourceGenerator
+ elif not isinstance(source_generator_class, SourceGenerator):
+ raise TypeError('source_generator_class should be a subclass of
SourceGenerator')
+ elif not callable(source_generator_class):
+ raise TypeError('source_generator_class should be a callable')
+ generator = source_generator_class(
+ indent_with, add_line_information, pretty_string)
generator.visit(node)
generator.result.append('\n')
if set(generator.result[0]) == set('\n'):
@@ -155,7 +165,6 @@
AST = ast.AST
visit = self.visit
- newline = self.newline
result = self.result
append = result.append
@@ -168,8 +177,6 @@
visit(item)
elif callable(item):
item()
- elif item == '\n':
- newline()
else:
if self.new_lines:
append('\n' * self.new_lines)
@@ -220,7 +227,7 @@
def else_body(self, elsewhat):
if elsewhat:
- self.write('\n', 'else:')
+ self.write(self.newline, 'else:')
self.body(elsewhat)
def body_or_else(self, node):
@@ -243,7 +250,14 @@
self.write(write_comma, arg)
self.conditional_write('=', default)
- loop_args(node.args, node.defaults)
+ posonlyargs = getattr(node, 'posonlyargs', [])
+ offset = 0
+ if posonlyargs:
+ offset += len(node.defaults) - len(node.args)
+ loop_args(posonlyargs, node.defaults[:offset])
+ self.write(write_comma, '/')
+
+ loop_args(node.args, node.defaults[offset:])
self.conditional_write(write_comma, '*', node.vararg)
kwonlyargs = self.get_kwonlyargs(node)
@@ -359,7 +373,7 @@
if len(else_) == 1 and isinstance(else_[0], ast.If):
node = else_[0]
set_precedence(node, node.test)
- self.write('\n', 'elif ', node.test, ':')
+ self.write(self.newline, 'elif ', node.test, ':')
self.body(node.body)
else:
self.else_body(else_)
@@ -400,8 +414,9 @@
self.write(node.context_expr)
self.conditional_write(' as ', node.optional_vars)
+ # deprecated in Python 3.8
def visit_NameConstant(self, node):
- self.write(str(node.value))
+ self.write(repr(node.value))
def visit_Pass(self, node):
self.statement(node, 'pass')
@@ -530,11 +545,25 @@
def visit_Name(self, node):
self.write(node.id)
- def visit_JoinedStr(self, node):
- self.visit_Str(node, True)
+ # ast.Constant is new in Python 3.6 and it replaces ast.Bytes,
+ # ast.Ellipsis, ast.NameConstant, ast.Num, ast.Str in Python 3.8
+ def visit_Constant(self, node):
+ value = node.value
+
+ if isinstance(value, (int, float, complex)):
+ with self.delimit(node):
+ self._handle_numeric_constant(value)
+ elif isinstance(value, str):
+ self._handle_string_constant(node, node.value)
+ elif value is Ellipsis:
+ self.write('...')
+ else:
+ self.write(repr(value))
- def visit_Str(self, node, is_joined=False):
+ def visit_JoinedStr(self, node):
+ self._handle_string_constant(node, None, is_joined=True)
+ def _handle_string_constant(self, node, value, is_joined=False):
# embedded is used to control when we might want
# to use a triple-quoted string. We determine
# if we are in an assignment and/or in an expression
@@ -557,8 +586,9 @@
current_line[0] = current_line[0][str_index:]
current_line = ''.join(current_line)
- if is_joined:
+ has_ast_constant = sys.version_info >= (3, 6)
+ if is_joined:
# Handle new f-strings. This is a bit complicated, because
# the tree can contain subnodes that recurse back to JoinedStr
# subnodes...
@@ -566,15 +596,22 @@
def recurse(node):
for value in node.values:
if isinstance(value, ast.Str):
- self.write(value.s)
+ # Double up braces to escape them.
+ self.write(value.s.replace('{', '{{').replace('}',
'}}'))
elif isinstance(value, ast.FormattedValue):
with self.delimit('{}'):
- self.visit(value.value)
+ # expr_text used for f-string debugging syntax.
+ if getattr(value, 'expr_text', None):
+ self.write(value.expr_text)
+ else:
+ self.visit(value.value)
if value.conversion != -1:
self.write('!%s' % chr(value.conversion))
if value.format_spec is not None:
self.write(':')
recurse(value.format_spec)
+ elif has_ast_constant and isinstance(value, ast.Constant):
+ self.write(value.value)
else:
kind = type(value).__name__
assert False, 'Invalid node %s inside JoinedStr' % kind
@@ -590,13 +627,17 @@
uni_lit = False # No formatted byte strings
else:
- mystr = node.s
+ assert value is not None, "Node value cannot be None"
+ mystr = value
uni_lit = self.using_unicode_literals
mystr = self.pretty_string(mystr, embedded, current_line, uni_lit)
if is_joined:
mystr = 'f' + mystr
+ elif getattr(node, 'kind', False):
+ # Constant.kind is a Python 3.8 addition.
+ mystr = node.kind + mystr
self.write(mystr)
@@ -604,39 +645,56 @@
if lf:
self.colinfo = len(result) - 1, lf
+ # deprecated in Python 3.8
+ def visit_Str(self, node):
+ self._handle_string_constant(node, node.s)
+
+ # deprecated in Python 3.8
def visit_Bytes(self, node):
self.write(repr(node.s))
- def visit_Num(self, node,
- # constants
- new=sys.version_info >= (3, 0)):
- with self.delimit(node) as delimiters:
- x = node.n
+ def _handle_numeric_constant(self, value):
+ x = value
- def part(p, imaginary):
- # Represent infinity as 1e1000 and NaN as 1e1000-1e1000.
- s = 'j' if imaginary else ''
+ def part(p, imaginary):
+ # Represent infinity as 1e1000 and NaN as 1e1000-1e1000.
+ s = 'j' if imaginary else ''
+ try:
if math.isinf(p):
if p < 0:
return '-1e1000' + s
return '1e1000' + s
if math.isnan(p):
return '(1e1000%s-1e1000%s)' % (s, s)
- return repr(p) + s
-
- real = part(x.real if isinstance(x, complex) else x,
imaginary=False)
- if isinstance(x, complex):
- imag = part(x.imag, imaginary=True)
- if x.real == 0:
- s = imag
- elif x.imag == 0:
- s = '(%s+0j)' % real
- else:
- # x has nonzero real and imaginary parts.
- s = '(%s%s%s)' % (real, ['+', ''][imag.startswith('-')],
imag)
+ except OverflowError:
+ # math.isinf will raise this when given an integer
+ # that's too large to convert to a float.
+ pass
+ return repr(p) + s
+
+ real = part(x.real if isinstance(x, complex) else x, imaginary=False)
+ if isinstance(x, complex):
+ imag = part(x.imag, imaginary=True)
+ if x.real == 0:
+ s = imag
+ elif x.imag == 0:
+ s = '(%s+0j)' % real
else:
- s = real
- self.write(s)
+ # x has nonzero real and imaginary parts.
+ s = '(%s%s%s)' % (real, ['+', ''][imag.startswith('-')], imag)
+ else:
+ s = real
+ self.write(s)
+
+ def visit_Num(self, node,
+ # constants
+ new=sys.version_info >= (3, 0)):
+ with self.delimit(node) as delimiters:
+ self._handle_numeric_constant(node.n)
+
+ # We can leave the delimiters handling in visit_Num
+ # since this is meant to handle a Python 2.x specific
+ # issue and ast.Constant exists only in 3.6+
# The Python 2.x compiler merges a unary minus
# with a number. This is a premature optimization
@@ -703,6 +761,22 @@
for op, right in zip(node.ops, node.comparators):
self.write(get_op_symbol(op, ' %s '), right)
+ # assignment expressions; new for Python 3.8
+ def visit_NamedExpr(self, node):
+ with self.delimit(node) as delimiters:
+ p = delimiters.p
+ set_precedence(p, node.target)
+ set_precedence(p + 1, node.value)
+ # Python is picky about delimiters for assignment
+ # expressions: it requires at least one pair in any
+ # statement that uses an assignment expression, even
+ # when not necessary according to the precedence
+ # rules. We address this with the kludge of forcing a
+ # pair of parentheses around every assignment
+ # expression.
+ delimiters.discard = False
+ self.write(node.target, ' := ', node.value)
+
def visit_UnaryOp(self, node):
with self.delimit(node, node.op) as delimiters:
set_precedence(delimiters.p, node.operand)
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/astor-0.7.1/astor/op_util.py
new/astor-0.8/astor/op_util.py
--- old/astor-0.7.1/astor/op_util.py 2018-07-06 10:03:51.000000000 +0200
+++ new/astor-0.8/astor/op_util.py 2019-05-19 18:51:56.000000000 +0200
@@ -36,6 +36,7 @@
Tuple 0
Comma 1
+ NamedExpr 1
Assert 0
Raise 0
call_one_arg 1
@@ -78,6 +79,7 @@
Pow ** 1
Await 1
Num 1
+ Constant 1
"""
op_data = [x.split() for x in op_data.splitlines()]
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/astor-0.7.1/astor/rtrip.py
new/astor-0.8/astor/rtrip.py
--- old/astor-0.7.1/astor/rtrip.py 2018-07-06 10:03:51.000000000 +0200
+++ new/astor-0.8/astor/rtrip.py 2019-05-19 18:51:56.000000000 +0200
@@ -81,7 +81,7 @@
try:
dsttxt = to_source(srcast)
- except:
+ except Exception:
if not ignore_exceptions:
raise
dsttxt = ''
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/astor-0.7.1/astor/source_repr.py
new/astor-0.8/astor/source_repr.py
--- old/astor-0.7.1/astor/source_repr.py 2018-07-06 10:03:51.000000000
+0200
+++ new/astor-0.8/astor/source_repr.py 2019-05-19 18:51:56.000000000 +0200
@@ -71,7 +71,7 @@
indentation = line[0]
lenfirst = len(indentation)
- indent = lenfirst - len(indentation.strip())
+ indent = lenfirst - len(indentation.lstrip())
assert indent in (0, lenfirst)
indentation = line.pop(0) if indent else ''
@@ -101,8 +101,8 @@
pos = indent + count(first)
indentation += ' '
indent += 4
- if indent >= maxline/2:
- maxline = maxline/2 + indent
+ if indent >= maxline / 2:
+ maxline = maxline / 2 + indent
for sg, nsg in zip(splittable, unsplittable[1:]):
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/astor-0.7.1/astor/string_repr.py
new/astor-0.8/astor/string_repr.py
--- old/astor-0.7.1/astor/string_repr.py 2018-07-06 10:03:51.000000000
+0200
+++ new/astor-0.8/astor/string_repr.py 2019-05-19 18:51:56.000000000 +0200
@@ -107,6 +107,6 @@
try:
if eval(fancy) == s and '\r' not in fancy:
return fancy
- except:
+ except Exception:
pass
return default
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/astor-0.7.1/astor/tree_walk.py
new/astor-0.8/astor/tree_walk.py
--- old/astor-0.7.1/astor/tree_walk.py 2018-07-06 10:03:51.000000000 +0200
+++ new/astor-0.8/astor/tree_walk.py 2019-05-19 18:51:56.000000000 +0200
@@ -30,6 +30,9 @@
if base not in newbases:
newdict.update(vars(base))
newdict.update(clsdict)
+ # These are class-bound, we should let Python recreate them.
+ newdict.pop('__dict__', None)
+ newdict.pop('__weakref__', None)
# Delegate the real work to type
return type.__new__(clstype, name, newbases, newdict)
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/astor-0.7.1/docs/changelog.rst
new/astor-0.8/docs/changelog.rst
--- old/astor-0.7.1/docs/changelog.rst 2018-07-06 10:03:51.000000000 +0200
+++ new/astor-0.8/docs/changelog.rst 2019-05-19 18:51:56.000000000 +0200
@@ -2,18 +2,81 @@
Release Notes
=============
-0.8.0 - in development
-----------------------
+0.8.0 - 2019-05-19
+------------------
New features
~~~~~~~~~~~~
-*
+* Support ``ast.Constant`` nodes being emitted by Python 3.8 (and initially
+ created in Python 3.6).
+ (Reported and fixed by Chris Rink in `Issue 120`_ and `PR 121`_.)
+
+.. _`Issue 120`: https://github.com/berkerpeksag/astor/issues/120
+.. _`PR 121`: https://github.com/berkerpeksag/astor/pull/121
+
+* Support Python 3.8's assignment expressions.
+ (Reported and fixed by Kodi Arfer in `Issue 126`_ and `PR 134`_.)
+
+.. _`Issue 126`: https://github.com/berkerpeksag/astor/issues/126
+.. _`PR 134`: https://github.com/berkerpeksag/astor/pull/134
+
+* Support Python 3.8's f-string debugging syntax.
+ (Reported and fixed by Batuhan Taskaya in `Issue 138`_ and `PR 139`_.)
+
+.. _`Issue 138`: https://github.com/berkerpeksag/astor/issues/138
+.. _`PR 139`: https://github.com/berkerpeksag/astor/pull/139
+
+* :func:`astor.to_source` now has a *source_generator_class* parameter to
+ customize source code generation.
+ (Reported and fixed by matham in `Issue 113`_ and `PR 114`_.)
+
+.. _`Issue 113`: https://github.com/berkerpeksag/astor/issues/113
+.. _`PR 114`: https://github.com/berkerpeksag/astor/pull/114
+
+* The :class:`~SourceGenerator` class can now be imported from the
+ :mod:`astor` package directly. Previously, the ``astor.code_gen``
+ submodule was needed to be imported.
+
+* Support Python 3.8's positional only arguments. See :pep:`570` for
+ more details.
+ (Reported and fixed by Batuhan Taskaya in `Issue 142`_ and `PR 143`_.)
+
+.. _`Issue 142`: https://github.com/berkerpeksag/astor/issues/142
+.. _`PR 143`: https://github.com/berkerpeksag/astor/pull/143
Bug fixes
~~~~~~~~~
-*
+* Fix string parsing when there is a newline inside an f-string. (Reported by
+ Adam Cécile in `Issue 119`_ and fixed by Felix Yan in `PR 123`_.)
+
+* Fixed code generation with escaped braces in f-strings.
+ (Reported by Felix Yan in `Issue 124`_ and fixed by Kodi Arfer in `PR 125`_.)
+
+.. _`Issue 119`: https://github.com/berkerpeksag/astor/issues/119
+.. _`PR 123`: https://github.com/berkerpeksag/astor/pull/123
+.. _`Issue 124`: https://github.com/berkerpeksag/astor/issues/124
+.. _`PR 125`: https://github.com/berkerpeksag/astor/pull/125
+
+* Fixed code generation with attributes of integer literals, and
+ with ``u``-prefixed string literals.
+ (Fixed by Kodi Arfer in `PR 133`_.)
+
+.. _`PR 133`: https://github.com/berkerpeksag/astor/pull/133
+
+* Fixed code generation with very large integers.
+ (Reported by Adam Kucz in `Issue 127`_ and fixed by Kodi Arfer in `PR 130`_.)
+
+.. _`Issue 127`: https://github.com/berkerpeksag/astor/issues/127
+.. _`PR 130`: https://github.com/berkerpeksag/astor/pull/130
+
+* Fixed :class:`astor.tree_walk.TreeWalk` when attempting to access attributes
+ created by Python's type system (such as ``__dict__`` and ``__weakref__``)
+ (Reported and fixed by esupoff in `Issue 136`_ and `PR 137`_.)
+
+.. _`Issue 136`: https://github.com/berkerpeksag/astor/issues/136
+.. _`PR 137`: https://github.com/berkerpeksag/astor/pull/137
0.7.1 - 2018-07-06
------------------
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/astor-0.7.1/docs/index.rst
new/astor-0.8/docs/index.rst
--- old/astor-0.7.1/docs/index.rst 2018-07-06 10:03:51.000000000 +0200
+++ new/astor-0.8/docs/index.rst 2019-05-19 18:51:56.000000000 +0200
@@ -129,7 +129,8 @@
*********
.. function:: to_source(source, indent_with=' ' * 4, \
- add_line_information=False)
+ add_line_information=False,
+ source_generator_class=astor.SourceGenerator)
Convert a node tree back into Python source code.
@@ -140,6 +141,13 @@
of the nodes are added to the output. This can be used to spot wrong line
number information of statement nodes.
+ *source_generator_class* defaults to :class:`astor.SourceGenerator`, and
+ specifies the class that will be instantiated and used to generate the
+ source code.
+
+ .. versionchanged:: 0.8
+ *source_generator_class* was added.
+
.. function:: codetoast
.. function:: code_to_ast(codeobj)
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/astor-0.7.1/requirements-tox.txt
new/astor-0.8/requirements-tox.txt
--- old/astor-0.7.1/requirements-tox.txt 2018-07-06 10:03:51.000000000
+0200
+++ new/astor-0.8/requirements-tox.txt 2019-05-19 18:51:56.000000000 +0200
@@ -1 +1,3 @@
nose>=1.3.0
+flake8>=3.7.0
+coverage>=4.5.0
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/astor-0.7.1/setup.cfg new/astor-0.8/setup.cfg
--- old/astor-0.7.1/setup.cfg 2018-07-06 10:03:51.000000000 +0200
+++ new/astor-0.8/setup.cfg 2019-05-19 18:51:56.000000000 +0200
@@ -3,7 +3,7 @@
description = Read/rewrite/write Python ASTs
long_description = file:README.rst
author = Patrick Maupin
-email = pmau...@gmail.com
+author_email = pmau...@gmail.com
platforms = Independent
url = https://github.com/berkerpeksag/astor
license = BSD-3-Clause
@@ -16,14 +16,13 @@
Operating System :: OS Independent
Programming Language :: Python
Programming Language :: Python :: 2
- Programming Language :: Python :: 2.6
Programming Language :: Python :: 2.7
Programming Language :: Python :: 3
- Programming Language :: Python :: 3.3
Programming Language :: Python :: 3.4
Programming Language :: Python :: 3.5
Programming Language :: Python :: 3.6
Programming Language :: Python :: 3.7
+ Programming Language :: Python :: 3.8
Programming Language :: Python :: Implementation
Programming Language :: Python :: Implementation :: CPython
Programming Language :: Python :: Implementation :: PyPy
@@ -35,6 +34,8 @@
include_package_data = True
packages = find:
python_requires = >=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*
+tests_requires = ["nose", "astunparse"]
+test_suite = nose.collector
[options.packages.find]
exclude = tests
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/astor-0.7.1/tests/check_astunparse.py
new/astor-0.8/tests/check_astunparse.py
--- old/astor-0.7.1/tests/check_astunparse.py 2018-07-06 10:03:51.000000000
+0200
+++ new/astor-0.8/tests/check_astunparse.py 2019-05-19 18:51:56.000000000
+0200
@@ -5,7 +5,7 @@
except ImportError:
import unittest
-import test_code_gen
+from . import test_code_gen
import astunparse
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/astor-0.7.1/tests/check_expressions.py
new/astor-0.8/tests/check_expressions.py
--- old/astor-0.7.1/tests/check_expressions.py 2018-07-06 10:03:51.000000000
+0200
+++ new/astor-0.8/tests/check_expressions.py 2019-05-19 18:51:56.000000000
+0200
@@ -33,7 +33,7 @@
import all_expr_2_6 as mymod
except ImportError:
print("Expression list does not exist -- building")
- import build_expressions
+ from . import build_expressions
build_expressions.makelib()
print("Expression list built")
import all_expr_2_6 as mymod
@@ -44,7 +44,7 @@
mymod = importlib.import_module(mymodname)
except ImportError:
print("Expression list does not exist -- building")
- import build_expressions
+ from . import build_expressions
build_expressions.makelib()
print("Expression list built")
mymod = importlib.import_module(mymodname)
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/astor-0.7.1/tests/test_code_gen.py
new/astor-0.8/tests/test_code_gen.py
--- old/astor-0.7.1/tests/test_code_gen.py 2018-07-06 10:03:51.000000000
+0200
+++ new/astor-0.8/tests/test_code_gen.py 2019-05-19 18:51:56.000000000
+0200
@@ -1,3 +1,4 @@
+# coding: utf-8
"""
Part of the astor library for Python AST manipulation
@@ -38,6 +39,9 @@
dmp2 = astor.dump_tree(ast2)
self.assertEqual(dmp1, dmp2)
+ def assertAstEqualsSource(self, tree, source):
+ self.assertEqual(self.to_source(tree).rstrip(), source)
+
def assertAstRoundtrips(self, srctxt):
"""This asserts that the reconstituted source
code can be compiled into the exact same AST
@@ -160,6 +164,21 @@
pass"""
self.assertSrcRoundtrips(source)
+ @unittest.skipUnless(sys.version_info >= (3, 8, 0, "alpha", 4),
+ "positional only arguments introduced in Python 3.8")
+ def test_positional_only_arguments(self):
+ source = """
+ def test(a, b, /, c, *, d, **kwargs):
+ pass
+
+ def test(a=3, b=4, /, c=7):
+ pass
+
+ def test(a, b=4, /, c=8, d=9):
+ pass
+ """
+ self.assertSrcRoundtrips(source)
+
def test_pass_arguments_node(self):
source = canonical("""
j = [1, 2, 3]
@@ -178,6 +197,10 @@
# Probably also works on < 3.4, but doesn't work on 2.7...
self.assertSrcRoundtripsGtVer(source, (3, 4), (2, 7))
+ def test_attribute(self):
+ self.assertSrcRoundtrips("x.foo")
+ self.assertSrcRoundtrips("(5).foo")
+
def test_matrix_multiplication(self):
for source in ("(a @ b)", "a @= b"):
self.assertAstRoundtripsGtVer(source, (3, 5))
@@ -259,6 +282,11 @@
"""
self.assertAstRoundtripsGtVer(source, (2, 7))
+ def test_huge_int(self):
+ for n in (10**10000,
+
0xdfa21cd2a530ccc8c870aa60d9feb3b35deeab81c3215a96557abbd683d21f4600f38e475d87100da9a4404220eeb3bb5584e5a2b5b48ffda58530ea19104a32577d7459d91e76aa711b241050f4cc6d5327ccee254f371bcad3be56d46eb5919b73f20dbdb1177b700f00891c5bf4ed128bb90ed541b778288285bcfa28432ab5cbcb8321b6e24760e998e0daa519f093a631e44276d7dd252ce0c08c75e2ab28a7349ead779f97d0f20a6d413bf3623cd216dc35375f6366690bcc41e3b2d5465840ec7ee0dc7e3f1c101d674a0c7dbccbc3942788b111396add2f8153b46a0e4b50d66e57ee92958f1c860dd97cc0e40e32febff915343ed53573142bdf4b):
+ self.assertEqual(astornum(n), n)
+
def test_complex(self):
source = """
(3) + (4j) + (1+2j) + (1+0j)
@@ -442,6 +470,126 @@
"""
self.assertSrcRoundtripsGtVer(source, (3, 6))
+ def test_assignment_expr(self):
+ cases = (
+ "(x := 3)",
+ "1 + (x := y)",
+ "x = (y := 0)",
+ "1 + (p := 1 if 2 else 3)",
+ "[y := f(x), y**2, y**3]",
+ "(2 ** 3 * 4 + 5 and 6, x := 2 ** 3 * 4 + 5 and 6)",
+ "foo(x := 3, cat='vector')",
+ "foo(x=(y := f(x)))",
+ "any(len(longline := line) >= 100 for line in lines)",
+ "[[y := f(x), x/y] for x in range(5)]",
+ "lambda: (x := 1)",
+ "def foo(answer=(p := 42)): pass",
+ "def foo(answer: (p := 42) = 5): pass",
+ "if reductor := dispatch_table.get(cls): pass",
+ "while line := fp.readline(): pass",
+ "while (command := input('> ')) != 'quit': pass")
+ for case in cases:
+ self.assertAstRoundtripsGtVer(case, (3, 8))
+
+ @unittest.skipUnless(sys.version_info <= (3, 3),
+ "ast.Name used for True, False, None until Python
3.4")
+ def test_deprecated_constants_as_name(self):
+ self.assertAstEqualsSource(
+ ast.Assign(targets=[ast.Name(id='spam')],
value=ast.Name(id='True')),
+ "spam = True")
+
+ self.assertAstEqualsSource(
+ ast.Assign(targets=[ast.Name(id='spam')],
value=ast.Name(id='False')),
+ "spam = False")
+
+ self.assertAstEqualsSource(
+ ast.Assign(targets=[ast.Name(id='spam')],
value=ast.Name(id='None')),
+ "spam = None")
+
+ @unittest.skipUnless(sys.version_info >= (3, 4),
+ "ast.NameConstant introduced in Python 3.4")
+ def test_deprecated_name_constants(self):
+ self.assertAstEqualsSource(
+ ast.Assign(targets=[ast.Name(id='spam')],
value=ast.NameConstant(value=True)),
+ "spam = True")
+
+ self.assertAstEqualsSource(
+ ast.Assign(targets=[ast.Name(id='spam')],
value=ast.NameConstant(value=False)),
+ "spam = False")
+
+ self.assertAstEqualsSource(
+ ast.Assign(targets=[ast.Name(id='spam')],
value=ast.NameConstant(value=None)),
+ "spam = None")
+
+ def test_deprecated_constant_nodes(self):
+ self.assertAstEqualsSource(
+ ast.Assign(targets=[ast.Name(id='spam')], value=ast.Num(3)),
+ "spam = 3")
+
+ self.assertAstEqualsSource(
+ ast.Assign(targets=[ast.Name(id='spam')], value=ast.Num(-93)),
+ "spam = -93")
+
+ self.assertAstEqualsSource(
+ ast.Assign(targets=[ast.Name(id='spam')], value=ast.Num(837.3888)),
+ "spam = 837.3888")
+
+ self.assertAstEqualsSource(
+ ast.Assign(targets=[ast.Name(id='spam')], value=ast.Num(-0.9877)),
+ "spam = -0.9877")
+
+ self.assertAstEqualsSource(ast.Ellipsis(), "...")
+
+ if sys.version_info >= (3, 0):
+ self.assertAstEqualsSource(
+ ast.Assign(targets=[ast.Name(id='spam')],
value=ast.Bytes(b"Bytes")),
+ "spam = b'Bytes'")
+
+ self.assertAstEqualsSource(
+ ast.Assign(targets=[ast.Name(id='spam')], value=ast.Str("String")),
+ "spam = 'String'")
+
+ @unittest.skipUnless(sys.version_info >= (3, 6),
+ "ast.Constant introduced in Python 3.6")
+ def test_constant_nodes(self):
+ self.assertAstEqualsSource(
+ ast.Assign(targets=[ast.Name(id='spam')],
value=ast.Constant(value=3)),
+ "spam = 3")
+
+ self.assertAstEqualsSource(
+ ast.Assign(targets=[ast.Name(id='spam')],
value=ast.Constant(value=-93)),
+ "spam = -93")
+
+ self.assertAstEqualsSource(
+ ast.Assign(targets=[ast.Name(id='spam')],
value=ast.Constant(value=837.3888)),
+ "spam = 837.3888")
+
+ self.assertAstEqualsSource(
+ ast.Assign(targets=[ast.Name(id='spam')],
value=ast.Constant(value=-0.9877)),
+ "spam = -0.9877")
+
+ self.assertAstEqualsSource(
+ ast.Assign(targets=[ast.Name(id='spam')],
value=ast.Constant(value=True)),
+ "spam = True")
+
+ self.assertAstEqualsSource(
+ ast.Assign(targets=[ast.Name(id='spam')],
value=ast.Constant(value=False)),
+ "spam = False")
+
+ self.assertAstEqualsSource(
+ ast.Assign(targets=[ast.Name(id='spam')],
value=ast.Constant(value=None)),
+ "spam = None")
+
+ self.assertAstEqualsSource(ast.Constant(value=Ellipsis), "...")
+
+ self.assertAstEqualsSource(
+ ast.Assign(targets=[ast.Name(id='spam')],
value=ast.Constant(b"Bytes")),
+ "spam = b'Bytes'")
+
+ self.assertAstEqualsSource(
+ ast.Assign(targets=[ast.Name(id='spam')],
value=ast.Constant(value="String")),
+ "spam = 'String'")
+
def test_annassign(self):
source = """
a: int
@@ -516,6 +664,34 @@
x = f"""{host}\n\t{port}\n"""
'''
self.assertSrcRoundtripsGtVer(source, (3, 6))
+ source = '''
+ if 1:
+ x = f'{host}\\n\\t{port}\\n'
+ '''
+ self.assertSrcRoundtripsGtVer(source, (3, 6))
+
+ def test_fstring_escaped_braces(self):
+ source = '''
+ x = f'{{hello world}}'
+ '''
+ self.assertSrcRoundtripsGtVer(source, (3, 6))
+ source = '''
+ x = f'{f.name}={{self.{f.name}!r}}'
+ '''
+ self.assertSrcRoundtripsGtVer(source, (3, 6))
+
+ @unittest.skipUnless(sys.version_info >= (3, 8, 0, "alpha", 4),
+ "f-string debugging introduced in Python 3.8")
+ def test_fstring_debugging(self):
+ source = """
+ x = f'{5=}'
+ y = f'{5=!r}'
+ z = f'{3*x+15=}'
+ f'{x=:}'
+ f'{x=:.2f}'
+ f'alpha α {pi=} ω omega'
+ """
+ self.assertAstRoundtripsGtVer(source, (3, 8))
def test_docstring_function(self):
source = '''
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/astor-0.7.1/tests/test_misc.py
new/astor-0.8/tests/test_misc.py
--- old/astor-0.7.1/tests/test_misc.py 2018-07-06 10:03:51.000000000 +0200
+++ new/astor-0.8/tests/test_misc.py 2019-05-19 18:51:56.000000000 +0200
@@ -46,6 +46,29 @@
'astor.code_gen module instead.'
)
+ def test_to_source_invalid_customize_generator(self):
+ class InvalidGenerator:
+ pass
+
+ node = ast.parse('spam = 42')
+
+ with self.assertRaises(TypeError) as cm:
+ astor.to_source(node, source_generator_class=InvalidGenerator)
+ self.assertEqual(
+ str(cm.exception),
+ 'source_generator_class should be a subclass of SourceGenerator',
+ )
+
+ with self.assertRaises(TypeError) as cm:
+ astor.to_source(
+ node,
+ source_generator_class=astor.SourceGenerator(indent_with=' ' *
4),
+ )
+ self.assertEqual(
+ str(cm.exception),
+ 'source_generator_class should be a callable',
+ )
+
class FastCompareTestCase(unittest.TestCase):
@@ -64,5 +87,17 @@
check('a = 3 - (3, 4, 5)', 'a = 3 - (3, 4, 6)')
+class TreeWalkTestCase(unittest.TestCase):
+
+ def test_auto_generated_attributes(self):
+ # See #136 for more details.
+ treewalk = astor.TreeWalk()
+ self.assertIsInstance(treewalk.__dict__, dict)
+ # Check that the inital state of the instance is empty.
+ self.assertEqual(treewalk.__dict__['nodestack'], [])
+ self.assertEqual(treewalk.__dict__['pre_handlers'], {})
+ self.assertEqual(treewalk.__dict__['post_handlers'], {})
+
+
if __name__ == '__main__':
unittest.main()
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/astor-0.7.1/tox.ini new/astor-0.8/tox.ini
--- old/astor-0.7.1/tox.ini 2018-07-06 10:03:51.000000000 +0200
+++ new/astor-0.8/tox.ini 2019-05-19 18:51:56.000000000 +0200
@@ -1,10 +1,26 @@
[tox]
-envlist = py27, py34, py35, py36, pypy, pypy3.5
+envlist =
+ py{27, 34, 35, 36, 37, 38, py, py3.5}
+ lint
skipsdist = True
+skip_missing_interpreters = true
[testenv]
usedevelop = True
-commands = nosetests -v --nocapture {posargs}
+commands =
+ coverage run {envbindir}/nosetests -v --nocapture {posargs}
+ coverage report
deps =
-rrequirements-tox.txt
py27,pypy: unittest2
+
+[testenv:lint]
+deps = flake8
+commands = flake8 astor/
+
+[flake8]
+ignore = E114, E116, E501, W504
+
+[travis]
+python =
+ 3.7: py37, lint
