Author: Matti Picus <matti.pi...@gmail.com>
Branch: unicode-utf8
Changeset: r95923:b96e9dd44e46
Date: 2019-02-09 17:36 +0100
http://bitbucket.org/pypy/pypy/changeset/b96e9dd44e46/
Log: merge default into branch
diff too long, truncating to 2000 out of 9094 lines
diff --git a/.hgignore b/.hgignore
--- a/.hgignore
+++ b/.hgignore
@@ -4,8 +4,10 @@
*~
.*.swp
.idea
+.mypy_cache
.project
.pydevproject
+.vscode
__pycache__
.cache/
diff --git a/.hgtags b/.hgtags
--- a/.hgtags
+++ b/.hgtags
@@ -58,3 +58,12 @@
3f6eaa010fce78cc7973bdc1dfdb95970f08fed2 release-pypy3.5-v5.10.1
ab0b9caf307db6592905a80b8faffd69b39005b8 release-pypy2.7-v6.0.0
fdd60ed87e941677e8ea11acf9f1819466521bf2 release-pypy3.5-v6.0.0
+9112c8071614108b1042bfef0713915107004d62 release-pypy2.7-v7.0.0
+1f86f25937b6ae6c8b25236c35228fac587678bf release-pypy3.5-v7.0.0
+dab365a465140aa79a5f3ba4db784c4af4d5c195 release-pypy3.6-v7.0.0
+9112c8071614108b1042bfef0713915107004d62 release-pypy2.7-v7.0.0
+c8805ee6d7846ca2722b106eeaa2f128c699aba3 release-pypy2.7-v7.0.0
+1f86f25937b6ae6c8b25236c35228fac587678bf release-pypy3.5-v7.0.0
+928a4f70d3de7d17449456946154c5da6e600162 release-pypy3.5-v7.0.0
+dab365a465140aa79a5f3ba4db784c4af4d5c195 release-pypy3.6-v7.0.0
+fb40f7a5524c77b80e6c468e087d621610137261 release-pypy3.6-v7.0.0
diff --git a/LICENSE b/LICENSE
--- a/LICENSE
+++ b/LICENSE
@@ -40,16 +40,16 @@
Armin Rigo
Maciej Fijalkowski
Carl Friedrich Bolz-Tereick
+ Antonio Cuni
Amaury Forgeot d'Arc
- Antonio Cuni
Matti Picus
Samuele Pedroni
Ronan Lamy
Alex Gaynor
Philip Jenvey
+ Richard Plangger
Brian Kearns
- Richard Plangger
- Michael Hudson
+ Michael Hudson-Doyle
Manuel Jacob
David Schneider
Holger Krekel
@@ -59,8 +59,8 @@
Anders Chrigstrom
Wim Lavrijsen
Eric van Riet Paap
+ Remi Meier
Richard Emslie
- Remi Meier
Alexander Schremmer
Dan Villiom Podlaski Christiansen
Lukas Diekmann
@@ -70,10 +70,10 @@
Niklaus Haldimann
Camillo Bruni
Laura Creighton
- Romain Guillebert
Toon Verwaest
Leonardo Santagada
Seo Sanghyeon
+ Romain Guillebert
Ronny Pfannschmidt
Justin Peel
Raffael Tfirst
@@ -114,12 +114,12 @@
Squeaky
Edd Barrett
Timo Paulssen
+ Laurence Tratt
Marius Gedminas
Nicolas Truessel
Alexandre Fayolle
Simon Burton
Martin Matusiak
- Laurence Tratt
Wenzhu Man
Konstantin Lopuhin
John Witulski
@@ -134,8 +134,9 @@
Jean-Philippe St. Pierre
Guido van Rossum
Pavel Vinogradov
+ Stefan Beyer
+ William Leslie
Paweł Piotr Przeradowski
- William Leslie
marky1991
Ilya Osadchiy
Tobias Oberstein
@@ -144,10 +145,10 @@
Taavi Burns
Adrian Kuhn
tav
+ Stian Andreassen
Georg Brandl
Joannah Nanjekye
Bert Freudenberg
- Stian Andreassen
Wanja Saatkamp
Mike Blume
Gerald Klix
@@ -163,6 +164,7 @@
Vasily Kuznetsov
Preston Timmons
David Ripton
+ Pieter Zieschang
Dusty Phillips
Lukas Renggli
Guenter Jantzen
@@ -176,6 +178,7 @@
Andrew Durdin
Ben Young
Michael Schneider
+ Yusuke Tsutsumi
Nicholas Riley
Jason Chu
Igor Trindade Oliveira
@@ -187,7 +190,6 @@
Mariano Anaya
anatoly techtonik
Karl Bartel
- Stefan Beyer
Gabriel Lavoie
Jared Grubb
Alecsandru Patrascu
@@ -198,7 +200,6 @@
Victor Stinner
Andrews Medina
Aaron Iles
- p_ziesch...@yahoo.de
Toby Watson
Daniel Patrick
Stuart Williams
@@ -210,6 +211,7 @@
Mikael Schönenberg
Stanislaw Halik
Mihnea Saracin
+ Matt Jackson
Berkin Ilbeyi
Gasper Zejn
Faye Zhao
@@ -217,12 +219,14 @@
Anders Qvist
Corbin Simpson
Chirag Jadwani
+ Pauli Virtanen
Jonathan David Riehl
Beatrice During
Alex Perry
Robert Zaremba
Alan McIntyre
Alexander Sedov
+ David C Ellis
Vaibhav Sood
Reuben Cummings
Attila Gobi
@@ -242,7 +246,6 @@
Arjun Naik
Aaron Gallagher
Alexis Daboville
- Pieter Zieschang
Karl Ramm
Lukas Vacek
Omer Katz
@@ -270,12 +273,15 @@
Catalin Gabriel Manciu
Jacob Oscarson
Ryan Gonzalez
+ Antoine Dupre
Kristjan Valur Jonsson
Lucio Torre
Richard Lancaster
Dan Buch
Lene Wagner
Tomo Cocoa
+ Miro Hrončok
+ Anthony Sottile
David Lievens
Neil Blakey-Milner
Henrik Vendelbo
@@ -290,10 +296,12 @@
Bobby Impollonia
Roberto De Ioris
Jeong YunWon
+ andrewjlawrence
Christopher Armstrong
Aaron Tubbs
Vasantha Ganesh K
Jason Michalski
+ Radu Ciorba
Markus Holtermann
Andrew Thompson
Yusei Tahara
@@ -301,28 +309,26 @@
Fabio Niephaus
Akira Li
Gustavo Niemeyer
- Rafał Gałczyński
+ Nate Bragg
Lucas Stadler
roberto@goyle
+ Carl Bordum Hansen
Matt Bogosian
Yury V. Zaytsev
florinpapa
Anders Sigfridsson
- Matt Jackson
Nikolay Zinov
rafalgalczyn...@gmail.com
Joshua Gilbert
Anna Katrina Dominguez
Kim Jin Su
Amber Brown
- Miro Hrončok
- Anthony Sottile
- Nate Bragg
+ Andrew Stepanov
+ Rafał Gałczyński
Ben Darnell
Juan Francisco Cantero Hurtado
Godefroid Chappelle
Julian Berman
- Michael Hudson-Doyle
Stephan Busemann
Dan Colish
timo
@@ -332,6 +338,7 @@
halgari
Jim Baker
Chris Lambacher
+ John Aldis
coolbutusel...@gmail.com
Mike Bayer
Rodrigo Araújo
@@ -340,6 +347,7 @@
OlivierBlanvillain
Jonas Pfannschmidt
Zearin
+ Johan Forsberg
Andrey Churin
Dan Crosta
reub...@gmail.com
@@ -349,8 +357,9 @@
Steve Papanik
Eli Stevens
Boglarka Vezer
- gabrielg
+ gabri...@ec2-54-146-239-158.compute-1.amazonaws.com
PavloKapyshin
+ Hervé Beraud
Tomer Chachamu
Christopher Groskopf
Asmo Soinio
@@ -364,7 +373,6 @@
Michael Chermside
Anna Ravencroft
remarkablerocket
- Pauli Virtanen
Petre Vijiac
Berker Peksag
Christian Muirhead
@@ -384,12 +392,13 @@
Zooko Wilcox-O Hearn
James Lan
jiaaro
+ Evgenii Gorinov
Markus Unterwaditzer
Kristoffer Kleine
Graham Markall
Dan Loewenherz
werat
- Andrew Stepanov
+ Filip Salomonsson
Niclas Olofsson
Chris Pressey
Tobias Diaz
diff --git a/extra_tests/cffi_tests/cffi0/test_function.py
b/extra_tests/cffi_tests/cffi0/test_function.py
--- a/extra_tests/cffi_tests/cffi0/test_function.py
+++ b/extra_tests/cffi_tests/cffi0/test_function.py
@@ -46,14 +46,15 @@
assert x != math.sin(1.23) # rounding effects
assert abs(x - math.sin(1.23)) < 1E-6
- def test_lround_no_return_value(self):
+ def test_getenv_no_return_value(self):
# check that 'void'-returning functions work too
ffi = FFI(backend=self.Backend())
ffi.cdef("""
- void lround(double x);
+ void getenv(char *);
""")
- m = ffi.dlopen(lib_m)
- x = m.lround(1.23)
+ needs_dlopen_none()
+ m = ffi.dlopen(None)
+ x = m.getenv(b"FOO")
assert x is None
def test_dlopen_filename(self):
diff --git a/extra_tests/cffi_tests/cffi1/test_pkgconfig.py
b/extra_tests/cffi_tests/cffi1/test_pkgconfig.py
new file mode 100644
--- /dev/null
+++ b/extra_tests/cffi_tests/cffi1/test_pkgconfig.py
@@ -0,0 +1,95 @@
+# Generated by pypy/tool/import_cffi.py
+import sys
+import subprocess
+import py
+import cffi.pkgconfig as pkgconfig
+from cffi import PkgConfigError
+
+
+def mock_call(libname, flag):
+ assert libname=="foobarbaz"
+ flags = {
+ "--cflags": "-I/usr/include/python3.6m -DABCD -DCFFI_TEST=1 -O42\n",
+ "--libs": "-L/usr/lib64 -lpython3.6 -shared\n",
+ }
+ return flags[flag]
+
+
+def test_merge_flags():
+ d1 = {"ham": [1, 2, 3], "spam" : ["a", "b", "c"], "foo" : []}
+ d2 = {"spam" : ["spam", "spam", "spam"], "bar" : ["b", "a", "z"]}
+
+ pkgconfig.merge_flags(d1, d2)
+ assert d1 == {
+ "ham": [1, 2, 3],
+ "spam" : ["a", "b", "c", "spam", "spam", "spam"],
+ "bar" : ["b", "a", "z"],
+ "foo" : []}
+
+
+def test_pkgconfig():
+ assert pkgconfig.flags_from_pkgconfig([]) == {}
+
+ saved = pkgconfig.call
+ try:
+ pkgconfig.call = mock_call
+ flags = pkgconfig.flags_from_pkgconfig(["foobarbaz"])
+ finally:
+ pkgconfig.call = saved
+ assert flags == {
+ 'include_dirs': ['/usr/include/python3.6m'],
+ 'library_dirs': ['/usr/lib64'],
+ 'libraries': ['python3.6'],
+ 'define_macros': [('ABCD', None), ('CFFI_TEST', '1')],
+ 'extra_compile_args': ['-O42'],
+ 'extra_link_args': ['-shared']
+ }
+
+class mock_subprocess:
+ PIPE = Ellipsis
+ class Popen:
+ def __init__(self, cmd, stdout, stderr):
+ if mock_subprocess.RESULT is None:
+ raise OSError("oops can't run")
+ assert cmd == ['pkg-config', '--print-errors', '--cflags',
'libfoo']
+ def communicate(self):
+ bout, berr, rc = mock_subprocess.RESULT
+ self.returncode = rc
+ return bout, berr
+
+def test_call():
+ saved = pkgconfig.subprocess
+ try:
+ pkgconfig.subprocess = mock_subprocess
+
+ mock_subprocess.RESULT = None
+ e = py.test.raises(PkgConfigError, pkgconfig.call, "libfoo",
"--cflags")
+ assert str(e.value) == "cannot run pkg-config: oops can't run"
+
+ mock_subprocess.RESULT = b"", "Foo error!\n", 1
+ e = py.test.raises(PkgConfigError, pkgconfig.call, "libfoo",
"--cflags")
+ assert str(e.value) == "Foo error!"
+
+ mock_subprocess.RESULT = b"abc\\def\n", "", 0
+ e = py.test.raises(PkgConfigError, pkgconfig.call, "libfoo",
"--cflags")
+ assert str(e.value).startswith("pkg-config --cflags libfoo returned an
"
+ "unsupported backslash-escaped output:")
+
+ mock_subprocess.RESULT = b"abc def\n", "", 0
+ result = pkgconfig.call("libfoo", "--cflags")
+ assert result == "abc def\n"
+
+ mock_subprocess.RESULT = b"abc def\n", "", 0
+ result = pkgconfig.call("libfoo", "--cflags")
+ assert result == "abc def\n"
+
+ if sys.version_info >= (3,):
+ mock_subprocess.RESULT = b"\xff\n", "", 0
+ e = py.test.raises(PkgConfigError, pkgconfig.call,
+ "libfoo", "--cflags", encoding="utf-8")
+ assert str(e.value) == (
+ "pkg-config --cflags libfoo returned bytes that cannot be "
+ "decoded with encoding 'utf-8':\nb'\\xff\\n'")
+
+ finally:
+ pkgconfig.subprocess = saved
diff --git a/lib_pypy/_csv.py b/lib_pypy/_csv.py
deleted file mode 100644
--- a/lib_pypy/_csv.py
+++ /dev/null
@@ -1,573 +0,0 @@
-"""CSV parsing and writing.
-
-This module provides classes that assist in the reading and writing
-of Comma Separated Value (CSV) files, and implements the interface
-described by PEP 305. Although many CSV files are simple to parse,
-the format is not formally defined by a stable specification and
-is subtle enough that parsing lines of a CSV file with something
-like line.split(\",\") is bound to fail. The module supports three
-basic APIs: reading, writing, and registration of dialects.
-
-
-DIALECT REGISTRATION:
-
-Readers and writers support a dialect argument, which is a convenient
-handle on a group of settings. When the dialect argument is a string,
-it identifies one of the dialects previously registered with the module.
-If it is a class or instance, the attributes of the argument are used as
-the settings for the reader or writer:
-
- class excel:
- delimiter = ','
- quotechar = '\"'
- escapechar = None
- doublequote = True
- skipinitialspace = False
- lineterminator = '\\r\\n'
- quoting = QUOTE_MINIMAL
-
-SETTINGS:
-
- * quotechar - specifies a one-character string to use as the
- quoting character. It defaults to '\"'.
- * delimiter - specifies a one-character string to use as the
- field separator. It defaults to ','.
- * skipinitialspace - specifies how to interpret whitespace which
- immediately follows a delimiter. It defaults to False, which
- means that whitespace immediately following a delimiter is part
- of the following field.
- * lineterminator - specifies the character sequence which should
- terminate rows.
- * quoting - controls when quotes should be generated by the writer.
- It can take on any of the following module constants:
-
- csv.QUOTE_MINIMAL means only when required, for example, when a
- field contains either the quotechar or the delimiter
- csv.QUOTE_ALL means that quotes are always placed around fields.
- csv.QUOTE_NONNUMERIC means that quotes are always placed around
- fields which do not parse as integers or floating point
- numbers.
- csv.QUOTE_NONE means that quotes are never placed around fields.
- * escapechar - specifies a one-character string used to escape
- the delimiter when quoting is set to QUOTE_NONE.
- * doublequote - controls the handling of quotes inside fields. When
- True, two consecutive quotes are interpreted as one during read,
- and when writing, each quote character embedded in the data is
- written as two quotes.
-"""
-
-__version__ = "1.0"
-
-QUOTE_MINIMAL, QUOTE_ALL, QUOTE_NONNUMERIC, QUOTE_NONE = range(4)
-_dialects = {}
-_field_limit = 128 * 1024 # max parsed field size
-
-class Error(Exception):
- pass
-
-class Dialect(object):
- """CSV dialect
-
- The Dialect type records CSV parsing and generation options."""
-
- __slots__ = ["_delimiter", "_doublequote", "_escapechar",
- "_lineterminator", "_quotechar", "_quoting",
- "_skipinitialspace", "_strict"]
-
- def __new__(cls, dialect, **kwargs):
-
- for name in kwargs:
- if '_' + name not in Dialect.__slots__:
- raise TypeError("unexpected keyword argument '%s'" %
- (name,))
-
- if dialect is not None:
- if isinstance(dialect, basestring):
- dialect = get_dialect(dialect)
-
- # Can we reuse this instance?
- if (isinstance(dialect, Dialect)
- and all(value is None for value in kwargs.itervalues())):
- return dialect
-
- self = object.__new__(cls)
-
-
- def set_char(x):
- if x is None:
- return None
- if isinstance(x, str) and len(x) <= 1:
- return x
- raise TypeError("%r must be a 1-character string" % (name,))
- def set_str(x):
- if isinstance(x, str):
- return x
- raise TypeError("%r must be a string" % (name,))
- def set_quoting(x):
- if x in range(4):
- return x
- raise TypeError("bad 'quoting' value")
-
- attributes = {"delimiter": (',', set_char),
- "doublequote": (True, bool),
- "escapechar": (None, set_char),
- "lineterminator": ("\r\n", set_str),
- "quotechar": ('"', set_char),
- "quoting": (QUOTE_MINIMAL, set_quoting),
- "skipinitialspace": (False, bool),
- "strict": (False, bool),
- }
-
- # Copy attributes
- notset = object()
- for name in Dialect.__slots__:
- name = name[1:]
- value = notset
- if name in kwargs:
- value = kwargs[name]
- elif dialect is not None:
- value = getattr(dialect, name, notset)
-
- # mapping by name: (default, converter)
- if value is notset:
- value = attributes[name][0]
- if name == 'quoting' and not self.quotechar:
- value = QUOTE_NONE
- else:
- converter = attributes[name][1]
- if converter:
- value = converter(value)
-
- setattr(self, '_' + name, value)
-
- if not self.delimiter:
- raise TypeError("delimiter must be set")
-
- if self.quoting != QUOTE_NONE and not self.quotechar:
- raise TypeError("quotechar must be set if quoting enabled")
-
- if not self.lineterminator:
- raise TypeError("lineterminator must be set")
-
- return self
-
- delimiter = property(lambda self: self._delimiter)
- doublequote = property(lambda self: self._doublequote)
- escapechar = property(lambda self: self._escapechar)
- lineterminator = property(lambda self: self._lineterminator)
- quotechar = property(lambda self: self._quotechar)
- quoting = property(lambda self: self._quoting)
- skipinitialspace = property(lambda self: self._skipinitialspace)
- strict = property(lambda self: self._strict)
-
-
-def _call_dialect(dialect_inst, kwargs):
- return Dialect(dialect_inst, **kwargs)
-
-def register_dialect(name, dialect=None, **kwargs):
- """Create a mapping from a string name to a dialect class.
- dialect = csv.register_dialect(name, dialect)"""
- if not isinstance(name, basestring):
- raise TypeError("dialect name must be a string or unicode")
-
- dialect = _call_dialect(dialect, kwargs)
- _dialects[name] = dialect
-
-def unregister_dialect(name):
- """Delete the name/dialect mapping associated with a string name.\n
- csv.unregister_dialect(name)"""
- try:
- del _dialects[name]
- except KeyError:
- raise Error("unknown dialect")
-
-def get_dialect(name):
- """Return the dialect instance associated with name.
- dialect = csv.get_dialect(name)"""
- try:
- return _dialects[name]
- except KeyError:
- raise Error("unknown dialect")
-
-def list_dialects():
- """Return a list of all know dialect names
- names = csv.list_dialects()"""
- return list(_dialects)
-
-class Reader(object):
- """CSV reader
-
- Reader objects are responsible for reading and parsing tabular data
- in CSV format."""
-
-
- (START_RECORD, START_FIELD, ESCAPED_CHAR, IN_FIELD,
- IN_QUOTED_FIELD, ESCAPE_IN_QUOTED_FIELD, QUOTE_IN_QUOTED_FIELD,
- EAT_CRNL) = range(8)
-
- def __init__(self, iterator, dialect=None, **kwargs):
- self.dialect = _call_dialect(dialect, kwargs)
- self.input_iter = iter(iterator)
- self.line_num = 0
-
- self._parse_reset()
-
- def _parse_reset(self):
- self.field = ''
- self.fields = []
- self.state = self.START_RECORD
- self.numeric_field = False
-
- def __iter__(self):
- return self
-
- def next(self):
- self._parse_reset()
- while True:
- try:
- line = next(self.input_iter)
- except StopIteration:
- # End of input OR exception
- if len(self.field) > 0:
- raise Error("newline inside string")
- raise
-
- self.line_num += 1
-
- if '\0' in line:
- raise Error("line contains NULL byte")
- pos = 0
- while pos < len(line):
- pos = self._parse_process_char(line, pos)
- self._parse_eol()
-
- if self.state == self.START_RECORD:
- break
-
- fields = self.fields
- self.fields = []
- return fields
-
- def _parse_process_char(self, line, pos):
- c = line[pos]
- if self.state == self.IN_FIELD:
- # in unquoted field
- pos2 = pos
- while True:
- if c in '\n\r':
- # end of line - return [fields]
- if pos2 > pos:
- self._parse_add_char(line[pos:pos2])
- pos = pos2
- self._parse_save_field()
- self.state = self.EAT_CRNL
- elif c == self.dialect.escapechar:
- # possible escaped character
- pos2 -= 1
- self.state = self.ESCAPED_CHAR
- elif c == self.dialect.delimiter:
- # save field - wait for new field
- if pos2 > pos:
- self._parse_add_char(line[pos:pos2])
- pos = pos2
- self._parse_save_field()
- self.state = self.START_FIELD
- else:
- # normal character - save in field
- pos2 += 1
- if pos2 < len(line):
- c = line[pos2]
- continue
- break
- if pos2 > pos:
- self._parse_add_char(line[pos:pos2])
- pos = pos2 - 1
-
- elif self.state == self.START_RECORD:
- if c in '\n\r':
- self.state = self.EAT_CRNL
- else:
- self.state = self.START_FIELD
- # restart process
- self._parse_process_char(line, pos)
-
- elif self.state == self.START_FIELD:
- if c in '\n\r':
- # save empty field - return [fields]
- self._parse_save_field()
- self.state = self.EAT_CRNL
- elif (c == self.dialect.quotechar
- and self.dialect.quoting != QUOTE_NONE):
- # start quoted field
- self.state = self.IN_QUOTED_FIELD
- elif c == self.dialect.escapechar:
- # possible escaped character
- self.state = self.ESCAPED_CHAR
- elif c == ' ' and self.dialect.skipinitialspace:
- # ignore space at start of field
- pass
- elif c == self.dialect.delimiter:
- # save empty field
- self._parse_save_field()
- else:
- # begin new unquoted field
- if self.dialect.quoting == QUOTE_NONNUMERIC:
- self.numeric_field = True
- self._parse_add_char(c)
- self.state = self.IN_FIELD
-
- elif self.state == self.ESCAPED_CHAR:
- self._parse_add_char(c)
- self.state = self.IN_FIELD
-
- elif self.state == self.IN_QUOTED_FIELD:
- if c == self.dialect.escapechar:
- # possible escape character
- self.state = self.ESCAPE_IN_QUOTED_FIELD
- elif (c == self.dialect.quotechar
- and self.dialect.quoting != QUOTE_NONE):
- if self.dialect.doublequote:
- # doublequote; " represented by ""
- self.state = self.QUOTE_IN_QUOTED_FIELD
- else:
- #end of quote part of field
- self.state = self.IN_FIELD
- else:
- # normal character - save in field
- self._parse_add_char(c)
-
- elif self.state == self.ESCAPE_IN_QUOTED_FIELD:
- self._parse_add_char(c)
- self.state = self.IN_QUOTED_FIELD
-
- elif self.state == self.QUOTE_IN_QUOTED_FIELD:
- # doublequote - seen a quote in a quoted field
- if (c == self.dialect.quotechar
- and self.dialect.quoting != QUOTE_NONE):
- # save "" as "
- self._parse_add_char(c)
- self.state = self.IN_QUOTED_FIELD
- elif c == self.dialect.delimiter:
- # save field - wait for new field
- self._parse_save_field()
- self.state = self.START_FIELD
- elif c in '\r\n':
- # end of line - return [fields]
- self._parse_save_field()
- self.state = self.EAT_CRNL
- elif not self.dialect.strict:
- self._parse_add_char(c)
- self.state = self.IN_FIELD
- else:
- raise Error("'%c' expected after '%c'" %
- (self.dialect.delimiter, self.dialect.quotechar))
-
- elif self.state == self.EAT_CRNL:
- if c not in '\r\n':
- raise Error("new-line character seen in unquoted field - "
- "do you need to open the file "
- "in universal-newline mode?")
-
- else:
- raise RuntimeError("unknown state: %r" % (self.state,))
-
- return pos + 1
-
- def _parse_eol(self):
- if self.state == self.EAT_CRNL:
- self.state = self.START_RECORD
- elif self.state == self.START_RECORD:
- # empty line - return []
- pass
- elif self.state == self.IN_FIELD:
- # in unquoted field
- # end of line - return [fields]
- self._parse_save_field()
- self.state = self.START_RECORD
- elif self.state == self.START_FIELD:
- # save empty field - return [fields]
- self._parse_save_field()
- self.state = self.START_RECORD
- elif self.state == self.ESCAPED_CHAR:
- self._parse_add_char('\n')
- self.state = self.IN_FIELD
- elif self.state == self.IN_QUOTED_FIELD:
- pass
- elif self.state == self.ESCAPE_IN_QUOTED_FIELD:
- self._parse_add_char('\n')
- self.state = self.IN_QUOTED_FIELD
- elif self.state == self.QUOTE_IN_QUOTED_FIELD:
- # end of line - return [fields]
- self._parse_save_field()
- self.state = self.START_RECORD
- else:
- raise RuntimeError("unknown state: %r" % (self.state,))
-
- def _parse_save_field(self):
- field, self.field = self.field, ''
- if self.numeric_field:
- self.numeric_field = False
- field = float(field)
- self.fields.append(field)
-
- def _parse_add_char(self, c):
- if len(self.field) + len(c) > _field_limit:
- raise Error("field larger than field limit (%d)" % (_field_limit))
- self.field += c
-
-
-class Writer(object):
- """CSV writer
-
- Writer objects are responsible for generating tabular data
- in CSV format from sequence input."""
-
- def __init__(self, file, dialect=None, **kwargs):
- if not (hasattr(file, 'write') and callable(file.write)):
- raise TypeError("argument 1 must have a 'write' method")
- self.writeline = file.write
- self.dialect = _call_dialect(dialect, kwargs)
-
- def _join_reset(self):
- self.rec = []
- self.num_fields = 0
-
- def _join_append(self, field, quoted, quote_empty):
- dialect = self.dialect
- # If this is not the first field we need a field separator
- if self.num_fields > 0:
- self.rec.append(dialect.delimiter)
-
- if dialect.quoting == QUOTE_NONE:
- need_escape = tuple(dialect.lineterminator) + (
- dialect.escapechar, # escapechar always first
- dialect.delimiter, dialect.quotechar)
-
- else:
- for c in tuple(dialect.lineterminator) + (
- dialect.delimiter, dialect.escapechar):
- if c and c in field:
- quoted = True
-
- need_escape = ()
- if dialect.quotechar in field:
- if dialect.doublequote:
- field = field.replace(dialect.quotechar,
- dialect.quotechar * 2)
- quoted = True
- else:
- need_escape = (dialect.quotechar,)
-
-
- for c in need_escape:
- if c and c in field:
- if not dialect.escapechar:
- raise Error("need to escape, but no escapechar set")
- field = field.replace(c, dialect.escapechar + c)
-
- # If field is empty check if it needs to be quoted
- if field == '' and quote_empty:
- if dialect.quoting == QUOTE_NONE:
- raise Error("single empty field record must be quoted")
- quoted = 1
-
- if quoted:
- field = dialect.quotechar + field + dialect.quotechar
-
- self.rec.append(field)
- self.num_fields += 1
-
-
-
- def writerow(self, row):
- dialect = self.dialect
- try:
- rowlen = len(row)
- except TypeError:
- raise Error("sequence expected")
-
- # join all fields in internal buffer
- self._join_reset()
-
- for field in row:
- quoted = False
- if dialect.quoting == QUOTE_NONNUMERIC:
- try:
- float(field)
- except:
- quoted = True
- # This changed since 2.5:
- # quoted = not isinstance(field, (int, long, float))
- elif dialect.quoting == QUOTE_ALL:
- quoted = True
-
- if field is None:
- value = ""
- elif isinstance(field, float):
- value = repr(field)
- else:
- value = str(field)
- self._join_append(value, quoted, rowlen == 1)
-
- # add line terminator
- self.rec.append(dialect.lineterminator)
-
- self.writeline(''.join(self.rec))
-
- def writerows(self, rows):
- for row in rows:
- self.writerow(row)
-
-def reader(*args, **kwargs):
- """
- csv_reader = reader(iterable [, dialect='excel']
- [optional keyword args])
- for row in csv_reader:
- process(row)
-
- The "iterable" argument can be any object that returns a line
- of input for each iteration, such as a file object or a list. The
- optional \"dialect\" parameter is discussed below. The function
- also accepts optional keyword arguments which override settings
- provided by the dialect.
-
- The returned object is an iterator. Each iteration returns a row
- of the CSV file (which can span multiple input lines)"""
-
- return Reader(*args, **kwargs)
-
-def writer(*args, **kwargs):
- """
- csv_writer = csv.writer(fileobj [, dialect='excel']
- [optional keyword args])
- for row in sequence:
- csv_writer.writerow(row)
-
- [or]
-
- csv_writer = csv.writer(fileobj [, dialect='excel']
- [optional keyword args])
- csv_writer.writerows(rows)
-
- The \"fileobj\" argument can be any object that supports the file API."""
- return Writer(*args, **kwargs)
-
-
-undefined = object()
-def field_size_limit(limit=undefined):
- """Sets an upper limit on parsed fields.
- csv.field_size_limit([limit])
-
- Returns old limit. If limit is not given, no new limit is set and
- the old limit is returned"""
-
- global _field_limit
- old_limit = _field_limit
-
- if limit is not undefined:
- if not isinstance(limit, (int, long)):
- raise TypeError("int expected, got %s" %
- (limit.__class__.__name__,))
- _field_limit = limit
-
- return old_limit
diff --git a/lib_pypy/cffi/__init__.py b/lib_pypy/cffi/__init__.py
--- a/lib_pypy/cffi/__init__.py
+++ b/lib_pypy/cffi/__init__.py
@@ -3,6 +3,7 @@
from .api import FFI
from .error import CDefError, FFIError, VerificationError, VerificationMissing
+from .error import PkgConfigError
__version__ = "1.12.0"
__version_info__ = (1, 12, 0)
diff --git a/lib_pypy/cffi/api.py b/lib_pypy/cffi/api.py
--- a/lib_pypy/cffi/api.py
+++ b/lib_pypy/cffi/api.py
@@ -592,7 +592,7 @@
if sys.platform == "win32":
# we need 'libpypy-c.lib'. Current distributions of
# pypy (>= 4.1) contain it as 'libs/python27.lib'.
- pythonlib = "python27"
+ pythonlib = "python{0[0]}{0[1]}".format(sys.version_info)
if hasattr(sys, 'prefix'):
ensure('library_dirs', os.path.join(sys.prefix, 'libs'))
else:
@@ -643,6 +643,16 @@
self._assigned_source = (str(module_name), source,
source_extension, kwds)
+ def set_source_pkgconfig(self, module_name, pkgconfig_libs, source,
+ source_extension='.c', **kwds):
+ from . import pkgconfig
+ if not isinstance(pkgconfig_libs, list):
+ raise TypeError("the pkgconfig_libs argument must be a list "
+ "of package names")
+ kwds2 = pkgconfig.flags_from_pkgconfig(pkgconfig_libs)
+ pkgconfig.merge_flags(kwds, kwds2)
+ self.set_source(module_name, source, source_extension, **kwds)
+
def distutils_extension(self, tmpdir='build', verbose=True):
from distutils.dir_util import mkpath
from .recompiler import recompile
diff --git a/lib_pypy/cffi/error.py b/lib_pypy/cffi/error.py
--- a/lib_pypy/cffi/error.py
+++ b/lib_pypy/cffi/error.py
@@ -1,8 +1,9 @@
class FFIError(Exception):
- pass
+ __module__ = 'cffi'
class CDefError(Exception):
+ __module__ = 'cffi'
def __str__(self):
try:
current_decl = self.args[1]
@@ -16,8 +17,15 @@
class VerificationError(Exception):
""" An error raised when verification fails
"""
+ __module__ = 'cffi'
class VerificationMissing(Exception):
""" An error raised when incomplete structures are passed into
cdef, but no verification has been done
"""
+ __module__ = 'cffi'
+
+class PkgConfigError(Exception):
+ """ An error raised for missing modules in pkg-config
+ """
+ __module__ = 'cffi'
diff --git a/lib_pypy/cffi/pkgconfig.py b/lib_pypy/cffi/pkgconfig.py
new file mode 100644
--- /dev/null
+++ b/lib_pypy/cffi/pkgconfig.py
@@ -0,0 +1,121 @@
+# pkg-config, https://www.freedesktop.org/wiki/Software/pkg-config/
integration for cffi
+import sys, os, subprocess
+
+from .error import PkgConfigError
+
+
+def merge_flags(cfg1, cfg2):
+ """Merge values from cffi config flags cfg2 to cf1
+
+ Example:
+ merge_flags({"libraries": ["one"]}, {"libraries": ["two"]})
+ {"libraries": ["one", "two"]}
+ """
+ for key, value in cfg2.items():
+ if key not in cfg1:
+ cfg1[key] = value
+ else:
+ if not isinstance(cfg1[key], list):
+ raise TypeError("cfg1[%r] should be a list of strings" %
(key,))
+ if not isinstance(value, list):
+ raise TypeError("cfg2[%r] should be a list of strings" %
(key,))
+ cfg1[key].extend(value)
+ return cfg1
+
+
+def call(libname, flag, encoding=sys.getfilesystemencoding()):
+ """Calls pkg-config and returns the output if found
+ """
+ a = ["pkg-config", "--print-errors"]
+ a.append(flag)
+ a.append(libname)
+ try:
+ pc = subprocess.Popen(a, stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
+ except EnvironmentError as e:
+ raise PkgConfigError("cannot run pkg-config: %s" % (str(e).strip(),))
+
+ bout, berr = pc.communicate()
+ if pc.returncode != 0:
+ try:
+ berr = berr.decode(encoding)
+ except Exception:
+ pass
+ raise PkgConfigError(berr.strip())
+
+ if sys.version_info >= (3,) and not isinstance(bout, str): # Python 3.x
+ try:
+ bout = bout.decode(encoding)
+ except UnicodeDecodeError:
+ raise PkgConfigError("pkg-config %s %s returned bytes that cannot "
+ "be decoded with encoding %r:\n%r" %
+ (flag, libname, encoding, bout))
+
+ if os.altsep != '\\' and '\\' in bout:
+ raise PkgConfigError("pkg-config %s %s returned an unsupported "
+ "backslash-escaped output:\n%r" %
+ (flag, libname, bout))
+ return bout
+
+
+def flags_from_pkgconfig(libs):
+ r"""Return compiler line flags for FFI.set_source based on pkg-config
output
+
+ Usage
+ ...
+ ffibuilder.set_source("_foo", pkgconfig = ["libfoo", "libbar >=
1.8.3"])
+
+ If pkg-config is installed on build machine, then arguments include_dirs,
+ library_dirs, libraries, define_macros, extra_compile_args and
+ extra_link_args are extended with an output of pkg-config for libfoo and
+ libbar.
+
+ Raises PkgConfigError in case the pkg-config call fails.
+ """
+
+ def get_include_dirs(string):
+ return [x[2:] for x in string.split() if x.startswith("-I")]
+
+ def get_library_dirs(string):
+ return [x[2:] for x in string.split() if x.startswith("-L")]
+
+ def get_libraries(string):
+ return [x[2:] for x in string.split() if x.startswith("-l")]
+
+ # convert -Dfoo=bar to list of tuples [("foo", "bar")] expected by
distutils
+ def get_macros(string):
+ def _macro(x):
+ x = x[2:] # drop "-D"
+ if '=' in x:
+ return tuple(x.split("=", 1)) # "-Dfoo=bar" => ("foo", "bar")
+ else:
+ return (x, None) # "-Dfoo" => ("foo", None)
+ return [_macro(x) for x in string.split() if x.startswith("-D")]
+
+ def get_other_cflags(string):
+ return [x for x in string.split() if not x.startswith("-I") and
+ not x.startswith("-D")]
+
+ def get_other_libs(string):
+ return [x for x in string.split() if not x.startswith("-L") and
+ not x.startswith("-l")]
+
+ # return kwargs for given libname
+ def kwargs(libname):
+ fse = sys.getfilesystemencoding()
+ all_cflags = call(libname, "--cflags")
+ all_libs = call(libname, "--libs")
+ return {
+ "include_dirs": get_include_dirs(all_cflags),
+ "library_dirs": get_library_dirs(all_libs),
+ "libraries": get_libraries(all_libs),
+ "define_macros": get_macros(all_cflags),
+ "extra_compile_args": get_other_cflags(all_cflags),
+ "extra_link_args": get_other_libs(all_libs),
+ }
+
+ # merge all arguments together
+ ret = {}
+ for libname in libs:
+ lib_flags = kwargs(libname)
+ merge_flags(ret, lib_flags)
+ return ret
diff --git a/lib_pypy/pwd.py b/lib_pypy/pwd.py
deleted file mode 100644
--- a/lib_pypy/pwd.py
+++ /dev/null
@@ -1,114 +0,0 @@
-# indirectly based on ctypes implementation: Victor Stinner, 2008-05-08
-"""
-This module provides access to the Unix password database.
-It is available on all Unix versions.
-
-Password database entries are reported as 7-tuples containing the following
-items from the password database (see `<pwd.h>'), in order:
-pw_name, pw_passwd, pw_uid, pw_gid, pw_gecos, pw_dir, pw_shell.
-The uid and gid items are integers, all others are strings. An
-exception is raised if the entry asked for cannot be found.
-"""
-
-from _pwdgrp_cffi import ffi, lib
-import _structseq
-import thread
-_lock = thread.allocate_lock()
-
-try: from __pypy__ import builtinify
-except ImportError: builtinify = lambda f: f
-
-
-class struct_passwd:
- """
- pwd.struct_passwd: Results from getpw*() routines.
-
- This object may be accessed either as a tuple of
- (pw_name,pw_passwd,pw_uid,pw_gid,pw_gecos,pw_dir,pw_shell)
- or via the object attributes as named in the above tuple.
- """
- __metaclass__ = _structseq.structseqtype
- name = "pwd.struct_passwd"
-
- pw_name = _structseq.structseqfield(0)
- pw_passwd = _structseq.structseqfield(1)
- pw_uid = _structseq.structseqfield(2)
- pw_gid = _structseq.structseqfield(3)
- pw_gecos = _structseq.structseqfield(4)
- pw_dir = _structseq.structseqfield(5)
- pw_shell = _structseq.structseqfield(6)
-
-
-def _mkpwent(pw):
- return struct_passwd([
- ffi.string(pw.pw_name),
- ffi.string(pw.pw_passwd),
- pw.pw_uid,
- pw.pw_gid,
- ffi.string(pw.pw_gecos),
- ffi.string(pw.pw_dir),
- ffi.string(pw.pw_shell)])
-
-@builtinify
-def getpwuid(uid):
- """
- getpwuid(uid) -> (pw_name,pw_passwd,pw_uid,
- pw_gid,pw_gecos,pw_dir,pw_shell)
- Return the password database entry for the given numeric user ID.
- See pwd.__doc__ for more on password database entries.
- """
- with _lock:
- pw = lib.getpwuid(uid)
- if not pw:
- raise KeyError("getpwuid(): uid not found: %s" % uid)
- return _mkpwent(pw)
-
-@builtinify
-def getpwnam(name):
- """
- getpwnam(name) -> (pw_name,pw_passwd,pw_uid,
- pw_gid,pw_gecos,pw_dir,pw_shell)
- Return the password database entry for the given user name.
- See pwd.__doc__ for more on password database entries.
- """
- if not isinstance(name, basestring):
- raise TypeError("expected string")
- name = str(name)
- with _lock:
- pw = lib.getpwnam(name)
- if not pw:
- raise KeyError("getpwname(): name not found: %s" % name)
- return _mkpwent(pw)
-
-@builtinify
-def getpwall():
- """
- getpwall() -> list_of_entries
- Return a list of all available password database entries, in arbitrary
order.
- See pwd.__doc__ for more on password database entries.
- """
- users = []
- with _lock:
- lib.setpwent()
- while True:
- pw = lib.getpwent()
- if not pw:
- break
- users.append(_mkpwent(pw))
- lib.endpwent()
- return users
-
-__all__ = ('struct_passwd', 'getpwuid', 'getpwnam', 'getpwall')
-
-if __name__ == "__main__":
-# Uncomment next line to test CPython implementation
-# from pwd import getpwuid, getpwnam, getpwall
- from os import getuid
- uid = getuid()
- pw = getpwuid(uid)
- print("uid %s: %s" % (pw.pw_uid, pw))
- name = pw.pw_name
- print("name %r: %s" % (name, getpwnam(name)))
- print("All:")
- for pw in getpwall():
- print(pw)
diff --git a/pypy/doc/build.rst b/pypy/doc/build.rst
--- a/pypy/doc/build.rst
+++ b/pypy/doc/build.rst
@@ -220,11 +220,12 @@
Making a debug build of PyPy
----------------------------
-If the Makefile is rerun with the lldebug or lldebug0 target, appropriate
-compilation flags are added to add debug info and reduce compiler optimizations
-to ``-O0`` respectively. If you stop in a debugger, you will see the
-very wordy machine-generated C code from the rpython translation step, which
-takes a little bit of reading to relate back to the rpython code.
+Rerun the ``Makefile`` with the ``make lldebug`` or ``make lldebug0`` target,
+which will build in a way that running under a debugger makes sense.
+Appropriate compilation flags are added to add debug info, and for ``lldebug0``
+compiler optimizations are fully disabled. If you stop in a debugger, you will
+see the very wordy machine-generated C code from the rpython translation step,
+which takes a little bit of reading to relate back to the rpython code.
Build cffi import libraries for the stdlib
------------------------------------------
diff --git a/pypy/doc/contributor.rst b/pypy/doc/contributor.rst
--- a/pypy/doc/contributor.rst
+++ b/pypy/doc/contributor.rst
@@ -7,16 +7,16 @@
Armin Rigo
Maciej Fijalkowski
Carl Friedrich Bolz-Tereick
+ Antonio Cuni
Amaury Forgeot d'Arc
- Antonio Cuni
Matti Picus
Samuele Pedroni
Ronan Lamy
Alex Gaynor
Philip Jenvey
+ Richard Plangger
Brian Kearns
- Richard Plangger
- Michael Hudson
+ Michael Hudson-Doyle
Manuel Jacob
David Schneider
Holger Krekel
@@ -26,8 +26,8 @@
Anders Chrigstrom
Wim Lavrijsen
Eric van Riet Paap
+ Remi Meier
Richard Emslie
- Remi Meier
Alexander Schremmer
Dan Villiom Podlaski Christiansen
Lukas Diekmann
@@ -37,10 +37,10 @@
Niklaus Haldimann
Camillo Bruni
Laura Creighton
- Romain Guillebert
Toon Verwaest
Leonardo Santagada
Seo Sanghyeon
+ Romain Guillebert
Ronny Pfannschmidt
Justin Peel
Raffael Tfirst
@@ -81,12 +81,12 @@
Squeaky
Edd Barrett
Timo Paulssen
+ Laurence Tratt
Marius Gedminas
Nicolas Truessel
Alexandre Fayolle
Simon Burton
Martin Matusiak
- Laurence Tratt
Wenzhu Man
Konstantin Lopuhin
John Witulski
@@ -101,8 +101,9 @@
Jean-Philippe St. Pierre
Guido van Rossum
Pavel Vinogradov
+ Stefan Beyer
+ William Leslie
Paweł Piotr Przeradowski
- William Leslie
marky1991
Ilya Osadchiy
Tobias Oberstein
@@ -111,10 +112,10 @@
Taavi Burns
Adrian Kuhn
tav
+ Stian Andreassen
Georg Brandl
Joannah Nanjekye
Bert Freudenberg
- Stian Andreassen
Wanja Saatkamp
Mike Blume
Gerald Klix
@@ -130,6 +131,7 @@
Vasily Kuznetsov
Preston Timmons
David Ripton
+ Pieter Zieschang
Dusty Phillips
Lukas Renggli
Guenter Jantzen
@@ -143,6 +145,7 @@
Andrew Durdin
Ben Young
Michael Schneider
+ Yusuke Tsutsumi
Nicholas Riley
Jason Chu
Igor Trindade Oliveira
@@ -154,7 +157,6 @@
Mariano Anaya
anatoly techtonik
Karl Bartel
- Stefan Beyer
Gabriel Lavoie
Jared Grubb
Alecsandru Patrascu
@@ -165,7 +167,6 @@
Victor Stinner
Andrews Medina
Aaron Iles
- p_ziesch...@yahoo.de
Toby Watson
Daniel Patrick
Stuart Williams
@@ -177,6 +178,7 @@
Mikael Schönenberg
Stanislaw Halik
Mihnea Saracin
+ Matt Jackson
Berkin Ilbeyi
Gasper Zejn
Faye Zhao
@@ -184,12 +186,14 @@
Anders Qvist
Corbin Simpson
Chirag Jadwani
+ Pauli Virtanen
Jonathan David Riehl
Beatrice During
Alex Perry
Robert Zaremba
Alan McIntyre
Alexander Sedov
+ David C Ellis
Vaibhav Sood
Reuben Cummings
Attila Gobi
@@ -209,7 +213,6 @@
Arjun Naik
Aaron Gallagher
Alexis Daboville
- Pieter Zieschang
Karl Ramm
Lukas Vacek
Omer Katz
@@ -237,12 +240,15 @@
Catalin Gabriel Manciu
Jacob Oscarson
Ryan Gonzalez
+ Antoine Dupre
Kristjan Valur Jonsson
Lucio Torre
Richard Lancaster
Dan Buch
Lene Wagner
Tomo Cocoa
+ Miro Hrončok
+ Anthony Sottile
David Lievens
Neil Blakey-Milner
Henrik Vendelbo
@@ -257,10 +263,12 @@
Bobby Impollonia
Roberto De Ioris
Jeong YunWon
+ andrewjlawrence
Christopher Armstrong
Aaron Tubbs
Vasantha Ganesh K
Jason Michalski
+ Radu Ciorba
Markus Holtermann
Andrew Thompson
Yusei Tahara
@@ -268,28 +276,26 @@
Fabio Niephaus
Akira Li
Gustavo Niemeyer
- Rafał Gałczyński
+ Nate Bragg
Lucas Stadler
roberto@goyle
+ Carl Bordum Hansen
Matt Bogosian
Yury V. Zaytsev
florinpapa
Anders Sigfridsson
- Matt Jackson
Nikolay Zinov
rafalgalczyn...@gmail.com
Joshua Gilbert
Anna Katrina Dominguez
Kim Jin Su
Amber Brown
- Miro Hrončok
- Anthony Sottile
- Nate Bragg
+ Andrew Stepanov
+ Rafał Gałczyński
Ben Darnell
Juan Francisco Cantero Hurtado
Godefroid Chappelle
Julian Berman
- Michael Hudson-Doyle
Stephan Busemann
Dan Colish
timo
@@ -299,6 +305,7 @@
halgari
Jim Baker
Chris Lambacher
+ John Aldis
coolbutusel...@gmail.com
Mike Bayer
Rodrigo Araújo
@@ -307,6 +314,7 @@
OlivierBlanvillain
Jonas Pfannschmidt
Zearin
+ Johan Forsberg
Andrey Churin
Dan Crosta
reub...@gmail.com
@@ -316,8 +324,9 @@
Steve Papanik
Eli Stevens
Boglarka Vezer
- gabrielg
+ gabri...@ec2-54-146-239-158.compute-1.amazonaws.com
PavloKapyshin
+ Hervé Beraud
Tomer Chachamu
Christopher Groskopf
Asmo Soinio
@@ -331,7 +340,6 @@
Michael Chermside
Anna Ravencroft
remarkablerocket
- Pauli Virtanen
Petre Vijiac
Berker Peksag
Christian Muirhead
@@ -351,12 +359,13 @@
Zooko Wilcox-O Hearn
James Lan
jiaaro
+ Evgenii Gorinov
Markus Unterwaditzer
Kristoffer Kleine
Graham Markall
Dan Loewenherz
werat
- Andrew Stepanov
+ Filip Salomonsson
Niclas Olofsson
Chris Pressey
Tobias Diaz
diff --git a/pypy/doc/how-to-release.rst b/pypy/doc/how-to-release.rst
--- a/pypy/doc/how-to-release.rst
+++ b/pypy/doc/how-to-release.rst
@@ -16,9 +16,6 @@
How to Create a PyPy Release
++++++++++++++++++++++++++++
-Overview
---------
-
As a meta rule setting up issues in the tracker for items here may help not
forgetting things. A set of todo files may also work.
@@ -28,17 +25,54 @@
Release Steps
--------------
+++++++++++++++
-* If needed, make a release branch
-* Bump the
- pypy version number in module/sys/version.py and in
- module/cpyext/include/patchlevel.h and in doc/conf.py. The branch
- will capture the revision number of this change for the release.
+Make the release branch
+------------------------
- Some of the next updates may be done before or after branching; make
- sure things are ported back to the trunk and to the branch as
- necessary.
+This is needed only in case you are doing a new major version; if not, you can
+probably reuse the existing release branch.
+
+We want to be able to freely merge default into the branch and vice-versa;
+thus we need to do a complicate dance to avoid to patch the version number
+when we do a merge::
+
+ $ hg up -r default
+ $ # edit the version to e.g. 7.0.0-final
+ $ hg ci
+ $ hg branch release-pypy2.7-7.x && hg ci
+ $ hg up -r default
+ $ # edit the version to 7.1.0-alpha0
+ $ hg ci
+ $ hg up -r release-pypy2.7-7.x
+ $ hg merge default
+ $ # edit the version to AGAIN 7.0.0-final
+ $ hg ci
+
+Then, we need to do the same for the 3.x branch::
+
+ $ hg up -r py3.5
+ $ hg merge default # this brings the version fo 7.1.0-alpha0
+ $ hg branch release-pypy3.5-7.x
+ $ # edit the version to 7.0.0-final
+ $ hg ci
+ $ hg up -r py3.5
+ $ hg merge release-pypy3.5-7.x
+ $ # edit the version to 7.1.0-alpha0
+ $ hg ci
+
+To change the version, you need to edit three files:
+
+ - ``module/sys/version.py``
+
+ - ``module/cpyext/include/patchlevel.h``
+
+ - ``doc/conf.py``
+
+
+Other steps
+-----------
+
* Make sure the RPython builds on the buildbot pass with no failures
diff --git a/pypy/doc/index-of-release-notes.rst
b/pypy/doc/index-of-release-notes.rst
--- a/pypy/doc/index-of-release-notes.rst
+++ b/pypy/doc/index-of-release-notes.rst
@@ -6,6 +6,7 @@
.. toctree::
+ release-v7.0.0.rst
release-v6.0.0.rst
release-v5.10.1.rst
release-v5.10.0.rst
diff --git a/pypy/doc/index-of-whatsnew.rst b/pypy/doc/index-of-whatsnew.rst
--- a/pypy/doc/index-of-whatsnew.rst
+++ b/pypy/doc/index-of-whatsnew.rst
@@ -7,6 +7,7 @@
.. toctree::
whatsnew-head.rst
+ whatsnew-pypy2-7.0.0.rst
whatsnew-pypy2-6.0.0.rst
whatsnew-pypy2-5.10.0.rst
whatsnew-pypy2-5.10.0.rst
@@ -41,6 +42,7 @@
.. toctree::
whatsnew-pypy3-head.rst
+ whatsnew-pypy3-7.0.0.rst
whatsnew-pypy3-5.9.0.rst
whatsnew-pypy3-5.8.0.rst
whatsnew-pypy3-5.7.0.rst
diff --git a/pypy/doc/interpreter.rst b/pypy/doc/interpreter.rst
--- a/pypy/doc/interpreter.rst
+++ b/pypy/doc/interpreter.rst
@@ -156,7 +156,7 @@
environment found in `Frames`. Frames and Functions have references
to a code object. Here is a list of Code attributes:
-* ``co_flags`` flags if this code object has nested scopes/generators
+* ``co_flags`` flags if this code object has nested scopes/generators/etc.
* ``co_stacksize`` the maximum depth the stack can reach while executing the
code
* ``co_code`` the actual bytecode string
diff --git a/pypy/doc/release-v7.0.0.rst b/pypy/doc/release-v7.0.0.rst
new file mode 100644
--- /dev/null
+++ b/pypy/doc/release-v7.0.0.rst
@@ -0,0 +1,151 @@
+======================================================
+PyPy v7.0.0: triple release of 2.7, 3.5 and 3.6-alpha
+======================================================
+
+The PyPy team is proud to release the version 7.0.0 of PyPy, which includes
+three different interpreters:
+
+ - PyPy2.7, which is an interpreter supporting the syntax and the features of
+ Python 2.7
+
+ - PyPy3.5, which supports Python 3.5
+
+ - PyPy3.6-alpha: this is the first official release of PyPy to support 3.6
+ features, although it is still considered alpha quality.
+
+All the interpreters are based on much the same codebase, thus the triple
+release.
+
+Until we can work with downstream providers to distribute builds with PyPy, we
+have made packages for some common packages `available as wheels`_.
+
+The GC `hooks`_ , which can be used to gain more insights into its
+performance, has been improved and it is now possible to manually manage the
+GC by using a combination of ``gc.disable`` and ``gc.collect_step``. See the
+`GC blog post`_.
+
+
+We updated the `cffi`_ module included in PyPy to version 1.12, and the
+`cppyy`_ backend to 1.4. Please use these to wrap your C and C++ code,
+respectively, for a JIT friendly experience.
+
+As always, this release is 100% compatible with the previous one and fixed
+several issues and bugs raised by the growing community of PyPy users.
+We strongly recommend updating.
+
+The PyPy3.6 release and the Windows PyPy3.5 release are still not production
+quality so your mileage may vary. There are open issues with incomplete
+compatibility and c-extension support.
+
+The utf8 branch that changes internal representation of unicode to utf8 did not
+make it into the release, so there is still more goodness coming.
+You can download the v7.0 releases here:
+
+ http://pypy.org/download.html
+
+We would like to thank our donors for the continued support of the PyPy
+project. If PyPy is not quite good enough for your needs, we are available for
+direct consulting work.
+
+We would also like to thank our contributors and encourage new people to join
+the project. PyPy has many layers and we need help with all of them: `PyPy`_
+and `RPython`_ documentation improvements, tweaking popular `modules`_ to run
+on pypy, or general `help`_ with making RPython's JIT even better.
+
+.. _`PyPy`: index.html
+.. _`RPython`: https://rpython.readthedocs.org
+.. _`help`: project-ideas.html
+.. _`cffi`: http://cffi.readthedocs.io
+.. _`cppyy`: https://cppyy.readthedocs.io
+.. _`available as wheels`: https://github.com/antocuni/pypy-wheels
+.. _`GC blog post`:
https://morepypy.blogspot.com/2019/01/pypy-for-low-latency-systems.html
+
+
+What is PyPy?
+=============
+
+PyPy is a very compliant Python interpreter, almost a drop-in replacement for
+CPython 2.7, 3.5 and 3.6. It's fast (`PyPy and CPython 2.7.x`_ performance
+comparison) due to its integrated tracing JIT compiler.
+
+We also welcome developers of other `dynamic languages`_ to see what RPython
+can do for them.
+
+The PyPy release supports:
+
+ * **x86** machines on most common operating systems
+ (Linux 32/64 bits, Mac OS X 64 bits, Windows 32 bits, OpenBSD, FreeBSD)
+
+ * big- and little-endian variants of **PPC64** running Linux,
+
+ * **s390x** running Linux
+
+Unfortunately at the moment of writing our ARM buildbots are out of service,
+so for now we are **not** releasing any binary for the ARM architecture.
+
+.. _`PyPy and CPython 2.7.x`: http://speed.pypy.org
+.. _`dynamic languages`: http://rpython.readthedocs.io/en/latest/examples.html
+
+
+Changelog
+=========
+
+If not specified, the changes are shared across versions
+
+* Support ``__set_name__``, ``__init_subclass__`` (Py3.6)
+* Support ``cppyy`` in Py3.5 and Py3.6
+* Use implementation-specific site directories in ``sysconfig`` (Py3.5, Py3.6)
+* Adding detection of gcc to ``sysconfig`` (Py3.5, Py3.6)
+* Fix multiprocessing regression on newer glibcs
+* Make sure 'blocking-ness' of socket is set along with default timeout
+* Include ``crypt.h`` for ``crypt()`` on Linux
+* Improve and re-organize the contributing_ documentation
+* Make the ``__module__`` attribute writable, fixing an incompatibility with
+ NumPy 1.16
+* Implement ``Py_ReprEnter``, ``Py_ReprLeave(),
``PyMarshal_ReadObjectFromString``,
+ ``PyMarshal_WriteObjectToString``, ``PyObject_DelItemString``,
+ ``PyMapping_DelItem``, ``PyMapping_DelItemString``, ``PyEval_GetFrame``,
+ ``PyOS_InputHook``, ``PyErr_FormatFromCause`` (Py3.6),
+* Implement new wordcode instruction encoding (Py3.6)
+* Log additional gc-minor and gc-collect-step info in the PYPYLOG
+* The ``reverse-debugger`` (revdb) branch has been merged to the default
+ branch, so it should always be up-to-date. You still need a special pypy
+ build, but you can compile it from the same source as the one we distribute
+ for the v7.0.0 release. For more information, see
+ https://bitbucket.org/pypy/revdb
+* Support underscores in numerical literals like ``'4_2'`` (Py3.6)
+* Pre-emptively raise MemoryError if the size of dequeue in
``_collections.deque``
+ is too large (Py3.5)
+* Fix multithreading issues in calls to ``os.setenv``
+* Add missing defines and typedefs for numpy and pandas on MSVC
+* Add CPython macros like ``Py_NAN`` to header files
+* Rename the ``MethodType`` to ``instancemethod``, like CPython
+* Better support for `async with` in generators (Py3.5, Py3.6)
+* Improve the performance of ``pow(a, b, c)`` if ``c`` is a large integer
+* Now ``vmprof`` works on FreeBSD
+* Support GNU Hurd, fixes for FreeBSD
+* Add deprecation warning if type of result of ``__float__`` is float inherited
+ class (Py3.6)
+* Fix async generator bug when yielding a ``StopIteration`` (Py3.6)
+* Speed up ``max(list-of-int)`` from non-jitted code
+* Fix Windows ``os.listdir()`` for some cases (see CPython #32539)
+* Add ``select.PIPE_BUF``
+* Use ``subprocess`` to avoid shell injection in ``shutil`` module - backport
+ of https://bugs.python.org/issue34540
+* Rename ``_Py_ZeroStruct`` to ``_Py_FalseStruct`` (Py3.5, Py3.6)
+* Remove some cpyext names for Py3.5, Py3.6
+* Enable use of unicode file names in ``dlopen``
+* Backport CPython fix for ``thread.RLock``
+* Make GC hooks measure time in seconds (as opposed to an opaque unit)
+* Refactor and reorganize tests in ``test_lib_pypy``
+* Check error values in ``socket.setblocking`` (Py3.6)
+* Add support for FsPath to os.unlink() (Py3.6)
+* Fix freezing builtin modules at translation
+* Tweak ``W_UnicodeDictionaryStrategy`` which speeds up dictionaries with only
+ unicode keys
+
+We also refactored many parts of the JIT bridge optimizations, as well as
cpyext
+internals, and together with new contributors fixed issues, added new
+documentation, and cleaned up the codebase.
+
+.. _contributing: http://doc.pypy.org/en/latest/contributing.html
diff --git a/pypy/doc/whatsnew-head.rst b/pypy/doc/whatsnew-head.rst
--- a/pypy/doc/whatsnew-head.rst
+++ b/pypy/doc/whatsnew-head.rst
@@ -1,78 +1,30 @@
==========================
-What's new in PyPy2.7 6.0+
+What's new in PyPy2.7 7.0+
==========================
-.. this is a revision shortly after release-pypy-6.0.0
-.. startrev: e50e11af23f1
+.. this is a revision shortly after release-pypy-7.0.0
+.. startrev: 481c69f7d81f
-.. branch: cppyy-packaging
+.. branch: zlib-copying-third-time-a-charm
-Main items: vastly better template resolution and improved performance. In
-detail: upgrade to backend 1.4, improved handling of templated methods and
-functions (in particular automatic deduction of types), improved pythonization
-interface, range of compatibility fixes for Python3, free functions now take
-fast libffi path when possible, moves for strings (incl. from Python str),
-easier/faster handling of std::vector by numpy, improved and faster object
-identity preservation
+Make sure zlib decompressobjs have their streams deallocated immediately
+on flush.
-.. branch: socket_default_timeout_blockingness
+.. branch: zlib-copying-redux
-Make sure 'blocking-ness' of socket is set along with default timeout
+Fix calling copy on already-flushed compressobjs.
-.. branch: crypt_h
+.. branch: zlib-copying
-Include crypt.h for crypt() on Linux
+The zlib module's compressobj and decompressobj now expose copy methods
+as they do on CPython.
-.. branch: gc-more-logging
-Log additional gc-minor and gc-collect-step info in the PYPYLOG
+.. branch: math-improvements
-.. branch: reverse-debugger
+Improve performance of long operations where one of the operands fits into
+an int.
-The reverse-debugger branch has been merged. For more information, see
-https://bitbucket.org/pypy/revdb
+.. branch: regalloc-playgrounds
-.. branch: pyparser-improvements-3
-
-Small refactorings in the Python parser.
-
-.. branch: fix-readme-typo
-
-.. branch: avoid_shell_injection_in_shutil
-
-.. branch: unicode-utf8-re
-.. branch: utf8-io
-
-Utf8 handling for unicode
-
-Backport CPython fix for possible shell injection issue in `distutils.spawn`,
-https://bugs.python.org/issue34540
-
-.. branch: cffi_dlopen_unicode
-
-Enable use of unicode file names in `dlopen`
-
-.. branch: rlock-in-rpython
-
-Backport CPython fix for `thread.RLock`
-
-
-.. branch: expose-gc-time
-
-Make GC hooks measure time in seconds (as opposed to an opaque unit).
-
-.. branch: cleanup-test_lib_pypy
-
-Update most test_lib_pypy/ tests and move them to extra_tests/.
-
-.. branch: gc-disable
-
-Make it possible to manually manage the GC by using a combination of
-gc.disable() and gc.collect_step(). Make sure to write a proper release
-announcement in which we explain that existing programs could leak memory if
-they run for too much time between a gc.disable()/gc.enable()
-
-.. branch: unicode-utf8
-
-Use utf8 internally to represent unicode
-
+Improve register allocation in the JIT.
diff --git a/pypy/doc/whatsnew-pypy2-5.10.0.rst
b/pypy/doc/whatsnew-pypy2-5.10.0.rst
--- a/pypy/doc/whatsnew-pypy2-5.10.0.rst
+++ b/pypy/doc/whatsnew-pypy2-5.10.0.rst
@@ -1,42 +1,42 @@
-==========================
-What's new in PyPy2.7 5.10
-==========================
-
-.. this is a revision shortly after release-pypy2.7-v5.9.0
-.. startrev:d56dadcef996
-
-
-.. branch: cppyy-packaging
-
-Cleanup and improve cppyy packaging
-
-.. branch: docs-osx-brew-openssl
-
-.. branch: keep-debug-symbols
-
-Add a smartstrip tool, which can optionally keep the debug symbols in a
-separate file, instead of just stripping them away. Use it in packaging
-
-.. branch: bsd-patches
-
-Fix failures on FreeBSD, contributed by David Naylor as patches on the issue
-tracker (issues 2694, 2695, 2696, 2697)
-
-.. branch: run-extra-tests
-
-Run extra_tests/ in buildbot
-
-.. branch: vmprof-0.4.10
-
-Upgrade the _vmprof backend to vmprof 0.4.10
-
-.. branch: fix-vmprof-stacklet-switch
-.. branch: fix-vmprof-stacklet-switch-2
-
-Fix a vmprof+continulets (i.e. greenelts, eventlet, gevent, ...)
-
-.. branch: win32-vcvars
-
-.. branch: rdict-fast-hash
-
-Make it possible to declare that the hash function of an r_dict is fast in
RPython.
_______________________________________________
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit
