Author: stian
Branch: math-improvements
Changeset: r95240:8679952ae1fd
Date: 2018-10-25 13:55 +0200
http://bitbucket.org/pypy/pypy/changeset/8679952ae1fd/
Log: Merge default
diff too long, truncating to 2000 out of 44960 lines
diff --git a/.hgtags b/.hgtags
--- a/.hgtags
+++ b/.hgtags
@@ -33,7 +33,12 @@
050d84dd78997f021acf0e133934275d63547cc0 release-pypy2.7-v5.4.1
050d84dd78997f021acf0e133934275d63547cc0 release-pypy2.7-v5.4.1
0e2d9a73f5a1818d0245d75daccdbe21b2d5c3ef release-pypy2.7-v5.4.1
+4909c06daf41ce88f87dc01c57959cadad4df4a8 RevDB-pypy2.7-v5.4.1
+4909c06daf41ce88f87dc01c57959cadad4df4a8 RevDB-pypy2.7-v5.4.1
+d7724c0a5700b895a47de44074cdf5fd659a988f RevDB-pypy2.7-v5.4.1
aff251e543859ce4508159dd9f1a82a2f553de00 release-pypy2.7-v5.6.0
+e90317857d27917bf840caf675832292ee070510 RevDB-pypy2.7-v5.6.1
+a24d6c7000c8099c73d3660857f7e3cee5ac045c RevDB-pypy2.7-v5.6.2
fa3249d55d15b9829e1be69cdf45b5a44cec902d release-pypy2.7-v5.7.0
b16a4363e930f6401bceb499b9520955504c6cb0 release-pypy3.5-v5.7.0
1aa2d8e03cdfab54b7121e93fda7e98ea88a30bf release-pypy2.7-v5.7.1
@@ -51,3 +56,5 @@
0000000000000000000000000000000000000000 release-pypy3.5-v5.10.0
09f9160b643e3f02ccb8c843b2fbb4e5cbf54082 release-pypy3.5-v5.10.0
3f6eaa010fce78cc7973bdc1dfdb95970f08fed2 release-pypy3.5-v5.10.1
+ab0b9caf307db6592905a80b8faffd69b39005b8 release-pypy2.7-v6.0.0
+fdd60ed87e941677e8ea11acf9f1819466521bf2 release-pypy3.5-v6.0.0
diff --git a/LICENSE b/LICENSE
--- a/LICENSE
+++ b/LICENSE
@@ -6,36 +6,36 @@
Except when otherwise stated (look for LICENSE files in directories or
information at the beginning of each file) all software and documentation in
the 'rpython', 'pypy', 'ctype_configure', 'dotviewer', 'demo', 'lib_pypy',
-'py', and '_pytest' directories is licensed as follows:
+'py', and '_pytest' directories is licensed as follows:
The MIT License
- Permission is hereby granted, free of charge, to any person
- obtaining a copy of this software and associated documentation
- files (the "Software"), to deal in the Software without
- restriction, including without limitation the rights to use,
- copy, modify, merge, publish, distribute, sublicense, and/or
- sell copies of the Software, and to permit persons to whom the
+ Permission is hereby granted, free of charge, to any person
+ obtaining a copy of this software and associated documentation
+ files (the "Software"), to deal in the Software without
+ restriction, including without limitation the rights to use,
+ copy, modify, merge, publish, distribute, sublicense, and/or
+ sell copies of the Software, and to permit persons to whom the
Software is furnished to do so, subject to the following conditions:
- The above copyright notice and this permission notice shall be included
+ The above copyright notice and this permission notice shall be included
in all copies or substantial portions of the Software.
- THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
- OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY,
- FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
- FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
+ OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
PyPy Copyright holders 2003-2018
------------------------------------
+--------------------------------
Except when otherwise stated (look for LICENSE files or information at
the beginning of each file) the files in the 'pypy' directory are each
-copyrighted by one or more of the following people and organizations:
+copyrighted by one or more of the following people and organizations:
Armin Rigo
Maciej Fijalkowski
@@ -89,13 +89,13 @@
Niko Matsakis
Alexander Hesse
Ludovic Aubry
+ stian
Jacob Hallen
Jason Creighton
Mark Young
Alex Martelli
Spenser Bauman
Michal Bendowski
- stian
Jan de Mooij
Tyler Wade
Vincent Legoll
@@ -123,10 +123,10 @@
Wenzhu Man
Konstantin Lopuhin
John Witulski
+ Jeremy Thurgood
Greg Price
Ivan Sichmann Freitas
Dario Bertini
- Jeremy Thurgood
Mark Pearse
Simon Cross
Tobias Pape
@@ -145,18 +145,19 @@
Adrian Kuhn
tav
Georg Brandl
+ Joannah Nanjekye
Bert Freudenberg
Stian Andreassen
Wanja Saatkamp
Mike Blume
- Joannah Nanjekye
Gerald Klix
Oscar Nierstrasz
Rami Chowdhury
Stefan H. Muller
+ Dodan Mihai
Tim Felgentreff
Eugene Oden
- Dodan Mihai
+ Colin Valliant
Jeff Terrace
Henry Mason
Vasily Kuznetsov
@@ -225,12 +226,14 @@
Vaibhav Sood
Reuben Cummings
Attila Gobi
+ Floris Bruynooghe
Christopher Pope
Tristan Arthur
Christian Tismer
Dan Stromberg
Carl Meyer
Florin Papa
+ Arianna Avanzini
Jens-Uwe Mager
Valentina Mukhamedzhanova
Stefano Parmesan
@@ -244,15 +247,18 @@
Lukas Vacek
Omer Katz
Jacek Generowicz
+ Tomasz Dziopa
Sylvain Thenault
Jakub Stasiak
Andrew Dalke
Alejandro J. Cura
Vladimir Kryachko
Gabriel
+ Thomas Hisch
Mark Williams
Kunal Grover
Nathan Taylor
+ Barry Hart
Travis Francis Athougies
Yasir Suhail
Sergey Kishchenko
@@ -260,6 +266,7 @@
Lutz Paelike
Ian Foote
Philipp Rustemeuer
+ Logan Chien
Catalin Gabriel Manciu
Jacob Oscarson
Ryan Gonzalez
@@ -295,19 +302,20 @@
Akira Li
Gustavo Niemeyer
Rafał Gałczyński
- Logan Chien
Lucas Stadler
roberto@goyle
Matt Bogosian
Yury V. Zaytsev
florinpapa
Anders Sigfridsson
+ Matt Jackson
Nikolay Zinov
rafalgalczyn...@gmail.com
Joshua Gilbert
Anna Katrina Dominguez
Kim Jin Su
Amber Brown
+ Miro Hrončok
Anthony Sottile
Nate Bragg
Ben Darnell
@@ -315,7 +323,6 @@
Godefroid Chappelle
Julian Berman
Michael Hudson-Doyle
- Floris Bruynooghe
Stephan Busemann
Dan Colish
timo
@@ -357,6 +364,7 @@
Michael Chermside
Anna Ravencroft
remarkablerocket
+ Pauli Virtanen
Petre Vijiac
Berker Peksag
Christian Muirhead
@@ -381,6 +389,7 @@
Graham Markall
Dan Loewenherz
werat
+ Andrew Stepanov
Niclas Olofsson
Chris Pressey
Tobias Diaz
@@ -395,14 +404,14 @@
m...@funkyhat.org
Stefan Marr
- Heinrich-Heine University, Germany
+ Heinrich-Heine University, Germany
Open End AB (formerly AB Strakt), Sweden
- merlinux GmbH, Germany
- tismerysoft GmbH, Germany
- Logilab Paris, France
- DFKI GmbH, Germany
+ merlinux GmbH, Germany
+ tismerysoft GmbH, Germany
+ Logilab Paris, France
+ DFKI GmbH, Germany
Impara, Germany
- Change Maker, Sweden
+ Change Maker, Sweden
University of California Berkeley, USA
Google Inc.
King's College London
@@ -410,14 +419,14 @@
The PyPy Logo as used by http://speed.pypy.org and others was created
by Samuel Reis and is distributed on terms of Creative Commons Share Alike
License.
-
-License for 'lib-python/2.7'
-============================
+
+License for 'lib-python/2.7, lib-python/3'
+==========================================
Except when otherwise stated (look for LICENSE files or copyright/license
-information at the beginning of each file) the files in the 'lib-python/2.7'
+information at the beginning of each file) the files in the 'lib-python'
directory are all copyrighted by the Python Software Foundation and licensed
-under the terms that you can find here: https://docs.python.org/2/license.html
+under the terms that you can find here: https://docs.python.org/3/license.html
License for 'pypy/module/unicodedata/'
======================================
@@ -441,9 +450,9 @@
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
-
+
http://www.apache.org/licenses/LICENSE-2.0
-
+
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
diff --git a/README.rst b/README.rst
--- a/README.rst
+++ b/README.rst
@@ -4,42 +4,40 @@
Welcome to PyPy!
-PyPy is both an implementation of the Python programming language, and
-an extensive compiler framework for dynamic language implementations.
-You can build self-contained Python implementations which execute
-independently from CPython.
+PyPy is an interpreter that implements the Python programming language, based
+on the RPython compiler framework for dynamic language implementations.
-The home page is:
+The home page for the interpreter is:
http://pypy.org/
-If you want to help developing PyPy, this document might help you:
+If you want to help developing PyPy, this documentation might help you:
http://doc.pypy.org/
-It will also point you to the rest of the documentation which is generated
-from files in the pypy/doc directory within the source repositories. Enjoy
-and send us feedback!
+More documentation about the RPython framework can be found here:
+
+ http://rpython.readthedocs.io/
+
+The source for the documentation is in the pypy/doc directory.
+
+
+Using PyPy instead of CPython
+-----------------------------
+
+Please read the information at http://pypy.org/ to find the correct way to
+download and use PyPy as an alternative to CPython.
+
+
+Building
+--------
+
+Building PyPy is not the recommended way to obtain the PyPy alternative python
+interpreter. It is time-consuming and requires significant computing resources.
+More information can be found here:
+
+ http://doc.pypy.org/en/latest/build.html
+
+Enjoy and send us feedback!
the pypy-dev team <pypy-...@python.org>
-
-
-Building
-========
-
-First switch to or download the correct branch. The basic choices are
-``default`` for Python 2.7 and, for Python 3.X, the corresponding py3.X
-branch (e.g. ``py3.5``).
-
-Build with:
-
-.. code-block:: console
-
- $ rpython/bin/rpython -Ojit pypy/goal/targetpypystandalone.py
-
-This ends up with a ``pypy-c`` or ``pypy3-c`` binary in the main pypy
-directory. We suggest to use virtualenv with the resulting
-pypy-c/pypy3-c as the interpreter; you can find more details about
-various installation schemes here:
-
- http://doc.pypy.org/en/latest/install.html
diff --git a/dotviewer/font/NOTICE b/dotviewer/font/COPYING.txt
rename from dotviewer/font/NOTICE
rename to dotviewer/font/COPYING.txt
diff --git a/pypy/module/test_lib_pypy/pyrepl/__init__.py
b/extra_tests/test_pyrepl/__init__.py
rename from pypy/module/test_lib_pypy/pyrepl/__init__.py
rename to extra_tests/test_pyrepl/__init__.py
--- a/pypy/module/test_lib_pypy/pyrepl/__init__.py
+++ b/extra_tests/test_pyrepl/__init__.py
@@ -1,3 +1,1 @@
-import sys
-import lib_pypy.pyrepl
-sys.modules['pyrepl'] = sys.modules['lib_pypy.pyrepl']
+
diff --git a/pypy/module/test_lib_pypy/pyrepl/infrastructure.py
b/extra_tests/test_pyrepl/infrastructure.py
rename from pypy/module/test_lib_pypy/pyrepl/infrastructure.py
rename to extra_tests/test_pyrepl/infrastructure.py
diff --git a/pypy/module/test_lib_pypy/pyrepl/test_basic.py
b/extra_tests/test_pyrepl/test_basic.py
rename from pypy/module/test_lib_pypy/pyrepl/test_basic.py
rename to extra_tests/test_pyrepl/test_basic.py
diff --git a/pypy/module/test_lib_pypy/pyrepl/test_bugs.py
b/extra_tests/test_pyrepl/test_bugs.py
rename from pypy/module/test_lib_pypy/pyrepl/test_bugs.py
rename to extra_tests/test_pyrepl/test_bugs.py
diff --git a/pypy/module/test_lib_pypy/pyrepl/test_functional.py
b/extra_tests/test_pyrepl/test_functional.py
rename from pypy/module/test_lib_pypy/pyrepl/test_functional.py
rename to extra_tests/test_pyrepl/test_functional.py
--- a/pypy/module/test_lib_pypy/pyrepl/test_functional.py
+++ b/extra_tests/test_pyrepl/test_functional.py
@@ -7,7 +7,8 @@
import sys
-def pytest_funcarg__child(request):
+@pytest.fixture()
+def child():
try:
import pexpect
except ImportError:
diff --git a/pypy/module/test_lib_pypy/pyrepl/test_keymap.py
b/extra_tests/test_pyrepl/test_keymap.py
rename from pypy/module/test_lib_pypy/pyrepl/test_keymap.py
rename to extra_tests/test_pyrepl/test_keymap.py
diff --git a/pypy/module/test_lib_pypy/pyrepl/test_reader.py
b/extra_tests/test_pyrepl/test_reader.py
rename from pypy/module/test_lib_pypy/pyrepl/test_reader.py
rename to extra_tests/test_pyrepl/test_reader.py
diff --git a/pypy/module/test_lib_pypy/pyrepl/test_readline.py
b/extra_tests/test_pyrepl/test_readline.py
rename from pypy/module/test_lib_pypy/pyrepl/test_readline.py
rename to extra_tests/test_pyrepl/test_readline.py
diff --git a/pypy/module/test_lib_pypy/pyrepl/test_wishes.py
b/extra_tests/test_pyrepl/test_wishes.py
rename from pypy/module/test_lib_pypy/pyrepl/test_wishes.py
rename to extra_tests/test_pyrepl/test_wishes.py
diff --git a/get_externals.py b/get_externals.py
new file mode 100644
--- /dev/null
+++ b/get_externals.py
@@ -0,0 +1,69 @@
+'''Get external dependencies for building PyPy
+they will end up in the platform.host().basepath, something like
repo-root/external
+'''
+
+from __future__ import print_function
+
+import argparse
+import os
+import zipfile
+from subprocess import Popen, PIPE
+from rpython.translator.platform import host
+
+def runcmd(cmd, verbose):
+ stdout = stderr = ''
+ report = False
+ try:
+ p = Popen(cmd, stdout=PIPE, stderr=PIPE)
+ stdout, stderr = p.communicate()
+ if p.wait() != 0 or verbose:
+ report = True
+ except Exception as e:
+ stderr = str(e) + '\n' + stderr
+ report = True
+ if report:
+ print('running "%s" returned\n%s\n%s' % (' '.join(cmd), stdout,
stderr))
+ if stderr:
+ raise RuntimeError(stderr)
+
+def checkout_repo(dest='externals', org='pypy', branch='default',
verbose=False):
+ url = 'https://bitbucket.org/{}/externals'.format(org)
+ if not os.path.exists(dest):
+ cmd = ['hg','clone',url,dest]
+ runcmd(cmd, verbose)
+ cmd = ['hg','-R', dest, 'update',branch]
+ runcmd(cmd, verbose)
+
+def extract_zip(externals_dir, zip_path):
+ with zipfile.ZipFile(os.fspath(zip_path)) as zf:
+ zf.extractall(os.fspath(externals_dir))
+ return externals_dir / zf.namelist()[0].split('/')[0]
+
+def parse_args():
+ p = argparse.ArgumentParser()
+ p.add_argument('-v', '--verbose', action='store_true')
+ p.add_argument('-O', '--organization',
+ help='Organization owning the deps repos', default='pypy')
+ p.add_argument('-e', '--externals', default=host.externals,
+ help='directory in which to store dependencies',
+ )
+ p.add_argument('-b', '--branch', default=host.externals_branch,
+ help='branch to check out',
+ )
+ p.add_argument('-p', '--platform', default=None,
+ help='someday support cross-compilation, ignore for now',
+ )
+ return p.parse_args()
+
+
+def main():
+ args = parse_args()
+ checkout_repo(
+ dest=args.externals,
+ org=args.organization,
+ branch=args.branch,
+ verbose=args.verbose,
+ )
+
+if __name__ == '__main__':
+ main()
diff --git a/lib-python/2.7/code.py b/lib-python/2.7/code.py
--- a/lib-python/2.7/code.py
+++ b/lib-python/2.7/code.py
@@ -104,6 +104,12 @@
except SystemExit:
raise
except:
+ if softspace(sys.stdout, 0):
+ print
+ try:
+ sys.stdout.flush()
+ except:
+ pass
self.showtraceback()
else:
if softspace(sys.stdout, 0):
diff --git a/lib-python/2.7/fractions.py b/lib-python/2.7/fractions.py
--- a/lib-python/2.7/fractions.py
+++ b/lib-python/2.7/fractions.py
@@ -517,8 +517,13 @@
# Get integers right.
return hash(self._numerator)
# Expensive check, but definitely correct.
- if self == float(self):
- return hash(float(self))
+ # PyPy: the following 4 lines used to be almost twice slower:
+ # if self == float(self):
+ # return hash(float(self))
+ f = float(self)
+ x, y = f.as_integer_ratio() # signs are correct: y is positive
+ if self._numerator == x and self._denominator == y:
+ return hash(f)
else:
# Use tuple's hash to avoid a high collision rate on
# simple fractions.
diff --git a/lib-python/2.7/hashlib.py b/lib-python/2.7/hashlib.py
--- a/lib-python/2.7/hashlib.py
+++ b/lib-python/2.7/hashlib.py
@@ -136,9 +136,14 @@
__get_hash = __get_openssl_constructor
algorithms_available = algorithms_available.union(
_hashlib.openssl_md_meth_names)
-except ImportError:
+except ImportError as e:
new = __py_new
__get_hash = __get_builtin_constructor
+ # added by PyPy
+ import warnings
+ warnings.warn("The _hashlib module is not available, falling back "
+ "to a much slower implementation (%s)" % str(e),
+ RuntimeWarning)
for __func_name in __always_supported:
# try them all, some may not work due to the OpenSSL
diff --git a/lib-python/2.7/opcode.py b/lib-python/2.7/opcode.py
--- a/lib-python/2.7/opcode.py
+++ b/lib-python/2.7/opcode.py
@@ -194,5 +194,6 @@
def_op('CALL_METHOD', 202) # #args not including 'self'
def_op('BUILD_LIST_FROM_ARG', 203)
jrel_op('JUMP_IF_NOT_DEBUG', 204) # jump over assert statements
+def_op('LOAD_REVDB_VAR', 205) # reverse debugger (syntax example: $5)
del def_op, name_op, jrel_op, jabs_op
diff --git a/lib-python/2.7/re.py b/lib-python/2.7/re.py
--- a/lib-python/2.7/re.py
+++ b/lib-python/2.7/re.py
@@ -225,7 +225,7 @@
_pattern_type = type(sre_compile.compile("", 0))
-_MAXCACHE = 100
+_MAXCACHE = 1000
def _compile(*key):
# internal: compile pattern
diff --git a/lib-python/2.7/shutil.py b/lib-python/2.7/shutil.py
--- a/lib-python/2.7/shutil.py
+++ b/lib-python/2.7/shutil.py
@@ -396,17 +396,21 @@
return archive_name
-def _call_external_zip(base_dir, zip_filename, verbose=False, dry_run=False):
+def _call_external_zip(base_dir, zip_filename, verbose, dry_run, logger):
# XXX see if we want to keep an external call here
if verbose:
zipoptions = "-r"
else:
zipoptions = "-rq"
- from distutils.errors import DistutilsExecError
- from distutils.spawn import spawn
+ cmd = ["zip", zipoptions, zip_filename, base_dir]
+ if logger is not None:
+ logger.info(' '.join(cmd))
+ if dry_run:
+ return
+ import subprocess
try:
- spawn(["zip", zipoptions, zip_filename, base_dir], dry_run=dry_run)
- except DistutilsExecError:
+ subprocess.check_call(cmd)
+ except subprocess.CalledProcessError:
# XXX really should distinguish between "couldn't find
# external 'zip' command" and "zip failed".
raise ExecError, \
@@ -440,7 +444,7 @@
zipfile = None
if zipfile is None:
- _call_external_zip(base_dir, zip_filename, verbose, dry_run)
+ _call_external_zip(base_dir, zip_filename, verbose, dry_run, logger)
else:
if logger is not None:
logger.info("creating '%s' and adding '%s' to it",
diff --git a/lib-python/2.7/test/test_eof.py b/lib-python/2.7/test/test_eof.py
--- a/lib-python/2.7/test/test_eof.py
+++ b/lib-python/2.7/test/test_eof.py
@@ -5,7 +5,7 @@
class EOFTestCase(unittest.TestCase):
def test_EOFC(self):
- expect = "EOL while scanning string literal (<string>, line 1)"
+ expect = "end of line (EOL) while scanning string literal (<string>,
line 1)"
try:
eval("""'this is a test\
""")
@@ -15,7 +15,7 @@
raise test_support.TestFailed
def test_EOFS(self):
- expect = ("EOF while scanning triple-quoted string literal "
+ expect = ("end of file (EOF) while scanning triple-quoted string
literal "
"(<string>, line 1)")
try:
eval("""'''this is a test""")
diff --git a/lib-python/2.7/test/test_generators.py
b/lib-python/2.7/test/test_generators.py
--- a/lib-python/2.7/test/test_generators.py
+++ b/lib-python/2.7/test/test_generators.py
@@ -398,7 +398,10 @@
0
>>> type(i.gi_frame)
<type 'frame'>
->>> i.gi_running = 42
+
+PyPy prints "readonly attribute 'gi_running'" so ignore the exception detail
+
+>>> i.gi_running = 42 # doctest: +IGNORE_EXCEPTION_DETAIL
Traceback (most recent call last):
...
TypeError: readonly attribute
diff --git a/lib-python/2.7/test/test_genexps.py
b/lib-python/2.7/test/test_genexps.py
--- a/lib-python/2.7/test/test_genexps.py
+++ b/lib-python/2.7/test/test_genexps.py
@@ -87,7 +87,7 @@
>>> dict(a = i for i in xrange(10))
Traceback (most recent call last):
...
- SyntaxError: invalid syntax
+ SyntaxError: invalid syntax (expected ')')
Verify that parenthesis are required when used as a keyword argument value
diff --git a/lib-python/2.7/test/test_inspect.py
b/lib-python/2.7/test/test_inspect.py
--- a/lib-python/2.7/test/test_inspect.py
+++ b/lib-python/2.7/test/test_inspect.py
@@ -45,6 +45,9 @@
git = mod.StupidGit()
+class ExampleClassWithSlot(object):
+ __slots__ = 'myslot'
+
class IsTestBase(unittest.TestCase):
predicates = set([inspect.isbuiltin, inspect.isclass, inspect.iscode,
inspect.isframe, inspect.isfunction, inspect.ismethod,
@@ -96,7 +99,11 @@
else:
self.assertFalse(inspect.isgetsetdescriptor(type(tb.tb_frame).f_locals))
if hasattr(types, 'MemberDescriptorType'):
- self.istest(inspect.ismemberdescriptor, 'type(lambda:
None).func_globals')
+ # App-level slots are member descriptors on both PyPy and
+ # CPython, but the various built-in attributes are all
+ # getsetdescriptors on PyPy. So check ismemberdescriptor()
+ # with an app-level slot.
+ self.istest(inspect.ismemberdescriptor,
'ExampleClassWithSlot.myslot')
else:
self.assertFalse(inspect.ismemberdescriptor(type(lambda:
None).func_globals))
diff --git a/lib-python/2.7/test/test_traceback.py
b/lib-python/2.7/test/test_traceback.py
--- a/lib-python/2.7/test/test_traceback.py
+++ b/lib-python/2.7/test/test_traceback.py
@@ -123,10 +123,7 @@
self.assertEqual(len(err), 4)
self.assertEqual(err[1].strip(), "print(2)")
self.assertIn("^", err[2])
- if check_impl_detail():
- self.assertEqual(err[1].find("p"), err[2].find("^"))
- if check_impl_detail(pypy=True):
- self.assertEqual(err[1].find("2)") + 1, err[2].find("^"))
+ self.assertEqual(err[1].find("p"), err[2].find("^"))
def test_base_exception(self):
# Test that exceptions derived from BaseException are formatted right
diff --git a/lib-python/2.7/threading.py b/lib-python/2.7/threading.py
--- a/lib-python/2.7/threading.py
+++ b/lib-python/2.7/threading.py
@@ -351,6 +351,21 @@
# forward-compatibility reasons we do the same.
waiter.acquire()
gotit = True
+ except AttributeError:
+ # someone patched the 'waiter' class, probably.
+ # Fall back to the standard CPython logic.
+ # See the CPython lib for the comments about it...
+ endtime = _time() + timeout
+ delay = 0.0005 # 500 us -> initial delay of 1 ms
+ while True:
+ gotit = waiter.acquire(0)
+ if gotit:
+ break
+ remaining = endtime - _time()
+ if remaining <= 0:
+ break
+ delay = min(delay * 2, remaining, .05)
+ _sleep(delay)
else:
gotit = waiter.acquire(False)
if not gotit:
diff --git a/lib-python/2.7/types.py b/lib-python/2.7/types.py
--- a/lib-python/2.7/types.py
+++ b/lib-python/2.7/types.py
@@ -83,9 +83,19 @@
DictProxyType = type(TypeType.__dict__)
NotImplementedType = type(NotImplemented)
-# For Jython, the following two types are identical
+#
+# On CPython, FunctionType.__code__ is a 'getset_descriptor', but
+# FunctionType.__globals__ is a 'member_descriptor', just like app-level
+# slots. On PyPy, all descriptors of built-in types are
+# 'getset_descriptor', but the app-level slots are 'member_descriptor'
+# as well. (On Jython the situation might still be different.)
+#
+# Note that MemberDescriptorType was equal to GetSetDescriptorType in
+# PyPy <= 6.0.
+#
GetSetDescriptorType = type(FunctionType.func_code)
-MemberDescriptorType = type(FunctionType.func_globals)
+class _C(object): __slots__ = 's'
+MemberDescriptorType = type(_C.s)
del sys, _f, _g, _C, _x # Not for export
diff --git a/lib_pypy/_ctypes/array.py b/lib_pypy/_ctypes/array.py
--- a/lib_pypy/_ctypes/array.py
+++ b/lib_pypy/_ctypes/array.py
@@ -82,8 +82,11 @@
def _CData_output(self, resarray, base=None, index=-1):
from _rawffi.alt import types
# If a char_p or unichar_p is received, skip the string interpretation
- if base._ffiargtype != types.Pointer(types.char_p) and \
- base._ffiargtype != types.Pointer(types.unichar_p):
+ try:
+ deref = type(base)._deref_ffiargtype()
+ except AttributeError:
+ deref = None
+ if deref != types.char_p and deref != types.unichar_p:
# this seems to be a string if we're array of char, surprise!
from ctypes import c_char, c_wchar
if self._type_ is c_char:
@@ -120,6 +123,12 @@
value = self(*value)
return _CDataMeta.from_param(self, value)
+ def _build_ffiargtype(self):
+ return _ffi.types.Pointer(self._type_.get_ffi_argtype())
+
+ def _deref_ffiargtype(self):
+ return self._type_.get_ffi_argtype()
+
def array_get_slice_params(self, index):
if hasattr(self, '_length_'):
start, stop, step = index.indices(self._length_)
@@ -174,6 +183,7 @@
self._buffer = self._ffiarray(self._length_, autofree=True)
for i, arg in enumerate(args):
self[i] = arg
+ _init_no_arg_ = __init__
def _fix_index(self, index):
if index < 0:
@@ -248,6 +258,5 @@
_type_ = base
)
cls = ArrayMeta(name, (Array,), tpdict)
- cls._ffiargtype = _ffi.types.Pointer(base.get_ffi_argtype())
ARRAY_CACHE[key] = cls
return cls
diff --git a/lib_pypy/_ctypes/basics.py b/lib_pypy/_ctypes/basics.py
--- a/lib_pypy/_ctypes/basics.py
+++ b/lib_pypy/_ctypes/basics.py
@@ -49,10 +49,13 @@
else:
return self.from_param(as_parameter)
+ def _build_ffiargtype(self):
+ return _shape_to_ffi_type(self._ffiargshape_)
+
def get_ffi_argtype(self):
if self._ffiargtype:
return self._ffiargtype
- self._ffiargtype = _shape_to_ffi_type(self._ffiargshape_)
+ self._ffiargtype = self._build_ffiargtype()
return self._ffiargtype
def _CData_output(self, resbuffer, base=None, index=-1):
@@ -117,7 +120,7 @@
raise ValueError(
"Buffer size too small (%d instead of at least %d bytes)"
% (len(buf) + offset, size + offset))
- result = self()
+ result = self._newowninstance_()
dest = result._buffer.buffer
try:
raw_addr = buf._pypy_raw_address()
@@ -128,6 +131,11 @@
memmove(dest, raw_addr, size)
return result
+ def _newowninstance_(self):
+ result = self.__new__(self)
+ result._init_no_arg_()
+ return result
+
class CArgObject(object):
""" simple wrapper around buffer, just for the case of freeing
@@ -159,6 +167,7 @@
def __init__(self, *args, **kwds):
raise TypeError("%s has no type" % (type(self),))
+ _init_no_arg_ = __init__
def _ensure_objects(self):
if '_objects' not in self.__dict__:
diff --git a/lib_pypy/_ctypes/function.py b/lib_pypy/_ctypes/function.py
--- a/lib_pypy/_ctypes/function.py
+++ b/lib_pypy/_ctypes/function.py
@@ -268,6 +268,7 @@
return
raise TypeError("Unknown constructor %s" % (args,))
+ _init_no_arg_ = __init__
def _wrap_callable(self, to_call, argtypes):
def f(*args):
@@ -485,6 +486,8 @@
return cobj, cobj._to_ffi_param(), type(cobj)
def _convert_args_for_callback(self, argtypes, args):
+ from _ctypes.structure import StructOrUnion
+ #
assert len(argtypes) == len(args)
newargs = []
for argtype, arg in zip(argtypes, args):
@@ -494,6 +497,10 @@
param = param._get_buffer_value()
elif self._is_primitive(argtype):
param = param.value
+ elif isinstance(param, StructOrUnion): # not a *pointer* to
struct
+ newparam = StructOrUnion.__new__(type(param))
+ param._copy_to(newparam._buffer.buffer)
+ param = newparam
newargs.append(param)
return newargs
@@ -557,7 +564,7 @@
keepalive, newarg, newargtype =
self._conv_param(argtype, defval)
else:
import ctypes
- val = argtype._type_()
+ val = argtype._type_._newowninstance_()
keepalive = None
newarg = ctypes.byref(val)
newargtype = type(newarg)
diff --git a/lib_pypy/_ctypes/pointer.py b/lib_pypy/_ctypes/pointer.py
--- a/lib_pypy/_ctypes/pointer.py
+++ b/lib_pypy/_ctypes/pointer.py
@@ -67,10 +67,18 @@
self._buffer = ffiarray(1, autofree=True)
if value is not None:
self.contents = value
+ def _init_no_arg_(self):
+ self._buffer = ffiarray(1, autofree=True)
self._ffiarray = ffiarray
self.__init__ = __init__
+ self._init_no_arg_ = _init_no_arg_
self._type_ = TP
- self._ffiargtype = _ffi.types.Pointer(TP.get_ffi_argtype())
+
+ def _build_ffiargtype(self):
+ return _ffi.types.Pointer(self._type_.get_ffi_argtype())
+
+ def _deref_ffiargtype(self):
+ return self._type_.get_ffi_argtype()
from_address = cdata_from_address
@@ -132,27 +140,21 @@
if not (isinstance(tp, _CDataMeta) and tp._is_pointer_like()):
raise TypeError("cast() argument 2 must be a pointer type, not %s"
% (tp,))
+ result = tp._newowninstance_()
if isinstance(obj, (int, long)):
- result = tp()
result._buffer[0] = obj
return result
elif obj is None:
- result = tp()
return result
elif isinstance(obj, Array):
- ptr = tp.__new__(tp)
- ptr._buffer = tp._ffiarray(1, autofree=True)
- ptr._buffer[0] = obj._buffer
- result = ptr
+ result._buffer[0] = obj._buffer
elif isinstance(obj, bytes):
- result = tp()
result._buffer[0] = buffer(obj)._pypy_raw_address()
return result
elif not (isinstance(obj, _CData) and type(obj)._is_pointer_like()):
raise TypeError("cast() argument 1 must be a pointer, not %s"
% (type(obj),))
else:
- result = tp()
result._buffer[0] = obj._buffer[0]
# The casted objects '_objects' member:
diff --git a/lib_pypy/_ctypes/primitive.py b/lib_pypy/_ctypes/primitive.py
--- a/lib_pypy/_ctypes/primitive.py
+++ b/lib_pypy/_ctypes/primitive.py
@@ -390,11 +390,14 @@
self._buffer = self._ffiarray(1, autofree=True)
if value is not DEFAULT_VALUE:
self.value = value
+ _init_no_arg_ = __init__
def _ensure_objects(self):
- if self._type_ not in 'zZP':
- assert self._objects is None
- return self._objects
+ # No '_objects' is the common case for primitives. Examples
+ # where there is an _objects is if _type in 'zZP', or if
+ # self comes from 'from_buffer(buf)'. See module/test_lib_pypy/
+ # ctypes_test/test_buffers.py: test_from_buffer_keepalive.
+ return getattr(self, '_objects', None)
def _getvalue(self):
return self._buffer[0]
diff --git a/lib_pypy/_ctypes/structure.py b/lib_pypy/_ctypes/structure.py
--- a/lib_pypy/_ctypes/structure.py
+++ b/lib_pypy/_ctypes/structure.py
@@ -160,6 +160,10 @@
raise AttributeError("_fields_ is final")
if self in [f[1] for f in value]:
raise AttributeError("Structure or union cannot contain itself")
+ if self._ffiargtype is not None:
+ raise NotImplementedError("Too late to set _fields_: we already "
+ "said to libffi that the structure type %s is opaque"
+ % (self,))
names_and_fields(
self,
value, self.__bases__[0],
@@ -277,6 +281,7 @@
self.__setattr__(name, arg)
for name, arg in kwds.items():
self.__setattr__(name, arg)
+ _init_no_arg_ = __init__
def _subarray(self, fieldtype, name):
"""Return a _rawffi array of length 1 whose address is the same as
diff --git a/lib_pypy/_pypy_testcapi.py b/lib_pypy/_pypy_testcapi.py
--- a/lib_pypy/_pypy_testcapi.py
+++ b/lib_pypy/_pypy_testcapi.py
@@ -2,7 +2,7 @@
import tempfile, binascii
-def get_hashed_dir(cfile):
+def _get_hashed_filename(cfile):
with open(cfile,'r') as fid:
content = fid.read()
# from cffi's Verifier()
@@ -21,10 +21,28 @@
username = os.environ['USERNAME'] #windows
except KeyError:
username = os.getuid()
- output_dir = tempfile.gettempdir() + os.path.sep + 'tmp_%s_%s%s' % (
+ return tempfile.gettempdir() + os.path.sep + 'testcapi_%s_%s%s' % (
username, k1, k2)
- if not os.path.exists(output_dir):
+
+def get_hashed_dir(cfile):
+ hashed_fn = _get_hashed_filename(cfile)
+ try:
+ with open(hashed_fn) as f:
+ dirname = f.read(1024)
+ except IOError:
+ dirname = ''
+ tmpdir = tempfile.gettempdir()
+ if (not dirname or '/' in dirname or '\\' in dirname or '\x00' in dirname
+ or not os.path.isdir(os.path.join(tmpdir, dirname))):
+ dirname = binascii.hexlify(os.urandom(8))
+ if not isinstance(dirname, str): # Python 3
+ dirname = dirname.decode('ascii')
+ dirname = 'testcapi_' + dirname
+ output_dir = os.path.join(tmpdir, dirname)
+ try:
os.mkdir(output_dir)
+ except OSError:
+ pass
return output_dir
@@ -34,13 +52,12 @@
return ext
-def compile_shared(csource, modulename, output_dir=None):
+def compile_shared(csource, modulename, output_dir):
"""Compile '_testcapi.c' or '_ctypes_test.c' into an extension module,
and import it.
"""
thisdir = os.path.dirname(__file__)
- if output_dir is None:
- output_dir = tempfile.mkdtemp()
+ assert output_dir is not None
from distutils.ccompiler import new_compiler
@@ -85,4 +102,16 @@
# Now import the newly created library, it will replace the original
# module in sys.modules
fp, filename, description = imp.find_module(modulename, path=[output_dir])
- imp.load_module(modulename, fp, filename, description)
+ with fp:
+ imp.load_module(modulename, fp, filename, description)
+
+ # If everything went fine up to now, write the name of this new
+ # directory to 'hashed_fn', for future processes (and to avoid a
+ # growing number of temporary directories that are not completely
+ # obvious to clean up on Windows)
+ hashed_fn = _get_hashed_filename(os.path.join(thisdir, csource))
+ try:
+ with open(hashed_fn, 'w') as f:
+ f.write(os.path.basename(output_dir))
+ except IOError:
+ pass
diff --git a/lib_pypy/_sqlite3.py b/lib_pypy/_sqlite3.py
--- a/lib_pypy/_sqlite3.py
+++ b/lib_pypy/_sqlite3.py
@@ -153,9 +153,10 @@
factory = Connection if not factory else factory
# an sqlite3 db seems to be around 100 KiB at least (doesn't matter if
# backed by :memory: or a file)
+ res = factory(database, timeout, detect_types, isolation_level,
+ check_same_thread, factory, cached_statements)
add_memory_pressure(100 * 1024)
- return factory(database, timeout, detect_types, isolation_level,
- check_same_thread, factory, cached_statements)
+ return res
def _unicode_text_factory(x):
diff --git a/lib_pypy/cffi.egg-info/PKG-INFO b/lib_pypy/cffi.egg-info/PKG-INFO
--- a/lib_pypy/cffi.egg-info/PKG-INFO
+++ b/lib_pypy/cffi.egg-info/PKG-INFO
@@ -1,6 +1,6 @@
Metadata-Version: 1.1
Name: cffi
-Version: 1.11.4
+Version: 1.12.0
Summary: Foreign Function Interface for Python calling C code.
Home-page: http://cffi.readthedocs.org
Author: Armin Rigo, Maciej Fijalkowski
diff --git a/lib_pypy/cffi/__init__.py b/lib_pypy/cffi/__init__.py
--- a/lib_pypy/cffi/__init__.py
+++ b/lib_pypy/cffi/__init__.py
@@ -4,8 +4,8 @@
from .api import FFI
from .error import CDefError, FFIError, VerificationError, VerificationMissing
-__version__ = "1.11.4"
-__version_info__ = (1, 11, 4)
+__version__ = "1.12.0"
+__version_info__ = (1, 12, 0)
# The verifier module file names are based on the CRC32 of a string that
# contains the following version number. It may be older than __version__
diff --git a/lib_pypy/cffi/_cffi_errors.h b/lib_pypy/cffi/_cffi_errors.h
--- a/lib_pypy/cffi/_cffi_errors.h
+++ b/lib_pypy/cffi/_cffi_errors.h
@@ -50,7 +50,9 @@
"import sys\n"
"class FileLike:\n"
" def write(self, x):\n"
- " of.write(x)\n"
+ " try:\n"
+ " of.write(x)\n"
+ " except: pass\n"
" self.buf += x\n"
"fl = FileLike()\n"
"fl.buf = ''\n"
diff --git a/lib_pypy/cffi/_cffi_include.h b/lib_pypy/cffi/_cffi_include.h
--- a/lib_pypy/cffi/_cffi_include.h
+++ b/lib_pypy/cffi/_cffi_include.h
@@ -8,20 +8,43 @@
the same works for the other two macros. Py_DEBUG implies them,
but not the other way around.
- Issue #350 is still open: on Windows, the code here causes it to link
- with PYTHON36.DLL (for example) instead of PYTHON3.DLL. A fix was
- attempted in 164e526a5515 and 14ce6985e1c3, but reverted: virtualenv
- does not make PYTHON3.DLL available, and so the "correctly" compiled
- version would not run inside a virtualenv. We will re-apply the fix
- after virtualenv has been fixed for some time. For explanation, see
- issue #355. For a workaround if you want PYTHON3.DLL and don't worry
- about virtualenv, see issue #350. See also 'py_limited_api' in
- setuptools_ext.py.
+ The implementation is messy (issue #350): on Windows, with _MSC_VER,
+ we have to define Py_LIMITED_API even before including pyconfig.h.
+ In that case, we guess what pyconfig.h will do to the macros above,
+ and check our guess after the #include.
+
+ Note that on Windows, with CPython 3.x, you need virtualenv version
+ >= 16.0.0. Older versions don't copy PYTHON3.DLL. As a workaround
+ you can remove the definition of Py_LIMITED_API here.
+
+ See also 'py_limited_api' in cffi/setuptools_ext.py.
*/
#if !defined(_CFFI_USE_EMBEDDING) && !defined(Py_LIMITED_API)
-# include <pyconfig.h>
-# if !defined(Py_DEBUG) && !defined(Py_TRACE_REFS) && !defined(Py_REF_DEBUG)
-# define Py_LIMITED_API
+# ifdef _MSC_VER
+# if !defined(_DEBUG) && !defined(Py_DEBUG) && !defined(Py_TRACE_REFS) &&
!defined(Py_REF_DEBUG)
+# define Py_LIMITED_API
+# endif
+# include <pyconfig.h>
+ /* sanity-check: Py_LIMITED_API will cause crashes if any of these
+ are also defined. Normally, the Python file PC/pyconfig.h does not
+ cause any of these to be defined, with the exception that _DEBUG
+ causes Py_DEBUG. Double-check that. */
+# ifdef Py_LIMITED_API
+# if defined(Py_DEBUG)
+# error "pyconfig.h unexpectedly defines Py_DEBUG, but Py_LIMITED_API
is set"
+# endif
+# if defined(Py_TRACE_REFS)
+# error "pyconfig.h unexpectedly defines Py_TRACE_REFS, but
Py_LIMITED_API is set"
+# endif
+# if defined(Py_REF_DEBUG)
+# error "pyconfig.h unexpectedly defines Py_REF_DEBUG, but
Py_LIMITED_API is set"
+# endif
+# endif
+# else
+# include <pyconfig.h>
+# if !defined(Py_DEBUG) && !defined(Py_TRACE_REFS) && !defined(Py_REF_DEBUG)
+# define Py_LIMITED_API
+# endif
# endif
#endif
diff --git a/lib_pypy/cffi/_embedding.h b/lib_pypy/cffi/_embedding.h
--- a/lib_pypy/cffi/_embedding.h
+++ b/lib_pypy/cffi/_embedding.h
@@ -146,32 +146,6 @@
PyGILState_STATE state;
PyObject *pycode=NULL, *global_dict=NULL, *x;
-#if PY_MAJOR_VERSION >= 3
- /* see comments in _cffi_carefully_make_gil() about the
- Python2/Python3 difference
- */
-#else
- /* Acquire the GIL. We have no threadstate here. If Python is
- already initialized, it is possible that there is already one
- existing for this thread, but it is not made current now.
- */
- PyEval_AcquireLock();
-
- _cffi_py_initialize();
-
- /* The Py_InitializeEx() sometimes made a threadstate for us, but
- not always. Indeed Py_InitializeEx() could be called and do
- nothing. So do we have a threadstate, or not? We don't know,
- but we can replace it with NULL in all cases.
- */
- (void)PyThreadState_Swap(NULL);
-
- /* Now we can release the GIL and re-acquire immediately using the
- logic of PyGILState(), which handles making or installing the
- correct threadstate.
- */
- PyEval_ReleaseLock();
-#endif
state = PyGILState_Ensure();
/* Call the initxxx() function from the present module. It will
@@ -247,7 +221,7 @@
if (f != NULL && f != Py_None) {
PyFile_WriteString("\nFrom: " _CFFI_MODULE_NAME
- "\ncompiled with cffi version: 1.11.4"
+ "\ncompiled with cffi version: 1.12.0"
"\n_cffi_backend module: ", f);
modules = PyImport_GetModuleDict();
mod = PyDict_GetItemString(modules, "_cffi_backend");
@@ -278,16 +252,14 @@
that we don't hold the GIL before (if it exists), and we don't
hold it afterwards.
- What it really does is completely different in Python 2 and
- Python 3.
+ (What it really does used to be completely different in Python 2
+ and Python 3, with the Python 2 solution avoiding the spin-lock
+ around the Py_InitializeEx() call. However, after recent changes
+ to CPython 2.7 (issue #358) it no longer works. So we use the
+ Python 3 solution everywhere.)
- Python 2
- ========
-
- Initialize the GIL, without initializing the rest of Python,
- by calling PyEval_InitThreads().
-
- PyEval_InitThreads() must not be called concurrently at all.
+ This initializes Python by calling Py_InitializeEx().
+ Important: this must not be called concurrently at all.
So we use a global variable as a simple spin lock. This global
variable must be from 'libpythonX.Y.so', not from this
cffi-based extension module, because it must be shared from
@@ -297,18 +269,6 @@
string "ENDMARKER". We change it temporarily to point to the
next character in that string. (Yes, I know it's REALLY
obscure.)
-
- Python 3
- ========
-
- In Python 3, PyEval_InitThreads() cannot be called before
- Py_InitializeEx() any more. So this function calls
- Py_InitializeEx() first. It uses the same obscure logic to
- make sure we never call it concurrently.
-
- Arguably, this is less good on the spinlock, because
- Py_InitializeEx() takes much longer to run than
- PyEval_InitThreads(). But I didn't find a way around it.
*/
#ifdef WITH_THREAD
@@ -332,8 +292,7 @@
}
#endif
-#if PY_MAJOR_VERSION >= 3
- /* Python 3: call Py_InitializeEx() */
+ /* call Py_InitializeEx() */
{
PyGILState_STATE state = PyGILState_UNLOCKED;
if (!Py_IsInitialized())
@@ -344,17 +303,6 @@
PyEval_InitThreads();
PyGILState_Release(state);
}
-#else
- /* Python 2: call PyEval_InitThreads() */
-# ifdef WITH_THREAD
- if (!PyEval_ThreadsInitialized()) {
- PyEval_InitThreads(); /* makes the GIL */
- PyEval_ReleaseLock(); /* then release it */
- }
- /* else: there is already a GIL, but we still needed to do the
- spinlock dance to make sure that we see it as fully ready */
-# endif
-#endif
#ifdef WITH_THREAD
/* release the lock */
diff --git a/lib_pypy/cffi/api.py b/lib_pypy/cffi/api.py
--- a/lib_pypy/cffi/api.py
+++ b/lib_pypy/cffi/api.py
@@ -96,18 +96,21 @@
self.CData, self.CType = backend._get_types()
self.buffer = backend.buffer
- def cdef(self, csource, override=False, packed=False):
+ def cdef(self, csource, override=False, packed=False, pack=None):
"""Parse the given C source. This registers all declared functions,
types, and global variables. The functions and global variables can
then be accessed via either 'ffi.dlopen()' or 'ffi.verify()'.
The types can be used in 'ffi.new()' and other functions.
If 'packed' is specified as True, all structs declared inside this
cdef are packed, i.e. laid out without any field alignment at all.
+ Alternatively, 'pack' can be a small integer, and requests for
+ alignment greater than that are ignored (pack=1 is equivalent to
+ packed=True).
"""
- self._cdef(csource, override=override, packed=packed)
+ self._cdef(csource, override=override, packed=packed, pack=pack)
- def embedding_api(self, csource, packed=False):
- self._cdef(csource, packed=packed, dllexport=True)
+ def embedding_api(self, csource, packed=False, pack=None):
+ self._cdef(csource, packed=packed, pack=pack, dllexport=True)
if self._embedding is None:
self._embedding = ''
@@ -143,6 +146,13 @@
self._libraries.append(lib)
return lib
+ def dlclose(self, lib):
+ """Close a library obtained with ffi.dlopen(). After this call,
+ access to functions or variables from the library will fail
+ (possibly with a segmentation fault).
+ """
+ type(lib).__cffi_close__(lib)
+
def _typeof_locked(self, cdecl):
# call me with the lock!
key = cdecl
@@ -898,6 +908,9 @@
return addressof_var(name)
raise AttributeError("cffi library has no function or "
"global variable named '%s'" % (name,))
+ def __cffi_close__(self):
+ backendlib.close_lib()
+ self.__dict__.clear()
#
if libname is not None:
try:
diff --git a/lib_pypy/cffi/backend_ctypes.py b/lib_pypy/cffi/backend_ctypes.py
--- a/lib_pypy/cffi/backend_ctypes.py
+++ b/lib_pypy/cffi/backend_ctypes.py
@@ -636,6 +636,10 @@
if isinstance(init, bytes):
init = [init[i:i+1] for i in range(len(init))]
else:
+ if isinstance(init, CTypesGenericArray):
+ if (len(init) != len(blob) or
+ not isinstance(init, CTypesArray)):
+ raise TypeError("length/type mismatch: %s" %
(init,))
init = tuple(init)
if len(init) > len(blob):
raise IndexError("too many initializers")
@@ -730,7 +734,8 @@
return self._new_struct_or_union('union', name, ctypes.Union)
def complete_struct_or_union(self, CTypesStructOrUnion, fields, tp,
- totalsize=-1, totalalignment=-1, sflags=0):
+ totalsize=-1, totalalignment=-1, sflags=0,
+ pack=0):
if totalsize >= 0 or totalalignment >= 0:
raise NotImplementedError("the ctypes backend of CFFI does not
support "
"structures completed by verify();
please "
@@ -751,6 +756,8 @@
bfield_types[fname] = Ellipsis
if sflags & 8:
struct_or_union._pack_ = 1
+ elif pack:
+ struct_or_union._pack_ = pack
struct_or_union._fields_ = cfields
CTypesStructOrUnion._bfield_types = bfield_types
#
diff --git a/lib_pypy/cffi/cparser.py b/lib_pypy/cffi/cparser.py
--- a/lib_pypy/cffi/cparser.py
+++ b/lib_pypy/cffi/cparser.py
@@ -306,11 +306,25 @@
msg = 'parse error\n%s' % (msg,)
raise CDefError(msg)
- def parse(self, csource, override=False, packed=False, dllexport=False):
+ def parse(self, csource, override=False, packed=False, pack=None,
+ dllexport=False):
+ if packed:
+ if packed != True:
+ raise ValueError("'packed' should be False or True; use "
+ "'pack' to give another value")
+ if pack:
+ raise ValueError("cannot give both 'pack' and 'packed'")
+ pack = 1
+ elif pack:
+ if pack & (pack - 1):
+ raise ValueError("'pack' must be a power of two, not %r" %
+ (pack,))
+ else:
+ pack = 0
prev_options = self._options
try:
self._options = {'override': override,
- 'packed': packed,
+ 'packed': pack,
'dllexport': dllexport}
self._internal_parse(csource)
finally:
diff --git a/lib_pypy/cffi/model.py b/lib_pypy/cffi/model.py
--- a/lib_pypy/cffi/model.py
+++ b/lib_pypy/cffi/model.py
@@ -342,7 +342,7 @@
fixedlayout = None
completed = 0
partial = False
- packed = False
+ packed = 0
def __init__(self, name, fldnames, fldtypes, fldbitsize, fldquals=None):
self.name = name
@@ -352,21 +352,20 @@
self.fldquals = fldquals
self.build_c_name_with_marker()
- def has_anonymous_struct_fields(self):
- if self.fldtypes is None:
- return False
- for name, type in zip(self.fldnames, self.fldtypes):
- if name == '' and isinstance(type, StructOrUnion):
- return True
- return False
+ def anonymous_struct_fields(self):
+ if self.fldtypes is not None:
+ for name, type in zip(self.fldnames, self.fldtypes):
+ if name == '' and isinstance(type, StructOrUnion):
+ yield type
- def enumfields(self):
+ def enumfields(self, expand_anonymous_struct_union=True):
fldquals = self.fldquals
if fldquals is None:
fldquals = (0,) * len(self.fldnames)
for name, type, bitsize, quals in zip(self.fldnames, self.fldtypes,
self.fldbitsize, fldquals):
- if name == '' and isinstance(type, StructOrUnion):
+ if (name == '' and isinstance(type, StructOrUnion)
+ and expand_anonymous_struct_union):
# nested anonymous struct/union
for result in type.enumfields():
yield result
@@ -415,11 +414,14 @@
fldtypes = [tp.get_cached_btype(ffi, finishlist)
for tp in self.fldtypes]
lst = list(zip(self.fldnames, fldtypes, self.fldbitsize))
- sflags = 0
+ extra_flags = ()
if self.packed:
- sflags = 8 # SF_PACKED
+ if self.packed == 1:
+ extra_flags = (8,) # SF_PACKED
+ else:
+ extra_flags = (0, self.packed)
ffi._backend.complete_struct_or_union(BType, lst, self,
- -1, -1, sflags)
+ -1, -1, *extra_flags)
#
else:
fldtypes = []
diff --git a/lib_pypy/cffi/recompiler.py b/lib_pypy/cffi/recompiler.py
--- a/lib_pypy/cffi/recompiler.py
+++ b/lib_pypy/cffi/recompiler.py
@@ -836,6 +836,10 @@
def _struct_collecttype(self, tp):
self._do_collect_type(tp)
+ if self.target_is_python:
+ # also requires nested anon struct/unions in ABI mode, recursively
+ for fldtype in tp.anonymous_struct_fields():
+ self._struct_collecttype(fldtype)
def _struct_decl(self, tp, cname, approxname):
if tp.fldtypes is None:
@@ -884,11 +888,17 @@
named_ptr not in self.ffi._parser._included_declarations)):
if tp.fldtypes is None:
pass # opaque
- elif tp.partial or tp.has_anonymous_struct_fields():
+ elif tp.partial or any(tp.anonymous_struct_fields()):
pass # field layout obtained silently from the C compiler
else:
flags.append("_CFFI_F_CHECK_FIELDS")
if tp.packed:
+ if tp.packed > 1:
+ raise NotImplementedError(
+ "%r is declared with 'pack=%r'; only 0 or 1 are "
+ "supported in API mode (try to use \"...;\", which "
+ "does not require a 'pack' declaration)" %
+ (tp, tp.packed))
flags.append("_CFFI_F_PACKED")
else:
flags.append("_CFFI_F_EXTERNAL")
@@ -896,7 +906,8 @@
flags = '|'.join(flags) or '0'
c_fields = []
if reason_for_not_expanding is None:
- enumfields = list(tp.enumfields())
+ expand_anonymous_struct_union = not self.target_is_python
+ enumfields = list(tp.enumfields(expand_anonymous_struct_union))
for fldname, fldtype, fbitsize, fqual in enumfields:
fldtype = self._field_type(tp, fldname, fldtype)
self._check_not_opaque(fldtype,
diff --git a/lib_pypy/cffi/setuptools_ext.py b/lib_pypy/cffi/setuptools_ext.py
--- a/lib_pypy/cffi/setuptools_ext.py
+++ b/lib_pypy/cffi/setuptools_ext.py
@@ -143,8 +143,8 @@
def _add_py_module(dist, ffi, module_name):
from distutils.dir_util import mkpath
- from distutils.command.build_py import build_py
- from distutils.command.build_ext import build_ext
+ from setuptools.command.build_py import build_py
+ from setuptools.command.build_ext import build_ext
from distutils import log
from cffi import recompiler
@@ -162,8 +162,31 @@
module_path = module_name.split('.')
module_path[-1] += '.py'
generate_mod(os.path.join(self.build_lib, *module_path))
+ def get_source_files(self):
+ # This is called from 'setup.py sdist' only. Exclude
+ # the generate .py module in this case.
+ saved_py_modules = self.py_modules
+ try:
+ if saved_py_modules:
+ self.py_modules = [m for m in saved_py_modules
+ if m != module_name]
+ return base_class.get_source_files(self)
+ finally:
+ self.py_modules = saved_py_modules
dist.cmdclass['build_py'] = build_py_make_mod
+ # distutils and setuptools have no notion I could find of a
+ # generated python module. If we don't add module_name to
+ # dist.py_modules, then things mostly work but there are some
+ # combination of options (--root and --record) that will miss
+ # the module. So we add it here, which gives a few apparently
+ # harmless warnings about not finding the file outside the
+ # build directory.
+ # Then we need to hack more in get_source_files(); see above.
+ if dist.py_modules is None:
+ dist.py_modules = []
+ dist.py_modules.append(module_name)
+
# the following is only for "build_ext -i"
base_class_2 = dist.cmdclass.get('build_ext', build_ext)
class build_ext_make_mod(base_class_2):
diff --git a/lib_pypy/dbm.py b/lib_pypy/dbm.py
--- a/lib_pypy/dbm.py
+++ b/lib_pypy/dbm.py
@@ -157,7 +157,14 @@
def open(filename, flag='r', mode=0666):
"open a DBM database"
if not isinstance(filename, str):
- raise TypeError("expected string")
+ if sys.version_info < (3,) and isinstance(filename, unicode):
+ # unlike CPython we'll encode 'filename' with filesystemencoding
+ # instead of defaultencoding, because that seems like a far
+ # better idea. But I'm also open for saying that we should
+ # rather go for bug-to-bug compatibility instead.
+ filename = filename.encode(sys.getfilesystemencoding())
+ else:
+ raise TypeError("expected string")
openflag = 0
diff --git a/lib_pypy/grp.py b/lib_pypy/grp.py
--- a/lib_pypy/grp.py
+++ b/lib_pypy/grp.py
@@ -4,6 +4,8 @@
from _pwdgrp_cffi import ffi, lib
import _structseq
+import thread
+_lock = thread.allocate_lock()
try: from __pypy__ import builtinify
except ImportError: builtinify = lambda f: f
@@ -33,32 +35,35 @@
@builtinify
def getgrgid(gid):
- res = lib.getgrgid(gid)
- if not res:
- # XXX maybe check error eventually
- raise KeyError(gid)
- return _group_from_gstruct(res)
+ with _lock:
+ res = lib.getgrgid(gid)
+ if not res:
+ # XXX maybe check error eventually
+ raise KeyError(gid)
+ return _group_from_gstruct(res)
@builtinify
def getgrnam(name):
if not isinstance(name, basestring):
raise TypeError("expected string")
name = str(name)
- res = lib.getgrnam(name)
- if not res:
- raise KeyError("'getgrnam(): name not found: %s'" % name)
- return _group_from_gstruct(res)
+ with _lock:
+ res = lib.getgrnam(name)
+ if not res:
+ raise KeyError("'getgrnam(): name not found: %s'" % name)
+ return _group_from_gstruct(res)
@builtinify
def getgrall():
- lib.setgrent()
lst = []
- while 1:
- p = lib.getgrent()
- if not p:
- break
- lst.append(_group_from_gstruct(p))
- lib.endgrent()
+ with _lock:
+ lib.setgrent()
+ while 1:
+ p = lib.getgrent()
+ if not p:
+ break
+ lst.append(_group_from_gstruct(p))
+ lib.endgrent()
return lst
__all__ = ('struct_group', 'getgrgid', 'getgrnam', 'getgrall')
diff --git a/lib_pypy/pwd.py b/lib_pypy/pwd.py
--- a/lib_pypy/pwd.py
+++ b/lib_pypy/pwd.py
@@ -12,6 +12,8 @@
from _pwdgrp_cffi import ffi, lib
import _structseq
+import thread
+_lock = thread.allocate_lock()
try: from __pypy__ import builtinify
except ImportError: builtinify = lambda f: f
@@ -55,10 +57,11 @@
Return the password database entry for the given numeric user ID.
See pwd.__doc__ for more on password database entries.
"""
- pw = lib.getpwuid(uid)
- if not pw:
- raise KeyError("getpwuid(): uid not found: %s" % uid)
- return _mkpwent(pw)
+ with _lock:
+ pw = lib.getpwuid(uid)
+ if not pw:
+ raise KeyError("getpwuid(): uid not found: %s" % uid)
+ return _mkpwent(pw)
@builtinify
def getpwnam(name):
@@ -71,10 +74,11 @@
if not isinstance(name, basestring):
raise TypeError("expected string")
name = str(name)
- pw = lib.getpwnam(name)
- if not pw:
- raise KeyError("getpwname(): name not found: %s" % name)
- return _mkpwent(pw)
+ with _lock:
+ pw = lib.getpwnam(name)
+ if not pw:
+ raise KeyError("getpwname(): name not found: %s" % name)
+ return _mkpwent(pw)
@builtinify
def getpwall():
@@ -84,13 +88,14 @@
See pwd.__doc__ for more on password database entries.
"""
users = []
- lib.setpwent()
- while True:
- pw = lib.getpwent()
- if not pw:
- break
- users.append(_mkpwent(pw))
- lib.endpwent()
+ with _lock:
+ lib.setpwent()
+ while True:
+ pw = lib.getpwent()
+ if not pw:
+ break
+ users.append(_mkpwent(pw))
+ lib.endpwent()
return users
__all__ = ('struct_passwd', 'getpwuid', 'getpwnam', 'getpwall')
diff --git a/lib_pypy/pyrepl/unix_console.py b/lib_pypy/pyrepl/unix_console.py
--- a/lib_pypy/pyrepl/unix_console.py
+++ b/lib_pypy/pyrepl/unix_console.py
@@ -70,8 +70,8 @@
pass
def register(self, fd, flag):
self.fd = fd
- def poll(self, timeout=None):
- r,w,e = select.select([self.fd],[],[],timeout)
+ def poll(self): # note: a 'timeout' argument would be *milliseconds*
+ r,w,e = select.select([self.fd],[],[])
return r
POLLIN = getattr(select, "POLLIN", None)
diff --git a/pypy/config/pypyoption.py b/pypy/config/pypyoption.py
--- a/pypy/config/pypyoption.py
+++ b/pypy/config/pypyoption.py
@@ -39,14 +39,10 @@
"_csv", "_cppyy", "_pypyjson", "_jitlog"
])
-from rpython.jit.backend import detect_cpu
-try:
- if detect_cpu.autodetect().startswith('x86'):
- if not sys.platform.startswith('openbsd'):
- working_modules.add('_vmprof')
- working_modules.add('faulthandler')
-except detect_cpu.ProcessorAutodetectError:
- pass
+import rpython.rlib.rvmprof.cintf
+if rpython.rlib.rvmprof.cintf.IS_SUPPORTED:
+ working_modules.add('_vmprof')
+ working_modules.add('faulthandler')
translation_modules = default_modules.copy()
translation_modules.update([
@@ -57,6 +53,11 @@
"termios", "_minimal_curses",
])
+reverse_debugger_disable_modules = set([
+ "_continuation", "_vmprof", "_multiprocessing",
+ "micronumpy",
+ ])
+
# XXX this should move somewhere else, maybe to platform ("is this posixish"
# check or something)
if sys.platform == "win32":
@@ -292,6 +293,9 @@
modules = working_modules.copy()
if config.translation.sandbox:
modules = default_modules
+ if config.translation.reverse_debugger:
+ for mod in reverse_debugger_disable_modules:
+ setattr(config.objspace.usemodules, mod, False)
# ignore names from 'essential_modules', notably 'exceptions', which
# may not be present in config.objspace.usemodules at all
modules = [name for name in modules if name not in essential_modules]
@@ -310,3 +314,4 @@
parser = to_optparse(config) #, useoptions=["translation.*"])
option, args = parser.parse_args()
print config
+ print working_modules
diff --git a/pypy/doc/architecture.rst b/pypy/doc/architecture.rst
--- a/pypy/doc/architecture.rst
+++ b/pypy/doc/architecture.rst
@@ -73,3 +73,63 @@
This division between bytecode evaluator and object space gives a lot of
flexibility. One can plug in different :doc:`object spaces <objspace>` to get
different or enriched behaviours of the Python objects.
+
+Layers
+------
+
+RPython
+~~~~~~~
+:ref:`RPython <rpython:language>` is the language in which we write
interpreters.
+Not the entire PyPy project is written in RPython, only the parts that are
+compiled in the translation process. The interesting point is that RPython
+has no parser, it's compiled from the live python objects, which makes it
+possible to do all kinds of metaprogramming during import time. In short,
+Python is a meta programming language for RPython.
+
+The RPython standard library is to be found in the ``rlib`` subdirectory.
+
+Consult `Getting Started with RPython`_ for further reading
+
+Translation
+~~~~~~~~~~~
+The translation toolchain - this is the part that takes care of translating
+RPython to flow graphs and then to C. There is more in the
+:doc:`architecture <architecture>` document written about it.
+
+It lives in the ``rpython`` directory: ``flowspace``, ``annotator``
+and ``rtyper``.
+
+PyPy Interpreter
+~~~~~~~~~~~~~~~~
+This is in the ``pypy`` directory. ``pypy/interpreter`` is a standard
+interpreter for Python written in RPython. The fact that it is
+RPython is not apparent at first. Built-in modules are written in
+``pypy/module/*``. Some modules that CPython implements in C are
+simply written in pure Python; they are in the top-level ``lib_pypy``
+directory. The standard library of Python (with a few changes to
+accomodate PyPy) is in ``lib-python``.
+
+JIT Compiler
+~~~~~~~~~~~~
+:ref:`Just-in-Time Compiler (JIT) <rpython:jit>`: we have a tracing JIT that
traces the
+interpreter written in RPython, rather than the user program that it
+interprets. As a result it applies to any interpreter, i.e. any
+language. But getting it to work correctly is not trivial: it
+requires a small number of precise "hints" and possibly some small
+refactorings of the interpreter. The JIT itself also has several
+almost-independent parts: the tracer itself in ``rpython/jit/metainterp``, the
+optimizer in ``rpython/jit/metainterp/optimizer`` that optimizes a list of
+residual operations, and the backend in ``rpython/jit/backend/<machine-name>``
+that turns it into machine code. Writing a new backend is a
+traditional way to get into the project.
+
+Garbage Collectors
+~~~~~~~~~~~~~~~~~~
+Garbage Collectors (GC): as you may notice if you are used to CPython's
+C code, there are no ``Py_INCREF/Py_DECREF`` equivalents in RPython code.
+:ref:`rpython:garbage-collection` is inserted
+during translation. Moreover, this is not reference counting; it is a real
+GC written as more RPython code. The best one we have so far is in
+``rpython/memory/gc/incminimark.py``.
+
+.. _`Getting started with RPython`:
http://rpython.readthedocs.org/en/latest/getting-started.html
diff --git a/pypy/doc/build.rst b/pypy/doc/build.rst
--- a/pypy/doc/build.rst
+++ b/pypy/doc/build.rst
@@ -267,14 +267,14 @@
* PyPy 2.5.1 or earlier: normal users would see permission errors.
Installers need to run ``pypy -c "import gdbm"`` and other similar
commands at install time; the exact list is in
- :source:`pypy/tool/release/package.py <package.py>`. Users
+ :source:`pypy/tool/release/package.py`. Users
seeing a broken installation of PyPy can fix it after-the-fact if they
have sudo rights, by running once e.g. ``sudo pypy -c "import gdbm``.
* PyPy 2.6 and later: anyone would get ``ImportError: no module named
_gdbm_cffi``. Installers need to run ``pypy _gdbm_build.py`` in the
``lib_pypy`` directory during the installation process (plus others;
- see the exact list in :source:`pypy/tool/release/package.py <package.py>`).
+ see the exact list in :source:`pypy/tool/release/package.py`).
Users seeing a broken
installation of PyPy can fix it after-the-fact, by running ``pypy
/path/to/lib_pypy/_gdbm_build.py``. This command produces a file
diff --git a/pypy/doc/coding-guide.rst b/pypy/doc/coding-guide.rst
--- a/pypy/doc/coding-guide.rst
+++ b/pypy/doc/coding-guide.rst
@@ -539,7 +539,7 @@
hg help branch
-.. _official wiki: http://mercurial.selenic.com/wiki/Branch
+.. _official wiki: https://www.mercurial-scm.org/wiki/
.. _using-development-tracker:
@@ -547,15 +547,7 @@
Using the development bug/feature tracker
-----------------------------------------
-We have a `development tracker`_, based on Richard Jones'
-`roundup`_ application. You can file bugs,
-feature requests or see what's going on
-for the next milestone, both from an E-Mail and from a
-web interface.
-
-.. _development tracker: https://bugs.pypy.org/
-.. _roundup: http://roundup.sourceforge.net/
-
+We use bitbucket for :source:`issues` tracking and :source:`pull-requests`.
.. _testing:
diff --git a/pypy/doc/commandline_ref.rst b/pypy/doc/commandline_ref.rst
--- a/pypy/doc/commandline_ref.rst
+++ b/pypy/doc/commandline_ref.rst
@@ -8,3 +8,4 @@
:maxdepth: 1
man/pypy.1.rst
+ man/pypy3.1.rst
diff --git a/pypy/doc/conf.py b/pypy/doc/conf.py
--- a/pypy/doc/conf.py
+++ b/pypy/doc/conf.py
@@ -66,9 +66,9 @@
# built documents.
#
# The short X.Y version.
-version = '5.8'
+version = '6.0'
# The full version, including alpha/beta/rc tags.
-release = '5.8.0'
+release = '6.0.0'
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
diff --git a/pypy/doc/config/objspace.disable_entrypoints.txt
b/pypy/doc/config/objspace.disable_entrypoints.txt
new file mode 100644
diff --git a/pypy/doc/config/objspace.fstrings.txt
b/pypy/doc/config/objspace.fstrings.txt
new file mode 100644
diff --git a/pypy/doc/config/objspace.hash.txt
b/pypy/doc/config/objspace.hash.txt
new file mode 100644
diff --git a/pypy/doc/config/objspace.usemodules._cppyy.txt
b/pypy/doc/config/objspace.usemodules._cppyy.txt
new file mode 100644
--- /dev/null
+++ b/pypy/doc/config/objspace.usemodules._cppyy.txt
@@ -0,0 +1,1 @@
+The internal backend for cppyy
diff --git a/pypy/doc/config/objspace.usemodules._frozen_importlib.txt
b/pypy/doc/config/objspace.usemodules._frozen_importlib.txt
new file mode 100644
diff --git a/pypy/doc/config/objspace.usemodules._jitlog.txt
b/pypy/doc/config/objspace.usemodules._jitlog.txt
new file mode 100644
diff --git a/pypy/doc/config/objspace.usemodules._rawffi.txt
b/pypy/doc/config/objspace.usemodules._rawffi.txt
--- a/pypy/doc/config/objspace.usemodules._rawffi.txt
+++ b/pypy/doc/config/objspace.usemodules._rawffi.txt
@@ -1,3 +1,3 @@
-An experimental module providing very low-level interface to
+A module providing very low-level interface to
C-level libraries, for use when implementing ctypes, not
-intended for a direct use at all.
\ No newline at end of file
+intended for a direct use at all.
diff --git a/pypy/doc/config/objspace.usemodules.cpyext.txt
b/pypy/doc/config/objspace.usemodules.cpyext.txt
--- a/pypy/doc/config/objspace.usemodules.cpyext.txt
+++ b/pypy/doc/config/objspace.usemodules.cpyext.txt
@@ -1,1 +1,1 @@
-Use (experimental) cpyext module, that tries to load and run CPython extension
modules
+Use cpyext module to load and run CPython extension modules
diff --git a/pypy/doc/config/objspace.usemodules.faulthandler.txt
b/pypy/doc/config/objspace.usemodules.faulthandler.txt
new file mode 100644
diff --git a/pypy/doc/config/translation.backendopt.replace_we_are_jitted.txt
b/pypy/doc/config/translation.backendopt.replace_we_are_jitted.txt
new file mode 100644
diff --git a/pypy/doc/config/translation.jit_opencoder_model.txt
b/pypy/doc/config/translation.jit_opencoder_model.txt
new file mode 100644
diff --git a/pypy/doc/config/translation.keepgoing.txt
b/pypy/doc/config/translation.keepgoing.txt
new file mode 100644
diff --git a/pypy/doc/config/translation.libname.txt
b/pypy/doc/config/translation.libname.txt
new file mode 100644
diff --git a/pypy/doc/config/translation.lto.txt
b/pypy/doc/config/translation.lto.txt
new file mode 100644
diff --git a/pypy/doc/config/translation.profoptargs.txt
b/pypy/doc/config/translation.profoptargs.txt
new file mode 100644
diff --git a/pypy/doc/config/translation.reverse_debugger.txt
b/pypy/doc/config/translation.reverse_debugger.txt
new file mode 100644
diff --git a/pypy/doc/config/translation.split_gc_address_space.txt
b/pypy/doc/config/translation.split_gc_address_space.txt
new file mode 100644
diff --git a/pypy/doc/contributing.rst b/pypy/doc/contributing.rst
new file mode 100644
--- /dev/null
+++ b/pypy/doc/contributing.rst
@@ -0,0 +1,472 @@
+Contributing Guidelines
+===========================
+
+.. contents::
+
+PyPy is a very large project that has a reputation of being hard to dive into.
+Some of this fame is warranted, some of it is purely accidental. There are
three
+important lessons that everyone willing to contribute should learn:
+
+* PyPy has layers. There are many pieces of architecture that are very well
+ separated from each other. More about this below, but often the manifestation
+ of this is that things are at a different layer than you would expect them
+ to be. For example if you are looking for the JIT implementation, you will
+ not find it in the implementation of the Python programming language.
+
+* Because of the above, we are very serious about Test Driven Development.
+ It's not only what we believe in, but also that PyPy's architecture is
+ working very well with TDD in mind and not so well without it. Often
+ development means progressing in an unrelated corner, one unittest
+ at a time; and then flipping a giant switch, bringing it all together.
+ (It generally works out of the box. If it doesn't, then we didn't
+ write enough unit tests.) It's worth repeating - PyPy's
+ approach is great if you do TDD, and not so great otherwise.
+
+* PyPy uses an entirely different set of tools - most of them included
+ in the PyPy repository. There is no Makefile, nor autoconf. More below.
+
+The first thing to remember is that PyPy project is very different than most
+projects out there. It's also different from a classic compiler project,
+so academic courses about compilers often don't apply or lead in the wrong
+direction. However, if you want to understand how designing & building a
runtime
+works in the real world then this is a great project!
+
+Getting involved
+^^^^^^^^^^^^^^^^
+
+PyPy employs a relatively standard open-source development process. You are
+encouraged as a first step to join our `pypy-dev mailing list`_ and IRC
channel,
+details of which can be found in our :ref:`contact <contact>` section. The
folks
+there are very friendly, and can point you in the right direction.
+
+We give out commit rights usually fairly liberally, so if you want to do
something
+with PyPy, you can become a committer. We also run frequent coding sprints
which
+are separately announced and often happen around Python conferences such as
+EuroPython or PyCon. Upcoming events are usually announced on `the blog`_.
+
+Further Reading: :ref:`Contact <contact>`
+
+.. _the blog: http://morepypy.blogspot.com
+.. _pypy-dev mailing list: http://mail.python.org/mailman/listinfo/pypy-dev
+
+
+Your first contribution
+^^^^^^^^^^^^^^^^^^^^^^^
+
+The first and most important rule how **not** to contribute to PyPy is
+"just hacking a feature". This won't work, and you'll find your PR will
typically
+require a lot of re-work. There are a few reasons why not:
+
+* build times are large
+* PyPy has very thick layer separation
+* context of the cPython runtime is often required
+
+Instead, reach out on the dev mailing list or the IRC channel, and we're more
+than happy to help! :)
+
+Some ideas for first contributions are:
+
+* Documentation - this will give you an understanding of the pypy architecture
+* Test failures - find a failing test in the `nightly builds`_, and fix it
+* Missing language features - these are listed in our `issue tracker`_
+
+.. _nightly builds: http://buildbot.pypy.org/nightly/
+.. _issue tracker: https://bitbucket.org/pypy/pypy/issues
_______________________________________________
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit
