Author: Richard Plangger <planri...@gmail.com> Branch: strbuf-as-buffer Changeset: r89201:e5ac7ec7ab17 Date: 2016-12-20 15:29 +0100 http://bitbucket.org/pypy/pypy/changeset/e5ac7ec7ab17/
Log: merge default diff too long, truncating to 2000 out of 16301 lines diff --git a/.hgignore b/.hgignore --- a/.hgignore +++ b/.hgignore @@ -77,3 +77,5 @@ ^.hypothesis/ ^release/ ^rpython/_cache$ + +pypy/module/cppyy/.+/*\.pcm diff --git a/lib_pypy/greenlet.egg-info b/lib_pypy/greenlet.egg-info --- a/lib_pypy/greenlet.egg-info +++ b/lib_pypy/greenlet.egg-info @@ -1,6 +1,6 @@ Metadata-Version: 1.0 Name: greenlet -Version: 0.4.10 +Version: 0.4.11 Summary: Lightweight in-process concurrent programming Home-page: https://github.com/python-greenlet/greenlet Author: Ralf Schmitt (for CPython), PyPy team diff --git a/lib_pypy/greenlet.py b/lib_pypy/greenlet.py --- a/lib_pypy/greenlet.py +++ b/lib_pypy/greenlet.py @@ -1,7 +1,7 @@ import sys import _continuation -__version__ = "0.4.10" +__version__ = "0.4.11" # ____________________________________________________________ # Exceptions diff --git a/pypy/config/pypyoption.py b/pypy/config/pypyoption.py --- a/pypy/config/pypyoption.py +++ b/pypy/config/pypyoption.py @@ -190,6 +190,12 @@ "make sure that all calls go through space.call_args", default=False), + BoolOption("disable_entrypoints", + "Disable external entry points, notably the" + " cpyext module and cffi's embedding mode.", + default=False, + requires=[("objspace.usemodules.cpyext", False)]), + OptionDescription("std", "Standard Object Space Options", [ BoolOption("withtproxy", "support transparent proxies", default=True), diff --git a/pypy/doc/contributor.rst b/pypy/doc/contributor.rst --- a/pypy/doc/contributor.rst +++ b/pypy/doc/contributor.rst @@ -1,3 +1,9 @@ +#encoding utf-8 + +Contributors +------------ +:: + Armin Rigo Maciej Fijalkowski Carl Friedrich Bolz @@ -307,7 +313,7 @@ Mads Kiilerich Antony Lee Jason Madden - Daniel Neuh�user + Daniel Neuhäuser reub...@gmail.com Yaroslav Fedevych Jim Hunziker diff --git a/pypy/doc/cppyy.rst b/pypy/doc/cppyy.rst --- a/pypy/doc/cppyy.rst +++ b/pypy/doc/cppyy.rst @@ -1,145 +1,61 @@ cppyy: C++ bindings for PyPy ============================ -The cppyy module creates, at run-time, Python-side classes and functions for -C++, by querying a C++ reflection system. -The default system used is `Reflex`_, which extracts the needed information -from C++ header files. -Another current backend is based on `CINT`_, and yet another, more important -one for the medium- to long-term will be based on `cling`_. -The latter sits on top of `llvm`_'s `clang`_, and will therefore allow the use -of C++11. -The work on the cling backend has so far been done only for CPython, but -bringing it to PyPy is a lot less work than developing it in the first place. +The cppyy module delivers dynamic Python-C++ bindings. +It is designed for automation, high performance, scale, interactivity, and +handling all of modern C++ (11, 14, etc.). +It is based on `Cling`_ which, through `LLVM`_/`clang`_, provides C++ +reflection and interactivity. +Reflection information is extracted from C++ header files. +Cppyy itself is built into PyPy (an alternative exists for CPython), but +it requires a `backend`_, installable through pip, to interface with Cling. -.. _Reflex: https://root.cern.ch/how/how-use-reflex -.. _CINT: https://root.cern.ch/introduction-cint -.. _cling: https://root.cern.ch/cling -.. _llvm: http://llvm.org/ +.. _Cling: https://root.cern.ch/cling +.. _LLVM: http://llvm.org/ .. _clang: http://clang.llvm.org/ - -This document describes the version of cppyy that lives in the main branch of -PyPy. -The development of cppyy happens in the "reflex-support" branch. - - -Motivation ----------- - -To provide bindings to another language in CPython, you program to a -generic C-API that exposes many of the interpreter features. -With PyPy, however, there is no such generic C-API, because several of the -interpreter features (e.g. the memory model) are pluggable and therefore -subject to change. -Furthermore, a generic API does not allow any assumptions about the calls -into another language, forcing the JIT to behave conservatively around these -calls and with the objects that cross language boundaries. -In contrast, cppyy does not expose an API, but expects one to be implemented -by a backend. -It makes strong assumptions about the semantics of the API that it uses and -that in turn allows the JIT to make equally strong assumptions. -This is possible, because the expected API is only for providing C++ language -bindings, and does not provide generic programmability. - -The cppyy module further offers two features, which result in improved -performance as well as better functionality and cross-language integration. -First, cppyy itself is written in RPython and therefore open to optimizations -by the JIT up until the actual point of call into C++. -This means for example, that if variables are already unboxed by the JIT, they -can be passed through directly to C++. -Second, a backend such as Reflex (and cling far more so) adds dynamic features -to C++, thus greatly reducing impedance mismatches between the two languages. -For example, Reflex is dynamic enough to allow writing runtime bindings -generation in python (as opposed to RPython) and this is used to create very -natural "pythonizations" of the bound code. -As another example, cling allows automatic instantiations of templates. - -See this description of the `cppyy architecture`_ for further details. - -.. _cppyy architecture: http://morepypy.blogspot.com/2012/06/architecture-of-cppyy.html +.. _backend: https://pypi.python.org/pypi/PyPy-cppyy-backend Installation ------------ -There are two ways of using cppyy, and the choice depends on how pypy-c was -built: the backend can be builtin, or dynamically loadable. -The former has the disadvantage of requiring pypy-c to be linked with external -C++ libraries (e.g. libReflex.so), but has the advantage of being faster in -some cases. -That advantage will disappear over time, however, with improvements in the -JIT. -Therefore, this document assumes that the dynamically loadable backend is -chosen (it is, by default). -See the :doc:`backend documentation <cppyy_backend>`. +This assumes PyPy2.7 v5.7 or later; earlier versions use a Reflex-based cppyy +module, which is no longer supported. +Both the tooling and user-facing Python codes are very backwards compatible, +however. +Further dependencies are cmake (for general build), Python2.7 (for LLVM), and +a modern C++ compiler (one that supports at least C++11). -A standalone version of Reflex that also provides the dynamically loadable -backend is available for `download`_. Note this is currently the only way to -get the dynamically loadable backend, so use this first. +Assuming you have a recent enough version of PyPy installed, use pip to +complete the installation of cppyy:: -That version, as well as any other distribution of Reflex (e.g. the one that -comes with `ROOT`_, which may be part of your Linux distribution as part of -the selection of scientific software) will also work for a build with the -builtin backend. + $ MAKE_NPROCS=4 pypy-c -m pip install --verbose PyPy-cppyy-backend -.. _download: http://cern.ch/wlav/reflex-2014-10-20.tar.bz2 -.. _ROOT: http://root.cern.ch/ +Set the number of parallel builds ('4' in this example, through the MAKE_NPROCS +environment variable) to a number appropriate for your machine. +The building process may take quite some time as it includes a customized +version of LLVM as part of Cling, which is why --verbose is recommended so that +you can see the build progress. -Besides Reflex, you probably need a version of `gccxml`_ installed, which is -most easily provided by the packager of your system. -If you read up on gccxml, you will probably notice that it is no longer being -developed and hence will not provide C++11 support. -That's why the medium term plan is to move to cling. -Note that gccxml is only needed to generate reflection libraries. -It is not needed to use them. - -.. _gccxml: http://www.gccxml.org - -To install the standalone version of Reflex, after download:: - - $ tar jxf reflex-2014-10-20.tar.bz2 - $ cd reflex-2014-10-20 - $ ./build/autogen - $ ./configure <usual set of options such as --prefix> - $ make && make install - -The usual rules apply: <prefix>/bin needs to be added to the ``PATH`` and -<prefix>/lib to the ``LD_LIBRARY_PATH`` environment variable. -For convenience, this document will assume that there is a ``REFLEXHOME`` -variable that points to <prefix>. -If you downloaded or built the whole of ROOT, ``REFLEXHOME`` should be equal -to ``ROOTSYS``. - -The following is optional, and is only to show how pypy-c can be build -:doc:`from source <build>`, for example to get at the main development branch of cppyy. -The :doc:`backend documentation <cppyy_backend>` has more details on the backend-specific -prerequisites. - -Then run the translation to build ``pypy-c``:: - - $ hg clone https://bitbucket.org/pypy/pypy - $ cd pypy - $ hg up reflex-support # optional - - # This example shows python, but using pypy-c is faster and uses less memory - $ python rpython/bin/rpython --opt=jit pypy/goal/targetpypystandalone --withmod-cppyy - -This will build a ``pypy-c`` that includes the cppyy module, and through that, -Reflex support. -Of course, if you already have a pre-built version of the ``pypy`` interpreter, -you can use that for the translation rather than ``python``. -If not, you may want :ref:`to obtain a binary distribution <prebuilt-pypy>` to speed up the -translation step. +The default installation will be under +$PYTHONHOME/site-packages/cppyy_backend/lib, +which needs to be added to your dynamic loader path (LD_LIBRARY_PATH). +If you need the dictionary and class map generation tools (used in the examples +below), you need to add $PYTHONHOME/site-packages/cppyy_backend/bin to your +executable path (PATH). Basic bindings example ---------------------- -Now test with a trivial example whether all packages are properly installed -and functional. -First, create a C++ header file with some class in it (note that all functions -are made inline for convenience; a real-world example would of course have a -corresponding source file):: +These examples assume that cppyy_backend is pointed to by the environment +variable CPPYYHOME, and that CPPYYHOME/lib is added to LD_LIBRARY_PATH and +CPPYYHOME/bin to PATH. + +Let's first test with a trivial example whether all packages are properly +installed and functional. +Create a C++ header file with some class in it (all functions are made inline +for convenience; if you have out-of-line code, link with it as appropriate):: $ cat MyClass.h class MyClass { @@ -153,11 +69,11 @@ int m_myint; }; -Then, generate the bindings using ``genreflex`` (part of ROOT), and compile the -code:: +Then, generate the bindings using ``genreflex`` (installed under +cppyy_backend/bin in site_packages), and compile the code:: $ genreflex MyClass.h - $ g++ -fPIC -rdynamic -O2 -shared -I$REFLEXHOME/include MyClass_rflx.cpp -o libMyClassDict.so -L$REFLEXHOME/lib -lReflex + $ g++ -std=c++11 -fPIC -rdynamic -O2 -shared -I$CPPYYHOME/include MyClass_rflx.cpp -o libMyClassDict.so -L$CPPYYHOME/lib -lCling Next, make sure that the library can be found through the dynamic lookup path (the ``LD_LIBRARY_PATH`` environment variable on Linux, ``PATH`` on Windows), @@ -209,7 +125,7 @@ For example:: $ genreflex MyClass.h --rootmap=libMyClassDict.rootmap --rootmap-lib=libMyClassDict.so - $ g++ -fPIC -rdynamic -O2 -shared -I$REFLEXHOME/include MyClass_rflx.cpp -o libMyClassDict.so -L$REFLEXHOME/lib -lReflex + $ g++ -std=c++11 -fPIC -rdynamic -O2 -shared -I$CPPYYHOME/include MyClass_rflx.cpp -o libMyClassDict.so -L$CPPYYHOME/lib -lCling where the first option (``--rootmap``) specifies the output file name, and the second option (``--rootmap-lib``) the name of the reflection library where @@ -311,7 +227,7 @@ Now the reflection info can be generated and compiled:: $ genreflex MyAdvanced.h --selection=MyAdvanced.xml - $ g++ -fPIC -rdynamic -O2 -shared -I$REFLEXHOME/include MyAdvanced_rflx.cpp -o libAdvExDict.so -L$REFLEXHOME/lib -lReflex + $ g++ -std=c++11 -fPIC -rdynamic -O2 -shared -I$CPPYYHOME/include MyAdvanced_rflx.cpp -o libAdvExDict.so -L$CPPYYHOME/lib -lCling and subsequently be used from PyPy:: @@ -370,7 +286,7 @@ bound using:: $ genreflex example.h --deep --rootmap=libexampleDict.rootmap --rootmap-lib=libexampleDict.so - $ g++ -fPIC -rdynamic -O2 -shared -I$REFLEXHOME/include example_rflx.cpp -o libexampleDict.so -L$REFLEXHOME/lib -lReflex + $ g++ -std=c++11 -fPIC -rdynamic -O2 -shared -I$CPPYYHOME/include example_rflx.cpp -o libexampleDict.so -L$CPPYYHOME/lib -lCling * **abstract classes**: Are represented as python classes, since they are needed to complete the inheritance hierarchies, but will raise an exception @@ -666,13 +582,10 @@ Templates --------- -A bit of special care needs to be taken for the use of templates. -For a templated class to be completely available, it must be guaranteed that -said class is fully instantiated, and hence all executable C++ code is -generated and compiled in. -The easiest way to fulfill that guarantee is by explicit instantiation in the -header file that is handed to ``genreflex``. -The following example should make that clear:: +Templates can be automatically instantiated, assuming the appropriate header +files have been loaded or are accessible to the class loader. +This is the case for example for all of STL. +For example:: $ cat MyTemplate.h #include <vector> @@ -686,68 +599,10 @@ int m_i; }; - #ifdef __GCCXML__ - template class std::vector<MyClass>; // explicit instantiation - #endif - -If you know for certain that all symbols will be linked in from other sources, -you can also declare the explicit template instantiation ``extern``. -An alternative is to add an object to an unnamed namespace:: - - namespace { - std::vector<MyClass> vmc; - } // unnamed namespace - -Unfortunately, this is not always enough for gcc. -The iterators of vectors, if they are going to be used, need to be -instantiated as well, as do the comparison operators on those iterators, as -these live in an internal namespace, rather than in the iterator classes. -Note that you do NOT need this iterators to iterator over a vector. -You only need them if you plan to explicitly call e.g. ``begin`` and ``end`` -methods, and do comparisons of iterators. -One way to handle this, is to deal with this once in a macro, then reuse that -macro for all ``vector`` classes. -Thus, the header above needs this (again protected with -``#ifdef __GCCXML__``), instead of just the explicit instantiation of the -``vector<MyClass>``:: - - #define STLTYPES_EXPLICIT_INSTANTIATION_DECL(STLTYPE, TTYPE) \ - template class std::STLTYPE< TTYPE >; \ - template class __gnu_cxx::__normal_iterator<TTYPE*, std::STLTYPE< TTYPE > >; \ - template class __gnu_cxx::__normal_iterator<const TTYPE*, std::STLTYPE< TTYPE > >;\ - namespace __gnu_cxx { \ - template bool operator==(const std::STLTYPE< TTYPE >::iterator&, \ - const std::STLTYPE< TTYPE >::iterator&); \ - template bool operator!=(const std::STLTYPE< TTYPE >::iterator&, \ - const std::STLTYPE< TTYPE >::iterator&); \ - } - - STLTYPES_EXPLICIT_INSTANTIATION_DECL(vector, MyClass) - -Then, still for gcc, the selection file needs to contain the full hierarchy as -well as the global overloads for comparisons for the iterators:: - - $ cat MyTemplate.xml - <lcgdict> - <class pattern="std::vector<*>" /> - <class pattern="std::vector<*>::iterator" /> - <function name="__gnu_cxx::operator=="/> - <function name="__gnu_cxx::operator!="/> - - <class name="MyClass" /> - </lcgdict> - Run the normal ``genreflex`` and compilation steps:: $ genreflex MyTemplate.h --selection=MyTemplate.xml - $ g++ -fPIC -rdynamic -O2 -shared -I$REFLEXHOME/include MyTemplate_rflx.cpp -o libTemplateDict.so -L$REFLEXHOME/lib -lReflex - -Note: this is a dirty corner that clearly could do with some automation, -even if the macro already helps. -Such automation is planned. -In fact, in the Cling world, the backend can perform the template -instantations and generate the reflection info on the fly, and none of the -above will any longer be necessary. + $ g++ -std=c++11 -fPIC -rdynamic -O2 -shared -I$CPPYYHOME/include MyTemplate_rflx.cpp -o libTemplateDict.so -L$CPPYYHOME/lib -lCling Subsequent use should be as expected. Note the meta-class style of "instantiating" the template:: @@ -764,8 +619,6 @@ 1 2 3 >>>> -Other templates work similarly, but are typically simpler, as there are no -similar issues with iterators for e.g. ``std::list``. The arguments to the template instantiation can either be a string with the full list of arguments, or the explicit classes. The latter makes for easier code writing if the classes passed to the @@ -775,95 +628,40 @@ The fast lane ------------- -The following is an experimental feature of cppyy. -It mostly works, but there are some known issues (e.g. with return-by-value). -Soon it should be the default mode, however. +By default, cppyy will use direct function pointers through `CFFI`_ whenever +possible. If this causes problems for you, you can disable it by setting the +CPPYY_DISABLE_FASTPATH environment variable. -With a slight modification of Reflex, it can provide function pointers for -C++ methods, and hence allow PyPy to call those pointers directly, rather than -calling C++ through a Reflex stub. +.. _CFFI: https://cffi.readthedocs.io/en/latest/ -The standalone version of Reflex `provided`_ has been patched, but if you get -Reflex from another source (most likely with a ROOT distribution), locate the -file `genreflex-methptrgetter.patch`_ in pypy/module/cppyy and apply it to -the genreflex python scripts found in ``$ROOTSYS/lib``:: - - $ cd $ROOTSYS/lib - $ patch -p2 < genreflex-methptrgetter.patch - -With this patch, ``genreflex`` will have grown the ``--with-methptrgetter`` -option. -Use this option when running ``genreflex``, and add the -``-Wno-pmf-conversions`` option to ``g++`` when compiling. -The rest works the same way: the fast path will be used transparently (which -also means that you can't actually find out whether it is in use, other than -by running a micro-benchmark or a JIT test). - -.. _provided: http://cern.ch/wlav/reflex-2014-10-20.tar.bz2 -.. _genreflex-methptrgetter.patch: https://bitbucket.org/pypy/pypy/src/default/pypy/module/cppyy/genreflex-methptrgetter.patch CPython ------- -Most of the ideas in cppyy come originally from the `PyROOT`_ project. -Although PyROOT does not support Reflex directly, it has an alter ego called -"PyCintex" that, in a somewhat roundabout way, does. -If you installed ROOT, rather than just Reflex, PyCintex should be available -immediately if you add ``$ROOTSYS/lib`` to the ``PYTHONPATH`` environment -variable. +Most of the ideas in cppyy come originally from the `PyROOT`_ project, which +contains a CPython-based cppyy.py module (with similar dependencies as the +one that comes with PyPy). +A standalone pip-installable version is planned, but for now you can install +ROOT through your favorite distribution installer (available in the science +section). .. _PyROOT: https://root.cern.ch/pyroot -There are a couple of minor differences between PyCintex and cppyy, most to do -with naming. -The one that you will run into directly, is that PyCintex uses a function -called ``loadDictionary`` rather than ``load_reflection_info`` (it has the -same rootmap-based class loader functionality, though, making this point -somewhat moot). -The reason for this is that Reflex calls the shared libraries that contain -reflection info "dictionaries." -However, in python, the name `dictionary` already has a well-defined meaning, -so a more descriptive name was chosen for cppyy. -In addition, PyCintex requires that the names of shared libraries so loaded -start with "lib" in their name. -The basic example above, rewritten for PyCintex thus goes like this:: - - $ python - >>> import PyCintex - >>> PyCintex.loadDictionary("libMyClassDict.so") - >>> myinst = PyCintex.gbl.MyClass(42) - >>> print myinst.GetMyInt() - 42 - >>> myinst.SetMyInt(33) - >>> print myinst.m_myint - 33 - >>> myinst.m_myint = 77 - >>> print myinst.GetMyInt() - 77 - >>> help(PyCintex.gbl.MyClass) # shows that normal python introspection works - -Other naming differences are such things as taking an address of an object. -In PyCintex, this is done with ``AddressOf`` whereas in cppyy the choice was -made to follow the naming as in ``ctypes`` and hence use ``addressof`` -(PyROOT/PyCintex predate ``ctypes`` by several years, and the ROOT project -follows camel-case, hence the differences). - -Of course, this is python, so if any of the naming is not to your liking, all -you have to do is provide a wrapper script that you import instead of -importing the ``cppyy`` or ``PyCintex`` modules directly. -In that wrapper script you can rename methods exactly the way you need it. - -In the cling world, all these differences will be resolved. +There are a couple of minor differences between the two versions of cppyy +(the CPython version has a few more features). +Work is on-going to integrate the nightly tests of both to make sure their +feature sets are equalized. Python3 ------- -To change versions of CPython (to Python3, another version of Python, or later -to the `Py3k`_ version of PyPy), the only part that requires recompilation is -the bindings module, be it ``cppyy`` or ``libPyROOT.so`` (in PyCintex). -Although ``genreflex`` is indeed a Python tool, the generated reflection -information is completely independent of Python. +The CPython version of cppyy supports Python3, assuming your packager has +build the backend for it. +The cppyy module has not been tested with the `Py3k`_ version of PyPy. +Note that the generated reflection information (from ``genreflex``) is fully +independent of Python, and does not need to be rebuild when switching versions +or interpreters. .. _Py3k: https://bitbucket.org/pypy/pypy/src/py3k @@ -871,5 +669,4 @@ .. toctree:: :hidden: - cppyy_backend cppyy_example diff --git a/pypy/doc/cppyy_backend.rst b/pypy/doc/cppyy_backend.rst deleted file mode 100644 --- a/pypy/doc/cppyy_backend.rst +++ /dev/null @@ -1,45 +0,0 @@ -Backends for cppyy -================== - -The cppyy module needs a backend to provide the C++ reflection information on -which the Python bindings are build. -The backend is called through a C-API, which can be found in the PyPy sources -in: :source:`pypy/module/cppyy/include/capi.h`. -There are two kinds of API calls: querying about reflection information, which -are used during the creation of Python-side constructs, and making the actual -calls into C++. -The objects passed around are all opaque: cppyy does not make any assumptions -about them, other than that the opaque handles can be copied. -Their definition, however, appears in two places: in the C code (in capi.h), -and on the RPython side (in :source:`capi_types.py <pypy/module/cppyy/capi/capi_types.py>`), so if they are changed, they -need to be changed on both sides. - -There are two places where selections in the RPython code affect the choice -(and use) of the backend. -The first is in :source:`pypy/module/cppyy/capi/__init__.py`:: - - # choose C-API access method: - from pypy.module.cppyy.capi.loadable_capi import * - #from pypy.module.cppyy.capi.builtin_capi import * - -The default is the loadable C-API. -Comment it and uncomment the builtin C-API line, to use the builtin version. - -Next, if the builtin C-API is chosen, the specific backend needs to be set as -well (default is Reflex). -This second choice is in :source:`pypy/module/cppyy/capi/builtin_capi.py`:: - - import reflex_capi as backend - #import cint_capi as backend - -After those choices have been made, built pypy-c as usual. - -When building pypy-c from source, keep the following in mind. -If the loadable_capi is chosen, no further prerequisites are needed. -However, for the build of the builtin_capi to succeed, the ``ROOTSYS`` -environment variable must point to the location of your ROOT (or standalone -Reflex in the case of the Reflex backend) installation, or the ``root-config`` -utility must be accessible through ``$PATH`` (e.g. by adding ``$ROOTSYS/bin`` -to ``PATH``). -In case of the former, include files are expected under ``$ROOTSYS/include`` -and libraries under ``$ROOTSYS/lib``. diff --git a/pypy/doc/extending.rst b/pypy/doc/extending.rst --- a/pypy/doc/extending.rst +++ b/pypy/doc/extending.rst @@ -12,7 +12,7 @@ * Write them in pure Python and use ctypes_. -* Write them in C++ and bind them through Reflex_. +* Write them in C++ and bind them through :doc:`cppyy <cppyy>` using Cling. * Write them in as `RPython mixed modules`_. @@ -61,11 +61,11 @@ .. _libffi: http://sourceware.org/libffi/ -Reflex ------- +Cling and cppyy +--------------- The builtin :doc:`cppyy <cppyy>` module uses reflection information, provided by -`Reflex`_ (which needs to be `installed separately`_), of C/C++ code to +`Cling`_ (which needs to be `installed separately`_), of C/C++ code to automatically generate bindings at runtime. In Python, classes and functions are always runtime structures, so when they are generated matters not for performance. @@ -76,11 +76,14 @@ The :doc:`cppyy <cppyy>` module is written in RPython, thus PyPy's JIT is able to remove most cross-language call overhead. -:doc:`Full details <cppyy>` are `available here <cppyy>`. +:doc:Full details are `available here <cppyy>`. -.. _installed separately: http://cern.ch/wlav/reflex-2013-08-14.tar.bz2 -.. _Reflex: https://root.cern.ch/how/how-use-reflex +.. _installed separately: https://pypi.python.org/pypi/PyPy-cppyy-backend +.. _Cling: https://root.cern.ch/cling +.. toctree:: + + cppyy RPython Mixed Modules --------------------- @@ -94,7 +97,3 @@ This is how the numpy module is being developed. -.. toctree:: - :hidden: - - cppyy diff --git a/pypy/doc/index-of-release-notes.rst b/pypy/doc/index-of-release-notes.rst --- a/pypy/doc/index-of-release-notes.rst +++ b/pypy/doc/index-of-release-notes.rst @@ -59,6 +59,7 @@ .. toctree:: + release-pypy3.3-v5.5.0.rst release-pypy3.3-v5.2-alpha1.rst CPython 3.2 compatible versions diff --git a/pypy/doc/whatsnew-head.rst b/pypy/doc/whatsnew-head.rst --- a/pypy/doc/whatsnew-head.rst +++ b/pypy/doc/whatsnew-head.rst @@ -45,3 +45,22 @@ Assign ``tp_doc`` to the new TypeObject's type dictionary ``__doc__`` key so it will be picked up by app-level objects of that type + +.. branch: cling-support + +Module cppyy now uses cling as its backend (Reflex has been removed). The +user-facing interface and main developer tools (genreflex, selection files, +class loader, etc.) remain the same. A libcppyy_backend.so library is still +needed but is now available through PyPI with pip: PyPy-cppyy-backend. + +The Cling-backend brings support for modern C++ (11, 14, etc.), dynamic +template instantations, and improved integration with CFFI for better +performance. It also provides interactive C++ (and bindings to that). + +.. branch: better-PyDict_Next + +Improve the performance of ``PyDict_Next``. When trying ``PyDict_Next`` on a +typedef dict, the test exposed a problem converting a ``GetSetProperty`` to a +``PyGetSetDescrObject``. The other direction seem to be fully implemented. +This branch made a minimal effort to convert the basic fields to avoid +segfaults, but trying to use the ``PyGetSetDescrObject`` will probably fail. diff --git a/pypy/goal/targetpypystandalone.py b/pypy/goal/targetpypystandalone.py --- a/pypy/goal/targetpypystandalone.py +++ b/pypy/goal/targetpypystandalone.py @@ -83,12 +83,18 @@ return 1 return exitcode + return entry_point, get_additional_entrypoints(space, w_initstdio) + + +def get_additional_entrypoints(space, w_initstdio): # register the minimal equivalent of running a small piece of code. This # should be used as sparsely as possible, just to register callbacks - from rpython.rlib.entrypoint import entrypoint_highlevel from rpython.rtyper.lltypesystem import rffi, lltype + if space.config.objspace.disable_entrypoints: + return {} + @entrypoint_highlevel('main', [rffi.CCHARP, rffi.INT], c_name='pypy_setup_home') def pypy_setup_home(ll_home, verbose): @@ -188,11 +194,11 @@ return -1 return 0 - return entry_point, {'pypy_execute_source': pypy_execute_source, - 'pypy_execute_source_ptr': pypy_execute_source_ptr, - 'pypy_init_threads': pypy_init_threads, - 'pypy_thread_attach': pypy_thread_attach, - 'pypy_setup_home': pypy_setup_home} + return {'pypy_execute_source': pypy_execute_source, + 'pypy_execute_source_ptr': pypy_execute_source_ptr, + 'pypy_init_threads': pypy_init_threads, + 'pypy_thread_attach': pypy_thread_attach, + 'pypy_setup_home': pypy_setup_home} # _____ Define and setup target ___ diff --git a/pypy/interpreter/baseobjspace.py b/pypy/interpreter/baseobjspace.py --- a/pypy/interpreter/baseobjspace.py +++ b/pypy/interpreter/baseobjspace.py @@ -428,6 +428,8 @@ make_finalizer_queue(W_Root, self) self._code_of_sys_exc_info = None + self._builtin_functions_by_identifier = {'': None} + # can be overridden to a subclass self.initialize() diff --git a/pypy/interpreter/function.py b/pypy/interpreter/function.py --- a/pypy/interpreter/function.py +++ b/pypy/interpreter/function.py @@ -247,16 +247,15 @@ def descr_function_repr(self): return self.getrepr(self.space, 'function %s' % (self.name,)) - # delicate - _all = {'': None} def _cleanup_(self): + # delicate from pypy.interpreter.gateway import BuiltinCode if isinstance(self.code, BuiltinCode): # we have been seen by other means so rtyping should not choke # on us identifier = self.code.identifier - previous = Function._all.get(identifier, self) + previous = self.space._builtin_functions_by_identifier.get(identifier, self) assert previous is self, ( "duplicate function ids with identifier=%r: %r and %r" % ( identifier, previous, self)) @@ -264,10 +263,10 @@ return False def add_to_table(self): - Function._all[self.code.identifier] = self + self.space._builtin_functions_by_identifier[self.code.identifier] = self - def find(identifier): - return Function._all[identifier] + def find(space, identifier): + return space._builtin_functions_by_identifier[identifier] find = staticmethod(find) def descr_function__reduce__(self, space): diff --git a/pypy/interpreter/gateway.py b/pypy/interpreter/gateway.py --- a/pypy/interpreter/gateway.py +++ b/pypy/interpreter/gateway.py @@ -671,10 +671,10 @@ return space.newtuple([builtin_code, space.newtuple([space.wrap(self.identifier)])]) - def find(indentifier): + @staticmethod + def find(space, identifier): from pypy.interpreter.function import Function - return Function._all[indentifier].code - find = staticmethod(find) + return Function.find(space, identifier).code def signature(self): return self.sig diff --git a/pypy/interpreter/pyframe.py b/pypy/interpreter/pyframe.py --- a/pypy/interpreter/pyframe.py +++ b/pypy/interpreter/pyframe.py @@ -4,6 +4,7 @@ import sys from rpython.rlib import jit from rpython.rlib.debug import make_sure_not_resized, check_nonneg +from rpython.rlib.debug import ll_assert_not_none from rpython.rlib.jit import hint from rpython.rlib.objectmodel import instantiate, specialize, we_are_translated from rpython.rlib.rarithmetic import intmask, r_uint @@ -298,7 +299,13 @@ # stack manipulation helpers def pushvalue(self, w_object): depth = self.valuestackdepth - self.locals_cells_stack_w[depth] = w_object + self.locals_cells_stack_w[depth] = ll_assert_not_none(w_object) + self.valuestackdepth = depth + 1 + + def pushvalue_none(self): + depth = self.valuestackdepth + # the entry is already None, and remains None + assert self.locals_cells_stack_w[depth] is None self.valuestackdepth = depth + 1 def _check_stack_index(self, index): @@ -311,6 +318,9 @@ return index >= stackstart def popvalue(self): + return ll_assert_not_none(self.popvalue_maybe_none()) + + def popvalue_maybe_none(self): depth = self.valuestackdepth - 1 assert self._check_stack_index(depth) assert depth >= 0 @@ -385,6 +395,9 @@ def peekvalue(self, index_from_top=0): # NOTE: top of the stack is peekvalue(0). # Contrast this with CPython where it's PEEK(-1). + return ll_assert_not_none(self.peekvalue_maybe_none(index_from_top)) + + def peekvalue_maybe_none(self, index_from_top=0): index_from_top = hint(index_from_top, promote=True) index = self.valuestackdepth + ~index_from_top assert self._check_stack_index(index) @@ -396,7 +409,7 @@ index = self.valuestackdepth + ~index_from_top assert self._check_stack_index(index) assert index >= 0 - self.locals_cells_stack_w[index] = w_object + self.locals_cells_stack_w[index] = ll_assert_not_none(w_object) @jit.unroll_safe def dropvaluesuntil(self, finaldepth): diff --git a/pypy/interpreter/test/test_pyframe.py b/pypy/interpreter/test/test_pyframe.py --- a/pypy/interpreter/test/test_pyframe.py +++ b/pypy/interpreter/test/test_pyframe.py @@ -580,3 +580,25 @@ pass sys.settrace(None) assert seen == ['call', 'exception', 'return'] + + def test_generator_trace_stopiteration(self): + import sys + def f(): + yield 5 + gen = f() + assert next(gen) == 5 + seen = [] + def trace_func(frame, event, *args): + print('TRACE:', frame, event, args) + seen.append(event) + return trace_func + def g(): + for x in gen: + never_entered + sys.settrace(trace_func) + g() + sys.settrace(None) + print 'seen:', seen + # on Python 3 we get an extra 'exception' when 'for' catches + # StopIteration + assert seen == ['call', 'line', 'call', 'return', 'return'] diff --git a/pypy/module/_cffi_backend/__init__.py b/pypy/module/_cffi_backend/__init__.py --- a/pypy/module/_cffi_backend/__init__.py +++ b/pypy/module/_cffi_backend/__init__.py @@ -1,6 +1,6 @@ import sys from pypy.interpreter.mixedmodule import MixedModule -from rpython.rlib import rdynload, clibffi, entrypoint +from rpython.rlib import rdynload, clibffi from rpython.rtyper.lltypesystem import rffi VERSION = "1.9.1" @@ -68,9 +68,14 @@ if has_stdcall: interpleveldefs['FFI_STDCALL'] = 'space.wrap(%d)' % FFI_STDCALL - def startup(self, space): - from pypy.module._cffi_backend import embedding - embedding.glob.space = space + def __init__(self, space, *args): + MixedModule.__init__(self, space, *args) + # + if not space.config.objspace.disable_entrypoints: + # import 'embedding', which has the side-effect of registering + # the 'pypy_init_embedded_cffi_module' entry point + from pypy.module._cffi_backend import embedding + embedding.glob.space = space def get_dict_rtld_constants(): @@ -85,11 +90,3 @@ for _name, _value in get_dict_rtld_constants().items(): Module.interpleveldefs[_name] = 'space.wrap(%d)' % _value - - -# write this entrypoint() here, to make sure it is registered early enough -@entrypoint.entrypoint_highlevel('main', [rffi.INT, rffi.VOIDP], - c_name='pypy_init_embedded_cffi_module') -def pypy_init_embedded_cffi_module(version, init_struct): - from pypy.module._cffi_backend import embedding - return embedding.pypy_init_embedded_cffi_module(version, init_struct) diff --git a/pypy/module/_cffi_backend/embedding.py b/pypy/module/_cffi_backend/embedding.py --- a/pypy/module/_cffi_backend/embedding.py +++ b/pypy/module/_cffi_backend/embedding.py @@ -1,4 +1,5 @@ import os +from rpython.rlib import entrypoint from rpython.rtyper.lltypesystem import lltype, rffi from rpython.translator.tool.cbuild import ExternalCompilationInfo @@ -46,6 +47,8 @@ glob = Global() +@entrypoint.entrypoint_highlevel('main', [rffi.INT, rffi.VOIDP], + c_name='pypy_init_embedded_cffi_module') def pypy_init_embedded_cffi_module(version, init_struct): # called from __init__.py name = "?" diff --git a/pypy/module/_pickle_support/maker.py b/pypy/module/_pickle_support/maker.py --- a/pypy/module/_pickle_support/maker.py +++ b/pypy/module/_pickle_support/maker.py @@ -77,7 +77,7 @@ def builtin_code(space, identifier): from pypy.interpreter import gateway try: - return gateway.BuiltinCode.find(identifier) + return gateway.BuiltinCode.find(space, identifier) except KeyError: raise oefmt(space.w_RuntimeError, "cannot unpickle builtin code: %s", identifier) @@ -86,7 +86,7 @@ def builtin_function(space, identifier): from pypy.interpreter import function try: - return function.Function.find(identifier) + return function.Function.find(space, identifier) except KeyError: raise oefmt(space.w_RuntimeError, "cannot unpickle builtin function: %s", identifier) diff --git a/pypy/module/_ssl/test/test_ssl.py b/pypy/module/_ssl/test/test_ssl.py --- a/pypy/module/_ssl/test/test_ssl.py +++ b/pypy/module/_ssl/test/test_ssl.py @@ -169,8 +169,8 @@ } def setup_method(self, method): - # https://www.verisign.net/ - ADDR = "www.verisign.net", 443 + # https://gmail.com/ + ADDR = "gmail.com", 443 self.w_s = self.space.appexec([self.space.wrap(ADDR)], """(ADDR): import socket diff --git a/pypy/module/cppyy/__init__.py b/pypy/module/cppyy/__init__.py --- a/pypy/module/cppyy/__init__.py +++ b/pypy/module/cppyy/__init__.py @@ -14,7 +14,6 @@ '_set_class_generator' : 'interp_cppyy.set_class_generator', '_set_function_generator': 'interp_cppyy.set_function_generator', '_register_class' : 'interp_cppyy.register_class', - '_is_static' : 'interp_cppyy.is_static', '_get_nullptr' : 'interp_cppyy.get_nullptr', 'CPPInstanceBase' : 'interp_cppyy.W_CPPInstance', 'addressof' : 'interp_cppyy.addressof', diff --git a/pypy/module/cppyy/bench/Makefile b/pypy/module/cppyy/bench/Makefile --- a/pypy/module/cppyy/bench/Makefile +++ b/pypy/module/cppyy/bench/Makefile @@ -26,4 +26,4 @@ bench02Dict_reflex.so: bench02.h bench02.cxx bench02.xml $(genreflex) bench02.h $(genreflexflags) --selection=bench02.xml -I$(ROOTSYS)/include - g++ -o $@ bench02.cxx bench02_rflx.cpp -I$(ROOTSYS)/include -shared -lReflex -lHistPainter `root-config --libs` $(cppflags) $(cppflags2) + g++ -o $@ bench02.cxx bench02_rflx.cpp -I$(ROOTSYS)/include -shared -std=c++11 -lHistPainter `root-config --libs` $(cppflags) $(cppflags2) diff --git a/pypy/module/cppyy/capi/builtin_capi.py b/pypy/module/cppyy/capi/builtin_capi.py --- a/pypy/module/cppyy/capi/builtin_capi.py +++ b/pypy/module/cppyy/capi/builtin_capi.py @@ -1,12 +1,11 @@ from rpython.rtyper.lltypesystem import rffi, lltype +from rpython.rlib.rarithmetic import intmask from rpython.rlib import jit -import reflex_capi as backend -#import cint_capi as backend +import cling_capi as backend from pypy.module.cppyy.capi.capi_types import C_SCOPE, C_TYPE, C_OBJECT,\ - C_METHOD, C_INDEX, C_INDEX_ARRAY, WLAVC_INDEX,\ - C_METHPTRGETTER, C_METHPTRGETTER_PTR + C_METHOD, C_INDEX, C_INDEX_ARRAY, WLAVC_INDEX, C_FUNC_PTR identify = backend.identify pythonize = backend.pythonize @@ -52,13 +51,6 @@ compilation_info=backend.eci) def c_get_scope_opaque(space, name): return _c_get_scope_opaque(name) -_c_get_template = rffi.llexternal( - "cppyy_get_template", - [rffi.CCHARP], C_TYPE, - releasegil=ts_reflect, - compilation_info=backend.eci) -def c_get_template(space, name): - return _c_get_template(name) _c_actual_class = rffi.llexternal( "cppyy_actual_class", [C_TYPE, C_OBJECT], C_TYPE, @@ -154,6 +146,13 @@ compilation_info=backend.eci) def c_call_d(space, cppmethod, cppobject, nargs, args): return _c_call_d(cppmethod, cppobject, nargs, args) +_c_call_ld = rffi.llexternal( + "cppyy_call_ld", + [C_METHOD, C_OBJECT, rffi.INT, rffi.VOIDP], rffi.LONGDOUBLE, + releasegil=ts_call, + compilation_info=backend.eci) +def c_call_ld(space, cppmethod, cppobject, nargs, args): + return _c_call_ld(cppmethod, cppobject, nargs, args) _c_call_r = rffi.llexternal( "cppyy_call_r", @@ -164,11 +163,17 @@ return _c_call_r(cppmethod, cppobject, nargs, args) _c_call_s = rffi.llexternal( "cppyy_call_s", - [C_METHOD, C_OBJECT, rffi.INT, rffi.VOIDP], rffi.CCHARP, + [C_METHOD, C_OBJECT, rffi.INT, rffi.VOIDP, rffi.SIZE_TP], rffi.CCHARP, releasegil=ts_call, compilation_info=backend.eci) def c_call_s(space, cppmethod, cppobject, nargs, args): - return _c_call_s(cppmethod, cppobject, nargs, args) + length = lltype.malloc(rffi.SIZE_TP.TO, 1, flavor='raw') + try: + cstr = _c_call_s(cppmethod, cppobject, nargs, args, length) + cstr_len = intmask(length[0]) + finally: + lltype.free(length, flavor='raw') + return cstr, cstr_len _c_constructor = rffi.llexternal( "cppyy_constructor", @@ -185,15 +190,14 @@ def c_call_o(space, method, cppobj, nargs, args, cppclass): return _c_call_o(method, cppobj, nargs, args, cppclass.handle) -_c_get_methptr_getter = rffi.llexternal( - "cppyy_get_methptr_getter", - [C_SCOPE, C_INDEX], C_METHPTRGETTER_PTR, +_c_get_function_address = rffi.llexternal( + "cppyy_get_function_address", + [C_SCOPE, C_INDEX], C_FUNC_PTR, releasegil=ts_reflect, compilation_info=backend.eci, - elidable_function=True, random_effects_on_gcobjs=False) -def c_get_methptr_getter(space, cppscope, index): - return _c_get_methptr_getter(cppscope.handle, index) +def c_get_function_address(space, cppscope, index): + return _c_get_function_address(cppscope.handle, index) # handling of function argument buffer --------------------------------------- _c_allocate_function_args = rffi.llexternal( @@ -215,8 +219,8 @@ [], rffi.SIZE_T, releasegil=ts_memory, compilation_info=backend.eci, - elidable_function=True, random_effects_on_gcobjs=False) +@jit.elidable def c_function_arg_sizeof(space): return _c_function_arg_sizeof() _c_function_arg_typeoffset = rffi.llexternal( @@ -224,8 +228,8 @@ [], rffi.SIZE_T, releasegil=ts_memory, compilation_info=backend.eci, - elidable_function=True, random_effects_on_gcobjs=False) +@jit.elidable def c_function_arg_typeoffset(space): return _c_function_arg_typeoffset() @@ -237,6 +241,20 @@ compilation_info=backend.eci) def c_is_namespace(space, scope): return _c_is_namespace(scope) +_c_is_template = rffi.llexternal( + "cppyy_is_template", + [rffi.CCHARP], rffi.INT, + releasegil=ts_reflect, + compilation_info=backend.eci) +def c_is_template(space, name): + return _c_is_template(name) +_c_is_abstract = rffi.llexternal( + "cppyy_is_abstract", + [C_SCOPE], rffi.INT, + releasegil=ts_reflect, + compilation_info=backend.eci) +def c_is_abstract(space, cpptype): + return _c_is_abstract(cpptype) _c_is_enum = rffi.llexternal( "cppyy_is_enum", [rffi.CCHARP], rffi.INT, @@ -286,9 +304,8 @@ [C_TYPE, C_TYPE], rffi.INT, releasegil=ts_reflect, compilation_info=backend.eci, - elidable_function=True, random_effects_on_gcobjs=False) -@jit.elidable_promote('2') +@jit.elidable def c_is_subtype(space, derived, base): if derived == base: return 1 @@ -296,12 +313,11 @@ _c_base_offset = rffi.llexternal( "cppyy_base_offset", - [C_TYPE, C_TYPE, C_OBJECT, rffi.INT], rffi.SIZE_T, + [C_TYPE, C_TYPE, C_OBJECT, rffi.INT], rffi.LONG, # actually ptrdiff_t releasegil=ts_reflect, compilation_info=backend.eci, - elidable_function=True, random_effects_on_gcobjs=False) -@jit.elidable_promote('1,2,4') +@jit.elidable def c_base_offset(space, derived, base, address, direction): if derived == base: return 0 @@ -340,7 +356,7 @@ i += 1 py_indices.append(index) index = indices[i] - c_free(rffi.cast(rffi.VOIDP, indices)) # c_free defined below + c_free(space, rffi.cast(rffi.VOIDP, indices)) # c_free defined below return py_indices _c_method_name = rffi.llexternal( @@ -474,7 +490,7 @@ return charp2str_free(space, _c_datamember_type(cppscope.handle, datamember_index)) _c_datamember_offset = rffi.llexternal( "cppyy_datamember_offset", - [C_SCOPE, rffi.INT], rffi.SIZE_T, + [C_SCOPE, rffi.INT], rffi.LONG, # actually ptrdiff_t releasegil=ts_reflect, compilation_info=backend.eci) def c_datamember_offset(space, cppscope, datamember_index): @@ -519,27 +535,29 @@ compilation_info=backend.eci) def c_strtoull(space, svalue): return _c_strtoull(svalue) -c_free = rffi.llexternal( +_c_free = rffi.llexternal( "cppyy_free", [rffi.VOIDP], lltype.Void, releasegil=ts_memory, compilation_info=backend.eci) +def c_free(space, voidp): + return _c_free(voidp) def charp2str_free(space, charp): string = rffi.charp2str(charp) voidp = rffi.cast(rffi.VOIDP, charp) - c_free(voidp) + _c_free(voidp) return string _c_charp2stdstring = rffi.llexternal( "cppyy_charp2stdstring", - [rffi.CCHARP], C_OBJECT, + [rffi.CCHARP, rffi.SIZE_T], C_OBJECT, releasegil=ts_helper, compilation_info=backend.eci) -def c_charp2stdstring(space, svalue): - with rffi.scoped_view_charp(svalue) as charp: - result = _c_charp2stdstring(charp) - return result +def c_charp2stdstring(space, pystr, sz): + with rffi.scoped_view_charp(pystr) as cstr: + cppstr = _c_charp2stdstring(cstr, sz) + return cppstr _c_stdstring2stdstring = rffi.llexternal( "cppyy_stdstring2stdstring", [C_OBJECT], C_OBJECT, @@ -547,3 +565,26 @@ compilation_info=backend.eci) def c_stdstring2stdstring(space, cppobject): return _c_stdstring2stdstring(cppobject) + +_c_stdvector_valuetype = rffi.llexternal( + "cppyy_stdvector_valuetype", + [rffi.CCHARP], rffi.CCHARP, + releasegil=ts_helper, + compilation_info=backend.eci) +def c_stdvector_valuetype(space, pystr): + cstr = rffi.str2charp(pystr) + result = _c_stdvector_valuetype(cstr) + rffi.free_charp(cstr) + if result: + return charp2str_free(space, result) + return "" +_c_stdvector_valuesize = rffi.llexternal( + "cppyy_stdvector_valuesize", + [rffi.CCHARP], rffi.SIZE_T, + releasegil=ts_helper, + compilation_info=backend.eci) +def c_stdvector_valuesize(space, pystr): + cstr = rffi.str2charp(pystr) + result = _c_stdvector_valuesize(cstr) + rffi.free_charp(cstr) + return result diff --git a/pypy/module/cppyy/capi/capi_types.py b/pypy/module/cppyy/capi/capi_types.py --- a/pypy/module/cppyy/capi/capi_types.py +++ b/pypy/module/cppyy/capi/capi_types.py @@ -18,5 +18,4 @@ C_INDEX_ARRAY = rffi.LONGP WLAVC_INDEX = rffi.LONG -C_METHPTRGETTER = lltype.FuncType([C_OBJECT], rffi.VOIDP) -C_METHPTRGETTER_PTR = lltype.Ptr(C_METHPTRGETTER) +C_FUNC_PTR = rffi.VOIDP diff --git a/pypy/module/cppyy/capi/cint_capi.py b/pypy/module/cppyy/capi/cint_capi.py deleted file mode 100644 --- a/pypy/module/cppyy/capi/cint_capi.py +++ /dev/null @@ -1,437 +0,0 @@ -import py, os, sys - -from pypy.interpreter.error import OperationError -from pypy.interpreter.gateway import interp2app, unwrap_spec -from pypy.interpreter.typedef import TypeDef -from pypy.interpreter.baseobjspace import W_Root - -from rpython.translator.tool.cbuild import ExternalCompilationInfo -from rpython.rtyper.lltypesystem import rffi, lltype -from rpython.rlib import libffi, rdynload -from rpython.tool.udir import udir - -from pypy.module.cppyy.capi.capi_types import C_OBJECT - - -__all__ = ['identify', 'std_string_name', 'eci', 'c_load_dictionary'] - -pkgpath = py.path.local(__file__).dirpath().join(os.pardir) -srcpath = pkgpath.join("src") -incpath = pkgpath.join("include") - -if os.environ.get("ROOTSYS"): - import commands - (stat, incdir) = commands.getstatusoutput("root-config --incdir") - if stat != 0: - rootincpath = [os.path.join(os.environ["ROOTSYS"], "include"), py.path.local(udir)] - rootlibpath = [os.path.join(os.environ["ROOTSYS"], "lib64"), os.path.join(os.environ["ROOTSYS"], "lib")] - else: - rootincpath = [incdir, py.path.local(udir)] - rootlibpath = commands.getoutput("root-config --libdir").split() -else: - rootincpath = [py.path.local(udir)] - rootlibpath = [] - -def identify(): - return 'CINT' - -ts_reflect = True -ts_call = True -ts_memory = False -ts_helper = False - -std_string_name = 'string' - -# force loading in global mode of core libraries, rather than linking with -# them as PyPy uses various version of dlopen in various places; note that -# this isn't going to fly on Windows (note that locking them in objects and -# calling dlclose in __del__ seems to come too late, so this'll do for now) -with rffi.scoped_str2charp('libCint.so') as ll_libname: - _cintdll = rdynload.dlopen(ll_libname, rdynload.RTLD_GLOBAL | rdynload.RTLD_NOW) -with rffi.scoped_str2charp('libCore.so') as ll_libname: - _coredll = rdynload.dlopen(ll_libname, rdynload.RTLD_GLOBAL | rdynload.RTLD_NOW) -with rffi.scoped_str2charp('libHist.so') as ll_libname: - _coredll = rdynload.dlopen(ll_libname, rdynload.RTLD_GLOBAL | rdynload.RTLD_NOW) - -eci = ExternalCompilationInfo( - separate_module_files=[srcpath.join("cintcwrapper.cxx")], - include_dirs=[incpath] + rootincpath, - includes=["cintcwrapper.h"], - library_dirs=rootlibpath, - libraries=["Hist", "Core", "Cint"], - use_cpp_linker=True, -) - -_c_load_dictionary = rffi.llexternal( - "cppyy_load_dictionary", - [rffi.CCHARP], rdynload.DLLHANDLE, - releasegil=False, - compilation_info=eci) - -def c_load_dictionary(name): - result = _c_load_dictionary(name) - # ignore result: libffi.CDLL(name) either returns a handle to the already - # open file, or will fail as well and produce a correctly formatted error - return libffi.CDLL(name) - - -# CINT-specific pythonizations =============================================== -_c_charp2TString = rffi.llexternal( - "cppyy_charp2TString", - [rffi.CCHARP], C_OBJECT, - releasegil=ts_helper, - compilation_info=eci) -def c_charp2TString(space, svalue): - with rffi.scoped_view_charp(svalue) as charp: - result = _c_charp2TString(charp) - return result -_c_TString2TString = rffi.llexternal( - "cppyy_TString2TString", - [C_OBJECT], C_OBJECT, - releasegil=ts_helper, - compilation_info=eci) -def c_TString2TString(space, cppobject): - return _c_TString2TString(cppobject) - -def _get_string_data(space, w_obj, m1, m2 = None): - from pypy.module.cppyy import interp_cppyy - obj = space.interp_w(interp_cppyy.W_CPPInstance, w_obj) - w_1 = obj.space.call_method(w_obj, m1) - if m2 is None: - return w_1 - return obj.space.call_method(w_1, m2) - -### TF1 ---------------------------------------------------------------------- -class State(object): - def __init__(self, space): - self.tfn_pyfuncs = [] - self.tfn_callbacks = [] - -_create_tf1 = rffi.llexternal( - "cppyy_create_tf1", - [rffi.CCHARP, rffi.ULONG, rffi.DOUBLE, rffi.DOUBLE, rffi.INT], C_OBJECT, - releasegil=False, - compilation_info=eci) - -@unwrap_spec(args_w='args_w') -def tf1_tf1(space, w_self, args_w): - """Pythonized version of TF1 constructor: - takes functions and callable objects, and allows a callback into them.""" - - from pypy.module.cppyy import interp_cppyy - tf1_class = interp_cppyy.scope_byname(space, "TF1") - - # expected signature: - # 1. (char* name, pyfunc, double xmin, double xmax, int npar = 0) - argc = len(args_w) - - try: - if argc < 4 or 5 < argc: - raise TypeError("wrong number of arguments") - - # first argument must be a name - funcname = space.str_w(args_w[0]) - - # last (optional) argument is number of parameters - npar = 0 - if argc == 5: npar = space.int_w(args_w[4]) - - # second argument must be a callable python object - w_callable = args_w[1] - if not space.is_true(space.callable(w_callable)): - raise TypeError("2nd argument is not a valid python callable") - - # generate a pointer to function - from pypy.module._cffi_backend import newtype, ctypefunc, func - - c_double = newtype.new_primitive_type(space, 'double') - c_doublep = newtype.new_pointer_type(space, c_double) - - # wrap the callable as the signature needs modifying - w_ifunc = interp_cppyy.get_interface_func(space, w_callable, npar) - - w_cfunc = ctypefunc.W_CTypeFunc(space, [c_doublep, c_doublep], c_double, False) - w_callback = func.callback(space, w_cfunc, w_ifunc, None) - funcaddr = rffi.cast(rffi.ULONG, w_callback.get_closure()) - - # so far, so good; leaves on issue: CINT is expecting a wrapper, but - # we need the overload that takes a function pointer, which is not in - # the dictionary, hence this helper: - newinst = _create_tf1(space.str_w(args_w[0]), funcaddr, - space.float_w(args_w[2]), space.float_w(args_w[3]), npar) - - # w_self is a null-ptr bound as TF1 - from pypy.module.cppyy.interp_cppyy import W_CPPInstance, memory_regulator - cppself = space.interp_w(W_CPPInstance, w_self, can_be_None=False) - cppself._rawobject = newinst - memory_regulator.register(cppself) - - # tie all the life times to the TF1 instance - space.setattr(w_self, space.wrap('_callback'), w_callback) - - # by definition for __init__ - return None - - except (OperationError, TypeError, IndexError) as e: - newargs_w = args_w[1:] # drop class - - # return control back to the original, unpythonized overload - ol = tf1_class.get_overload("TF1") - return ol.call(None, newargs_w) - -### TTree -------------------------------------------------------------------- -_ttree_Branch = rffi.llexternal( - "cppyy_ttree_Branch", - [rffi.VOIDP, rffi.CCHARP, rffi.CCHARP, rffi.VOIDP, rffi.INT, rffi.INT], C_OBJECT, - releasegil=False, - compilation_info=eci) - -@unwrap_spec(args_w='args_w') -def ttree_Branch(space, w_self, args_w): - """Pythonized version of TTree::Branch(): takes proxy objects and by-passes - the CINT-manual layer.""" - - from pypy.module.cppyy import interp_cppyy - tree_class = interp_cppyy.scope_byname(space, "TTree") - - # sigs to modify (and by-pass CINT): - # 1. (const char*, const char*, T**, Int_t=32000, Int_t=99) - # 2. (const char*, T**, Int_t=32000, Int_t=99) - argc = len(args_w) - - # basic error handling of wrong arguments is best left to the original call, - # so that error messages etc. remain consistent in appearance: the following - # block may raise TypeError or IndexError to break out anytime - - try: - if argc < 2 or 5 < argc: - raise TypeError("wrong number of arguments") - - tree = space.interp_w(interp_cppyy.W_CPPInstance, w_self, can_be_None=True) - if (tree is None) or (tree.cppclass != tree_class): - raise TypeError("not a TTree") - - # first argument must always always be cont char* - branchname = space.str_w(args_w[0]) - - # if args_w[1] is a classname, then case 1, else case 2 - try: - classname = space.str_w(args_w[1]) - addr_idx = 2 - w_address = args_w[addr_idx] - except (OperationError, TypeError): - addr_idx = 1 - w_address = args_w[addr_idx] - - bufsize, splitlevel = 32000, 99 - if addr_idx+1 < argc: bufsize = space.c_int_w(args_w[addr_idx+1]) - if addr_idx+2 < argc: splitlevel = space.c_int_w(args_w[addr_idx+2]) - - # now retrieve the W_CPPInstance and build other stub arguments - space = tree.space # holds the class cache in State - cppinstance = space.interp_w(interp_cppyy.W_CPPInstance, w_address) - address = rffi.cast(rffi.VOIDP, cppinstance.get_rawobject()) - klassname = cppinstance.cppclass.full_name() - vtree = rffi.cast(rffi.VOIDP, tree.get_rawobject()) - - # call the helper stub to by-pass CINT - vbranch = _ttree_Branch(vtree, branchname, klassname, address, bufsize, splitlevel) - branch_class = interp_cppyy.scope_byname(space, "TBranch") - w_branch = interp_cppyy.wrap_cppobject(space, vbranch, branch_class) - return w_branch - except (OperationError, TypeError, IndexError): - pass - - # return control back to the original, unpythonized overload - ol = tree_class.get_overload("Branch") - return ol.call(w_self, args_w) - -def activate_branch(space, w_branch): - w_branches = space.call_method(w_branch, "GetListOfBranches") - for i in range(space.r_longlong_w(space.call_method(w_branches, "GetEntriesFast"))): - w_b = space.call_method(w_branches, "At", space.wrap(i)) - activate_branch(space, w_b) - space.call_method(w_branch, "SetStatus", space.wrap(1)) - space.call_method(w_branch, "ResetReadEntry") - -c_ttree_GetEntry = rffi.llexternal( - "cppyy_ttree_GetEntry", - [rffi.VOIDP, rffi.LONGLONG], rffi.LONGLONG, - releasegil=False, - compilation_info=eci) - -@unwrap_spec(args_w='args_w') -def ttree_getattr(space, w_self, args_w): - """Specialized __getattr__ for TTree's that allows switching on/off the - reading of individual branchs.""" - - from pypy.module.cppyy import interp_cppyy - tree = space.interp_w(interp_cppyy.W_CPPInstance, w_self) - - space = tree.space # holds the class cache in State - - # prevent recursion - attr = space.str_w(args_w[0]) - if attr and attr[0] == '_': - raise OperationError(space.w_AttributeError, args_w[0]) - - # try the saved cdata (for builtin types) - try: - w_cdata = space.getattr(w_self, space.wrap('_'+attr)) - from pypy.module._cffi_backend import cdataobj - cdata = space.interp_w(cdataobj.W_CData, w_cdata, can_be_None=False) - return cdata.convert_to_object() - except OperationError: - pass - - # setup branch as a data member and enable it for reading - w_branch = space.call_method(w_self, "GetBranch", args_w[0]) - if not space.is_true(w_branch): - raise OperationError(space.w_AttributeError, args_w[0]) - activate_branch(space, w_branch) - - # figure out from where we're reading - entry = space.r_longlong_w(space.call_method(w_self, "GetReadEntry")) - if entry == -1: - entry = 0 - - # setup cache structure - w_klassname = space.call_method(w_branch, "GetClassName") - if space.is_true(w_klassname): - # some instance - klass = interp_cppyy.scope_byname(space, space.str_w(w_klassname)) - w_obj = klass.construct() - # 0x10000 = kDeleteObject; reset because we own the object - space.call_method(w_branch, "ResetBit", space.wrap(0x10000)) - space.call_method(w_branch, "SetObject", w_obj) - space.call_method(w_branch, "GetEntry", space.wrap(entry)) - space.setattr(w_self, args_w[0], w_obj) - return w_obj - else: - # builtin data - w_leaf = space.call_method(w_self, "GetLeaf", args_w[0]) - space.call_method(w_branch, "GetEntry", space.wrap(entry)) - - # location - w_address = space.call_method(w_leaf, "GetValuePointer") - buf = space.getarg_w('s*', w_address) - from pypy.module._rawffi import buffer - assert isinstance(buf, buffer.RawFFIBuffer) - address = rffi.cast(rffi.CCHARP, buf.datainstance.ll_buffer) - - # placeholder - w_typename = space.call_method(w_leaf, "GetTypeName" ) - from pypy.module.cppyy import capi - typename = capi.c_resolve_name(space, space.str_w(w_typename)) - if typename == 'bool': typename = '_Bool' - w_address = space.call_method(w_leaf, "GetValuePointer") - from pypy.module._cffi_backend import cdataobj, newtype - cdata = cdataobj.W_CData(space, address, newtype.new_primitive_type(space, typename)) - - # cache result - space.setattr(w_self, space.wrap('_'+attr), space.wrap(cdata)) - return space.getattr(w_self, args_w[0]) - -class W_TTreeIter(W_Root): - def __init__(self, space, w_tree): - from pypy.module.cppyy import interp_cppyy - tree = space.interp_w(interp_cppyy.W_CPPInstance, w_tree) - self.vtree = rffi.cast(rffi.VOIDP, tree.get_cppthis(tree.cppclass)) - self.w_tree = w_tree - - self.current = 0 - self.maxentry = space.r_longlong_w(space.call_method(w_tree, "GetEntriesFast")) - - space = self.space = tree.space # holds the class cache in State - space.call_method(w_tree, "SetBranchStatus", space.wrap("*"), space.wrap(0)) - - def iter_w(self): - return self.space.wrap(self) - - def next_w(self): - if self.current == self.maxentry: - raise OperationError(self.space.w_StopIteration, self.space.w_None) - # TODO: check bytes read? - c_ttree_GetEntry(self.vtree, self.current) - self.current += 1 - return self.w_tree - -W_TTreeIter.typedef = TypeDef( - 'TTreeIter', - __iter__ = interp2app(W_TTreeIter.iter_w), - next = interp2app(W_TTreeIter.next_w), -) - -def ttree_iter(space, w_self): - """Allow iteration over TTree's. Also initializes branch data members and - sets addresses, if needed.""" - w_treeiter = W_TTreeIter(space, w_self) - return w_treeiter - -# setup pythonizations for later use at run-time -_pythonizations = {} -def register_pythonizations(space): - "NOT_RPYTHON" - - allfuncs = [ - - ### TF1 - tf1_tf1, - - ### TTree - ttree_Branch, ttree_iter, ttree_getattr, - ] - - for f in allfuncs: - _pythonizations[f.__name__] = space.wrap(interp2app(f)) - -def _method_alias(space, w_pycppclass, m1, m2): - space.setattr(w_pycppclass, space.wrap(m1), - space.getattr(w_pycppclass, space.wrap(m2))) - -# callback coming in when app-level bound classes have been created -def pythonize(space, name, w_pycppclass): - - if name == "TCollection": - _method_alias(space, w_pycppclass, "append", "Add") - _method_alias(space, w_pycppclass, "__len__", "GetSize") - - elif name == "TF1": - space.setattr(w_pycppclass, space.wrap("__init__"), _pythonizations["tf1_tf1"]) - - elif name == "TFile": - _method_alias(space, w_pycppclass, "__getattr__", "Get") - - elif name == "TObjString": - _method_alias(space, w_pycppclass, "__str__", "GetName") - _method_alias(space, w_pycppclass, "_cppyy_as_builtin", "GetString") - - elif name == "TString": - _method_alias(space, w_pycppclass, "__str__", "Data") - _method_alias(space, w_pycppclass, "__len__", "Length") - _method_alias(space, w_pycppclass, "__cmp__", "CompareTo") - _method_alias(space, w_pycppclass, "_cppyy_as_builtin", "Data") - - elif name == "TTree": - _method_alias(space, w_pycppclass, "_unpythonized_Branch", "Branch") - - space.setattr(w_pycppclass, space.wrap("Branch"), _pythonizations["ttree_Branch"]) - space.setattr(w_pycppclass, space.wrap("__iter__"), _pythonizations["ttree_iter"]) - space.setattr(w_pycppclass, space.wrap("__getattr__"), _pythonizations["ttree_getattr"]) - - elif name[0:8] == "TVectorT": # TVectorT<> template - _method_alias(space, w_pycppclass, "__len__", "GetNoElements") - -# destruction callback (needs better solution, but this is for CINT -# only and should not appear outside of ROOT-specific uses) -from pypy.module.cpyext.api import cpython_api, CANNOT_FAIL - -@cpython_api([rffi.VOIDP], lltype.Void, error=CANNOT_FAIL) -def _Py_cppyy_recursive_remove(space, cppobject): - from pypy.module.cppyy.interp_cppyy import memory_regulator - from pypy.module.cppyy.capi import C_OBJECT, C_NULL_OBJECT - - obj = memory_regulator.retrieve(rffi.cast(C_OBJECT, cppobject)) - if obj is not None: - memory_regulator.unregister(obj) - obj._rawobject = C_NULL_OBJECT diff --git a/pypy/module/cppyy/capi/cling_capi.py b/pypy/module/cppyy/capi/cling_capi.py --- a/pypy/module/cppyy/capi/cling_capi.py +++ b/pypy/module/cppyy/capi/cling_capi.py @@ -1,8 +1,17 @@ import py, os +from pypy.objspace.std.iterobject import W_AbstractSeqIterObject + +from pypy.interpreter.error import OperationError +from pypy.interpreter.gateway import interp2app + from rpython.translator.tool.cbuild import ExternalCompilationInfo -from rpython.rtyper.lltypesystem import rffi -from rpython.rlib import libffi, rdynload +from rpython.rtyper.lltypesystem import rffi, lltype +from rpython.rlib.rarithmetic import intmask +from rpython.rlib import jit, libffi, rdynload + +from pypy.module._rawffi.array import W_ArrayInstance +from pypy.module.cppyy.capi.capi_types import C_OBJECT __all__ = ['identify', 'std_string_name', 'eci', 'c_load_dictionary'] @@ -16,7 +25,8 @@ if os.environ.get("ROOTSYS"): if config_stat != 0: # presumably Reflex-only rootincpath = [os.path.join(os.environ["ROOTSYS"], "interpreter/cling/include"), - os.path.join(os.environ["ROOTSYS"], "interpreter/llvm/inst/include")] + os.path.join(os.environ["ROOTSYS"], "interpreter/llvm/inst/include"), + os.path.join(os.environ["ROOTSYS"], "include"),] rootlibpath = [os.path.join(os.environ["ROOTSYS"], "lib64"), os.path.join(os.environ["ROOTSYS"], "lib")] else: rootincpath = [incdir] @@ -39,13 +49,21 @@ std_string_name = 'std::basic_string<char>' +# force loading (and exposure) of libCore symbols +with rffi.scoped_str2charp('libCore.so') as ll_libname: + _coredll = rdynload.dlopen(ll_libname, rdynload.RTLD_GLOBAL | rdynload.RTLD_NOW) + +# require local translator path to pickup common defs +from rpython.translator import cdir +translator_c_dir = py.path.local(cdir) + eci = ExternalCompilationInfo( separate_module_files=[srcpath.join("clingcwrapper.cxx")], - include_dirs=[incpath] + rootincpath, + include_dirs=[incpath, translator_c_dir] + rootincpath, includes=["clingcwrapper.h"], library_dirs=rootlibpath, libraries=["Cling"], - compile_extra=["-fno-strict-aliasing"], + compile_extra=["-fno-strict-aliasing", "-std=c++11"], use_cpp_linker=True, ) @@ -59,11 +77,120 @@ pch = _c_load_dictionary(name) return pch +_c_stdstring2charp = rffi.llexternal( + "cppyy_stdstring2charp", + [C_OBJECT, rffi.SIZE_TP], rffi.CCHARP, + releasegil=ts_helper, + compilation_info=eci) +def c_stdstring2charp(space, cppstr): + sz = lltype.malloc(rffi.SIZE_TP.TO, 1, flavor='raw') + try: + cstr = _c_stdstring2charp(cppstr, sz) + cstr_len = intmask(sz[0]) + finally: + lltype.free(sz, flavor='raw') + return rffi.charpsize2str(cstr, cstr_len) -# Cling-specific pythonizations +# TODO: factor these out ... +# pythonizations + +# +# std::string behavior +def stdstring_c_str(space, w_self): + """Return a python string taking into account \0""" + + from pypy.module.cppyy import interp_cppyy + cppstr = space.interp_w(interp_cppyy.W_CPPInstance, w_self, can_be_None=False) + return space.wrap(c_stdstring2charp(space, cppstr._rawobject)) + +# +# std::vector behavior +class W_STLVectorIter(W_AbstractSeqIterObject): + _immutable_fields_ = ['overload', 'len']#'data', 'converter', 'len', 'stride', 'vector'] + + def __init__(self, space, w_vector): + W_AbstractSeqIterObject.__init__(self, w_vector) + # TODO: this should live in rpythonize.py or something so that the + # imports can move to the top w/o getting circles + from pypy.module.cppyy import interp_cppyy + assert isinstance(w_vector, interp_cppyy.W_CPPInstance) + vector = space.interp_w(interp_cppyy.W_CPPInstance, w_vector) + self.overload = vector.cppclass.get_overload("__getitem__") + + from pypy.module.cppyy import capi + v_type = capi.c_stdvector_valuetype(space, vector.cppclass.name) + v_size = capi.c_stdvector_valuesize(space, vector.cppclass.name) + + if not v_type or not v_size: + raise NotImplementedError # fallback on getitem + + w_arr = vector.cppclass.get_overload("data").call(w_vector, []) + arr = space.interp_w(W_ArrayInstance, w_arr, can_be_None=True) + if not arr: + raise OperationError(space.w_StopIteration, space.w_None) + + self.data = rffi.cast(rffi.VOIDP, space.uint_w(arr.getbuffer(space))) + + from pypy.module.cppyy import converter + self.converter = converter.get_converter(space, v_type, '') + self.len = space.uint_w(vector.cppclass.get_overload("size").call(w_vector, [])) + self.stride = v_size + + def descr_next(self, space): + if self.w_seq is None: + raise OperationError(space.w_StopIteration, space.w_None) + if self.len <= self.index: + self.w_seq = None + raise OperationError(space.w_StopIteration, space.w_None) + try: + from pypy.module.cppyy import capi # TODO: refector + offset = capi.direct_ptradd(rffi.cast(C_OBJECT, self.data), self.index*self.stride) + w_item = self.converter.from_memory(space, space.w_None, space.w_None, offset) + except OperationError as e: + self.w_seq = None + if not e.match(space, space.w_IndexError): + raise + raise OperationError(space.w_StopIteration, space.w_None) + self.index += 1 + return w_item + +def stdvector_iter(space, w_self): + return W_STLVectorIter(space, w_self) + +# setup pythonizations for later use at run-time +_pythonizations = {} def register_pythonizations(space): "NOT_RPYTHON" - pass + + allfuncs = [ + + ### std::string + stdstring_c_str, + + ### std::vector + stdvector_iter, + + ] + + for f in allfuncs: + _pythonizations[f.__name__] = space.wrap(interp2app(f)) + +def _method_alias(space, w_pycppclass, m1, m2): + space.setattr(w_pycppclass, space.wrap(m1), + space.getattr(w_pycppclass, space.wrap(m2))) def pythonize(space, name, w_pycppclass): - pass + if name == "string": + space.setattr(w_pycppclass, space.wrap("c_str"), _pythonizations["stdstring_c_str"]) + _method_alias(space, w_pycppclass, "_cppyy_as_builtin", "c_str") + _method_alias(space, w_pycppclass, "__str__", "c_str") + + if "vector" in name[:11]: # len('std::vector') == 11 + from pypy.module.cppyy import capi + v_type = capi.c_stdvector_valuetype(space, name) + if v_type: + space.setattr(w_pycppclass, space.wrap("value_type"), space.wrap(v_type)) + v_size = capi.c_stdvector_valuesize(space, name) + if v_size: + space.setattr(w_pycppclass, space.wrap("value_size"), space.wrap(v_size)) + space.setattr(w_pycppclass, space.wrap("__iter__"), _pythonizations["stdvector_iter"]) diff --git a/pypy/module/cppyy/capi/loadable_capi.py b/pypy/module/cppyy/capi/loadable_capi.py --- a/pypy/module/cppyy/capi/loadable_capi.py +++ b/pypy/module/cppyy/capi/loadable_capi.py @@ -1,14 +1,18 @@ from rpython.rtyper.lltypesystem import rffi, lltype +from rpython.rlib.rarithmetic import intmask from rpython.rlib import jit, jit_libffi, libffi, rdynload, objectmodel from rpython.rlib.rarithmetic import r_singlefloat from rpython.tool import leakfinder +from pypy.interpreter.gateway import interp2app from pypy.interpreter.error import oefmt from pypy.module._cffi_backend import ctypefunc, ctypeprim, cdataobj, misc +from pypy.module._cffi_backend import newtype +from pypy.module.cppyy import ffitypes from pypy.module.cppyy.capi.capi_types import C_SCOPE, C_TYPE, C_OBJECT,\ - C_METHOD, C_INDEX, C_INDEX_ARRAY, WLAVC_INDEX, C_METHPTRGETTER_PTR + C_METHOD, C_INDEX, C_INDEX_ARRAY, WLAVC_INDEX, C_FUNC_PTR reflection_library = 'libcppyy_backend.so' @@ -21,11 +25,32 @@ class _Arg: # poor man's union _immutable_ = True - def __init__(self, h = 0, l = -1, s = '', vp = rffi.cast(rffi.VOIDP, 0)): + def __init__(self, tc, h = 0, l = -1, s = '', p = rffi.cast(rffi.VOIDP, 0)): + self.tc = tc self._handle = h self._long = l self._string = s - self._voidp = vp + self._voidp = p + +class _ArgH(_Arg): + _immutable_ = True + def __init__(self, val): + _Arg.__init__(self, 'h', h = val) + +class _ArgL(_Arg): + _immutable_ = True + def __init__(self, val): + _Arg.__init__(self, 'l', l = val) + +class _ArgS(_Arg): + _immutable_ = True + def __init__(self, val): + _Arg.__init__(self, 's', s = val) + +class _ArgP(_Arg): + _immutable_ = True + def __init__(self, val): + _Arg.__init__(self, 'p', p = val) # For the loadable CAPI, the calls start and end in RPython. Therefore, the standard # _call of W_CTypeFunc, which expects wrapped objects, does not quite work: some @@ -55,14 +80,18 @@ argtype = self.fargs[i] # the following is clumsy, but the data types used as arguments are # very limited, so it'll do for now - if isinstance(argtype, ctypeprim.W_CTypePrimitiveSigned): + if obj.tc == 'l': + assert isinstance(argtype, ctypeprim.W_CTypePrimitiveSigned) misc.write_raw_signed_data(data, rffi.cast(rffi.LONG, obj._long), argtype.size) - elif isinstance(argtype, ctypeprim.W_CTypePrimitiveUnsigned): + elif obj.tc == 'h': + assert isinstance(argtype, ctypeprim.W_CTypePrimitiveUnsigned) misc.write_raw_unsigned_data(data, rffi.cast(rffi.ULONG, obj._handle), argtype.size) - elif obj._voidp != rffi.cast(rffi.VOIDP, 0): + elif obj.tc == 'p': + assert obj._voidp != rffi.cast(rffi.VOIDP, 0) data = rffi.cast(rffi.VOIDPP, data) data[0] = obj._voidp else: # only other use is sring + assert obj.tc == 's' n = len(obj._string) assert raw_string == rffi.cast(rffi.CCHARP, 0) # XXX could use rffi.get_nonmovingbuffer_final_null() @@ -89,35 +118,36 @@ self.library = None self.capi_calls = {} - import pypy.module._cffi_backend.newtype as nt + nt = newtype # module from _cffi_backend + state = space.fromcache(ffitypes.State) # factored out common types # TODO: the following need to match up with the globally defined C_XYZ low-level # types (see capi/__init__.py), but by using strings here, that isn't guaranteed - c_opaque_ptr = nt.new_primitive_type(space, 'unsigned long') + c_opaque_ptr = state.c_ulong - c_scope = c_opaque_ptr - c_type = c_scope - c_object = c_opaque_ptr - c_method = c_opaque_ptr - c_index = nt.new_primitive_type(space, 'long') + c_scope = c_opaque_ptr + c_type = c_scope + c_object = c_opaque_ptr + c_method = c_opaque_ptr + c_index = state.c_long + c_index_array = state.c_voidp - c_void = nt.new_void_type(space) - c_char = nt.new_primitive_type(space, 'char') - c_uchar = nt.new_primitive_type(space, 'unsigned char') - c_short = nt.new_primitive_type(space, 'short') - c_int = nt.new_primitive_type(space, 'int') - c_long = nt.new_primitive_type(space, 'long') - c_llong = nt.new_primitive_type(space, 'long long') - c_ullong = nt.new_primitive_type(space, 'unsigned long long') - c_float = nt.new_primitive_type(space, 'float') - c_double = nt.new_primitive_type(space, 'double') + c_void = state.c_void + c_char = state.c_char + c_uchar = state.c_uchar + c_short = state.c_short + c_int = state.c_int + c_long = state.c_long + c_llong = state.c_llong + c_ullong = state.c_ullong + c_float = state.c_float + c_double = state.c_double + c_ldouble = state.c_ldouble - c_ccharp = nt.new_pointer_type(space, c_char) - c_index_array = nt.new_pointer_type(space, c_void) + c_ccharp = state.c_ccharp + c_voidp = state.c_voidp - c_voidp = nt.new_pointer_type(space, c_void) c_size_t = nt.new_primitive_type(space, 'size_t') - c_ptrdiff_t = nt.new_primitive_type(space, 'ptrdiff_t') self.capi_call_ifaces = { @@ -127,7 +157,6 @@ 'resolve_name' : ([c_ccharp], c_ccharp), 'get_scope' : ([c_ccharp], c_scope), - 'get_template' : ([c_ccharp], c_type), 'actual_class' : ([c_type, c_object], c_type), # memory management @@ -146,14 +175,16 @@ 'call_ll' : ([c_method, c_object, c_int, c_voidp], c_llong), 'call_f' : ([c_method, c_object, c_int, c_voidp], c_float), 'call_d' : ([c_method, c_object, c_int, c_voidp], c_double), + 'call_ld' : ([c_method, c_object, c_int, c_voidp], c_ldouble), 'call_r' : ([c_method, c_object, c_int, c_voidp], c_voidp), - 'call_s' : ([c_method, c_object, c_int, c_voidp], c_ccharp), + # call_s actually takes an size_t* as last parameter, but this will do + 'call_s' : ([c_method, c_object, c_int, c_voidp, c_voidp], c_ccharp), 'constructor' : ([c_method, c_object, c_int, c_voidp], c_object), 'call_o' : ([c_method, c_object, c_int, c_voidp, c_type], c_object), - 'get_methptr_getter' : ([c_scope, c_index], c_voidp), # TODO: verify + 'get_function_address' : ([c_scope, c_index], c_voidp), # TODO: verify # handling of function argument buffer 'allocate_function_args' : ([c_int], c_voidp), @@ -163,6 +194,8 @@ # scope reflection information _______________________________________________ pypy-commit mailing list pypy-commit@python.org https://mail.python.org/mailman/listinfo/pypy-commit
