Author: Carl Friedrich Bolz <[email protected]>
Branch: extradoc
Changeset: r3999:a88377852aa3
Date: 2012-01-03 14:30 +0100
http://bitbucket.org/pypy/extradoc/changeset/a88377852aa3/
Log: merge
diff --git a/blog/draft/matplotlib.rst b/blog/draft/matplotlib.rst
new file mode 100644
--- /dev/null
+++ b/blog/draft/matplotlib.rst
@@ -0,0 +1,86 @@
+===================================
+Plotting using matplotlib from PyPy
+===================================
+
+**Big fat warning** This is just a proof of concept. It barely works. There are
+missing pieces left and right, which were replaced with hacks so I can get this
+to run and prove it's possible. Don't try this at home, especially your home.
+You have been warned.
+
+There has been a lot of talking about PyPy not integrating well with the
+current scientific Python ecosystem, and ``numpypy`` (a NumPy reimplementation
+on top of pypy) was dubbed "a fancy array library". I'm going to show that
+integration with this ecosystem is possible with our design.
+
+First, `the demo`_::
+
+ #!/usr/bin/env pypy
+
+ # numpy, pypy version
+ import numpypy as numpy
+ # DRAGONS LIVE THERE (fortunately hidden)
+ from embed.emb import import_mod
+
+ pylab = import_mod('matplotlib.pylab')
+
+ if __name__ == '__main__':
+ a = numpy.arange(100, dtype=int)
+ b = numpy.sin(a)
+ pylab.plot(a, b)
+ pylab.show()
+
+And you get:
+
+ XXX pic
+
+Now, how to reproduce it:
+
+* You need a PyPy without cpyext, I did not find a linker that would support
+ overriding symbols. Right now there are no nightlies like this, so you have
+ to compile it yourself, like::
+
+ ./translate.py -Ojit targetpypystandalone.py --withoutmod-cpyext
+
+ That would give you a PyPy that's unable to load some libraries like PIL, but
+ perfectly working otherwise.
+
+* Speaking of which, you need a reasonably recent PyPy.
+
+* The approach is generally portable, however the implementation has been
+ tested only on 64bit linux. Few tweaks might be required.
+
+* You need to install python2.6, the python2.6 development headers, and have
+ numpy and matplotlib installed on that python.
+
+* You need a checkout of my `hacks directory`_ and put embedded on your
+ ``PYTHONPATH``, your pypy checkout also has to be on the ``PYTHONPATH``.
+
+Er wait, what happened?
+-----------------------
+
+What didn't happen is we did not reimplement matplotlib on top of PyPy. What
+did happen is we embed CPython inside of PyPy using ctypes. We instantiate it.
+and follow the `embedding`_ tutorial for CPython. Since numpy arrays are not
+movable, we're able to pass around an integer that's represents the memory
+address of the array data and reconstruct it in the embedded interpreter. Hence
+with a relatively little effort we managed to reuse the same array data on both
+sides to plot at array. Easy, no?
+
+This approach can be extended to support anything that's not too tied with
+python objects. SciPy and matplotlib both fall into the same category
+but probably the same strategy can be applied to anything, like GTK or QT.
+It's just a matter of extending a hack into a working library.
+
+To summarize, while we're busy making numpypy better and faster, it seems
+that all external libraries on the C side can be done using an embedded Python
+interpreter with relatively little effort. To get to that point, I spent
+a day and a half to learn how to embed CPython, with very little prior
+experience in the CPython APIs. Of course you should still keep as much as
+possible in PyPy to make it nice and fast :)
+
+Cheers,
+fijal
+
+.. _`hacks directory`: https://bitbucket.org/fijal/hack2
+.. _`the demo`:
https://bitbucket.org/fijal/hack2/src/default/embed/embed/matplotwrapper.py
+.. _`embedding`: http://docs.python.org/extending/embedding.html
diff --git a/blog/draft/pycon-2012-teaser.rst b/blog/draft/pycon-2012-teaser.rst
new file mode 100644
--- /dev/null
+++ b/blog/draft/pycon-2012-teaser.rst
@@ -0,0 +1,38 @@
+Come see us at PyCon 2012
+=========================
+
+`PyCon 2012`_ is coming up in just a few short months, and PyPy will be well
+represented there. We'll be delivering a tutorial, two talks, plus we'll be
+around for the sprints.
+
+Here are the abstracts for the tutorials and talks:
+
+* **How to get the most out of your PyPy**, by Maciej Fijalkowski, Alex Gaynor
+ and Armin Rigo: For many applications PyPy can provide performance benefits
+ right out of the box. However, little details can push your application to
+ perform much better. In this tutorial we'll give you insights on how to push
+ PyPy to it's limits. We'll focus on understanding the performance
+ characteristics of PyPy, and learning the analysis tools in order to maximize
+ your applications performance. *This is the tutorial.*
+
+* **Why PyPy by example**, by Maciej Fijalkowski, Alex Gaynor and Armin Rigo:
+ One of the goals of PyPy is to make existing Python code faster, however an
+ even broader goal was to make it possible to write things in Python that
+ previous would needed to be written in C or other low-level language. This
+ talk will show examples of this, and describe how they represent the
+ tremendous progress PyPy has made, and what it means for people looking to
+ use PyPy.
+
+* **How the PyPy JIT works**, by Benjamin Peterson: The Python community is
+ abuzz about the major speed gains PyPy can offer pure Python code. But how
+ does PyPy JIT actually work? This talk will discuss how the PyPy JIT is
+ implemented. It will include descriptions of the tracing, optimization, and
+ assembly generation phases. I will demonstrate each step with a example loop.
+
+If you have any questions let us know! We look forward to seeing people at
+PyCon and chatting about PyPy and the entire Python ecosystem.
+
+See you there,
+Maciej Fijalkowski, Alex Gaynor, Benjamin Peterson, Armin Rigo, and the entire
PyPy team
+
+.. _`PyCon 2012`: https://us.pycon.org/2012/
diff --git a/blog/draft/screen0.png b/blog/draft/screen0.png
new file mode 100644
index
0000000000000000000000000000000000000000..0f6d5f2e8bdb961c6fc893e6d745910f98792684
GIT binary patch
[cut]
diff --git a/planning/micronumpy.txt b/planning/micronumpy.txt
--- a/planning/micronumpy.txt
+++ b/planning/micronumpy.txt
@@ -1,10 +1,6 @@
NEW TASKS
---------
-- add in numpy.generic and the various subclasses, use them in returning
- instances from subscripting (and possibly internally), also make them valid
- for the dtype arguments (numpy-dtype-refactor branch)
-
- astype
- a good sort function
@@ -13,16 +9,20 @@
- endianness
-- scalar types like numpy.int8 (numpy-dtype-refacotr branch)
-
-- add multi-dim arrays (numpy-multidim-shards branch)
-
- - will need to refactor some functions
-
- frompyfunc to create ufuncs from python functions
- more ufuncs
-- arange/linspace/other ranges
+- linspace/other ranges
-- numpy.flatiter array.flat and friends
+- more attributes/methods on numpy.flatiter
+
+- axis= parameter to various methods
+
+- expose ndarray.ctypes
+
+- subclassing ndarray (instantiating subcalsses curently returns the wrong
type)
+
+ * keep subclass type when slicing, __array_finalize__
+
+ * ndarray.view
diff --git a/sprintinfo/leysin-winter-2011/announcement.txt
b/sprintinfo/leysin-winter-2012/announcement.txt
copy from sprintinfo/leysin-winter-2011/announcement.txt
copy to sprintinfo/leysin-winter-2012/announcement.txt
--- a/sprintinfo/leysin-winter-2011/announcement.txt
+++ b/sprintinfo/leysin-winter-2012/announcement.txt
@@ -1,30 +1,23 @@
=====================================================================
- PyPy Leysin Winter Sprint (16-22nd January 2011)
+ PyPy Leysin Winter Sprint (15-22nd January 2012)
=====================================================================
The next PyPy sprint will be in Leysin, Switzerland, for the
-seventh time. This is a fully public sprint: newcomers and topics
+eighth time. This is a fully public sprint: newcomers and topics
other than those proposed below are welcome.
------------------------------
Goals and topics of the sprint
------------------------------
-* Now that we have released 1.4, and plan to release 1.4.1 soon
- (possibly before the sprint), the sprint itself is going to be
- mainly working on fixing issues reported by various users. Of
- course this does not prevent people from showing up with a more
- precise interest in mind. If there are newcomers, we will gladly
- give introduction talks.
+* Py3k: work towards supporting Python 3 in PyPy
-* We will also work on polishing and merging the long-standing
- branches that are around, which could eventually lead to the
- next PyPy release. These branches are notably:
+* NumPyPy: work towards supporting the numpy module in PyPy
- - fast-forward (Python 2.7 support, by Benjamin, Amaury, and others)
- - jit-unroll-loops (improve JITting of smaller loops, by Hakan)
- - arm-backend (a JIT backend for ARM, by David)
- - jitypes2 (fast ctypes calls with the JIT, by Antonio).
+* JIT backends: integrate tests for ARM; look at the PowerPC 64;
+ maybe try again to write an LLVM- or GCC-based one
+
+* STM and STM-related topics; or the Concurrent Mark-n-Sweep GC
* And as usual, the main side goal is to have fun in winter sports :-)
We can take a day off for ski.
@@ -33,8 +26,9 @@
Exact times
-----------
-The work days should be 16-22 January 2011. People may arrive on
-the 15th already and/or leave on the 23rd.
+The work days should be 15-21 January 2011 (Sunday-Saturday). The
+official plans are for people to arrive on the 14th or the 15th, and to
+leave on the 22nd.
-----------------------
Location & Accomodation
@@ -56,13 +50,14 @@
expensive) and maybe the possibility to get a single room if you really want
to.
-Please register by svn:
+Please register by Mercurial::
-
http://codespeak.net/svn/pypy/extradoc/sprintinfo/leysin-winter-2011/people.txt
+ https://bitbucket.org/pypy/extradoc/
+
https://bitbucket.org/pypy/extradoc/raw/extradoc/sprintinfo/leysin-winter-2012
-or on the pypy-sprint mailing list if you do not yet have check-in rights:
+or on the pypy-dev mailing list if you do not yet have check-in rights:
- http://codespeak.net/mailman/listinfo/pypy-sprint
+ http://mail.python.org/mailman/listinfo/pypy-dev
You need a Swiss-to-(insert country here) power adapter. There will be
some Swiss-to-EU adapters around -- bring a EU-format power strip if you
diff --git a/sprintinfo/leysin-winter-2012/people.txt
b/sprintinfo/leysin-winter-2012/people.txt
new file mode 100644
--- /dev/null
+++ b/sprintinfo/leysin-winter-2012/people.txt
@@ -0,0 +1,58 @@
+
+People coming to the Leysin sprint Winter 2011
+==================================================
+
+People who have a ``?`` in their arrive/depart or accomodation
+column are known to be coming but there are no details
+available yet from them.
+
+
+==================== ============== =======================
+ Name Arrive/Depart Accomodation
+==================== ============== =======================
+Armin Rigo private
+David Schneider 17/22 ermina
+==================== ============== =======================
+
+
+People on the following list were present at previous sprints:
+
+==================== ============== =====================
+ Name Arrive/Depart Accomodation
+==================== ============== =====================
+Antonio Cuni ? ?
+Michael Foord ? ?
+Maciej Fijalkowski ? ?
+David Schneider ? ?
+Jacob Hallen ? ?
+Laura Creighton ? ?
+Hakan Ardo ? ?
+Carl Friedrich Bolz ? ?
+Samuele Pedroni ? ?
+Anders Hammarquist ? ?
+Christian Tismer ? ?
+Niko Matsakis ? ?
+Toby Watson ? ?
+Paul deGrandis ? ?
+Michael Hudson ? ?
+Anders Lehmann ? ?
+Niklaus Haldimann ? ?
+Lene Wagner ? ?
+Amaury Forgeot d'Arc ? ?
+Valentino Volonghi ? ?
+Boris Feigin ? ?
+Andrew Thompson ? ?
+Bert Freudenberg ? ?
+Beatrice Duering ? ?
+Richard Emslie ? ?
+Johan Hahn ? ?
+Stephan Diehl ? ?
+Alexander Schremmer ? ?
+Anders Chrigstroem ? ?
+Eric van Riet Paap ? ?
+Holger Krekel ? ?
+Guido Wesdorp ? ?
+Leonardo Santagada ? ?
+Alexandre Fayolle ? ?
+Sylvain Th�nault ? ?
+==================== ============== =====================
diff --git a/talk/iwtc11/benchmarks/convolution/convolution.py
b/talk/iwtc11/benchmarks/convolution/convolution.py
--- a/talk/iwtc11/benchmarks/convolution/convolution.py
+++ b/talk/iwtc11/benchmarks/convolution/convolution.py
@@ -57,6 +57,19 @@
self[x, y] = data[y][x]
return self
+class NumpyArray(Array2D):
+ def __init__(self, w, h):
+ self.width = w
+ self.height = h
+ import numpypy
+ self.data = numpypy.zeros([h, w], 'd')
+
+ def __getitem__(self, (x, y)):
+ return self.data[y, x]
+
+ def __setitem__(self, (x, y), val):
+ self.data[y, x] = val
+
def _conv3x3(a, b, k):
assert k.width == k.height == 3
for y in xrange(1, a.height-1):
@@ -88,6 +101,13 @@
_conv3x3(a, b, Array2D(3,3))
return 'conv3x3(Array2D(%sx%s))' % tuple(args)
+def conv3x3_numpy(args):
+ a = NumpyArray(int(args[0]), int(args[1]))
+ b = NumpyArray(a.width, a.height)
+ for i in range(10):
+ _conv3x3(a, b, NumpyArray(3,3))
+ return 'conv3x3(NumpyArray(%sx%s))' % tuple(args)
+
def dilate3x3(args):
a = Array2D(int(args[0]), int(args[1]))
b = Array2D(a.width, a.height)
diff --git a/talk/iwtc11/benchmarks/image/io.py
b/talk/iwtc11/benchmarks/image/io.py
--- a/talk/iwtc11/benchmarks/image/io.py
+++ b/talk/iwtc11/benchmarks/image/io.py
@@ -1,4 +1,6 @@
import os, re, array
+from subprocess import Popen, PIPE, STDOUT
+
def mplayer(Image, fn='tv://', options=''):
f = os.popen('mplayer -really-quiet -noframedrop ' + options + ' '
@@ -19,18 +21,18 @@
def view(self, img):
assert img.typecode == 'B'
if not self.width:
- self.mplayer = os.popen('mplayer -really-quiet -noframedrop - ' +
- '2> /dev/null ', 'w')
- self.mplayer.write('YUV4MPEG2 W%d H%d F100:1 Ip A1:1\n' %
- (img.width, img.height))
+ w, h = img.width, img.height
+ self.mplayer = Popen(['mplayer', '-', '-benchmark',
+ '-demuxer', 'rawvideo',
+ '-rawvideo', 'w=%d:h=%d:format=y8' % (w, h),
+ '-really-quiet'],
+ stdin=PIPE, stdout=PIPE, stderr=PIPE)
+
self.width = img.width
self.height = img.height
- self.color_data = array.array('B', [127]) * (img.width *
img.height / 2)
assert self.width == img.width
assert self.height == img.height
- self.mplayer.write('FRAME\n')
- img.tofile(self.mplayer)
- self.color_data.tofile(self.mplayer)
+ img.tofile(self.mplayer.stdin)
default_viewer = MplayerViewer()
diff --git a/talk/iwtc11/licm.pdf b/talk/iwtc11/licm.pdf
new file mode 100644
index
0000000000000000000000000000000000000000..ff2a7bf547f542771702ac86ea8531f8ba16cc28
GIT binary patch
[cut]
diff --git a/talk/sea2012/abstract.rst b/talk/sea2012/abstract.rst
new file mode 100644
--- /dev/null
+++ b/talk/sea2012/abstract.rst
@@ -0,0 +1,18 @@
+Fast numeric in Python - NumPy and PyPy
+=======================================
+
+Python increasingly is being utilized as a powerful scientific
+processing language. It successfully has been used as a glue language
+to drive simulations written in C, Fortran or the array
+manipulation language provided by the NumPy package. Originally
+Python only was used as a glue language because the original Python
+implementation was relatively slow. With the recent progress in the
+PyPy project that is showing significant performance
+improvements in each release, Python is nearing performance comparable
+to native C language implementations. In this talk I will
+describe three stages: how to use it right now, in the near future and
+(xxx give before this line a hint that "we" or "our" means "pypy developers")
+our plans to provide a very robust infrastructure for implementing
+numerical computations. I also will spend some time exploring ideas
+how dynamic compilation eventually can outperform static compilation
+and how a high-level language helps accomplish this.
_______________________________________________
pypy-commit mailing list
[email protected]
http://mail.python.org/mailman/listinfo/pypy-commit
