[pypy-commit] extradoc extradoc: merge
Author: Maciej Fijalkowski
Branch: extradoc
Changeset: r5540:ea1511561764
Date: 2015-07-15 12:51 +0200
http://bitbucket.org/pypy/extradoc/changeset/ea1511561764/
Log:merge
diff --git a/talk/ep2015/pypy-abstract.rst b/talk/ep2015/pypy-abstract.rst
new file mode 100644
--- /dev/null
+++ b/talk/ep2015/pypy-abstract.rst
@@ -0,0 +1,22 @@
+=
+PyPy for mediocre programmers
+=
+
+This is a talk for mediocre Python programmers by a mediocre programmer. PyPy
+is an alternative implementation of Python. It is notorious for being fast, but
+also for using clever algorithms pertaining to advanced concepts such as type
+inference, garbage collection, just-in-time compilation, etc. So, can we,
+mediocre programmers, realistically use PyPy?
+
+Yes, absolutely. In fact, PyPy developers did all that hard work so that we
+wouldn't have to. As we'll see, it runs most Python code exactly like CPython
+does, save that it magically makes it faster.
+
+Porting existing applications is always more involved than running a simple
+script, so we'll also examine likely difficulties such as code relying on
+CPython implementation details, and dependencies on C extensions, and explore
+simple principles to let PyPy run your code even faster.
+
+Finally, we'll have a glimpse of the future by looking at what's brewing in
+the PyPy lair, such as software transactional memory, new speed optimisations,
+better support for Python 3 and NumPy, ...
diff --git a/talk/pycon-italy-2015/Makefile b/talk/pycon-italy-2015/Makefile
new file mode 100644
--- /dev/null
+++ b/talk/pycon-italy-2015/Makefile
@@ -0,0 +1,18 @@
+# you can find rst2beamer.py here:
+# http://codespeak.net/svn/user/antocuni/bin/rst2beamer.py
+
+# WARNING: to work, it needs this patch for docutils
+#
https://sourceforge.net/tracker/?func=detail&atid=422032&aid=1459707&group_id=38414
+
+talk.pdf: talk.rst author.latex title.latex stylesheet.latex
+ python `which rst2beamer.py` --stylesheet=stylesheet.latex
--documentoptions=14pt talk.rst talk.latex || exit
+ #/home/antocuni/.virtualenvs/rst2beamer/bin/python `which
rst2beamer.py` --stylesheet=stylesheet.latex --documentoptions=14pt talk.rst
talk.latex || exit
+ sed 's/\\date{}/\\input{author.latex}/' -i talk.latex || exit
+ #sed 's/\\maketitle/\\input{title.latex}/' -i talk.latex || exit
+ pdflatex talk.latex || exit
+
+view: talk.pdf
+ evince talk.pdf &
+
+xpdf: talk.pdf
+ xpdf talk.pdf &
diff --git a/talk/pycon-italy-2015/author.latex
b/talk/pycon-italy-2015/author.latex
new file mode 100644
--- /dev/null
+++ b/talk/pycon-italy-2015/author.latex
@@ -0,0 +1,8 @@
+\definecolor{rrblitbackground}{rgb}{0.0, 0.0, 0.0}
+
+\title[PyPy JIT]{PyPy JIT (not) for dummies}
+\author[antocuni]
+{Antonio Cuni}
+
+\institute{PyCon Sei}
+\date{April 17, 2015}
diff --git a/talk/pycon-italy-2015/beamerdefs.txt
b/talk/pycon-italy-2015/beamerdefs.txt
new file mode 100644
--- /dev/null
+++ b/talk/pycon-italy-2015/beamerdefs.txt
@@ -0,0 +1,108 @@
+.. colors
+.. ===
+
+.. role:: green
+.. role:: red
+
+
+.. general useful commands
+.. ===
+
+.. |pause| raw:: latex
+
+ \pause
+
+.. |small| raw:: latex
+
+ {\small
+
+.. |end_small| raw:: latex
+
+ }
+
+.. |scriptsize| raw:: latex
+
+ {\scriptsize
+
+.. |end_scriptsize| raw:: latex
+
+ }
+
+.. |strike<| raw:: latex
+
+ \sout{
+
+.. closed bracket
+.. ===
+
+.. |>| raw:: latex
+
+ }
+
+
+.. example block
+.. ===
+
+.. |example<| raw:: latex
+
+ \begin{exampleblock}{
+
+
+.. |end_example| raw:: latex
+
+ \end{exampleblock}
+
+
+
+.. alert block
+.. ===
+
+.. |alert<| raw:: latex
+
+ \begin{alertblock}{
+
+
+.. |end_alert| raw:: latex
+
+ \end{alertblock}
+
+
+
+.. columns
+.. ===
+
+.. |column1| raw:: latex
+
+ \begin{columns}
+ \begin{column}{0.45\textwidth}
+
+.. |column2| raw:: latex
+
+ \end{column}
+ \begin{column}{0.45\textwidth}
+
+
+.. |end_columns| raw:: latex
+
+ \end{column}
+ \end{columns}
+
+
+
+.. |snake| image:: ../../img/py-web-new.png
+ :scale: 15%
+
+
+
+.. nested blocks
+.. ===
+
+.. |nested| raw:: latex
+
+ \begin{columns}
+ \begin{column}{0.85\textwidth}
+
+.. |end_nested| raw:: latex
+
+ \end{column}
+ \end{columns}
diff --git a/talk/pycon-italy-2015/jit-overview.odg
b/talk/pycon-italy-2015/jit-overview.odg
new file mode 100644
index
..43aec7c16da3c72bec87a6be52ce41969f4db8c7
GIT binary patch
[cut]
diff --git a/talk/pycon-italy-2015/jit-overview1.pdf
b/talk/pycon-italy-2015/jit-overview1.pdf
new file mode 100644
index
..6b511590edd06e9deb157eb02cecb7cf64c61926
GIT binary patch
[cut]
diff --git a/talk/pycon-italy-2015/jit-overview2.pdf
b/talk/pyco
[pypy-commit] extradoc extradoc: add a proposal
Author: Maciej Fijalkowski Branch: extradoc Changeset: r5539:5b5d9a9ac7bb Date: 2015-07-15 12:44 +0200 http://bitbucket.org/pypy/extradoc/changeset/5b5d9a9ac7bb/ Log:add a proposal diff --git a/talk/pyconpl-2015/proposal.rst b/talk/pyconpl-2015/proposal.rst new file mode 100644 --- /dev/null +++ b/talk/pyconpl-2015/proposal.rst @@ -0,0 +1,29 @@ +Python & PyPy performance (not) for dummies +--- + +In this talk we would like to talk a bit about the performance +characteristics of python programs, how to (or when not to) write benchmarks +and how to measure performance of the existing python programs both +under CPython (the standard python interpreter) and PyPy. + +One of the key points will be presentation of vmprof, a low-overhead +Python statistical profiler written by us. + +The talk will cover: + +* the basics of statistics and benchmarking + +* tools available for measuring the performance of python programs + +* common strategies to improve the bottlenecks + +About author: +- + +Maciej is a freelancer working mostly on PyPy for the past several +years. He's a core developer since 2006, working on all kinds of parts +in the entire codebase including JIT, GC and assembler +backends. Maciej has been going to many conferences, advertising PyPy +to a broader audience for the past several years, including a keynote +at Pycon 2010. He's also the main maintainer of jitviewer and vmprof, tools +analyzing performance of your python programs under PyPy. ___ pypy-commit mailing list pypy-commit@python.org https://mail.python.org/mailman/listinfo/pypy-commit
[pypy-commit] extradoc extradoc: start working on a talk
Author: Maciej Fijalkowski Branch: extradoc Changeset: r5541:d0835161a1e9 Date: 2015-07-15 16:35 +0200 http://bitbucket.org/pypy/extradoc/changeset/d0835161a1e9/ Log:start working on a talk diff --git a/talk/ep2015/performance/talk.rst b/talk/ep2015/performance/talk.rst new file mode 100644 --- /dev/null +++ b/talk/ep2015/performance/talk.rst @@ -0,0 +1,84 @@ + +Python & PyPy performance +- + +What is performance? + + +* it's a metric + +* usually, time spent doing task X + +* sometimes number of requests, latency, etc. + +* some statistical properties about that metric (average, minimum, maximum) + +Do you have a performance problem? +-- + +* define the metric + +* measure it (production, benchmarks, etc.) + +* see if Python is the cause here (if it's not, we can't help you, + but I'm sure someone help) + +* make sure you can change and test stuff quickly (e.g. benchmarks are better + than changing stuff in production) + +We have a python problem + + +* tools, timers etc. + +* systems are too complicated to **guess** which will be faster + +* find your bottlenecks + +* 20/80 (but 20% of million lines is 200 000 lines, remember that) + +Profilers landscape +--- + +* cProfile, runSnakeRun (high overhead) - exact profiler + +* plop, vmprof - statistical profiler + +* cProfile & vmprof work on pypy + +vmprof +-- + +XXXxxx + +using vmprof + + +yyy + +interpreting the results + + + + +using vmprof in production +-- + +demo + + +let's optimize some code + + +let's optimize some more complex code +- + +Extras: what's cool what's not cool on cpython and pypy + +CPython vs PyPy +--- + +* very different performance characteristics + +* XXX list them + ___ pypy-commit mailing list pypy-commit@python.org https://mail.python.org/mailman/listinfo/pypy-commit
[pypy-commit] pypy vecopt: rewriting parts of the guard strength reduction to eliminate array bound checks
Author: Richard Plangger
Branch: vecopt
Changeset: r78555:961725ecb559
Date: 2015-07-15 17:32 +0200
http://bitbucket.org/pypy/pypy/changeset/961725ecb559/
Log:rewriting parts of the guard strength reduction to eliminate array
bound checks
diff --git a/rpython/jit/metainterp/optimizeopt/guard.py
b/rpython/jit/metainterp/optimizeopt/guard.py
--- a/rpython/jit/metainterp/optimizeopt/guard.py
+++ b/rpython/jit/metainterp/optimizeopt/guard.py
@@ -9,34 +9,34 @@
MemoryRef, Node, IndexVar)
from rpython.jit.metainterp.resoperation import (rop, ResOperation, GuardResOp)
from rpython.jit.metainterp.history import (ConstInt, BoxVector,
-BoxFloat, BoxInt, ConstFloat, Box)
+BoxFloat, BoxInt, ConstFloat, Box, Const)
+from rpython.rlib.objectmodel import we_are_translated
class Guard(object):
""" An object wrapper around a guard. Helps to determine
if one guard implies another
"""
-def __init__(self, index, op, cmp_op, lhs, lhs_arg, rhs, rhs_arg):
+def __init__(self, index, op, cmp_op, lhs_arg, rhs_arg):
self.index = index
self.op = op
self.cmp_op = cmp_op
-self.lhs = lhs
-self.rhs = rhs
self.lhs_arg = lhs_arg
self.rhs_arg = rhs_arg
-self.implied = False
-self.stronger = False
+self.lhs_key = None
+self.rhs_key = None
def implies(self, guard, opt):
if self.op.getopnum() != guard.op.getopnum():
return False
-my_key = opt._get_key(self.cmp_op)
-ot_key = opt._get_key(guard.cmp_op)
-
-if my_key[1] == ot_key[1]:
+if self.lhs_key == guard.lhs_key:
# same operation
-lc = self.compare(self.lhs, guard.lhs)
-rc = self.compare(self.rhs, guard.rhs)
+valid, lc = self.compare(self.lhs, guard.lhs)
+if not valid:
+return False
+valid, rc = self.compare(self.rhs, guard.rhs)
+if not valid:
+return False
opnum = self.get_compare_opnum()
if opnum == -1:
return False
@@ -65,40 +65,43 @@
otherop = other.op
assert isinstance(otherop, GuardResOp)
assert isinstance(myop, GuardResOp)
-self.stronger = True
self.index = other.index
descr = myop.getdescr()
-descr.copy_all_attributes_from(other.op.getdescr())
-myop.rd_frame_info_list = otherop.rd_frame_info_list
-myop.rd_snapshot = otherop.rd_snapshot
-myop.setfailargs(otherop.getfailargs())
+if we_are_translated():
+descr.copy_all_attributes_from(other.op.getdescr())
+myop.rd_frame_info_list = otherop.rd_frame_info_list
+myop.rd_snapshot = otherop.rd_snapshot
+myop.setfailargs(otherop.getfailargs())
def compare(self, key1, key2):
if isinstance(key1, Box):
-assert isinstance(key2, Box)
-assert key1 is key2 # key of hash enforces this
-return 0
+if isinstance(key2, Box) and key1 is key2:
+return True, 0
+return False, 0
#
if isinstance(key1, ConstInt):
-assert isinstance(key2, ConstInt)
+if not isinstance(key2, ConstInt):
+return False, 0
v1 = key1.value
v2 = key2.value
if v1 == v2:
-return 0
+return True, 0
elif v1 < v2:
-return -1
+return True, -1
else:
-return 1
+return True, 1
#
if isinstance(key1, IndexVar):
assert isinstance(key2, IndexVar)
-return key1.compare(key2)
+return True, key1.compare(key2)
#
raise AssertionError("cannot compare: " + str(key1) + " <=> " +
str(key2))
def emit_varops(self, opt, var, old_arg):
if isinstance(var, IndexVar):
+if var.is_identity():
+return var.var
box = var.emit_operations(opt)
opt.renamer.start_renaming(old_arg, box)
return box
@@ -117,105 +120,122 @@
guard.setarg(0, box_result)
opt.emit_operation(guard)
-class GuardStrengthenOpt(object):
-def __init__(self, index_vars):
-self.index_vars = index_vars
-self._newoperations = []
-self._same_as = {}
-self.strength_reduced = 0 # how many guards could be removed?
+def update_keys(self, index_vars):
+self.lhs = index_vars.get(self.lhs_arg, self.lhs_arg)
+if isinstance(self.lhs, IndexVar):
+self.lhs = self.lhs.var
+self.lhs_key = self.lhs
+#
+self.rhs = index_vars.get(self.rhs_arg, self.rhs_arg)
+if isinstance(self.rhs, IndexVar):
+self.rhs = self.rhs.var
+self.rhs_key = self.rhs
-def find_co
[pypy-commit] pypy dtypes-compatability: merge default into branch
Author: mattip
Branch: dtypes-compatability
Changeset: r78556:b07b520cb8f4
Date: 2015-07-13 00:39 +0300
http://bitbucket.org/pypy/pypy/changeset/b07b520cb8f4/
Log:merge default into branch
diff --git a/pypy/module/micronumpy/concrete.py
b/pypy/module/micronumpy/concrete.py
--- a/pypy/module/micronumpy/concrete.py
+++ b/pypy/module/micronumpy/concrete.py
@@ -617,15 +617,17 @@
self.impl = impl
self.readonly = readonly
-def getitem(self, item):
-return raw_storage_getitem(lltype.Char, self.impl.storage, item)
+def getitem(self, index):
+return raw_storage_getitem(lltype.Char, self.impl.storage,
+ index + self.impl.start)
-def setitem(self, item, v):
-raw_storage_setitem(self.impl.storage, item,
+def setitem(self, index, v):
+raw_storage_setitem(self.impl.storage, index + self.impl.start,
rffi.cast(lltype.Char, v))
def getlength(self):
-return self.impl.size
+return self.impl.size - self.impl.start
def get_raw_address(self):
-return self.impl.storage
+from rpython.rtyper.lltypesystem import rffi
+return rffi.ptradd(self.impl.storage, self.impl.start)
diff --git a/pypy/module/micronumpy/strides.py
b/pypy/module/micronumpy/strides.py
--- a/pypy/module/micronumpy/strides.py
+++ b/pypy/module/micronumpy/strides.py
@@ -77,7 +77,7 @@
stop = 1
step = 1
lgt = 1
-axis_step = 0
+axis_step = 0 # both skip this axis in calculate_slice_strides and set
stride => 0
def __init__(self):
pass
@@ -127,7 +127,7 @@
except IndexError:
continue
if chunk.step != 0:
-rstrides[j] = s_i * chunk.step
+rstrides[j] = s_i * chunk.step * chunk.axis_step
rbackstrides[j] = s_i * max(0, chunk.lgt - 1) * chunk.step
rshape[j] = chunk.lgt
j += 1
diff --git a/pypy/module/micronumpy/test/test_ndarray.py
b/pypy/module/micronumpy/test/test_ndarray.py
--- a/pypy/module/micronumpy/test/test_ndarray.py
+++ b/pypy/module/micronumpy/test/test_ndarray.py
@@ -764,6 +764,8 @@
assert (a[1:] == b).all()
assert (a[1:,newaxis] == d).all()
assert (a[newaxis,1:] == c).all()
+assert a.strides == (8,)
+assert a[:, newaxis].strides == (8, 0)
def test_newaxis_assign(self):
from numpy import array, newaxis
@@ -2345,6 +2347,7 @@
assert a[1] == 0xff
assert len(a.data) == 16
assert type(a.data) is buffer
+assert a[1:].data._pypy_raw_address() - a.data._pypy_raw_address() ==
a.strides[0]
def test_explicit_dtype_conversion(self):
from numpy import array
diff --git a/rpython/jit/metainterp/optimizeopt/test/test_optimizeopt.py
b/rpython/jit/metainterp/optimizeopt/test/test_optimizeopt.py
--- a/rpython/jit/metainterp/optimizeopt/test/test_optimizeopt.py
+++ b/rpython/jit/metainterp/optimizeopt/test/test_optimizeopt.py
@@ -8637,5 +8637,27 @@
"""
self.optimize_loop(ops, expected, preamble)
+def test_getfield_proven_constant(self):
+py.test.skip("not working")
+ops = """
+[p0]
+i1 = getfield_gc(p0, descr=valuedescr)
+guard_value(i1, 13) []
+escape(i1)
+jump(p0)
+"""
+expected = """
+[p0]
+escape(13)
+jump(p0)
+"""
+expected_short = """
+[p0]
+i1 = getfield_gc(p0, descr=valuedescr)
+guard_value(i1, 13) []
+jump(p0)
+"""
+self.optimize_loop(ops, expected, expected_short=expected_short)
+
class TestLLtype(OptimizeOptTest, LLtypeMixin):
pass
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit
[pypy-commit] pypy dtypes-compatability: hack at dict-based vs. list-based repr(dtype) for compatibility, fix union dtypes, fix translation
Author: mattip
Branch: dtypes-compatability
Changeset: r78557:8937dc4116db
Date: 2015-07-15 22:19 +0300
http://bitbucket.org/pypy/pypy/changeset/8937dc4116db/
Log:hack at dict-based vs. list-based repr(dtype) for compatibility, fix
union dtypes, fix translation
diff --git a/pypy/module/micronumpy/descriptor.py
b/pypy/module/micronumpy/descriptor.py
--- a/pypy/module/micronumpy/descriptor.py
+++ b/pypy/module/micronumpy/descriptor.py
@@ -182,14 +182,27 @@
name = name[:-1]
return name
-def descr_get_name(self, space):
-name = self.get_name()
+def descr_get_name(self, space, quote=False):
+if quote:
+name = "'" + self.get_name() + "'"
+else:
+name = self.get_name()
if self.is_flexible() and self.elsize != 0:
return space.wrap(name + str(self.elsize * 8))
return space.wrap(name)
-def descr_get_str(self, space):
-return space.wrap(self.get_str())
+def descr_get_str(self, space, ignore='|', simple=True):
+if not simple and self.fields and len(self.fields) > 0:
+return self.descr_get_descr(space)
+total = 0
+for s in self.shape:
+total += s
+if not simple and total > 0:
+return space.newtuple(
+[space.wrap(self.subdtype.get_str(ignore='')),
+ space.newtuple([space.wrap(s) for s in self.shape]),
+])
+return space.wrap(self.get_str(ignore=ignore))
def get_str(self, ignore='|'):
basic = self.kind
@@ -203,11 +216,14 @@
size >>= 2
return "%s%s%s" % (endian, basic, size)
-def descr_get_descr(self, space, style='descr'):
+def descr_get_descr(self, space, style='descr', force_dict=False):
+simple = False
+if style == 'descr':
+simple = True
if not self.is_record():
return space.newlist([space.newtuple([space.wrap(""),
- self.descr_get_str(space)])])
-elif self.alignment >= 0 and style != 'descr':
+ self.descr_get_str(space,
simple=simple)])])
+elif (self.alignment > 1 and style != 'descr') or force_dict:
# we need to force a sorting order for the keys,
# so return a string instead of a dict. Also, numpy formats
# the lists without spaces between elements, so we cannot simply
@@ -217,6 +233,9 @@
offsets = "'offsets':["
titles = "'titles':["
use_titles = False
+show_offsets = False
+offsets_n = []
+total = 0
for name, title in self.names:
offset, subdtype = self.fields[name]
if subdtype.is_record():
@@ -232,20 +251,37 @@
titles += "'" + str(title) + "',"
if title is not None:
use_titles = True
+if total != offset:
+show_offsets = True
+total += subdtype.elsize
+# make sure offsets_n is sorted
+i = 0
+for i in range(len(offsets_n)):
+if offset < offsets_n[i]:
+break
+offsets_n.insert(i, offset)
+total = 0
+for i in range(len(offsets_n)):
+if offsets_n[i] != self.alignment * i:
+show_offsets = True
formats = formats[:-1] + ']'
offsets = offsets[:-1] + ']'
names = names[:-1] + ']'
titles = titles[:-1] + ']'
-if style == 'str':
+if self.alignment < 2:
+suffix = "}"
+elif style == 'str':
suffix = ", 'aligned':True}"
elif style == 'substr':
suffix = '}'
else:
suffix = "}, align=True"
-if use_titles:
+if use_titles and not show_offsets:
+return self.descr_get_descr(space, style='descr')
+elif use_titles:
return space.wrap('{' + names + ', ' + formats + ', ' +
-offsets + ', ' + "'itemsize':" + str(self.elsize) +
-titles + ', ' + suffix)
+offsets + ', ' + titles + ", 'itemsize':" +
+str(self.elsize) + suffix)
else:
return space.wrap('{' + names + ', ' + formats + ', ' +
offsets + ', ' + "'itemsize':" + str(self.elsize) +
@@ -253,18 +289,31 @@
else:
descr = []
+total = 0
for name, title in self.names:
-subdtype = self.fields[name][1]
-subdescr = [space.wrap(name)]
+offset, subdtype = self.fields[name]
+
