[pypy-commit] pypy memop-simplify3: merged default

2016-02-29 Thread plan_rich 
Author: Richard Plangger 
Branch: memop-simplify3
Changeset: r82622:6593ec12ff85
Date: 2016-03-01 08:16 +0100
http://bitbucket.org/pypy/pypy/changeset/6593ec12ff85/

Log:merged default

diff too long, truncating to 2000 out of 25290 lines

diff --git a/.hgignore b/.hgignore
--- a/.hgignore
+++ b/.hgignore
@@ -22,6 +22,7 @@
 ^pypy/module/cpyext/test/.+\.obj$
 ^pypy/module/cpyext/test/.+\.manifest$
 ^pypy/module/test_lib_pypy/ctypes_tests/.+\.o$
+^pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test\.o$
 ^pypy/module/cppyy/src/.+\.o$
 ^pypy/module/cppyy/bench/.+\.so$
 ^pypy/module/cppyy/bench/.+\.root$
@@ -35,7 +36,6 @@
 ^pypy/module/test_lib_pypy/cffi_tests/__pycache__.+$
 ^pypy/doc/.+\.html$
 ^pypy/doc/config/.+\.rst$
-^pypy/doc/basicblock\.asc$
 ^pypy/doc/.+\.svninfo$
 ^rpython/translator/c/src/libffi_msvc/.+\.obj$
 ^rpython/translator/c/src/libffi_msvc/.+\.dll$
@@ -45,52 +45,33 @@
 ^rpython/translator/c/src/cjkcodecs/.+\.obj$
 ^rpython/translator/c/src/stacklet/.+\.o$
 ^rpython/translator/c/src/.+\.o$
-^rpython/translator/jvm/\.project$
-^rpython/translator/jvm/\.classpath$
-^rpython/translator/jvm/eclipse-bin$
-^rpython/translator/jvm/src/pypy/.+\.class$
-^rpython/translator/benchmark/docutils$
-^rpython/translator/benchmark/templess$
-^rpython/translator/benchmark/gadfly$
-^rpython/translator/benchmark/mako$
-^rpython/translator/benchmark/bench-custom\.benchmark_result$
-^rpython/translator/benchmark/shootout_benchmarks$
+^rpython/translator/llvm/.+\.so$
 ^rpython/translator/goal/target.+-c$
 ^rpython/translator/goal/.+\.exe$
 ^rpython/translator/goal/.+\.dll$
 ^pypy/goal/pypy-translation-snapshot$
 ^pypy/goal/pypy-c
-^pypy/goal/pypy-jvm
-^pypy/goal/pypy-jvm.jar
 ^pypy/goal/.+\.exe$
 ^pypy/goal/.+\.dll$
 ^pypy/goal/.+\.lib$
 ^pypy/_cache$
-^pypy/doc/statistic/.+\.html$
-^pypy/doc/statistic/.+\.eps$
-^pypy/doc/statistic/.+\.pdf$
-^rpython/translator/cli/src/pypylib\.dll$
-^rpython/translator/cli/src/query\.exe$
-^rpython/translator/cli/src/main\.exe$
+^lib-python/2.7/lib2to3/.+\.pickle$
 ^lib_pypy/__pycache__$
 ^lib_pypy/ctypes_config_cache/_.+_cache\.py$
 ^lib_pypy/ctypes_config_cache/_.+_.+_\.py$
-^rpython/translator/cli/query-descriptions$
+^lib_pypy/_libmpdec/.+.o$
 ^pypy/doc/discussion/.+\.html$
 ^include/.+\.h$
 ^include/.+\.inl$
 ^pypy/doc/_build/.*$
 ^pypy/doc/config/.+\.html$
 ^pypy/doc/config/style\.css$
-^pypy/doc/jit/.+\.html$
-^pypy/doc/jit/style\.css$
 ^pypy/doc/image/lattice1\.png$
 ^pypy/doc/image/lattice2\.png$
 ^pypy/doc/image/lattice3\.png$
 ^pypy/doc/image/stackless_informal\.png$
 ^pypy/doc/image/parsing_example.+\.png$
 ^rpython/doc/_build/.*$
-^pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test\.o$
 ^compiled
 ^.git/
 ^release/
diff --git a/LICENSE b/LICENSE
--- a/LICENSE
+++ b/LICENSE
@@ -41,29 +41,29 @@
   Amaury Forgeot d'Arc
   Antonio Cuni
   Samuele Pedroni
+  Matti Picus
   Alex Gaynor
   Brian Kearns
-  Matti Picus
   Philip Jenvey
   Michael Hudson
+  Ronan Lamy
   David Schneider
+  Manuel Jacob
   Holger Krekel
   Christian Tismer
   Hakan Ardo
-  Manuel Jacob
-  Ronan Lamy
   Benjamin Peterson
+  Richard Plangger
   Anders Chrigstrom
   Eric van Riet Paap
   Wim Lavrijsen
-  Richard Plangger
   Richard Emslie
   Alexander Schremmer
   Dan Villiom Podlaski Christiansen
+  Remi Meier
   Lukas Diekmann
   Sven Hager
   Anders Lehmann
-  Remi Meier
   Aurelien Campeas
   Niklaus Haldimann
   Camillo Bruni
@@ -72,8 +72,8 @@
   Romain Guillebert
   Leonardo Santagada
   Seo Sanghyeon
+  Ronny Pfannschmidt
   Justin Peel
-  Ronny Pfannschmidt
   David Edelsohn
   Anders Hammarquist
   Jakub Gustak
@@ -95,6 +95,7 @@
   Tyler Wade
   Michael Foord
   Stephan Diehl
+  Vincent Legoll
   Stefan Schwarzer
   Valentino Volonghi
   Tomek Meka
@@ -105,9 +106,9 @@
   Jean-Paul Calderone
   Timo Paulssen
   Squeaky
+  Marius Gedminas
   Alexandre Fayolle
   Simon Burton
-  Marius Gedminas
   Martin Matusiak
   Konstantin Lopuhin
   Wenzhu Man
@@ -116,16 +117,20 @@
   Ivan Sichmann Freitas
   Greg Price
   Dario Bertini
+  Stefano Rivera
   Mark Pearse
   Simon Cross
   Andreas Sthrk
-  Stefano Rivera
+  Edd Barrett
   Jean-Philippe St. Pierre
   Guido van Rossum
   Pavel Vinogradov
+  Jeremy Thurgood
   Pawe Piotr Przeradowski
+  Spenser Bauman
   Paul deGrandis
   Ilya Osadchiy
+  marky1991
   Tobias Oberstein
   Adrian Kuhn
   Boris Feigin
@@ -134,14 +139,12 @@
   Georg Brandl
   Bert Freudenberg
   Stian Andreassen
-  Edd Barrett
+  Tobias Pape
   Wanja Saatkamp
   Gerald Klix
   Mike Blume
-  Tobias Pape
   Oscar Nierstrasz
   Stefan H. Muller
-  Jeremy Thurgood
   Rami Chowdhury
   Eugene Oden
   Henry Mason
@@ -153,6 +156,8 @@
   Lukas Renggli
   Guenter Jantzen
   Ned Batchelder
+  Tim Felgentreff
+  Anton Gulenko
   Amit Regmi
   Ben Young
   Nicolas Chauvat
@@ -162,12 +167,12 @@
   Nicholas Riley
   Jason Chu
   Igor Trindade Oliveira
-  Tim Felgentreff
+  Yichao Yu
   Rocco Moretti
   Gintautas Miliauskas
   Michael Twomey
   Lucian Branescu Mihaila
-

[pypy-commit] pypy memop-simplify3: calculate index value (const scale, const offset) before emiting the load for ConstPtrs

2016-02-29 Thread plan_rich 
Author: Richard Plangger 
Branch: memop-simplify3
Changeset: r82623:6f7d32f89861
Date: 2016-03-01 08:32 +0100
http://bitbucket.org/pypy/pypy/changeset/6f7d32f89861/

Log:calculate index value (const scale, const offset) before emiting the
load for ConstPtrs

diff --git a/rpython/jit/backend/llsupport/gc.py 
b/rpython/jit/backend/llsupport/gc.py
--- a/rpython/jit/backend/llsupport/gc.py
+++ b/rpython/jit/backend/llsupport/gc.py
@@ -164,13 +164,11 @@
 array_index = moving_obj_tracker.get_array_index(v)
 
 size, offset, _ = 
unpack_arraydescr(moving_obj_tracker.ptr_array_descr)
-scale = size
+array_index = array_index * size + offset
 args = [moving_obj_tracker.const_ptr_gcref_array,
 ConstInt(array_index),
-ConstInt(scale),
-ConstInt(offset),
 ConstInt(size)]
-load_op = ResOperation(rop.GC_LOAD_INDEXED_R, args)
+load_op = ResOperation(rop.GC_LOAD_R, args)
 newops.append(load_op)
 op.setarg(arg_i, load_op)
 #
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit

[pypy-commit] pypy memop-simplify3: removed getfield_gc_pure_* which where removed on default some time ago

2016-02-29 Thread plan_rich 
Author: Richard Plangger 
Branch: memop-simplify3
Changeset: r82624:08f7c1c3571d
Date: 2016-03-01 08:34 +0100
http://bitbucket.org/pypy/pypy/changeset/08f7c1c3571d/

Log:removed getfield_gc_pure_* which where removed on default some time
ago

diff --git a/rpython/jit/backend/llsupport/rewrite.py 
b/rpython/jit/backend/llsupport/rewrite.py
--- a/rpython/jit/backend/llsupport/rewrite.py
+++ b/rpython/jit/backend/llsupport/rewrite.py
@@ -243,7 +243,6 @@
 self.emit_gc_store_or_indexed(op, ptr_box, index_box, value_box,
   fieldsize, itemsize, ofs)
 elif opnum in (rop.GETFIELD_GC_I, rop.GETFIELD_GC_F, rop.GETFIELD_GC_R,
-   rop.GETFIELD_GC_PURE_I, rop.GETFIELD_GC_PURE_F, 
rop.GETFIELD_GC_PURE_R,
rop.GETFIELD_RAW_I, rop.GETFIELD_RAW_F, 
rop.GETFIELD_RAW_R):
 ofs, itemsize, sign = unpack_fielddescr(op.getdescr())
 ptr_box = op.getarg(0)
@@ -534,8 +533,6 @@
 # See emit_pending_zeros().  (This optimization is done by
 # hacking the object 'o' in-place: e.g., o.getarg(1) may be
 # replaced with another constant greater than 0.)
-#o = ResOperation(rop.ZERO_ARRAY, [v_arr, self.c_zero, v_length],
-# descr=arraydescr)
 assert isinstance(arraydescr, ArrayDescr)
 scale = arraydescr.itemsize
 v_length_scaled = v_length
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit

[pypy-commit] pypy s390x-backend: list append, insert, del. indexing functions do not generate byte code if IndexError would be raised

2016-02-29 Thread plan_rich 
Author: Richard Plangger 
Branch: s390x-backend
Changeset: r82621:53b6b15a4a97
Date: 2016-03-01 08:05 +0100
http://bitbucket.org/pypy/pypy/changeset/53b6b15a4a97/

Log:list append, insert, del. indexing functions do not generate byte
code if IndexError would be raised

diff --git a/rpython/jit/backend/llsupport/tl/code.py 
b/rpython/jit/backend/llsupport/tl/code.py
--- a/rpython/jit/backend/llsupport/tl/code.py
+++ b/rpython/jit/backend/llsupport/tl/code.py
@@ -14,14 +14,16 @@
 
 LIST_TYP = 'l'
 INT_TYP = 'i'
+OBJ_TYP = 'o'
+STR_TYP = 's'
+VAL_TYP = 'v' # either one of the earlier
+
+all_types = [INT_TYP, LIST_TYP, STR_TYP] # TODO OBJ_TYP
+
 SHORT_TYP = 'h'
 BYTE_TYP = 'b'
-OBJ_TYP = 'o'
-STR_TYP = 's'
 COND_TYP = 'c'
-VAL_TYP = 'v' # either one of the earlier
-
-all_types = [INT_TYP, LIST_TYP, STR_TYP]
+IDX_TYP = 'x'
 
 
 def unique_code():
@@ -146,6 +148,27 @@
 ctx.append_byte(self.BYTE_CODE)
 ctx.append_short(self.size)
 
+@requires_stack(LIST_TYP, IDX_TYP, INT_TYP) # TODO VAL_TYP
+@leaves_on_stack(LIST_TYP)
+class InsertList(ByteCode):
+BYTE_CODE = unique_code()
+def __init__(self):
+pass
+
+@requires_stack(LIST_TYP, IDX_TYP)
+@leaves_on_stack(LIST_TYP)
+class DelList(ByteCode):
+BYTE_CODE = unique_code()
+def __init__(self):
+pass
+
+@requires_stack(LIST_TYP, INT_TYP) # TODO VAL_TYP)
+@leaves_on_stack(LIST_TYP)
+class AppendList(ByteCode):
+BYTE_CODE = unique_code()
+def __init__(self):
+pass
+
 
 # remove comment one by one!
 
@@ -176,33 +199,6 @@
 #pass
 #
 
-#@requires_stack(LIST_TYP, INT_TYP, INT_TYP) # TODO VAL_TYP
-#class InsertList(ByteCode):
-#BYTE_CODE = unique_code()
-#@requires_param(INT_TYP)
-#def __init__(self, index):
-#self.index = index
-#def encode(self, ctx):
-#ctx.append_byte(self.BYTE_CODE)
-#ctx.append_int(self.index)
-#
-#@requires_stack(LIST_TYP, INT_TYP)
-#@leaves_on_stack(LIST_TYP)
-#class DelList(ByteCode):
-#BYTE_CODE = unique_code()
-#@requires_param(INT_TYP)
-#def __init__(self, index):
-#self.index = index
-#def encode(self, ctx):
-#ctx.append_byte(self.BYTE_CODE)
-#ctx.append_int(self.index)
-#
-#@requires_stack(LIST_TYP, INT_TYP, INT_TYP) # TODO VAL_TYP)
-#class AppendList(ByteCode):
-#BYTE_CODE = unique_code()
-#def __init__(self):
-#pass
-#
 #@requires_stack(LIST_TYP)
 #@leaves_on_stack(LIST_TYP, INT_TYP)
 #class LenList(ByteCode):
diff --git a/rpython/jit/backend/llsupport/tl/interp.py 
b/rpython/jit/backend/llsupport/tl/interp.py
--- a/rpython/jit/backend/llsupport/tl/interp.py
+++ b/rpython/jit/backend/llsupport/tl/interp.py
@@ -84,6 +84,21 @@
 size = runpack('h', bytecode[i+1:i+3])
 stack.append(space.wrap([None] * size))
 i += 2
+elif opcode == code.AppendList.BYTE_CODE:
+w_val = stack.pop()
+w_lst = stack.peek(0)
+w_lst.items.append(w_val)
+elif opcode == code.InsertList.BYTE_CODE:
+w_val = stack.pop()
+w_idx = stack.pop()
+w_lst = stack.peek(0)
+w_lst.items[w_idx.value] = w_val
+# index error, just crash here!
+elif opcode == code.DelList.BYTE_CODE:
+w_idx = stack.pop()
+w_lst = stack.peek(0)
+del w_lst.items[w_idx.value]
+# index error, just crash the machine!!
 else:
 raise NotImplementedError
 return i + 1
diff --git a/rpython/jit/backend/llsupport/tl/stack.py 
b/rpython/jit/backend/llsupport/tl/stack.py
--- a/rpython/jit/backend/llsupport/tl/stack.py
+++ b/rpython/jit/backend/llsupport/tl/stack.py
@@ -17,6 +17,12 @@
 self.stack[self.stackpos] = elem
 self.stackpos += 1
 
+def peek(self, i):
+stackpos = self.stackpos - i - 1
+if stackpos < 0:
+raise IndexError
+return self.stack[stackpos]
+
 def pop(self):
 stackpos = self.stackpos - 1
 if stackpos < 0:
diff --git a/rpython/jit/backend/llsupport/tl/test/code_strategies.py 
b/rpython/jit/backend/llsupport/tl/test/code_strategies.py
--- a/rpython/jit/backend/llsupport/tl/test/code_strategies.py
+++ b/rpython/jit/backend/llsupport/tl/test/code_strategies.py
@@ -1,15 +1,18 @@
 from hypothesis import strategies as st
+from hypothesis.control import assume
 from hypothesis.strategies import defines_strategy, composite
 from rpython.jit.backend.llsupport.tl import code, interp, stack
 from rpython.jit.backend.llsupport.tl.code import (all_types,
 INT_TYP, STR_TYP, LIST_TYP, SHORT_TYP, BYTE_TYP,
-COND_TYP)
+COND_TYP, IDX_TYP)
 from hypothesis.searchstrategy.strategies import OneOfStrategy
 from hypothesis.searchstrategy.collections import TupleStrategy
 
 def get_strategy_for(typ):
 if typ == INT_TYP:
 return st.integers(min_value=-2**31, max_value=2**31-1)
+elif typ == IDX_TYP:
+return st.integers(min_value=-2**31, max_value=2**31-1)
 elif typ ==

[pypy-commit] pypy default: merge

2016-02-29 Thread fijal 
Author: fijal
Branch: 
Changeset: r82620:b72f032521e3
Date: 2016-02-29 23:21 +0100
http://bitbucket.org/pypy/pypy/changeset/b72f032521e3/

Log:merge

diff --git a/rpython/rtyper/test/test_rdict.py 
b/rpython/rtyper/test/test_rdict.py
--- a/rpython/rtyper/test/test_rdict.py
+++ b/rpython/rtyper/test/test_rdict.py
@@ -1,8 +1,9 @@
+import sys
 from contextlib import contextmanager
 import signal
 
 from rpython.translator.translator import TranslationContext
-from rpython.annotator import model as annmodel
+from rpython.annotator.model import SomeInteger, SomeString
 from rpython.annotator.dictdef import DictKey, DictValue
 from rpython.rtyper.lltypesystem import lltype, rffi
 from rpython.rtyper.lltypesystem.rstr import string_repr
@@ -13,7 +14,20 @@
 from rpython.rlib.rarithmetic import r_int, r_uint, r_longlong, r_ulonglong
 
 import py
-py.log.setconsumer("rtyper", py.log.STDOUT)
+from hypothesis.strategies import builds, sampled_from, binary, just, integers
+from hypothesis.stateful import GenericStateMachine, run_state_machine_as_test
+
+def ann2strategy(s_value):
+if isinstance(s_value, SomeString):
+if s_value.can_be_None:
+return binary() | just(None)
+else:
+return binary()
+elif isinstance(s_value, SomeInteger):
+return integers(min_value=~sys.maxint, max_value=sys.maxint)
+else:
+raise TypeError("Cannot convert annotation %s to a strategy" % s_value)
+
 
 if hasattr(signal, 'alarm'):
 @contextmanager
@@ -35,7 +49,6 @@
 yield
 
 
-
 def not_really_random():
 """A random-ish generator, which also generates nice patterns from time to 
time.
 Could be useful to detect problems associated with specific usage 
patterns."""
@@ -1031,28 +1044,6 @@
 assert r_AB_dic.lowleveltype == r_BA_dic.lowleveltype
 
 
-def test_dict_resize(self):
-py.test.skip("test written for non-ordered dicts, update or kill")
-# XXX we no longer automatically resize on 'del'.  We need to
-# hack a bit in this test to trigger a resize by continuing to
-# fill the dict's table while keeping the actual size very low
-# in order to force a resize to shrink the table back
-def func(want_empty):
-d = self.newdict()
-for i in range(rdict.DICT_INITSIZE << 1):
-d[chr(ord('a') + i)] = i
-if want_empty:
-for i in range(rdict.DICT_INITSIZE << 1):
-del d[chr(ord('a') + i)]
-for i in range(rdict.DICT_INITSIZE << 3):
-d[chr(ord('A') - i)] = i
-del d[chr(ord('A') - i)]
-return d
-res = self.interpret(func, [0])
-assert len(res.entries) > rdict.DICT_INITSIZE
-res = self.interpret(func, [1])
-assert len(res.entries) == rdict.DICT_INITSIZE
-
 def test_opt_dummykeymarker(self):
 def f():
 d = {"hello": None}
@@ -1144,238 +1135,101 @@
 DICT = lltype.typeOf(llres.item1)
 assert sorted(DICT.TO.entries.TO.OF._flds) == ['f_hash', 'key', 
'value']
 
-def test_deleted_entry_reusage_with_colliding_hashes(self):
-py.test.skip("test written for non-ordered dicts, update or kill")
-def lowlevelhash(value):
-p = rstr.mallocstr(len(value))
-for i in range(len(value)):
-p.chars[i] = value[i]
-return rstr.LLHelpers.ll_strhash(p)
 
-def func(c1, c2):
-c1 = chr(c1)
-c2 = chr(c2)
-d = self.newdict()
-d[c1] = 1
-d[c2] = 2
-del d[c1]
-return d[c2]
-
-char_by_hash = {}
-base = rdict.DICT_INITSIZE
-for y in range(0, 256):
-y = chr(y)
-y_hash = lowlevelhash(y) % base
-char_by_hash.setdefault(y_hash, []).append(y)
-
-x, y = char_by_hash[0][:2]   # find a collision
-
-res = self.interpret(func, [ord(x), ord(y)])
-assert res == 2
-
-def func2(c1, c2):
-c1 = chr(c1)
-c2 = chr(c2)
-d = self.newdict()
-d[c1] = 1
-d[c2] = 2
-del d[c1]
-d[c1] = 3
-return d
-
-res = self.interpret(func2, [ord(x), ord(y)])
-for i in range(len(res.entries)):
-assert not (res.entries.everused(i) and not res.entries.valid(i))
-
-def func3(c0, c1, c2, c3, c4, c5, c6, c7):
-d = self.newdict()
-c0 = chr(c0) ; d[c0] = 1; del d[c0]
-c1 = chr(c1) ; d[c1] = 1; del d[c1]
-c2 = chr(c2) ; d[c2] = 1; del d[c2]
-c3 = chr(c3) ; d[c3] = 1; del d[c3]
-c4 = chr(c4) ; d[c4] = 1; del d[c4]
-c5 = chr(c5) ; d[c5] = 1; del d[c5]
-c6 = chr(c6) ; d[c6] = 1; del d[c6]
-c7 = chr(c7) ; d[c7] = 1; del d[c7]
-return d
-
-if rdict.DICT_INITSIZE != 8:

[pypy-commit] pypy default: fix embedding tests on OS X (they might or might not work on linux, too lazy to check), sem_init is deprecated

2016-02-29 Thread fijal 
Author: fijal
Branch: 
Changeset: r82619:38deea741bed
Date: 2016-02-29 23:20 +0100
http://bitbucket.org/pypy/pypy/changeset/38deea741bed/

Log:fix embedding tests on OS X (they might or might not work on linux,
too lazy to check), sem_init is deprecated

diff --git a/pypy/module/test_lib_pypy/cffi_tests/embedding/perf-test.c 
b/pypy/module/test_lib_pypy/cffi_tests/embedding/perf-test.c
--- a/pypy/module/test_lib_pypy/cffi_tests/embedding/perf-test.c
+++ b/pypy/module/test_lib_pypy/cffi_tests/embedding/perf-test.c
@@ -1,10 +1,12 @@
 #include 
 #include 
+#include 
 #include 
+#include 
 #ifdef PTEST_USE_THREAD
 # include 
 # include 
-static sem_t done;
+sem_t *done;
 #endif
 
 
@@ -54,7 +56,7 @@
 printf("time per call: %.3g\n", t);
 
 #ifdef PTEST_USE_THREAD
-int status = sem_post();
+int status = sem_post(done);
 assert(status == 0);
 #endif
 
@@ -68,8 +70,8 @@
 start_routine(0);
 #else
 pthread_t th;
-int i, status = sem_init(, 0, 0);
-assert(status == 0);
+done = sem_open("perf-test", O_CREAT, 0777, 0);
+int i, status;
 
 add1(0, 0);   /* this is the main thread */
 
@@ -78,7 +80,9 @@
 assert(status == 0);
 }
 for (i = 0; i < PTEST_USE_THREAD; i++) {
-status = sem_wait();
+status = sem_wait(done);
+if (status)
+fprintf(stderr, "%s\n", strerror(errno));
 assert(status == 0);
 }
 #endif
diff --git a/pypy/module/test_lib_pypy/cffi_tests/embedding/thread1-test.c 
b/pypy/module/test_lib_pypy/cffi_tests/embedding/thread1-test.c
--- a/pypy/module/test_lib_pypy/cffi_tests/embedding/thread1-test.c
+++ b/pypy/module/test_lib_pypy/cffi_tests/embedding/thread1-test.c
@@ -7,7 +7,7 @@
 
 extern int add1(int, int);
 
-static sem_t done;
+static sem_t* done;
 
 
 static void *start_routine(void *arg)
@@ -16,7 +16,7 @@
 x = add1(40, 2);
 assert(x == 42);
 
-status = sem_post();
+status = sem_post(done);
 assert(status == 0);
 
 return arg;
@@ -25,8 +25,8 @@
 int main(void)
 {
 pthread_t th;
-int i, status = sem_init(, 0, 0);
-assert(status == 0);
+int i, status;
+done = sem_open("thread1-test", O_CREAT, 0777, 0);
 
 printf("starting\n");
 fflush(stdout);
@@ -35,7 +35,7 @@
 assert(status == 0);
 }
 for (i = 0; i < NTHREADS; i++) {
-status = sem_wait();
+status = sem_wait(done);
 assert(status == 0);
 }
 printf("done\n");
diff --git a/pypy/module/test_lib_pypy/cffi_tests/embedding/thread2-test.c 
b/pypy/module/test_lib_pypy/cffi_tests/embedding/thread2-test.c
--- a/pypy/module/test_lib_pypy/cffi_tests/embedding/thread2-test.c
+++ b/pypy/module/test_lib_pypy/cffi_tests/embedding/thread2-test.c
@@ -5,7 +5,7 @@
 extern int add1(int, int);
 extern int add2(int, int, int);
 
-static sem_t done;
+static sem_t* done;
 
 
 static void *start_routine_1(void *arg)
@@ -14,7 +14,7 @@
 x = add1(40, 2);
 assert(x == 42);
 
-status = sem_post();
+status = sem_post(done);
 assert(status == 0);
 
 return arg;
@@ -29,7 +29,7 @@
 x = add2(1000, 200, 30);
 assert(x == 1230);
 
-status = sem_post();
+status = sem_post(done);
 assert(status == 0);
 
 return arg;
@@ -38,8 +38,8 @@
 int main(void)
 {
 pthread_t th;
-int i, status = sem_init(, 0, 0);
-assert(status == 0);
+int i, status;
+done = sem_open("thread2-test", O_CREAT, 0777, 0);
 
 printf("starting\n");
 fflush(stdout);
@@ -49,7 +49,7 @@
 assert(status == 0);
 
 for (i = 0; i < 2; i++) {
-status = sem_wait();
+status = sem_wait(done);
 assert(status == 0);
 }
 printf("done\n");
diff --git a/pypy/module/test_lib_pypy/cffi_tests/embedding/thread3-test.c 
b/pypy/module/test_lib_pypy/cffi_tests/embedding/thread3-test.c
--- a/pypy/module/test_lib_pypy/cffi_tests/embedding/thread3-test.c
+++ b/pypy/module/test_lib_pypy/cffi_tests/embedding/thread3-test.c
@@ -5,7 +5,7 @@
 extern int add2(int, int, int);
 extern int add3(int, int, int, int);
 
-static sem_t done;
+static sem_t* done;
 
 
 static void *start_routine_2(void *arg)
@@ -14,7 +14,7 @@
 x = add2(40, 2, 100);
 assert(x == 142);
 
-status = sem_post();
+status = sem_post(done);
 assert(status == 0);
 
 return arg;
@@ -26,7 +26,7 @@
 x = add3(1000, 200, 30, 4);
 assert(x == 1234);
 
-status = sem_post();
+status = sem_post(done);
 assert(status == 0);
 
 return arg;
@@ -35,8 +35,8 @@
 int main(void)
 {
 pthread_t th;
-int i, status = sem_init(, 0, 0);
-assert(status == 0);
+int i, status;
+done = sem_open("thread-test3", O_CREAT, 0777, 0);
 
 printf("starting\n");
 fflush(stdout);
@@ -47,7 +47,7 @@
 assert(status == 0);
 }
 for (i = 0; i < 20; i++) {
-status = sem_wait();
+status = sem_wait(done);
 assert(status == 0);
 }
 printf("done\n");
diff --git

[pypy-commit] pypy default: kill dead and/or superseded tests

2016-02-29 Thread rlamy 
Author: Ronan Lamy 
Branch: 
Changeset: r82618:51611a2b96f6
Date: 2016-02-29 22:08 +
http://bitbucket.org/pypy/pypy/changeset/51611a2b96f6/

Log:kill dead and/or superseded tests

diff --git a/rpython/rtyper/test/test_rdict.py 
b/rpython/rtyper/test/test_rdict.py
--- a/rpython/rtyper/test/test_rdict.py
+++ b/rpython/rtyper/test/test_rdict.py
@@ -14,7 +14,20 @@
 from rpython.rlib.rarithmetic import r_int, r_uint, r_longlong, r_ulonglong
 
 import py
-py.log.setconsumer("rtyper", py.log.STDOUT)
+from hypothesis.strategies import builds, sampled_from, binary, just, integers
+from hypothesis.stateful import GenericStateMachine, run_state_machine_as_test
+
+def ann2strategy(s_value):
+if isinstance(s_value, SomeString):
+if s_value.can_be_None:
+return binary() | just(None)
+else:
+return binary()
+elif isinstance(s_value, SomeInteger):
+return integers(min_value=~sys.maxint, max_value=sys.maxint)
+else:
+raise TypeError("Cannot convert annotation %s to a strategy" % s_value)
+
 
 if hasattr(signal, 'alarm'):
 @contextmanager
@@ -36,7 +49,6 @@
 yield
 
 
-
 def not_really_random():
 """A random-ish generator, which also generates nice patterns from time to 
time.
 Could be useful to detect problems associated with specific usage 
patterns."""
@@ -1032,28 +1044,6 @@
 assert r_AB_dic.lowleveltype == r_BA_dic.lowleveltype
 
 
-def test_dict_resize(self):
-py.test.skip("test written for non-ordered dicts, update or kill")
-# XXX we no longer automatically resize on 'del'.  We need to
-# hack a bit in this test to trigger a resize by continuing to
-# fill the dict's table while keeping the actual size very low
-# in order to force a resize to shrink the table back
-def func(want_empty):
-d = self.newdict()
-for i in range(rdict.DICT_INITSIZE << 1):
-d[chr(ord('a') + i)] = i
-if want_empty:
-for i in range(rdict.DICT_INITSIZE << 1):
-del d[chr(ord('a') + i)]
-for i in range(rdict.DICT_INITSIZE << 3):
-d[chr(ord('A') - i)] = i
-del d[chr(ord('A') - i)]
-return d
-res = self.interpret(func, [0])
-assert len(res.entries) > rdict.DICT_INITSIZE
-res = self.interpret(func, [1])
-assert len(res.entries) == rdict.DICT_INITSIZE
-
 def test_opt_dummykeymarker(self):
 def f():
 d = {"hello": None}
@@ -1145,170 +1135,6 @@
 DICT = lltype.typeOf(llres.item1)
 assert sorted(DICT.TO.entries.TO.OF._flds) == ['f_hash', 'key', 
'value']
 
-def test_deleted_entry_reusage_with_colliding_hashes(self):
-py.test.skip("test written for non-ordered dicts, update or kill")
-def lowlevelhash(value):
-p = rstr.mallocstr(len(value))
-for i in range(len(value)):
-p.chars[i] = value[i]
-return rstr.LLHelpers.ll_strhash(p)
-
-def func(c1, c2):
-c1 = chr(c1)
-c2 = chr(c2)
-d = self.newdict()
-d[c1] = 1
-d[c2] = 2
-del d[c1]
-return d[c2]
-
-char_by_hash = {}
-base = rdict.DICT_INITSIZE
-for y in range(0, 256):
-y = chr(y)
-y_hash = lowlevelhash(y) % base
-char_by_hash.setdefault(y_hash, []).append(y)
-
-x, y = char_by_hash[0][:2]   # find a collision
-
-res = self.interpret(func, [ord(x), ord(y)])
-assert res == 2
-
-def func2(c1, c2):
-c1 = chr(c1)
-c2 = chr(c2)
-d = self.newdict()
-d[c1] = 1
-d[c2] = 2
-del d[c1]
-d[c1] = 3
-return d
-
-res = self.interpret(func2, [ord(x), ord(y)])
-for i in range(len(res.entries)):
-assert not (res.entries.everused(i) and not res.entries.valid(i))
-
-def func3(c0, c1, c2, c3, c4, c5, c6, c7):
-d = self.newdict()
-c0 = chr(c0) ; d[c0] = 1; del d[c0]
-c1 = chr(c1) ; d[c1] = 1; del d[c1]
-c2 = chr(c2) ; d[c2] = 1; del d[c2]
-c3 = chr(c3) ; d[c3] = 1; del d[c3]
-c4 = chr(c4) ; d[c4] = 1; del d[c4]
-c5 = chr(c5) ; d[c5] = 1; del d[c5]
-c6 = chr(c6) ; d[c6] = 1; del d[c6]
-c7 = chr(c7) ; d[c7] = 1; del d[c7]
-return d
-
-if rdict.DICT_INITSIZE != 8:
-py.test.skip("make dict tests more indepdent from initsize")
-res = self.interpret(func3, [ord(char_by_hash[i][0])
-   for i in range(rdict.DICT_INITSIZE)])
-count_frees = 0
-for i in range(len(res.entries)):
-if not res.entries.everused(i):
-count_frees += 1
-assert

[pypy-commit] pypy default: Make hypothesis test parametric in the key and value types

2016-02-29 Thread rlamy 
Author: Ronan Lamy 
Branch: 
Changeset: r82617:959fe6807849
Date: 2016-02-29 21:57 +
http://bitbucket.org/pypy/pypy/changeset/959fe6807849/

Log:Make hypothesis test parametric in the key and value types

diff --git a/rpython/rtyper/test/test_rdict.py 
b/rpython/rtyper/test/test_rdict.py
--- a/rpython/rtyper/test/test_rdict.py
+++ b/rpython/rtyper/test/test_rdict.py
@@ -1,8 +1,9 @@
+import sys
 from contextlib import contextmanager
 import signal
 
 from rpython.translator.translator import TranslationContext
-from rpython.annotator import model as annmodel
+from rpython.annotator.model import SomeInteger, SomeString
 from rpython.annotator.dictdef import DictKey, DictValue
 from rpython.rtyper.lltypesystem import lltype, rffi
 from rpython.rtyper.lltypesystem.rstr import string_repr
@@ -1213,8 +1214,8 @@
 
 def test_stress():
 dictrepr = rdict.DictRepr(None, rint.signed_repr, rint.signed_repr,
-DictKey(None, annmodel.SomeInteger()),
-DictValue(None, annmodel.SomeInteger()))
+DictKey(None, SomeInteger()),
+DictValue(None, SomeInteger()))
 dictrepr.setup()
 l_dict = rdict.ll_newdict(dictrepr.DICT)
 reference = {}
@@ -1257,8 +1258,8 @@
 class PseudoRTyper:
 cache_dummy_values = {}
 dictrepr = rdict.DictRepr(PseudoRTyper(), string_repr, string_repr,
-DictKey(None, annmodel.SomeString(key_can_be_none)),
-DictValue(None, annmodel.SomeString(value_can_be_none)))
+DictKey(None, SomeString(key_can_be_none)),
+DictValue(None, SomeString(value_can_be_none)))
 dictrepr.setup()
 l_dict = rdict.ll_newdict(dictrepr.DICT)
 reference = {}
@@ -1295,87 +1296,114 @@
 assert l_dict.num_items == len(reference)
 complete_check()
 
-from hypothesis.strategies import builds, sampled_from, binary, just
+from hypothesis.strategies import builds, sampled_from, binary, just, integers
+from hypothesis.stateful import GenericStateMachine, run_state_machine_as_test
+
+def ann2strategy(s_value):
+if isinstance(s_value, SomeString):
+if s_value.can_be_None:
+return binary() | just(None)
+else:
+return binary()
+elif isinstance(s_value, SomeInteger):
+return integers(min_value=~sys.maxint, max_value=sys.maxint)
+else:
+raise TypeError("Cannot convert annotation %s to a strategy" % s_value)
+
 
 class Action(object):
-pass
+def __repr__(self):
+return "%s()" % self.__class__.__name__
 
-class SetItem(Action):
-def __init__(self, key, value):
-self.key = key
-self.value = value
+class PseudoRTyper:
+cache_dummy_values = {}
 
-def __repr__(self):
-return 'SetItem(%r, %r)' % (self.key, self.value)
+# XXX: None keys crash the test, but translation sort-of allows it
+@py.test.mark.parametrize('s_key',
+[SomeString(), SomeInteger()])
+@py.test.mark.parametrize('s_value',
+[SomeString(can_be_None=True), SomeString(), SomeInteger()])
+def test_hypothesis(s_key, s_value):
+rtyper = PseudoRTyper()
+r_key = s_key.rtyper_makerepr(rtyper)
+r_value = s_value.rtyper_makerepr(rtyper)
+dictrepr = rdict.DictRepr(rtyper, r_key, r_value,
+DictKey(None, s_key),
+DictValue(None, s_value))
+dictrepr.setup()
 
-def execute(self, state):
-ll_key = string_repr.convert_const(self.key)
-ll_value = string_repr.convert_const(self.value)
-rdict.ll_dict_setitem(state.l_dict, ll_key, ll_value)
-state.reference[self.key] = self.value
-assert rdict.ll_contains(state.l_dict, ll_key)
+_ll_key = r_key.convert_const
+_ll_value = r_value.convert_const
 
-class DelItem(Action):
-def __init__(self, key):
-self.key = key
+class SetItem(Action):
+def __init__(self, key, value):
+self.key = key
+self.value = value
 
-def __repr__(self):
-return 'DelItem(%r)' % (self.key)
+def __repr__(self):
+return 'SetItem(%r, %r)' % (self.key, self.value)
 
-def execute(self, state):
-ll_key = string_repr.convert_const(self.key)
-rdict.ll_dict_delitem(state.l_dict, ll_key)
-del state.reference[self.key]
-assert not rdict.ll_contains(state.l_dict, ll_key)
+def execute(self, state):
+ll_key = _ll_key(self.key)
+ll_value = _ll_value(self.value)
+rdict.ll_dict_setitem(state.l_dict, ll_key, ll_value)
+state.reference[self.key] = self.value
+assert rdict.ll_contains(state.l_dict, ll_key)
 
-class CopyDict(Action):
-def execute(self, state):
-state.l_dict = rdict.ll_copy(state.l_dict)
+class DelItem(Action):
+def __init__(self, key):
+self.key = key
 
-class

[pypy-commit] pypy cpyext-ext: fix test from 22fa8dfd0a2d by creating yet more alloc() functions

2016-02-29 Thread mattip 
Author: mattip 
Branch: cpyext-ext
Changeset: r82616:d3f9504cf268
Date: 2016-02-29 16:57 -0500
http://bitbucket.org/pypy/pypy/changeset/d3f9504cf268/

Log:fix test from 22fa8dfd0a2d by creating yet more alloc() functions

diff --git a/pypy/module/cpyext/api.py b/pypy/module/cpyext/api.py
--- a/pypy/module/cpyext/api.py
+++ b/pypy/module/cpyext/api.py
@@ -952,6 +952,8 @@
 cpyext_type_init = self.cpyext_type_init
 self.cpyext_type_init = None
 for pto, w_type in cpyext_type_init:
+if space.is_w(w_type, space.w_str):
+pto.c_tp_itemsize = 1
 finish_type_1(space, pto)
 finish_type_2(space, pto, w_type)
 
diff --git a/pypy/module/cpyext/pyobject.py b/pypy/module/cpyext/pyobject.py
--- a/pypy/module/cpyext/pyobject.py
+++ b/pypy/module/cpyext/pyobject.py
@@ -92,7 +92,7 @@
 
 if tp_alloc:
 def allocate(self, space, w_type, itemcount=0):
-return tp_alloc(space, w_type)
+return tp_alloc(space, w_type, itemcount)
 
 if tp_dealloc:
 def get_dealloc(self, space):
diff --git a/pypy/module/cpyext/stringobject.py 
b/pypy/module/cpyext/stringobject.py
--- a/pypy/module/cpyext/stringobject.py
+++ b/pypy/module/cpyext/stringobject.py
@@ -6,7 +6,7 @@
 from pypy.module.cpyext.pyerrors import PyErr_BadArgument
 from pypy.module.cpyext.pyobject import (
 PyObject, PyObjectP, Py_DecRef, make_ref, from_ref, track_reference,
-make_typedescr, get_typedescr)
+make_typedescr, get_typedescr, as_pyobj)
 
 ##
 ## Implementation of PyStringObject
@@ -62,12 +62,43 @@
 "Type description of PyStringObject"
 make_typedescr(space.w_str.layout.typedef,
basestruct=PyStringObject.TO,
+   alloc = string_alloc,
attach=string_attach,
dealloc=string_dealloc,
realize=string_realize)
 
 PyString_Check, PyString_CheckExact = build_type_checkers("String", "w_str")
 
+def string_alloc(space, w_type, length):
+'''
+Yet another way to allocate a PyObject, this time a
+PyStringObject. The first bit is copied from 
+BaseCpyTypedescr.allocate, the bit after length>0
+from string_attach. This is used as the tp_alloc function
+for PyStringObject
+'''
+from pypy.module.cpyext.typeobjectdefs import PyTypeObjectPtr
+pytype = as_pyobj(space, w_type)
+pytype = rffi.cast(PyTypeObjectPtr, pytype)
+assert pytype
+size = pytype.c_tp_basicsize
+buf = lltype.malloc(rffi.VOIDP.TO, size,
+flavor='raw', zero=True)
+py_str = rffi.cast(PyStringObject, buf)
+py_str.c_ob_refcnt = 1
+py_str.c_ob_type = pytype
+if length > 0:
+py_str.c_buffer = lltype.malloc(rffi.CCHARP.TO, length+1,
+flavor='raw', zero=True)
+py_str.c_size = length
+py_str.c_ob_sstate = rffi.cast(rffi.INT, 0) # SSTATE_NOT_INTERNED
+s = rffi.charpsize2str(py_str.c_buffer, length+1)
+w_obj = space.wrap(s)
+py_str.c_ob_shash = space.hash_w(w_obj)
+py_str.c_ob_sstate = rffi.cast(rffi.INT, 1) # SSTATE_INTERNED_MORTAL
+track_reference(space, rffi.cast(PyObject, py_str), w_obj)
+return rffi.cast(PyObject, py_str)
+
 def new_empty_str(space, length):
 """
 Allocate a PyStringObject and its buffer, but without a corresponding
diff --git a/pypy/module/cpyext/test/test_stringobject.py 
b/pypy/module/cpyext/test/test_stringobject.py
--- a/pypy/module/cpyext/test/test_stringobject.py
+++ b/pypy/module/cpyext/test/test_stringobject.py
@@ -99,6 +99,8 @@
 char * p_str;
 base = PyString_FromString("test");
 type = base->ob_type;
+if (type->tp_itemsize != 1)
+return PyLong_FromLong(type->tp_itemsize);
 obj = (PyStringObject*)type->tp_alloc(type, 10);
 if (PyString_GET_SIZE(obj) == 0)
 return PyLong_FromLong(-1);
diff --git a/pypy/module/cpyext/typeobject.py b/pypy/module/cpyext/typeobject.py
--- a/pypy/module/cpyext/typeobject.py
+++ b/pypy/module/cpyext/typeobject.py
@@ -494,7 +494,7 @@
 PyObject_dealloc(space, obj)
 
 
-def type_alloc(space, w_metatype):
+def type_alloc(space, w_metatype, itemsize=0):
 metatype = rffi.cast(PyTypeObjectPtr, make_ref(space, w_metatype))
 # Don't increase refcount for non-heaptypes
 if metatype:
diff --git a/pypy/module/cpyext/unicodeobject.py 
b/pypy/module/cpyext/unicodeobject.py
--- a/pypy/module/cpyext/unicodeobject.py
+++ b/pypy/module/cpyext/unicodeobject.py
@@ -30,6 +30,7 @@
 def init_unicodeobject(space):
 make_typedescr(space.w_unicode.layout.typedef,
basestruct=PyUnicodeObject.TO,
+   alloc = unicode_alloc,
attach=unicode_attach,
dealloc=unicode_dealloc,

[pypy-commit] pypy.org extradoc: update the values

2016-02-29 Thread arigo 
Author: Armin Rigo 
Branch: extradoc
Changeset: r712:545a2ef714d5
Date: 2016-02-29 21:19 +0100
http://bitbucket.org/pypy/pypy.org/changeset/545a2ef714d5/

Log:update the values

diff --git a/don4.html b/don4.html
--- a/don4.html
+++ b/don4.html
@@ -17,7 +17,7 @@
2nd call:

-   $30403 of $8 (38.0%)
+   $30423 of $8 (38.0%)


 
@@ -25,7 +25,7 @@
   
   This donation goes towards supporting the 
Transactional Memory in PyPy.
   Current status:
-we have $23126 left
+we have $23143 left
   in the account. Read proposal (2nd 
call)
   
   
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit

[pypy-commit] pypy default: Improve hypothesis test: catch infinite loops (posix only) and do the complete check once and only once, at the end of the run.

2016-02-29 Thread rlamy 
Author: Ronan Lamy 
Branch: 
Changeset: r82615:721ef287647d
Date: 2016-02-29 18:36 +
http://bitbucket.org/pypy/pypy/changeset/721ef287647d/

Log:Improve hypothesis test: catch infinite loops (posix only) and do
the complete check once and only once, at the end of the run.

diff --git a/rpython/rtyper/test/test_rdict.py 
b/rpython/rtyper/test/test_rdict.py
--- a/rpython/rtyper/test/test_rdict.py
+++ b/rpython/rtyper/test/test_rdict.py
@@ -1,3 +1,6 @@
+from contextlib import contextmanager
+import signal
+
 from rpython.translator.translator import TranslationContext
 from rpython.annotator import model as annmodel
 from rpython.annotator.dictdef import DictKey, DictValue
@@ -12,6 +15,27 @@
 import py
 py.log.setconsumer("rtyper", py.log.STDOUT)
 
+if hasattr(signal, 'alarm'):
+@contextmanager
+def signal_timeout(n):
+"""A flaky context manager that throws an exception if the body of the
+`with` block runs for longer than `n` seconds.
+"""
+def handler(signum, frame):
+raise RuntimeError('timeout')
+signal.signal(signal.SIGALRM, handler)
+signal.alarm(n)
+try:
+yield
+finally:
+signal.alarm(0)
+else:
+@contextmanager
+def signal_timeout(n):
+yield
+
+
+
 def not_really_random():
 """A random-ish generator, which also generates nice patterns from time to 
time.
 Could be useful to detect problems associated with specific usage 
patterns."""
@@ -1313,12 +1337,6 @@
 rdict.ll_clear(state.l_dict)
 state.reference.clear()
 
-class CompleteCheck(Action):
-def execute(self, state):
-assert state.l_dict.num_items == len(state.reference)
-for key, value in state.reference.iteritems():
-assert rdict.ll_dict_getitem(state.l_dict, _ll(key)) == _ll(value)
-
 st_keys = binary()
 st_values = binary()
 st_setitem = builds(SetItem, st_keys, st_values)
@@ -1342,7 +1360,7 @@
 self.reference = {}
 
 def steps(self):
-global_actions = [CopyDict(), ClearDict(), CompleteCheck()]
+global_actions = [CopyDict(), ClearDict()]
 if self.reference:
 return (
 st_setitem | st_delitem(self.reference) |
@@ -1351,6 +1369,13 @@
 return (st_setitem | sampled_from(global_actions))
 
 def execute_step(self, action):
-action.execute(self)
+with signal_timeout(1):  # catches infinite loops
+action.execute(self)
+
+def teardown(self):
+assert rdict.ll_dict_len(self.l_dict) == len(self.reference)
+for key, value in self.reference.iteritems():
+assert rdict.ll_dict_getitem(self.l_dict, _ll(key)) == _ll(value)
+
 
 TestHyp = StressTest.TestCase
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit

[pypy-commit] pypy default: Move execute_step() implementation to the Action subclasses

2016-02-29 Thread rlamy 
Author: Ronan Lamy 
Branch: 
Changeset: r82613:a1d15704499c
Date: 2016-02-29 15:24 +
http://bitbucket.org/pypy/pypy/changeset/a1d15704499c/

Log:Move execute_step() implementation to the Action subclasses

diff --git a/rpython/rtyper/test/test_rdict.py 
b/rpython/rtyper/test/test_rdict.py
--- a/rpython/rtyper/test/test_rdict.py
+++ b/rpython/rtyper/test/test_rdict.py
@@ -1284,6 +1284,13 @@
 def __repr__(self):
 return 'SetItem(%r, %r)' % (self.key, self.value)
 
+def execute(self, state):
+ll_key = string_repr.convert_const(self.key)
+ll_value = string_repr.convert_const(self.value)
+rdict.ll_dict_setitem(state.l_dict, ll_key, ll_value)
+state.reference[self.key] = self.value
+assert rdict.ll_contains(state.l_dict, ll_key)
+
 class DelItem(Action):
 def __init__(self, key):
 self.key = key
@@ -1291,8 +1298,17 @@
 def __repr__(self):
 return 'DelItem(%r)' % (self.key)
 
+def execute(self, state):
+ll_key = string_repr.convert_const(self.key)
+rdict.ll_dict_delitem(state.l_dict, ll_key)
+del state.reference[self.key]
+assert not rdict.ll_contains(state.l_dict, ll_key)
+
 class CompleteCheck(Action):
-pass
+def execute(self, state):
+assert state.l_dict.num_items == len(state.reference)
+for key, value in state.reference.iteritems():
+assert rdict.ll_dict_getitem(state.l_dict, _ll(key)) == _ll(value)
 
 st_keys = binary()
 st_values = binary()
@@ -1320,22 +1336,6 @@
 return (st_setitem | st_delitem(self.reference) | 
just(CompleteCheck())) if self.reference else (st_setitem | 
just(CompleteCheck()))
 
 def execute_step(self, action):
-if isinstance(action, SetItem):
-ll_key = string_repr.convert_const(action.key)
-ll_value = string_repr.convert_const(action.value)
-rdict.ll_dict_setitem(self.l_dict, ll_key, ll_value)
-self.reference[action.key] = action.value
-assert rdict.ll_contains(self.l_dict, ll_key)
-elif isinstance(action, DelItem):
-ll_key = string_repr.convert_const(action.key)
-rdict.ll_dict_delitem(self.l_dict, ll_key)
-del self.reference[action.key]
-assert not rdict.ll_contains(self.l_dict, ll_key)
-elif isinstance(action, CompleteCheck):
-assert self.l_dict.num_items == len(self.reference)
-for key, value in self.reference.iteritems():
-assert rdict.ll_dict_getitem(self.l_dict, _ll(key)) == 
_ll(value)
-
+action.execute(self)
 
 TestHyp = StressTest.TestCase
-
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit

[pypy-commit] pypy s390x-backend: list create, list concat

2016-02-29 Thread plan_rich 
Author: Richard Plangger 
Branch: s390x-backend
Changeset: r82612:df3c1340c16f
Date: 2016-02-29 14:58 +0100
http://bitbucket.org/pypy/pypy/changeset/df3c1340c16f/

Log:list create, list concat

diff --git a/rpython/jit/backend/llsupport/tl/code.py 
b/rpython/jit/backend/llsupport/tl/code.py
--- a/rpython/jit/backend/llsupport/tl/code.py
+++ b/rpython/jit/backend/llsupport/tl/code.py
@@ -128,6 +128,25 @@
 def __init__(self):
 pass
 
+@requires_stack(LIST_TYP, LIST_TYP)
+@leaves_on_stack(LIST_TYP)
+class AddList(ByteCode):
+BYTE_CODE = unique_code()
+def __init__(self):
+pass
+
+@requires_stack()
+@leaves_on_stack(LIST_TYP)
+class CreateList(ByteCode):
+BYTE_CODE = unique_code()
+@requires_param(BYTE_TYP)
+def __init__(self, size=8):
+self.size = size
+def encode(self, ctx):
+ctx.append_byte(self.BYTE_CODE)
+ctx.append_short(self.size)
+
+
 # remove comment one by one!
 
 #@requires_stack()
@@ -157,24 +176,6 @@
 #pass
 #
 
-#
-#@requires_stack(LIST_TYP, LIST_TYP)
-#@leaves_on_stack(LIST_TYP)
-#class AddList(ByteCode):
-#BYTE_CODE = unique_code()
-#def __init__(self):
-#pass
-#
-#@requires_stack()
-#class CreateList(ByteCode):
-#BYTE_CODE = unique_code()
-#@requires_param(BYTE_TYP)
-#def __init__(self, size=8):
-#self.size = size
-#def encode(self, ctx):
-#ctx.append_byte(self.BYTE_CODE)
-#ctx.append_short(self.size)
-#
 #@requires_stack(LIST_TYP, INT_TYP, INT_TYP) # TODO VAL_TYP
 #class InsertList(ByteCode):
 #BYTE_CODE = unique_code()
diff --git a/rpython/jit/backend/llsupport/tl/interp.py 
b/rpython/jit/backend/llsupport/tl/interp.py
--- a/rpython/jit/backend/llsupport/tl/interp.py
+++ b/rpython/jit/backend/llsupport/tl/interp.py
@@ -6,8 +6,12 @@
 pass
 
 class W_ListObject(W_Root):
-def __init__(self):
-self.items = []
+def __init__(self, items):
+self.items = items
+
+def concat(self, w_lst):
+assert isinstance(w_lst, W_ListObject)
+return self.items + w_lst.items
 
 class W_IntObject(W_Root):
 def __init__(self, value):
@@ -36,6 +40,8 @@
 return W_StrObject(val)
 if isinstance(val, unicode):
 return W_StrObject(val.encode('utf-8'))
+if isinstance(val, list):
+return W_ListObject(val)
 raise NotImplementedError("cannot handle: " + str(val) + 
str(type(val)))
 
 def entry_point(argv):
@@ -70,6 +76,14 @@
 w_str2 = stack.pop()
 w_str1 = stack.pop()
 stack.append(space.wrap(w_str1.concat(w_str2)))
+elif opcode == code.AddList.BYTE_CODE:
+w_lst2 = stack.pop()
+w_lst1 = stack.pop()
+stack.append(space.wrap(w_lst1.concat(w_lst2)))
+elif opcode == code.CreateList.BYTE_CODE:
+size = runpack('h', bytecode[i+1:i+3])
+stack.append(space.wrap([None] * size))
+i += 2
 else:
 raise NotImplementedError
 return i + 1
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit

[pypy-commit] pypy s390x-backend: renamed module, first hypothesis test that ensures that each interp function works as expected

2016-02-29 Thread plan_rich 
Author: Richard Plangger 
Branch: s390x-backend
Changeset: r82610:061da05db893
Date: 2016-02-29 14:27 +0100
http://bitbucket.org/pypy/pypy/changeset/061da05db893/

Log:renamed module, first hypothesis test that ensures that each interp
function works as expected

diff --git a/.hgignore b/.hgignore
--- a/.hgignore
+++ b/.hgignore
@@ -94,5 +94,6 @@
 ^pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test\.o$
 ^compiled
 ^.git/
+^.hypothesis/
 ^release/
 ^rpython/_cache$
diff --git a/rpython/jit/backend/llsupport/tl/__init__.py 
b/rpython/jit/backend/llsupport/tl/__init__.py
new file mode 100644
diff --git a/rpython/jit/backend/llsupport/tl/code.py 
b/rpython/jit/backend/llsupport/tl/code.py
new file mode 100644
--- /dev/null
+++ b/rpython/jit/backend/llsupport/tl/code.py
@@ -0,0 +1,219 @@
+
+import struct
+
+class ByteCode(object):
+def encode(self, ctx):
+ctx.append_byte(self.BYTE_CODE)
+
+@classmethod
+def create_from(self, draw, get_strategy_for):
+pt = getattr(self.__init__, '_param_types', [])
+return self(*[draw(get_strategy_for(t)) for t in pt])
+
+_c = 0
+
+LIST_TYP = 'l'
+INT_TYP = 'i'
+SHORT_TYP = 'h'
+BYTE_TYP = 'b'
+OBJ_TYP = 'o'
+STR_TYP = 's'
+COND_TYP = 'c'
+VAL_TYP = 'v' # either one of the earlier
+
+all_types = [INT_TYP, LIST_TYP, STR_TYP]
+
+
+def unique_code():
+global _c
+v = _c
+_c = v + 1
+return v
+
+class Context(object):
+def __init__(self):
+self.consts = {}
+self.const_idx = 0
+self.bytecode = []
+
+def append_byte(self, byte):
+self.bytecode.append(('b', byte))
+
+def get_byte(self, i):
+typ, byte = self.bytecode[i]
+assert typ == 'b'
+return byte
+
+def get_short(self, i):
+typ, int = self.bytecode[i]
+assert typ == 'h'
+return int
+
+def append_short(self, byte):
+self.bytecode.append(('h', byte))
+
+def append_int(self, byte):
+self.bytecode.append(('i', byte))
+
+def const_str(self, str):
+self.consts[self.const_idx] = str
+self.append_short(self.const_idx)
+self.const_idx += 1
+
+def to_string(self):
+code = []
+for typ, nmr in self.bytecode:
+code.append(struct.pack(typ, nmr))
+return ''.join(code)
+
+def transform(self, code_objs):
+for code_obj in code_objs:
+code_obj.encode(self)
+
+return self.to_string(), self.consts
+
+
+def requires_stack(*types):
+def method(clazz):
+clazz._stack_types = tuple(types)
+return clazz
+return method
+
+def leaves_on_stack(*types):
+def method(clazz):
+clazz._return_on_stack_types = tuple(types)
+return clazz
+return method
+
+
+def requires_param(*types):
+def method(m):
+m._param_types = tuple(types)
+return m
+return method
+
+@requires_stack()
+@leaves_on_stack(INT_TYP)
+class PutInt(ByteCode):
+BYTE_CODE = unique_code()
+@requires_param(INT_TYP)
+def __init__(self, value):
+self.integral = value
+def encode(self, ctx):
+ctx.append_byte(self.BYTE_CODE)
+ctx.append_int(self.integral)
+
+@requires_stack(INT_TYP, INT_TYP)
+@leaves_on_stack(INT_TYP)
+class CompareInt(ByteCode):
+BYTE_CODE = unique_code()
+def __init__(self):
+pass
+
+@requires_stack()
+@leaves_on_stack(STR_TYP)
+class LoadStr(ByteCode):
+BYTE_CODE = unique_code()
+@requires_param(STR_TYP)
+def __init__(self, string):
+self.string = string
+def encode(self, ctx):
+ctx.append_byte(self.BYTE_CODE)
+ctx.const_str(self.string)
+
+@requires_stack(STR_TYP, STR_TYP)
+@leaves_on_stack(STR_TYP)
+class AddStr(ByteCode):
+BYTE_CODE = unique_code()
+def __init__(self):
+pass
+
+# remove comment one by one!
+
+#@requires_stack()
+#@leaves_on_stack(INT_TYP)
+#class CondJump(ByteCode):
+#BYTE_CODE = unique_code()
+#
+#COND_EQ = 0
+#COND_LT = 1
+#COND_GT = 2
+#COND_LE = 3
+#COND_GE = 4
+#
+#@requires_param(COND_TYP)
+#def __init__(self, cond):
+#self.cond = cond
+#
+#def encode(self, ctx):
+#ctx.append_byte(self.BYTE_CODE)
+#ctx.append_byte(self.cond)
+#
+#@requires_stack()
+#@leaves_on_stack()
+#class Jump(ByteCode):
+#BYTE_CODE = unique_code()
+#def __init__(self):
+#pass
+#
+
+#
+#@requires_stack(LIST_TYP, LIST_TYP)
+#@leaves_on_stack(LIST_TYP)
+#class AddList(ByteCode):
+#BYTE_CODE = unique_code()
+#def __init__(self):
+#pass
+#
+#@requires_stack()
+#class CreateList(ByteCode):
+#BYTE_CODE = unique_code()
+#@requires_param(BYTE_TYP)
+#def __init__(self, size=8):
+#self.size = size
+#def encode(self, ctx):
+#ctx.append_byte(self.BYTE_CODE)
+#ctx.append_short(self.size)
+#
+#@requires_stack(LIST_TYP, INT_TYP, INT_TYP) # TODO VAL_TYP
+#class InsertList(ByteCode):
+#BYTE_CODE

[pypy-commit] pypy s390x-backend: removed some old files that where moved in the last commit

2016-02-29 Thread plan_rich 
Author: Richard Plangger 
Branch: s390x-backend
Changeset: r82611:4560bc0454eb
Date: 2016-02-29 14:30 +0100
http://bitbucket.org/pypy/pypy/changeset/4560bc0454eb/

Log:removed some old files that where moved in the last commit

diff --git a/rpython/jit/backend/llsupport/gcstress/__init__.py 
b/rpython/jit/backend/llsupport/gcstress/__init__.py
deleted file mode 100644
diff --git a/rpython/jit/backend/llsupport/gcstress/code.py 
b/rpython/jit/backend/llsupport/gcstress/code.py
deleted file mode 100644
--- a/rpython/jit/backend/llsupport/gcstress/code.py
+++ /dev/null
@@ -1,160 +0,0 @@
-
-import struct
-
-class ByteCode(object):
-def encode(self, ctx):
-ctx.append_byte(self.BYTE_CODE)
-
-_c = 0
-
-LIST_TYP = 'l'
-INT_TYP = 'i'
-OBJ_TYP = 'o'
-STR_TYP = 's'
-VAL_TYP = 'v' # either one of the earlier
-
-def unique_code():
-global _c
-v = _c
-_c = v + 1
-return v
-
-class Context(object):
-def __init__(self):
-self.consts = {}
-self.const_idx = 0
-self.bytecode = []
-
-def append_byte(self, byte):
-self.bytecode.append(('b', byte))
-
-def get_byte(self, i):
-typ, byte = self.bytecode[i]
-assert typ == 'b'
-return byte
-
-def get_short(self, i):
-typ, int = self.bytecode[i]
-assert typ == 'h'
-return int
-
-def append_short(self, byte):
-self.bytecode.append(('h', byte))
-
-def append_int(self, byte):
-self.bytecode.append(('i', byte))
-
-def const_str(self, str):
-self.consts[self.const_idx] = str
-self.append_short(self.const_idx)
-self.const_idx += 1
-
-def to_string(self):
-code = []
-for typ, nmr in self.bytecode:
-code.append(struct.pack(typ, nmr))
-return ''.join(code)
-
-def requires_stack(*types):
-def method(clazz):
-clazz._stack_types = tuple(types)
-return clazz
-
-return method
-
-@requires_stack()
-class CondJump(ByteCode):
-BYTE_CODE = unique_code()
-
-COND_EQ = 0
-COND_LT = 1
-COND_GT = 2
-COND_LE = 3
-COND_GE = 4
-
-def __init__(self, cond):
-self.cond = cond
-def encode(self, ctx):
-ctx.append_byte(self.BYTE_CODE)
-ctx.append_byte(self.cond)
-
-@requires_stack()
-class Jump(ByteCode):
-BYTE_CODE = unique_code()
-def __init__(self):
-pass
-
-@requires_stack()
-class LoadStr(ByteCode):
-BYTE_CODE = unique_code()
-def __init__(self, string):
-self.string = string
-def encode(self, ctx):
-ctx.append_byte(self.BYTE_CODE)
-ctx.const_str(self.string)
-
-@requires_stack(STR_TYP, STR_TYP)
-class AddStr(ByteCode):
-BYTE_CODE = unique_code()
-def __init__(self):
-pass
-
-@requires_stack(LIST_TYP, LIST_TYP)
-class AddList(ByteCode):
-BYTE_CODE = unique_code()
-def __init__(self):
-pass
-
-@requires_stack()
-class CreateList(ByteCode):
-BYTE_CODE = unique_code()
-def __init__(self, size=8):
-self.size = size
-def encode(self, ctx):
-ctx.append_byte(self.BYTE_CODE)
-ctx.append_short(self.size)
-
-@requires_stack()
-class PutInt(ByteCode):
-BYTE_CODE = unique_code()
-def __init__(self, value):
-self.integral = value
-def encode(self, ctx):
-ctx.append_byte(self.BYTE_CODE)
-ctx.append_short(self.integral)
-
-@requires_stack(LIST_TYP, INT_TYP, VAL_TYP)
-class InsertList(ByteCode):
-BYTE_CODE = unique_code()
-def __init__(self, index):
-self.index = index
-def encode(self, ctx):
-ctx.append_byte(self.BYTE_CODE)
-ctx.append_int(self.index)
-
-@requires_stack(LIST_TYP, INT_TYP)
-class DelList(ByteCode):
-BYTE_CODE = unique_code()
-def __init__(self, index):
-self.index = index
-def encode(self, ctx):
-ctx.append_byte(self.BYTE_CODE)
-ctx.append_int(self.index)
-
-@requires_stack(LIST_TYP, INT_TYP, VAL_TYP)
-class AppendList(ByteCode):
-BYTE_CODE = unique_code()
-def __init__(self):
-pass
-
-@requires_stack(LIST_TYP)
-class LenList(ByteCode):
-BYTE_CODE = unique_code()
-def __init__(self):
-self.required_stack('l')
-
-@requires_stack(INT_TYP, INT_TYP)
-class CompareInt(ByteCode):
-BYTE_CODE = unique_code()
-def __init__(self):
-pass
diff --git a/rpython/jit/backend/llsupport/gcstress/interp.py 
b/rpython/jit/backend/llsupport/gcstress/interp.py
deleted file mode 100644
--- a/rpython/jit/backend/llsupport/gcstress/interp.py
+++ /dev/null
@@ -1,23 +0,0 @@
-class W_Root(object):
-pass
-
-class W_ListObject(W_Root):
-def __init__(self):
-self.items = []
-
-def entry_point(argv):
-pass
-#bytecode = argv[0]
-#pc = 0
-#end = len(bytecode)
-#stack = Stack(512)
-#while i < end:
-#opcode = ord(bytecode[i])
-#if opcode == 0x0:
-#stack.push(space.new_list())
-#elif

[pypy-commit] pypy.org extradoc: update the values

2016-02-29 Thread arigo 
Author: Armin Rigo 
Branch: extradoc
Changeset: r711:74d29efbfbe9
Date: 2016-02-29 12:32 +0100
http://bitbucket.org/pypy/pypy.org/changeset/74d29efbfbe9/

Log:update the values

diff --git a/don1.html b/don1.html
--- a/don1.html
+++ b/don1.html
@@ -9,13 +9,13 @@
 
   $(function() {
 $("#progressbar").progressbar({
-  value: 59.9
+  value: 60.0
});
   });
 
 

-   $62936 of $105000 (59.9%)
+   $62984 of $105000 (60.0%)


 
@@ -23,7 +23,7 @@
   
   This donation goes towards supporting Python 3 in 
PyPy.
   Current status:
-we have $8093 left
+we have $8136 left
   in the account. Read proposal
   
   
diff --git a/don4.html b/don4.html
--- a/don4.html
+++ b/don4.html
@@ -17,7 +17,7 @@
2nd call:

-   $30393 of $8 (38.0%)
+   $30403 of $8 (38.0%)


 
@@ -25,7 +25,7 @@
   
   This donation goes towards supporting the 
Transactional Memory in PyPy.
   Current status:
-we have $23117 left
+we have $23126 left
   in the account. Read proposal (2nd 
call)
   
   
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit

[pypy-commit] extradoc extradoc: add the result of discussions from leysin (unedited)

2016-02-29 Thread fijal 
Author: fijal
Branch: extradoc
Changeset: r5612:de191b0da0b9
Date: 2016-02-29 10:51 +0100
http://bitbucket.org/pypy/extradoc/changeset/de191b0da0b9/

Log:add the result of discussions from leysin (unedited)

diff --git a/planning/sprint-leysin-2016-notes.rst 
b/planning/sprint-leysin-2016-notes.rst
new file mode 100644
--- /dev/null
+++ b/planning/sprint-leysin-2016-notes.rst
@@ -0,0 +1,323 @@
+Tasks
+=
+
+- mercurial benchmarks on PyPy runner exists, some benchmarks
+- mercurial porting C extensions to cffi MORE PROGRESS (fijal)
+- fix multiple inheritance resolution in cpyext (arigo, cfbolz around)
+- py3k work FIXING EVEN MORE TESTS, MERGED DEFAULT (AGAIN) (manuel, ronan)
+- register allocator, more information is now available, FIRST PROTOTYPE 
(remi, richard if remi has time), created an issue
+- clean up project lists (ronan, fijal)
+- test optimizeopt chain with hypothesis (cfbolz, fijal to discuss)
+- try fix speed center (richard, fijal to get him access), created issue
+- go skiing (marmoute)
+- go shopping
+- turn won't manage into issues (all)
+- start with new binary jit-log-opt (richard, fijal)
+- fixing stm (remi)
+- fix buffer API (arigo, fijal)
+
+
+
+
+won't manage
+
+
+- VMProf on OS X, fix bugs (can't reproduce)
+- jit leaner frontend
+- live ranges in JIT viewer
+- fix logging to not take time MESS
+- continuing to refactoring annotator
+- add warmup time to VMProf
+- use probablistic data structures for guard_value, WE HAVE A PLAN
+- single-run branch progress
+- update setup.py and upload rpython to pip
+
+
+done:
+-
+- dict-proxy with cpyext DONE
+- fix bug in FixedSizeArray DONE
+- compress resume data more, play with hypothesis (cfbolz, arigo, fijal) DONE
+- maps reordering DONE
+- take funding calls off the website, write blog post DONE
+- fix lxml on cpyext-gc-support-2 (ronan, arigo) DONE, MERGED
+- apply vmprof to a non pypy lang (cfbolz, fijal around) DONE
+- talk benchmark statistics (cfbolz, mattip, ronan) DONE
+- merging default into stm (remi) LESS MESS, MERGING DONE
+- cpyext-gc-support-2 blog post (mattip, arigo) DONE
+- get data about test duration DONE
+- start a bit of a dev process document
+- merging cpyext-ext, numpy-on-cpyext NEXT NEXT SEGFAULT, IMPORTS NUMPY WITH 
ONE HACK
+- fix tests
+- a script to upload to bitbucket IN PROGRESS
+- have a test in rpython that checks against imports from pypy (cfbolz)
+- make snowperson (cfbolz, fijal)
+- general wizardry (cfbolz, arigo, samuele not around) 
+
+
+too many bridges
+-
+
+Problems:
+ -  pypy py.test is slow
+ - most bridges come from guard_value(map) (then guard_class)
+
+Steps:
+ - detect the situation (cardinality estimation)
+ - trace a general version
+ - look at all promotes in pypy, to see whether the general version is good
+ - in particular, we need to general version for maps
+ - make maps give up if the object is too big
+
+Research:
+ - how to deal with method invocations of the same method on different classes
+ - 
+
+
+Python3
+=
+  - add more rposix features, use less replacements of os.XXX
+  - merge py3.3 -> py3k and create py3.5
+  - solve the speed issue
+  - utf8 & unicode problems
+   - list of things we suspect are slow on pypy3k:
+ * unicode & utf8 strings and dictionaries of those strings, potential 
solution
+   is not to use rpython unicode type
+ * itertools stuff is slower than python 3
+  - manuel & ronan go and work and SFC
+  - what to do with crowdfunding
+
+
+Idea around Mercurial
+==
+(notes about "new" feature that could be useful in pypy
+
+- clone bundle,
+- share.pool,
+- people version,
+
+
+
+summer of code
+=
+
+- volunteers from the pypy side: fijal, ronan, richard, remi, backup: armin
+- looking for students: richard, remi
+- unicode stuff as project
+
+
+
+cpyext+numpy
+
+
+- two approaches:
+  - micronumpy: basically works, but no story for cpyext, bit of a dead end
+  - using numpy code with cpyext, with hooks into micronumpy
+
+- safe (but maybe slow) default, everything just works
+- hard part: hijack some of the functionality and replace it with micronumpy 
code
+- > Bucharest?
+
+
+
+tooling
+===
+
+technical problems:
+- too many tools (vmprof, jitviewer, stmlog)
+- too many output formats (vmprof, jit-log-opt, stmlog*2)
+- jit-log-opt output format is brittle
+- parsing debug_merge_point is brittle
+- not good fallbacks
+- a lot of pypy-specific
+- identifying traces is not unique
+
+
+consolidation goals:
+- better format for jit-log-opt (keeping a way to show the old ascii output)
+- having a programmatic way to turn on trace dumps
+- combining vmprof/jitviewer
+- documentation/tutorial
+
+future cool features:
+- memory
+- warmup time
+- extensible events
+- web app changes respectively
+- navigation in jitviewer
+- way to compare runs
+- rpython functions where ops are coming from
+- threading and forking support
+
+
+volunteers:
+-