date:20160222

[pypy-commit] pypy default: (cfbolz, fijal around) kill test belonging to 40f823984bbf

2016-02-22 Thread cfbolz

Author: Carl Friedrich Bolz 
Branch: 
Changeset: r82383:d883e5c610d6
Date: 2016-02-22 11:03 +0100
http://bitbucket.org/pypy/pypy/changeset/d883e5c610d6/

Log:(cfbolz, fijal around) kill test belonging to 40f823984bbf

diff --git a/pypy/module/pypyjit/test_pypy_c/test_string.py 
b/pypy/module/pypyjit/test_pypy_c/test_string.py
--- a/pypy/module/pypyjit/test_pypy_c/test_string.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_string.py
@@ -142,43 +142,6 @@
 jump(..., descr=...)
 """)
 
-def test_getattr_promote(self):
-def main(n):
-class A(object):
-def meth_a(self):
-return 1
-def meth_b(self):
-return 2
-a = A()
-
-l = ['a', 'b']
-s = 0
-for i in range(n):
-name = 'meth_' + l[i & 1]
-meth = getattr(a, name) # ID: getattr
-s += meth()
-return s
-
-log = self.run(main, [1000])
-assert log.result == main(1000)
-loops = log.loops_by_filename(self.filepath)
-assert len(loops) == 1
-for loop in loops:
-assert loop.match_by_id('getattr','''
-guard_not_invalidated?
-i32 = strlen(p31)
-i34 = int_add(5, i32)
-p35 = newstr(i34)
-strsetitem(p35, 0, 109)
-strsetitem(p35, 1, 101)
-strsetitem(p35, 2, 116)
-strsetitem(p35, 3, 104)
-strsetitem(p35, 4, 95)
-copystrcontent(p31, p35, 0, 5, i32)
-i49 = 
call_i(ConstClass(_ll_2_str_eq_nonnull__rpy_stringPtr_rpy_stringPtr), p35, 
ConstPtr(ptr48), descr=)
-guard_value(i49, 1, descr=...)
-''')
-
 def test_remove_duplicate_method_calls(self):
 def main(n):
 lst = []
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit

[pypy-commit] pypy default: fix fallout from 4ee6b2b67a6b

2016-02-22 Thread cfbolz

Author: Carl Friedrich Bolz 
Branch: 
Changeset: r82382:47e7d8d28c74
Date: 2016-02-22 11:00 +0100
http://bitbucket.org/pypy/pypy/changeset/47e7d8d28c74/

Log:fix fallout from 4ee6b2b67a6b

diff --git a/pypy/module/pypyjit/test_pypy_c/test_string.py 
b/pypy/module/pypyjit/test_pypy_c/test_string.py
--- a/pypy/module/pypyjit/test_pypy_c/test_string.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_string.py
@@ -28,7 +28,6 @@
 guard_true(i14, descr=...)
 guard_not_invalidated(descr=...)
 i16 = int_eq(i6, %d)
-guard_false(i16, descr=...)
 i15 = int_mod(i6, i10)
 i17 = int_rshift(i15, %d)
 i18 = int_and(i10, i17)
@@ -68,7 +67,6 @@
 guard_true(i11, descr=...)
 guard_not_invalidated(descr=...)
 i13 = int_eq(i6, %d) # value provided below
-guard_false(i13, descr=...)
 i15 = int_mod(i6, 10)
 i17 = int_rshift(i15, %d)# value provided below
 i18 = int_and(10, i17)
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit

[pypy-commit] pypy.org extradoc: close numpy donation campaign

2016-02-22 Thread mattip

Author: mattip 
Branch: extradoc
Changeset: r705:62fbad1ffe3d
Date: 2016-02-22 11:44 +0100
http://bitbucket.org/pypy/pypy.org/changeset/62fbad1ffe3d/

Log:close numpy donation campaign

diff --git a/don1.html b/don1.html
--- a/don1.html
+++ b/don1.html
@@ -3,7 +3,7 @@
 Donate towards STM in 
pypy
 Donate towards py3k 
in pypy
 Donate towards 
general pypy progress
-Donate towards NumPy 
in pypy
+

[pypy-commit] pypy reorder-map-attributes: close to-be-merged branch

2016-02-22 Thread cfbolz

Author: Carl Friedrich Bolz 
Branch: reorder-map-attributes
Changeset: r82385:8a729799f9ef
Date: 2016-02-22 11:49 +0100
http://bitbucket.org/pypy/pypy/changeset/8a729799f9ef/

Log:close to-be-merged branch

___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit

[pypy-commit] pypy reorder-map-attributes: what's new entry

2016-02-22 Thread cfbolz

Author: Carl Friedrich Bolz 
Branch: reorder-map-attributes
Changeset: r82384:435c07e47188
Date: 2016-02-22 11:14 +0100
http://bitbucket.org/pypy/pypy/changeset/435c07e47188/

Log:what's new entry

diff --git a/pypy/doc/whatsnew-head.rst b/pypy/doc/whatsnew-head.rst
--- a/pypy/doc/whatsnew-head.rst
+++ b/pypy/doc/whatsnew-head.rst
@@ -123,3 +123,10 @@
 .. branch: fix-cpython-ssl-tests-2.7
 
 Fix SSL tests by importing cpython's patch
+
+
+.. branch: reorder-map-attributes
+
+When creating instances and adding attributes in several different orders
+depending on some condition, the JIT would create too much code. This is now
+fixed.
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit

[pypy-commit] pypy default: merge reorder-map-attributes:

2016-02-22 Thread cfbolz

Author: Carl Friedrich Bolz 
Branch: 
Changeset: r82386:572984f4ec26
Date: 2016-02-22 11:53 +0100
http://bitbucket.org/pypy/pypy/changeset/572984f4ec26/

Log:merge reorder-map-attributes:

when creating an instance with attributes in a different order than
some previously existing ordering, switch to that ordering. this
reduces the number of bridges in some situations.

diff --git a/pypy/doc/whatsnew-head.rst b/pypy/doc/whatsnew-head.rst
--- a/pypy/doc/whatsnew-head.rst
+++ b/pypy/doc/whatsnew-head.rst
@@ -128,6 +128,7 @@
 
 Fix SSL tests by importing cpython's patch
 
+
 .. branch: remove-getfield-pure
 
 Remove pure variants of ``getfield_gc_*`` operations from the JIT. Relevant
@@ -163,3 +164,10 @@
 .. branch: windows-vmprof-support
 
 vmprof should work on Windows.
+
+
+.. branch: reorder-map-attributes
+
+When creating instances and adding attributes in several different orders
+depending on some condition, the JIT would create too much code. This is now
+fixed.
\ No newline at end of file
diff --git a/pypy/objspace/std/mapdict.py b/pypy/objspace/std/mapdict.py
--- a/pypy/objspace/std/mapdict.py
+++ b/pypy/objspace/std/mapdict.py
@@ -1,4 +1,4 @@
-import weakref
+import weakref, sys
 
 from rpython.rlib import jit, objectmodel, debug, rerased
 from rpython.rlib.rarithmetic import intmask, r_uint
@@ -12,6 +12,11 @@
 from pypy.objspace.std.typeobject import MutableCell
 
 
+erase_item, unerase_item = rerased.new_erasing_pair("mapdict storage item")
+erase_map,  unerase_map = rerased.new_erasing_pair("map")
+erase_list, unerase_list = rerased.new_erasing_pair("mapdict storage list")
+
+
 # 
 # attribute shapes
 
@@ -20,6 +25,7 @@
 # note: we use "x * NUM_DIGITS_POW2" instead of "x << NUM_DIGITS" because
 # we want to propagate knowledge that the result cannot be negative
 
+
 class AbstractAttribute(object):
 _immutable_fields_ = ['terminator']
 cache_attrs = None
@@ -151,29 +157,124 @@
 cache[name, index] = attr
 return attr
 
+@jit.elidable
+def _get_cache_attr(self, name, index):
+key = name, index
+# this method is not actually elidable, but it's fine anyway
+if self.cache_attrs is not None:
+return self.cache_attrs.get(key, None)
+return None
+
+def add_attr(self, obj, name, index, w_value):
+self._reorder_and_add(obj, name, index, w_value)
+if not jit.we_are_jitted():
+oldattr = self
+attr = obj._get_mapdict_map()
+size_est = (oldattr._size_estimate + attr.size_estimate()
+   - oldattr.size_estimate())
+assert size_est >= (oldattr.length() * NUM_DIGITS_POW2)
+oldattr._size_estimate = size_est
+
+def _add_attr_without_reordering(self, obj, name, index, w_value):
+attr = self._get_new_attr(name, index)
+attr._switch_map_and_write_storage(obj, w_value)
+
+@jit.unroll_safe
+def _switch_map_and_write_storage(self, obj, w_value):
+if self.length() > obj._mapdict_storage_length():
+# note that self.size_estimate() is always at least self.length()
+new_storage = [None] * self.size_estimate()
+for i in range(obj._mapdict_storage_length()):
+new_storage[i] = obj._mapdict_read_storage(i)
+obj._set_mapdict_storage_and_map(new_storage, self)
+
+# the order is important here: first change the map, then the storage,
+# for the benefit of the special subclasses
+obj._set_mapdict_map(self)
+obj._mapdict_write_storage(self.storageindex, w_value)
+
+
+@jit.elidable
+def _find_branch_to_move_into(self, name, index):
+# walk up the map chain to find an ancestor with lower order that
+# already has the current name as a child inserted
+current_order = sys.maxint
+number_to_readd = 0
+current = self
+key = (name, index)
+while True:
+attr = None
+if current.cache_attrs is not None:
+attr = current.cache_attrs.get(key, None)
+if attr is None or attr.order > current_order:
+# we reached the top, so we didn't find it anywhere,
+# just add it to the top attribute
+if not isinstance(current, PlainAttribute):
+return 0, self._get_new_attr(name, index)
+
+else:
+return number_to_readd, attr
+# if not found try parent
+number_to_readd += 1
+current_order = current.order
+current = current.back
+
 @jit.look_inside_iff(lambda self, obj, name, index, w_value:
 jit.isconstant(self) and
 jit.isconstant(name) and
 jit.isconstant(index))
-def add_attr(self, obj, name, index, w_value):
-attr = self._get_new_attr(nam

[pypy-commit] pypy vlen-resume: finish the refactoring

2016-02-22 Thread fijal

Author: fijal
Branch: vlen-resume
Changeset: r82387:8e1dbed9f261
Date: 2016-02-22 12:00 +0100
http://bitbucket.org/pypy/pypy/changeset/8e1dbed9f261/

Log:finish the refactoring

diff --git a/rpython/jit/metainterp/resume.py b/rpython/jit/metainterp/resume.py
--- a/rpython/jit/metainterp/resume.py
+++ b/rpython/jit/metainterp/resume.py
@@ -134,7 +134,10 @@
snapshot_storage):
 n = len(framestack) - 1
 if virtualizable_boxes is not None:
-virtualizable_boxes = virtualizable_boxes[:]
+virtualizable_boxes = ([virtualizable_boxes[-1]] +
+virtualizable_boxes[:-1])
+else:
+virtualizable_boxes = []
 virtualref_boxes = virtualref_boxes[:]
 if n >= 0:
 top = framestack[n]
@@ -1131,49 +1134,42 @@
 self.boxes_f = boxes_f
 self._prepare_next_section(info)
 
-def consume_virtualizable_boxes(self, vinfo):
+def consume_virtualizable_boxes(self, vinfo, index):
 # we have to ignore the initial part of 'nums' (containing vrefs),
 # find the virtualizable from nums[-1], and use it to know how many
 # boxes of which type we have to return.  This does not write
 # anything into the virtualizable.
 numb = self.numb
-first_snapshot_size = rffi.cast(lltype.Signed, 
numb.first_snapshot_size)
-item, _ = resumecode.numb_next_item(numb, first_snapshot_size - 1)
+item, index = resumecode.numb_next_item(numb, index)
 virtualizablebox = self.decode_ref(item)
-index = first_snapshot_size - 
vinfo.get_total_size(virtualizablebox.getref_base()) - 1
 virtualizable = vinfo.unwrap_virtualizable_box(virtualizablebox)
 return vinfo.load_list_of_boxes(virtualizable, self, virtualizablebox,
 numb, index)
 
-def consume_virtualref_boxes(self, end):
+def consume_virtualref_boxes(self, index):
 # Returns a list of boxes, assumed to be all BoxPtrs.
 # We leave up to the caller to call vrefinfo.continue_tracing().
-assert (end & 1) == 0
+size, index = resumecode.numb_next_item(self.numb, index)
+if size == 0:
+return [], index
 lst = []
-self.cur_index = 0
-for i in range(end):
-item, self.cur_index = resumecode.numb_next_item(self.numb,
-self.cur_index)
+for i in range(size * 2):
+item, index = resumecode.numb_next_item(self.numb, index)
 lst.append(self.decode_ref(item))
-return lst
+return lst, index
 
 def consume_vref_and_vable_boxes(self, vinfo, ginfo):
-first_snapshot_size = rffi.cast(lltype.Signed,
-self.numb.first_snapshot_size)
+vable_size, index = resumecode.numb_next_item(self.numb, 0)
 if vinfo is not None:
-virtualizable_boxes = self.consume_virtualizable_boxes(vinfo)
-end = first_snapshot_size - len(virtualizable_boxes)
+virtualizable_boxes, index = 
self.consume_virtualizable_boxes(vinfo,
+  
index)
 elif ginfo is not None:
-xx
-item, self.cur_index = resumecode.numb_next_item(self.numb,
-first_snapshot_size - 1)
+item, index = resumecode.numb_next_item(self.numb, index)
 virtualizable_boxes = [self.decode_ref(item)]
-end = first_snapshot_size - 1
 else:
-end = first_snapshot_size
 virtualizable_boxes = None
-virtualref_boxes = self.consume_virtualref_boxes(end)
-self.cur_index = rffi.cast(lltype.Signed, 
self.numb.first_snapshot_size)
+virtualref_boxes, index = self.consume_virtualref_boxes(index)
+self.cur_index = index
 return virtualizable_boxes, virtualref_boxes
 
 def allocate_with_vtable(self, descr=None):
@@ -1452,37 +1448,32 @@
 # we have to decode a list of references containing pairs
 # [..., virtual, vref, ...] and returns the index at the end
 size, index = resumecode.numb_next_item(self.numb, index)
-if vrefinfo is None:
+if vrefinfo is None or size == 0:
 assert size == 0
 return index
-
-assert (end & 1) == 0
-self.cur_index = 0
-for i in range(0, end, 2):
-virtual_item, self.cur_index = resumecode.numb_next_item(
-self.numb, self.cur_index)
-vref_item, self.cur_index = resumecode.numb_next_item(
-self.numb, self.cur_index)
+for i in range(size):
+virtual_item, index = resumecode.numb_next_item(
+self.numb, index)
+vref_item, index = resumecode.numb_next_item(
+self.numb, index)
 virtual = self.decode_ref(virtual_item)
 vref = self.decode_ref(vref_item)

[pypy-commit] pypy.org pypy3-update: update pypy3 and kill pypy3 campaign, pending a new pypy3 campaign in the futue

2016-02-22 Thread mattip

Author: mattip 
Branch: pypy3-update
Changeset: r706:e54d381be09f
Date: 2016-02-22 12:32 +0100
http://bitbucket.org/pypy/pypy.org/changeset/e54d381be09f/

Log:update pypy3 and kill pypy3 campaign, pending a new pypy3 campaign
in the futue

diff --git a/don1.html b/don1.html
--- a/don1.html
+++ b/don1.html
@@ -2,7 +2,7 @@
   
 Donate towards STM in 
pypy
 Donate towards py3k 
in pypy
-Donate towards 
general pypy progress
+

[pypy-commit] pypy default: add requirements.txt for untranslated tests

2016-02-22 Thread mattip

Author: mattip 
Branch: 
Changeset: r82390:b46f8fd85897
Date: 2016-02-22 14:03 +0100
http://bitbucket.org/pypy/pypy/changeset/b46f8fd85897/

Log:add requirements.txt for untranslated tests

diff --git a/requirements.txt b/requirements.txt
new file mode 100644
--- /dev/null
+++ b/requirements.txt
@@ -0,0 +1,3 @@
+# hypothesis is used for test generation on untranslated jit tests
+hypothesis
+
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit

[pypy-commit] pypy cpyext-ext: (rlamy) fix for testing - cache the function

2016-02-22 Thread mattip

Author: mattip 
Branch: cpyext-ext
Changeset: r82389:a795cb8e3138
Date: 2016-02-21 18:29 +0100
http://bitbucket.org/pypy/pypy/changeset/a795cb8e3138/

Log:(rlamy) fix for testing - cache the function

diff --git a/pypy/module/cpyext/dictobject.py b/pypy/module/cpyext/dictobject.py
--- a/pypy/module/cpyext/dictobject.py
+++ b/pypy/module/cpyext/dictobject.py
@@ -230,6 +230,12 @@
 
 @specialize.memo()
 def make_frozendict(space):
+if space not in _frozendict_cache:
+_frozendict_cache[space] = _make_frozendict(space)
+return _frozendict_cache[space]
+
+_frozendict_cache = {}
+def _make_frozendict(space):
 return space.appexec([], '''():
 import _abcoll
 class FrozenDict(_abcoll.Mapping):
diff --git a/pypy/module/cpyext/test/test_dictobject.py 
b/pypy/module/cpyext/test/test_dictobject.py
--- a/pypy/module/cpyext/test/test_dictobject.py
+++ b/pypy/module/cpyext/test/test_dictobject.py
@@ -161,17 +161,13 @@
 
 def test_dictproxy(self, space, api):
 w_dict = space.sys.get('modules')
+w_proxy = api.PyDictProxy_New(w_dict)
 assert space.contains_w(w_proxy, space.wrap('sys'))
 raises(OperationError, space.setitem,
w_proxy, space.wrap('sys'), space.w_None)
 raises(OperationError, space.delitem,
w_proxy, space.wrap('sys'))
 raises(OperationError, space.call_method, w_proxy, 'clear')
-
-
-@py.test.mark.xfail(reason='make_frozendict memoize only works translated')
-def test_dictproxy(self, space, api):
-w_proxy = api.PyDictProxy_New(w_dict)
 assert api.PyDictProxy_Check(w_proxy)
 
 class AppTestDictObject(AppTestCpythonExtensionBase):
@@ -187,7 +183,6 @@
  return NULL;
  proxydict = PyDictProxy_New(dict);
  Py_DECREF(dict);
- /* when memoize works untranslated, add these tests
  if (!PyDictProxy_Check(proxydict)) {
 Py_DECREF(proxydict);
 PyErr_SetNone(PyExc_ValueError);
@@ -198,7 +193,6 @@
 PyErr_SetNone(PyExc_ValueError);
 return NULL;
  }
- */
  i = PyObject_Size(proxydict);
  Py_DECREF(proxydict);
  return PyLong_FromLong(i); 
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit

[pypy-commit] pypy cpyext-ext: fix, test PyDictProxy object

2016-02-22 Thread mattip

Author: mattip 
Branch: cpyext-ext
Changeset: r82388:b1fc26f1766b
Date: 2016-02-21 18:00 +0100
http://bitbucket.org/pypy/pypy/changeset/b1fc26f1766b/

Log:fix, test PyDictProxy object

diff --git a/pypy/module/cpyext/api.py b/pypy/module/cpyext/api.py
--- a/pypy/module/cpyext/api.py
+++ b/pypy/module/cpyext/api.py
@@ -473,7 +473,7 @@
 "PyUnicode_Type": "space.w_unicode",
 "PyBaseString_Type": "space.w_basestring",
 "PyDict_Type": "space.w_dict",
-#"PyDictProxy_Type": "space.type(space.w_NotImplemented)",
+"PyDictProxy_Type": "cpyext.dictobject.make_frozendict(space)",
 "PyTuple_Type": "space.w_tuple",
 "PyList_Type": "space.w_list",
 "PySet_Type": "space.w_set",
diff --git a/pypy/module/cpyext/dictobject.py b/pypy/module/cpyext/dictobject.py
--- a/pypy/module/cpyext/dictobject.py
+++ b/pypy/module/cpyext/dictobject.py
@@ -248,3 +248,15 @@
 w_frozendict = make_frozendict(space)
 return space.call_function(w_frozendict, w_dict)
 
+@cpython_api([PyObject], rffi.INT_real, error=CANNOT_FAIL)
+def PyDictProxy_Check(space, w_obj):
+w_typ = make_frozendict(space)
+print 'check', w_typ, space.type(w_obj)
+return space.isinstance_w(w_obj, w_typ)
+
+@cpython_api([PyObject], rffi.INT_real, error=CANNOT_FAIL)
+def PyDictProxy_CheckExact(space, w_obj):
+w_typ = make_frozendict(space)
+print 'exact', w_typ, w_obj
+return space.is_w(space.type(w_obj), w_typ)
+
diff --git a/pypy/module/cpyext/test/test_dictobject.py 
b/pypy/module/cpyext/test/test_dictobject.py
--- a/pypy/module/cpyext/test/test_dictobject.py
+++ b/pypy/module/cpyext/test/test_dictobject.py
@@ -1,8 +1,10 @@
+import py
 from rpython.rtyper.lltypesystem import rffi, lltype
 from pypy.module.cpyext.test.test_api import BaseApiTest
 from pypy.module.cpyext.api import Py_ssize_tP, PyObjectP
 from pypy.module.cpyext.pyobject import make_ref, from_ref
 from pypy.interpreter.error import OperationError
+from pypy.module.cpyext.test.test_cpyext import AppTestCpythonExtensionBase
 
 class TestDictObject(BaseApiTest):
 def test_dict(self, space, api):
@@ -159,7 +161,6 @@
 
 def test_dictproxy(self, space, api):
 w_dict = space.sys.get('modules')
-w_proxy = api.PyDictProxy_New(w_dict)
 assert space.contains_w(w_proxy, space.wrap('sys'))
 raises(OperationError, space.setitem,
w_proxy, space.wrap('sys'), space.w_None)
@@ -167,6 +168,40 @@
w_proxy, space.wrap('sys'))
 raises(OperationError, space.call_method, w_proxy, 'clear')
 
-def test_dictproxytype(self, space, api):
-# XXX test PyDictProxy_Type, currently space.NotImplemented
-assert False
+
+@py.test.mark.xfail(reason='make_frozendict memoize only works translated')
+def test_dictproxy(self, space, api):
+w_proxy = api.PyDictProxy_New(w_dict)
+assert api.PyDictProxy_Check(w_proxy)
+
+class AppTestDictObject(AppTestCpythonExtensionBase):
+#@py.test.mark.xfail(reason='make_frozendict memoize only works 
translated')
+def test_dictproxytype(self):
+module = self.import_extension('foo', [
+("dict_proxy", "METH_VARARGS",
+ """
+ PyObject * dict;
+ PyObject * proxydict;
+ int i;
+ if (!PyArg_ParseTuple(args, "O", &dict))
+ return NULL;
+ proxydict = PyDictProxy_New(dict);
+ Py_DECREF(dict);
+ /* when memoize works untranslated, add these tests
+ if (!PyDictProxy_Check(proxydict)) {
+Py_DECREF(proxydict);
+PyErr_SetNone(PyExc_ValueError);
+return NULL;
+ } 
+ if (!PyDictProxy_CheckExact(proxydict)) {
+Py_DECREF(proxydict);
+PyErr_SetNone(PyExc_ValueError);
+return NULL;
+ }
+ */
+ i = PyObject_Size(proxydict);
+ Py_DECREF(proxydict);
+ return PyLong_FromLong(i); 
+ """),
+])
+assert module.dict_proxy({'a': 1, 'b': 2}) == 2
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit

[pypy-commit] pypy default: (cfbolz, arigato, plan_rich) progress on fixing the new introduced bug in ll2ctypes, added two tests

2016-02-22 Thread plan_rich

Author: Richard Plangger 
Branch: 
Changeset: r82391:fd5943132b71
Date: 2016-02-22 14:15 +0100
http://bitbucket.org/pypy/pypy/changeset/fd5943132b71/

Log:(cfbolz, arigato, plan_rich) progress on fixing the new introduced
bug in ll2ctypes, added two tests

diff --git a/rpython/rtyper/lltypesystem/ll2ctypes.py 
b/rpython/rtyper/lltypesystem/ll2ctypes.py
--- a/rpython/rtyper/lltypesystem/ll2ctypes.py
+++ b/rpython/rtyper/lltypesystem/ll2ctypes.py
@@ -426,7 +426,12 @@
 else:
 n = None
 cstruct = cls._malloc(n)
-add_storage(container, _struct_mixin, ctypes.pointer(cstruct))
+
+if isinstance(container, lltype._fixedsizearray):
+cls_mixin = _fixedsizedarray_mixin
+else:
+cls_mixin = _struct_mixin
+add_storage(container, cls_mixin, ctypes.pointer(cstruct))
 
 if delayed_converters is None:
 delayed_converters_was_None = True
@@ -506,7 +511,11 @@
 def struct_use_ctypes_storage(container, ctypes_storage):
 STRUCT = container._TYPE
 assert isinstance(STRUCT, lltype.Struct)
-add_storage(container, _struct_mixin, ctypes_storage)
+if isinstance(container, lltype._fixedsizearray):
+cls_mixin = _fixedsizedarray_mixin
+else:
+cls_mixin = _struct_mixin
+add_storage(container, cls_mixin, ctypes_storage)
 remove_regular_struct_content(container)
 for field_name in STRUCT._names:
 FIELDTYPE = getattr(STRUCT, field_name)
@@ -645,11 +654,44 @@
 cobj = lltype2ctypes(value)
 setattr(self._storage.contents, field_name, cobj)
 
+class _fixedsizedarray_mixin(_parentable_mixin):
+"""Mixin added to _fixedsizearray containers when they become 
ctypes-based."""
+__slots__ = ()
+
+def __getattr__(self, field_name):
+if hasattr(self, '_items'):
+obj = lltype._fixedsizearray.__getattr__.im_func(self, field_name)
+return obj
+else:
+cobj = getattr(self._storage.contents, field_name)
+T = getattr(self._TYPE, field_name)
+return ctypes2lltype(T, cobj)
+
+def __setattr__(self, field_name, value):
+if field_name.startswith('_'):
+object.__setattr__(self, field_name, value)  # '_xxx' attributes
+else:
+cobj = lltype2ctypes(value)
+if hasattr(self, '_items'):
+lltype._fixedsizearray.__setattr__.im_func(self, field_name, 
cobj)
+else:
+setattr(self._storage.contents, field_name, cobj)
+
+
 def getitem(self, index, uninitialized_ok=False):
-return getattr(self, "item%s" % index)
+if hasattr(self, '_items'):
+obj = lltype._fixedsizearray.getitem.im_func(self, 
+ index, uninitialized_ok=uninitialized_ok)
+return obj
+else:
+return getattr(self, 'item%d' % index)
 
 def setitem(self, index, value):
-setattr(self, "item%s" % index, value)
+cobj = lltype2ctypes(value)
+if hasattr(self, '_items'):
+lltype._fixedsizearray.setitem.im_func(self, index, value)
+else:
+setattr(self, 'item%d' % index, cobj)
 
 class _array_mixin(_parentable_mixin):
 """Mixin added to _array containers when they become ctypes-based."""
diff --git a/rpython/rtyper/lltypesystem/test/test_ll2ctypes.py 
b/rpython/rtyper/lltypesystem/test/test_ll2ctypes.py
--- a/rpython/rtyper/lltypesystem/test/test_ll2ctypes.py
+++ b/rpython/rtyper/lltypesystem/test/test_ll2ctypes.py
@@ -1461,3 +1461,20 @@
 assert a[3].a == 17
 #lltype.free(a, flavor='raw')
 py.test.skip("free() not working correctly here...")
+
+def test_fixedsizedarray_to_ctypes(self):
+T = lltype.Ptr(rffi.CFixedArray(rffi.INT, 1))
+inst = lltype.malloc(T.TO, flavor='raw')
+inst[0] = rffi.cast(rffi.INT, 42)
+assert inst[0] == 42
+cinst = lltype2ctypes(inst)
+assert rffi.cast(lltype.Signed, inst[0]) == 42
+assert cinst.contents.item0 == 42
+lltype.free(inst, flavor='raw')
+
+def test_fixedsizedarray_to_ctypes(self):
+T = lltype.Ptr(rffi.CFixedArray(rffi.CHAR, 123))
+inst = lltype.malloc(T.TO, flavor='raw', zero=True)
+cinst = lltype2ctypes(inst)
+assert cinst.contents.item0 == 0
+lltype.free(inst, flavor='raw')
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit

[pypy-commit] pypy py3.3: Remove unneeded and broken hack.

2016-02-22 Thread mjacob

Author: Manuel Jacob 
Branch: py3.3
Changeset: r82392:9b79c1399d57
Date: 2016-02-22 16:36 +0100
http://bitbucket.org/pypy/pypy/changeset/9b79c1399d57/

Log:Remove unneeded and broken hack.

For some reason the hack resulted in later tests failing (e.g.
AppTestUnicodeFormat.test_oldstyle_custom_format()).

diff --git a/pypy/objspace/std/test/test_iterobject.py 
b/pypy/objspace/std/test/test_iterobject.py
--- a/pypy/objspace/std/test/test_iterobject.py
+++ b/pypy/objspace/std/test/test_iterobject.py
@@ -91,8 +91,6 @@
 raises(TypeError, len, iter(iterable))
 
 def test_no_len_on_UserList_iter_reversed(self):
-import sys, collections.abc
-sys.modules['collections'] = collections.abc
 class UserList(object):
 def __init__(self, i):
 self.i = i
@@ -101,7 +99,6 @@
 iterable = UserList([1,2,3,4])
 raises(TypeError, len, iter(iterable))
 raises(TypeError, reversed, iterable)
-del sys.modules['collections']
 
 def test_no_len_on_UserList_reversed(self):
 iterable = [1,2,3,4]
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit

[pypy-commit] pypy stmgc-c8: Partial merge with default

2016-02-22 Thread Raemi

Author: Remi Meier 
Branch: stmgc-c8
Changeset: r82394:616abaac8a22
Date: 2016-02-21 16:32 +0100
http://bitbucket.org/pypy/pypy/changeset/616abaac8a22/

Log:Partial merge with default

diff --git a/pypy/doc/whatsnew-head.rst b/pypy/doc/whatsnew-head.rst
--- a/pypy/doc/whatsnew-head.rst
+++ b/pypy/doc/whatsnew-head.rst
@@ -73,3 +73,7 @@
 Move wrappers for OS functions from `rpython/rtyper` to `rpython/rlib` and 
 turn them into regular RPython functions. Most RPython-compatible `os.*` 
 functions are now directly accessible as `rpython.rposix.*`.
+
+.. branch: always-enable-gil
+
+Simplify a bit the GIL handling in non-jitted code.  Fixes issue #2205.
diff --git a/pypy/module/_cffi_backend/call_python.py 
b/pypy/module/_cffi_backend/call_python.py
--- a/pypy/module/_cffi_backend/call_python.py
+++ b/pypy/module/_cffi_backend/call_python.py
@@ -40,10 +40,9 @@
at least 8 bytes in size.
 """
 from pypy.module._cffi_backend.ccallback import reveal_callback
+from rpython.rlib import rgil
 
-after = rffi.aroundstate.after
-if after:
-after()
+rgil.acquire()
 rffi.stackcounter.stacks_counter += 1
 llop.gc_stack_bottom(lltype.Void)   # marker for trackgcroot.py
 
@@ -71,9 +70,7 @@
 cerrno._errno_before(rffi.RFFI_ERR_ALL | rffi.RFFI_ALT_ERRNO)
 
 rffi.stackcounter.stacks_counter -= 1
-before = rffi.aroundstate.before
-if before:
-before()
+rgil.release()
 
 
 def get_ll_cffi_call_python():
diff --git a/pypy/module/cpyext/api.py b/pypy/module/cpyext/api.py
--- a/pypy/module/cpyext/api.py
+++ b/pypy/module/cpyext/api.py
@@ -602,6 +602,7 @@
 # Make the wrapper for the cases (1) and (2)
 def make_wrapper(space, callable, gil=None):
 "NOT_RPYTHON"
+from rpython.rlib import rgil
 names = callable.api_func.argnames
 argtypes_enum_ui = 
unrolling_iterable(enumerate(zip(callable.api_func.argtypes,
 [name.startswith("w_") for name in names])))
@@ -617,9 +618,7 @@
 # we hope that malloc removal removes the newtuple() that is
 # inserted exactly here by the varargs specializer
 if gil_acquire:
-after = rffi.aroundstate.after
-if after:
-after()
+rgil.acquire()
 rffi.stackcounter.stacks_counter += 1
 llop.gc_stack_bottom(lltype.Void)   # marker for trackgcroot.py
 retval = fatal_value
@@ -692,9 +691,7 @@
 pypy_debug_catch_fatal_exception()
 rffi.stackcounter.stacks_counter -= 1
 if gil_release:
-before = rffi.aroundstate.before
-if before:
-before()
+rgil.release()
 return retval
 callable._always_inline_ = 'try'
 wrapper.__name__ = "wrapper for %r" % (callable, )
diff --git a/pypy/module/signal/__init__.py b/pypy/module/signal/__init__.py
--- a/pypy/module/signal/__init__.py
+++ b/pypy/module/signal/__init__.py
@@ -48,3 +48,6 @@
   use_bytecode_counter=False)
 space.actionflag.__class__ = interp_signal.SignalActionFlag
 # xxx yes I know the previous line is a hack
+
+def startup(self, space):
+space.check_signal_action.startup(space)
diff --git a/pypy/module/signal/interp_signal.py 
b/pypy/module/signal/interp_signal.py
--- a/pypy/module/signal/interp_signal.py
+++ b/pypy/module/signal/interp_signal.py
@@ -67,19 +67,25 @@
 AsyncAction.__init__(self, space)
 self.pending_signal = -1
 self.fire_in_another_thread = False
-if self.space.config.objspace.usemodules.thread:
-from pypy.module.thread import gil
-gil.after_thread_switch = self._after_thread_switch
+#
+@rgc.no_collect
+def _after_thread_switch():
+if self.fire_in_another_thread:
+if self.space.threadlocals.signals_enabled():
+self.fire_in_another_thread = False
+self.space.actionflag.rearm_ticker()
+# this occurs when we just switched to the main thread
+# and there is a signal pending: we force the ticker to
+# -1, which should ensure perform() is called quickly.
+self._after_thread_switch = _after_thread_switch
+# ^^^ so that 'self._after_thread_switch' can be annotated as a
+# constant
 
-@rgc.no_collect
-def _after_thread_switch(self):
-if self.fire_in_another_thread:
-if self.space.threadlocals.signals_enabled():
-self.fire_in_another_thread = False
-self.space.actionflag.rearm_ticker()
-# this occurs when we just switched to the main thread
-# and there is a signal pending: we force the ticker to
-# -1, which should ensure perform() is called quickly.
+def startup(self, space):
+# this is translated
+if space.config.objspace.usemodules.thre

[pypy-commit] pypy stmgc-c8: Merge with default

2016-02-22 Thread Raemi

Author: Remi Meier 
Branch: stmgc-c8
Changeset: r82393:0ae4b8d1eed4
Date: 2016-02-21 15:43 +0100
http://bitbucket.org/pypy/pypy/changeset/0ae4b8d1eed4/

Log:Merge with default

diff too long, truncating to 2000 out of 24060 lines

diff --git a/.hgtags b/.hgtags
--- a/.hgtags
+++ b/.hgtags
@@ -17,3 +17,4 @@
 295ee98b69288471b0fcf2e0ede82ce5209eb90b release-2.6.0
 f3ad1e1e1d6215e20d34bb65ab85ff9188c9f559 release-2.6.1
 850edf14b2c75573720f59e95767335fb1affe55 release-4.0.0
+5f8302b8bf9f53056e40426f10c72151564e5b19 release-4.0.1
diff --git a/Makefile b/Makefile
--- a/Makefile
+++ b/Makefile
@@ -1,5 +1,5 @@
 
-all: pypy-c
+all: pypy-c cffi_imports
 
 PYPY_EXECUTABLE := $(shell which pypy)
 URAM := $(shell python -c "import sys; print 4.5 if sys.maxint>1<<32 else 2.5")
@@ -10,6 +10,8 @@
 RUNINTERP = $(PYPY_EXECUTABLE)
 endif
 
+.PHONY: cffi_imports
+
 pypy-c:
@echo
@echo 
""
@@ -36,3 +38,6 @@
 # replaced with an opaque --jobserver option by the time this Makefile
 # runs.  We cannot get their original value either:
 # http://lists.gnu.org/archive/html/help-make/2010-08/msg00106.html
+
+cffi_imports:
+   PYTHONPATH=. ./pypy-c pypy/tool/build_cffi_imports.py
diff --git a/lib-python/2.7/collections.py b/lib-python/2.7/collections.py
--- a/lib-python/2.7/collections.py
+++ b/lib-python/2.7/collections.py
@@ -18,9 +18,9 @@
 assert '__pypy__' not in _sys.builtin_module_names
 newdict = lambda _ : {}
 try:
-from __pypy__ import reversed_dict
+from __pypy__ import reversed_dict as _reversed_dict
 except ImportError:
-reversed_dict = lambda d: reversed(d.keys())
+_reversed_dict = None # don't have ordered dicts
 
 try:
 from thread import get_ident as _get_ident
@@ -46,7 +46,7 @@
 '''
 
 def __reversed__(self):
-return reversed_dict(self)
+return _reversed_dict(self)
 
 def popitem(self, last=True):
 '''od.popitem() -> (k, v), return and remove a (key, value) pair.
@@ -116,6 +116,178 @@
 return ItemsView(self)
 
 
+def _compat_with_unordered_dicts():
+# This returns the methods needed in OrderedDict in case the base
+# 'dict' class is not actually ordered, like on top of CPython or
+# old PyPy or PyPy-STM.
+
+# = Original comments and code follows  =
+# = The unmodified methods are not repeated =
+
+# An inherited dict maps keys to values.
+# The inherited dict provides __getitem__, __len__, __contains__, and get.
+# The remaining methods are order-aware.
+# Big-O running times for all methods are the same as regular dictionaries.
+
+# The internal self.__map dict maps keys to links in a doubly linked list.
+# The circular doubly linked list starts and ends with a sentinel element.
+# The sentinel element never gets deleted (this simplifies the algorithm).
+# Each link is stored as a list of length three:  [PREV, NEXT, KEY].
+
+def __init__(self, *args, **kwds):
+'''Initialize an ordered dictionary.  The signature is the same as
+regular dictionaries, but keyword arguments are not recommended because
+their insertion order is arbitrary.
+
+'''
+if len(args) > 1:
+raise TypeError('expected at most 1 arguments, got %d' % len(args))
+try:
+self.__root
+except AttributeError:
+self.__root = root = [] # sentinel node
+root[:] = [root, root, None]
+self.__map = {}
+self.__update(*args, **kwds)
+
+def __setitem__(self, key, value, dict_setitem=dict.__setitem__):
+'od.__setitem__(i, y) <==> od[i]=y'
+# Setting a new item creates a new link at the end of the linked list,
+# and the inherited dictionary is updated with the new key/value pair.
+if key not in self:
+root = self.__root
+last = root[0]
+last[1] = root[0] = self.__map[key] = [last, root, key]
+return dict_setitem(self, key, value)
+
+def __delitem__(self, key, dict_delitem=dict.__delitem__):
+'od.__delitem__(y) <==> del od[y]'
+# Deleting an existing item uses self.__map to find the link which gets
+# removed by updating the links in the predecessor and successor nodes.
+dict_delitem(self, key)
+link_prev, link_next, _ = self.__map.pop(key)
+link_prev[1] = link_next# update 
link_prev[NEXT]
+link_next[0] = link_prev# update 
link_next[PREV]
+
+def __iter__(self):
+'od.__iter__() <==> iter(od)'
+# Traverse the linked list in order.
+root = self.__root
+curr = root[1]  # start at the first 
node
+while curr is not root:
+yield curr[2]   # yield the curr[KEY]
+curr = curr[1]

[pypy-commit] pypy stmgc-c8: fix several things from the merges (wip)

2016-02-22 Thread Raemi

Author: Remi Meier 
Branch: stmgc-c8
Changeset: r82395:8c5f6193d317
Date: 2016-02-22 16:26 +0100
http://bitbucket.org/pypy/pypy/changeset/8c5f6193d317/

Log:fix several things from the merges (wip)

mostly always-enable-gil and memop-simplify related

diff --git a/rpython/jit/backend/llsupport/assembler.py 
b/rpython/jit/backend/llsupport/assembler.py
--- a/rpython/jit/backend/llsupport/assembler.py
+++ b/rpython/jit/backend/llsupport/assembler.py
@@ -432,8 +432,12 @@
 # 'rpy_fastgil' contains only zero or non-zero, and this is only
 # called when the old value stored in 'rpy_fastgil' was non-zero
 # (i.e. still locked, must wait with the regular mutex)
-from rpython.rlib import rgil
-rgil.acquire()
+if rgc.stm_is_enabled():
+from rpython.rlib import rstm
+rstm.after_external_call()
+else:
+from rpython.rlib import rgil
+rgil.acquire()
 
 _REACQGIL0_FUNC = lltype.Ptr(lltype.FuncType([], lltype.Void))
 _REACQGIL2_FUNC = lltype.Ptr(lltype.FuncType([rffi.CCHARP, lltype.Signed],
diff --git a/rpython/jit/backend/llsupport/gc.py 
b/rpython/jit/backend/llsupport/gc.py
--- a/rpython/jit/backend/llsupport/gc.py
+++ b/rpython/jit/backend/llsupport/gc.py
@@ -499,9 +499,10 @@
 
 def _initialize_for_tests(self):
 self.layoutbuilder = None
-self.fielddescr_tid = FieldDescr("test_tid",0,8,0)
+from rpython.jit.backend.llsupport.descr import FieldDescr
+self.fielddescr_tid = FieldDescr("test_tid", 0, 8, 0)
 if self.stm:
-self.fielddescr_stmflags = AbstractDescr()
+self.fielddescr_stmflags = FieldDescr("test_stmflags", 4, 4, 0)
 self.max_size_of_young_obj = 1000
 self.GCClass = None
 self.gcheaderbuilder = None
diff --git a/rpython/jit/backend/llsupport/rewrite.py 
b/rpython/jit/backend/llsupport/rewrite.py
--- a/rpython/jit/backend/llsupport/rewrite.py
+++ b/rpython/jit/backend/llsupport/rewrite.py
@@ -583,7 +583,6 @@
 self.gen_initialize_tid(frame, descrs.arraydescr.tid)
 # we need to explicitely zero all the gc fields, because
 # of the unusal malloc pattern
-
 length = self.emit_getfield(ConstInt(frame_info),
 descr=descrs.jfi_frame_depth, raw=True)
 self.emit_setfield(frame, self.c_zero,
@@ -603,30 +602,32 @@
 return self.get_box_replacement(frame)
 else:
 # jfi_frame_size not set in STM!
-length = ResOperation(rop.GETFIELD_RAW_I,
-  [history.ConstInt(frame_info)],
-  descr=descrs.jfi_frame_depth)
-self.emit_op(length)
+length = self.emit_getfield(ConstInt(frame_info),
+descr=descrs.jfi_frame_depth, raw=True)
+# ofs, size, sign = unpack_fielddescr(descrs.jfi_frame_depth)
+# if sign:
+# size = -size
+# args = [ConstInt(frame_info), ConstInt(0), ConstInt(1),
+# ConstInt(ofs), ConstInt(size)]
+# length = ResOperation(rop.GC_LOAD_INDEXED_I, args)
+# self.emit_op(length)
 frame = self.gen_malloc_nursery_varsize_frame(length)
 self.gen_initialize_tid(frame, descrs.arraydescr.tid)
 # we need to explicitely zero all the gc fields, because
 # of the unusal malloc pattern
-extra_ops = [
-ResOperation(rop.SETFIELD_GC, [frame, self.c_zero],
- descr=descrs.jf_extra_stack_depth),
-ResOperation(rop.SETFIELD_GC, [frame, self.c_null],
- descr=descrs.jf_savedata),
-ResOperation(rop.SETFIELD_GC, [frame, self.c_null],
- descr=descrs.jf_force_descr),
-ResOperation(rop.SETFIELD_GC, [frame, self.c_null],
- descr=descrs.jf_descr),
-ResOperation(rop.SETFIELD_GC, [frame, self.c_null],
- descr=descrs.jf_guard_exc),
-ResOperation(rop.SETFIELD_GC, [frame, self.c_null],
- descr=descrs.jf_forward),
-]
-for op in extra_ops:
-self.emit_op(op)
+self.emit_setfield(frame, self.c_zero,
+   descr=descrs.jf_extra_stack_depth)
+self.emit_setfield(frame, self.c_null,
+   descr=descrs.jf_savedata)
+self.emit_setfield(frame, self.c_null,
+   descr=descrs.jf_force_descr)
+self.emit_setfield(frame, self.c_null,
+   descr=descrs.jf_descr)
+self.emit_setfield(frame, self.c_null,
+   descr=descrs.jf_guard_exc)
+self.emit_setfiel

[pypy-commit] pypy stmgc-c8: readd changes lost during a merge

2016-02-22 Thread Raemi

Author: Remi Meier 
Branch: stmgc-c8
Changeset: r82396:97bc4a194b25
Date: 2016-02-22 16:38 +0100
http://bitbucket.org/pypy/pypy/changeset/97bc4a194b25/

Log:readd changes lost during a merge

diff --git a/rpython/jit/backend/x86/regalloc.py 
b/rpython/jit/backend/x86/regalloc.py
--- a/rpython/jit/backend/x86/regalloc.py
+++ b/rpython/jit/backend/x86/regalloc.py
@@ -1030,9 +1030,22 @@
 gc_ll_descr.get_nursery_top_addr(),
 lengthloc, itemsize, maxlength, gcmap, arraydescr)
 
+def extract_raw_stm_location(self):
+if self.stm_location is not None:
+num = rffi.cast(lltype.Signed, self.stm_location.num)
+ref = rffi.cast(lltype.Signed, self.stm_location.ref)
+else:
+num = 0
+ref = 0
+return (num, ref)
+
+def get_empty_gcmap(self, frame_depth):
+return allocate_gcmap(self.assembler, frame_depth,
+  JITFRAME_FIXED_SIZE)
+
 def get_gcmap(self, forbidden_regs=[], noregs=False):
 frame_depth = self.fm.get_frame_depth()
-gcmap = allocate_gcmap(self.assembler, frame_depth, 
JITFRAME_FIXED_SIZE)
+gcmap = self.get_empty_gcmap(frame_depth)
 for box, loc in self.rm.reg_bindings.iteritems():
 if loc in forbidden_regs:
 continue
@@ -1187,6 +1200,8 @@
 dstaddr_loc = self.rm.force_allocate_reg(dstaddr_box, forbidden_vars)
 self._gen_address_inside_string(base_loc, ofs_loc, dstaddr_loc,
 is_unicode=is_unicode)
+# for stm: convert the addresses from %gs-based to linear
+self.assembler.convert_addresses_to_linear(srcaddr_loc, dstaddr_loc)
 # compute the length in bytes
 length_box = args[4]
 length_loc = self.loc(length_box)
@@ -1288,6 +1303,11 @@
 if isinstance(loc, FrameLoc):
 self.fm.hint_frame_pos[box] = self.fm.get_loc_index(loc)
 
+
+def consider_stm_should_break_transaction(self, op):
+resloc = self.force_allocate_reg_or_cc(op)
+self.perform(op, [], resloc)
+
 def consider_jump(self, op):
 assembler = self.assembler
 assert self.jump_target_descr is None
@@ -1405,6 +1425,20 @@
 def consider_keepalive(self, op):
 pass
 
+def consider_stm_read(self, op):
+loc_src = self.loc(op.getarg(0))
+self.possibly_free_vars_for_op(op)
+# this will get in 'loc_tmp' a register that is the same as
+# 'loc_src' if the op.getarg(0) is freed now
+if (isinstance(loc_src, ImmedLoc) and
+rx86.fits_in_32bits(loc_src.value >> 4)):
+loc_tmp = None
+else:
+tmpxvar = TempVar()
+loc_tmp = self.rm.force_allocate_reg(tmpxvar)
+self.rm.possibly_free_var(tmpxvar)
+self.perform_discard(op, [loc_src, loc_tmp])
+
 def consider_zero_array(self, op):
 itemsize, baseofs, _ = unpack_arraydescr(op.getdescr())
 length_box = op.getarg(2)
@@ -1442,6 +1476,8 @@
 dstaddr_loc, startindex_loc, itemsize_loc,
 base_loc, imm(baseofs))
 self.assembler.mc.LEA(dstaddr_loc, dst_addr)
+# for stm: convert the address from %gs-based to linear
+self.assembler.convert_addresses_to_linear(dstaddr_loc)
 #
 if constbytes >= 0:
 length_loc = imm(constbytes)
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit

[pypy-commit] pypy py3.3: cpyext: Add defines for PyExc_OSError aliases.

2016-02-22 Thread mjacob

Author: Manuel Jacob 
Branch: py3.3
Changeset: r82397:573e529840d7
Date: 2016-02-22 17:12 +0100
http://bitbucket.org/pypy/pypy/changeset/573e529840d7/

Log:cpyext: Add defines for PyExc_OSError aliases.

diff --git a/pypy/module/cpyext/include/Python.h 
b/pypy/module/cpyext/include/Python.h
--- a/pypy/module/cpyext/include/Python.h
+++ b/pypy/module/cpyext/include/Python.h
@@ -80,6 +80,11 @@
 
 #include 
 
+#define PyExc_EnvironmentError PyExc_OSError
+#define PyExc_IOError PyExc_OSError
+// TODO: fix windows support
+// #define PyExc_WindowsError PyExc_OSError
+
 #include "patchlevel.h"
 #include "pyconfig.h"
 
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit

[pypy-commit] buildbot default: run own tests in a virtualenv, document the virtualenv requirement

2016-02-22 Thread mattip

Author: mattip 
Branch: 
Changeset: r988:d0965d2a7c68
Date: 2016-02-22 17:03 +0100
http://bitbucket.org/pypy/buildbot/changeset/d0965d2a7c68/

Log:run own tests in a virtualenv, document the virtualenv requirement

diff --git a/README_BUILDSLAVE b/README_BUILDSLAVE
--- a/README_BUILDSLAVE
+++ b/README_BUILDSLAVE
@@ -2,7 +2,8 @@
 ==
 
 The recommended setup is to have a "pypy" in your path that will translate and
-a "python" (cpython) in your path that will run the test suites.
+a "python" (cpython) in your path that will run the test suites. You will also
+need a "virtualenv" in your path that will create a cpython virtualenv.
 
 Then you will need to install the ``buildbot-slave`` package, which will
 install many other packages like twised, so you may prefer to run the
diff --git a/bot2/pypybuildbot/builds.py b/bot2/pypybuildbot/builds.py
--- a/bot2/pypybuildbot/builds.py
+++ b/bot2/pypybuildbot/builds.py
@@ -406,7 +406,7 @@
 # If target_tmpdir is empty, crash.
 tmp_or_crazy = '%(prop:target_tmpdir:-crazy/name/so/mkdir/fails/)s'
 pytest = "pytest"
-factory.addStep(PytestCmd( 
+factory.addStep(ShellCmd( 
 description="mkdir for tests",
 command=['python', '-c', Interpolate("import os;  os.mkdir(r'" + \
 tmp_or_crazy + pytest + "') if not os.path.exists(r'" + \
@@ -505,7 +505,7 @@
 # If target_tmpdir is empty, crash.
 tmp_or_crazy = '%(prop:target_tmpdir:-crazy/name/so/mkdir/fails/)s'
 pytest = "pytest"
-self.addStep(PytestCmd( 
+self.addStep(ShellCmd( 
 description="mkdir for tests",
 command=['python', '-c', Interpolate("import os;  os.mkdir(r'" + \
 tmp_or_crazy + pytest + "') if not os.path.exists(r'" 
+ \
@@ -527,9 +527,26 @@
 haltOnFailure=False,
 ))
 
+self.addStep(ShellCmd(
+description="create virtualenv for tests",
+command=['virtualenv', 'virt_test'],
+haltOnFailure=True,
+))
+
+if platform == 'win32':
+virt_python = 'virt_test/Scripts/python'
+else:
+virt_python = 'virt_test/bin/python'
+
+self.addStep(ShellCmd(
+description="install requirments to virtual environment",
+command=[virt_python, '-mpip', 'install', '-r', 
'requirements.txt'],
+haltOnFailure=True,
+))
+
 self.addStep(PytestCmd(
 description="pytest pypy",
-command=["python", "testrunner/runner.py",
+command=[virt_python, "testrunner/runner.py",
  "--logfile=testrun.log",
  "--config=pypy/testrunner_cfg.py",
  "--config=~/machine_cfg.py",
@@ -544,7 +561,7 @@
 
 self.addStep(PytestCmd(
 description="pytest rpython",
-command=["python", "testrunner/runner.py",
+command=[virt_python, "testrunner/runner.py",
  "--logfile=testrun.log",
  "--config=pypy/testrunner_cfg.py",
  "--config=~/machine_cfg.py",
@@ -707,6 +724,7 @@
 
 self.addStep(Translate(translationArgs, targetArgs,
interpreter=interpreter))
+
 name = build_name(platform, flags=translationArgs)
 self.addStep(ShellCmd(
 description="compress pypy-c",
@@ -809,18 +827,14 @@
  '--upload-executable', 'pypy-c' + postfix,
  '--upload-project', 'PyPy',
  '--revision', WithProperties('%(got_revision)s'),
- # HACK: branches are not uploaded any more, so that
- # codespeed will display it, even if not "default"
- #'--branch', WithProperties('%(branch)s'),
+ '--branch', WithProperties('%(branch)s'),
  '--upload-urls', 'http://speed.pypy.org/',
  '--upload-baseline',
  '--upload-baseline-executable', 'pypy-c-jit' + postfix,
  '--upload-baseline-project', 'PyPy',
  '--upload-baseline-revision',
  WithProperties('%(got_revision)s'),
- # HACK: branches are not uploaded any more, so that
- # codespeed will display it, even if not "default"
- #'--upload-baseline-branch', WithProperties('%(branch)s'),
+ '--upload-baseline-branch', WithProperties('%(branch)s'),
  '--upload-baseline-urls', 'http://speed.pypy.org/',
  ],
 workdir='./benchmarks',
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit

[pypy-commit] pypy cpyext-gc-support-2: merge heads

2016-02-22 Thread arigo

Author: Armin Rigo 
Branch: cpyext-gc-support-2
Changeset: r82399:029c2b1fb8e6
Date: 2016-02-22 17:22 +0100
http://bitbucket.org/pypy/pypy/changeset/029c2b1fb8e6/

Log:merge heads

diff too long, truncating to 2000 out of 7656 lines

diff --git a/.hgignore b/.hgignore
--- a/.hgignore
+++ b/.hgignore
@@ -75,6 +75,7 @@
 ^lib_pypy/__pycache__$
 ^lib_pypy/ctypes_config_cache/_.+_cache\.py$
 ^lib_pypy/ctypes_config_cache/_.+_.+_\.py$
+^lib_pypy/_libmpdec/.+.o$
 ^rpython/translator/cli/query-descriptions$
 ^pypy/doc/discussion/.+\.html$
 ^include/.+\.h$
diff --git a/dotviewer/drawgraph.py b/dotviewer/drawgraph.py
--- a/dotviewer/drawgraph.py
+++ b/dotviewer/drawgraph.py
@@ -14,12 +14,661 @@
 FONT = os.path.join(this_dir, 'font', 'DroidSans.ttf')
 FIXEDFONT = os.path.join(this_dir, 'font', 'DroidSansMono.ttf')
 COLOR = {
-'black': (0,0,0),
-'white': (255,255,255),
-'red': (255,0,0),
-'green': (0,255,0),
-'blue': (0,0,255),
-'yellow': (255,255,0),
+'aliceblue': (240, 248, 255),
+'antiquewhite': (250, 235, 215),
+'antiquewhite1': (255, 239, 219),
+'antiquewhite2': (238, 223, 204),
+'antiquewhite3': (205, 192, 176),
+'antiquewhite4': (139, 131, 120),
+'aquamarine': (127, 255, 212),
+'aquamarine1': (127, 255, 212),
+'aquamarine2': (118, 238, 198),
+'aquamarine3': (102, 205, 170),
+'aquamarine4': (69, 139, 116),
+'azure': (240, 255, 255),
+'azure1': (240, 255, 255),
+'azure2': (224, 238, 238),
+'azure3': (193, 205, 205),
+'azure4': (131, 139, 139),
+'beige': (245, 245, 220),
+'bisque': (255, 228, 196),
+'bisque1': (255, 228, 196),
+'bisque2': (238, 213, 183),
+'bisque3': (205, 183, 158),
+'bisque4': (139, 125, 107),
+'black': (0, 0, 0),
+'blanchedalmond': (255, 235, 205),
+'blue': (0, 0, 255),
+'blue1': (0, 0, 255),
+'blue2': (0, 0, 238),
+'blue3': (0, 0, 205),
+'blue4': (0, 0, 139),
+'blueviolet': (138, 43, 226),
+'brown': (165, 42, 42),
+'brown1': (255, 64, 64),
+'brown2': (238, 59, 59),
+'brown3': (205, 51, 51),
+'brown4': (139, 35, 35),
+'burlywood': (222, 184, 135),
+'burlywood1': (255, 211, 155),
+'burlywood2': (238, 197, 145),
+'burlywood3': (205, 170, 125),
+'burlywood4': (139, 115, 85),
+'cadetblue': (95, 158, 160),
+'cadetblue1': (152, 245, 255),
+'cadetblue2': (142, 229, 238),
+'cadetblue3': (122, 197, 205),
+'cadetblue4': (83, 134, 139),
+'chartreuse': (127, 255, 0),
+'chartreuse1': (127, 255, 0),
+'chartreuse2': (118, 238, 0),
+'chartreuse3': (102, 205, 0),
+'chartreuse4': (69, 139, 0),
+'chocolate': (210, 105, 30),
+'chocolate1': (255, 127, 36),
+'chocolate2': (238, 118, 33),
+'chocolate3': (205, 102, 29),
+'chocolate4': (139, 69, 19),
+'coral': (255, 127, 80),
+'coral1': (255, 114, 86),
+'coral2': (238, 106, 80),
+'coral3': (205, 91, 69),
+'coral4': (139, 62, 47),
+'cornflowerblue': (100, 149, 237),
+'cornsilk': (255, 248, 220),
+'cornsilk1': (255, 248, 220),
+'cornsilk2': (238, 232, 205),
+'cornsilk3': (205, 200, 177),
+'cornsilk4': (139, 136, 120),
+'crimson': (220, 20, 60),
+'cyan': (0, 255, 255),
+'cyan1': (0, 255, 255),
+'cyan2': (0, 238, 238),
+'cyan3': (0, 205, 205),
+'cyan4': (0, 139, 139),
+'darkgoldenrod': (184, 134, 11),
+'darkgoldenrod1': (255, 185, 15),
+'darkgoldenrod2': (238, 173, 14),
+'darkgoldenrod3': (205, 149, 12),
+'darkgoldenrod4': (139, 101, 8),
+'darkgreen': (0, 100, 0),
+'darkkhaki': (189, 183, 107),
+'darkolivegreen': (85, 107, 47),
+'darkolivegreen1': (202, 255, 112),
+'darkolivegreen2': (188, 238, 104),
+'darkolivegreen3': (162, 205, 90),
+'darkolivegreen4': (110, 139, 61),
+'darkorange': (255, 140, 0),
+'darkorange1': (255, 127, 0),
+'darkorange2': (238, 118, 0),
+'darkorange3': (205, 102, 0),
+'darkorange4': (139, 69, 0),
+'darkorchid': (153, 50, 204),
+'darkorchid1': (191, 62, 255),
+'darkorchid2': (178, 58, 238),
+'darkorchid3': (154, 50, 205),
+'darkorchid4': (104, 34, 139),
+'darksalmon': (233, 150, 122),
+'darkseagreen': (143, 188, 143),
+'darkseagreen1': (193, 255, 193),
+'darkseagreen2': (180, 238, 180),
+'darkseagreen3': (155, 205, 155),
+'darkseagreen4': (105, 139, 105),
+'darkslateblue': (72, 61, 139),
+'darkslategray': (47, 79, 79),
+'darkslategray1': (151, 255, 255),
+'darkslategray2': (141, 238, 238),
+'darkslategray3': (121, 205, 205),
+'darkslategray4': (82, 139, 139),
+'darkslategrey': (47, 79, 79),
+'darkturquoise': (0, 206, 209),
+'darkviolet': (148, 0, 211),
+'deeppink': (255, 20, 147),
+'deeppink1': (255, 20, 147),
+'deeppink2': (238, 18, 137),
+'deeppink3': (205, 16, 118),
+'deeppink4': (139, 10, 80),
+'deepskyblue': (0, 191, 255),
+'deepskyblue1': (0, 191, 255),
+'d

[pypy-commit] pypy cpyext-gc-support-2: (ronan, arigo)

2016-02-22 Thread arigo

Author: Armin Rigo 
Branch: cpyext-gc-support-2
Changeset: r82398:e4cdc09604fe
Date: 2016-02-22 17:20 +0100
http://bitbucket.org/pypy/pypy/changeset/e4cdc09604fe/

Log:(ronan, arigo)

refactor and made more explicit the "layout" of W_TypeObjects, to be
used by cpyext

diff --git a/pypy/interpreter/typedef.py b/pypy/interpreter/typedef.py
--- a/pypy/interpreter/typedef.py
+++ b/pypy/interpreter/typedef.py
@@ -262,7 +262,7 @@
 def user_setup(self, space, w_subtype):
 self.space = space
 self.w__class__ = w_subtype
-self.user_setup_slots(w_subtype.nslots)
+self.user_setup_slots(w_subtype.layout.nslots)
 
 def user_setup_slots(self, nslots):
 assert nslots == 0
diff --git a/pypy/module/cpyext/typeobject.py b/pypy/module/cpyext/typeobject.py
--- a/pypy/module/cpyext/typeobject.py
+++ b/pypy/module/cpyext/typeobject.py
@@ -523,7 +523,7 @@
 def best_base(space, bases_w):
 if not bases_w:
 return None
-return find_best_base(space, bases_w)
+return find_best_base(bases_w)
 
 def inherit_slots(space, pto, w_base):
 # XXX missing: nearly everything
diff --git a/pypy/objspace/std/objspace.py b/pypy/objspace/std/objspace.py
--- a/pypy/objspace/std/objspace.py
+++ b/pypy/objspace/std/objspace.py
@@ -359,7 +359,8 @@
 subcls = get_subclass_of_correct_size(self, cls, w_subtype)
 else:
 subcls = get_unique_interplevel_subclass(
-self.config, cls, w_subtype.hasdict, w_subtype.nslots 
!= 0,
+self.config, cls, w_subtype.hasdict,
+w_subtype.layout.nslots != 0,
 w_subtype.needsdel, w_subtype.weakrefable)
 instance = instantiate(subcls)
 assert isinstance(instance, cls)
diff --git a/pypy/objspace/std/transparent.py b/pypy/objspace/std/transparent.py
--- a/pypy/objspace/std/transparent.py
+++ b/pypy/objspace/std/transparent.py
@@ -62,7 +62,7 @@
 return W_TransparentGenerator(space, w_type, w_controller)
 if space.is_true(space.issubtype(w_type, 
space.gettypeobject(PyCode.typedef))):
 return W_TransparentCode(space, w_type, w_controller)
-if w_type.instancetypedef is space.w_object.instancetypedef:
+if w_type.layout.typedef is space.w_object.layout.typedef:
 return W_Transparent(space, w_type, w_controller)
 else:
 raise OperationError(space.w_TypeError, space.wrap("type expected as 
first argument"))
diff --git a/pypy/objspace/std/typeobject.py b/pypy/objspace/std/typeobject.py
--- a/pypy/objspace/std/typeobject.py
+++ b/pypy/objspace/std/typeobject.py
@@ -87,6 +87,29 @@
 for i in range(len(self.lookup_where)):
 self.lookup_where[i] = None_None
 
+
+class Layout(object):
+"""A Layout is attached to every W_TypeObject to represent the
+layout of instances.  Some W_TypeObjects share the same layout.
+If a W_TypeObject is a base of another, then the layout of
+the first is either the same or a parent layout of the second.
+The Layouts have single inheritance, unlike W_TypeObjects.
+"""
+_immutable_ = True
+
+def __init__(self, typedef, nslots, base_layout=None):
+self.typedef = typedef
+self.nslots = nslots
+self.base_layout = base_layout
+
+def issublayout(self, parent):
+while self is not parent:
+self = self.base_layout
+if self is None:
+return False
+return True
+
+
 # possible values of compares_by_identity_status
 UNKNOWN = 0
 COMPARES_BY_IDENTITY = 1
@@ -106,8 +129,7 @@
   'needsdel',
   'weakrefable',
   'hasdict',
-  'nslots',
-  'instancetypedef',
+  'layout',
   'terminator',
   '_version_tag?',
   'name?',
@@ -131,7 +153,6 @@
 w_self.name = name
 w_self.bases_w = bases_w
 w_self.dict_w = dict_w
-w_self.nslots = 0
 w_self.hasdict = False
 w_self.needsdel = False
 w_self.weakrefable = False
@@ -141,13 +162,12 @@
 w_self.flag_cpytype = False
 w_self.flag_abstract = False
 w_self.flag_sequence_bug_compat = False
-w_self.instancetypedef = overridetypedef
 
 if overridetypedef is not None:
-setup_builtin_type(w_self)
+layout = setup_builtin_type(w_self, overridetypedef)
 else:
-setup_user_defined_type(w_self)
-w_self.w_same_layout_as = get_parent_layout(w_self)
+layout = setup_user_defined_type(w_self)
+w_self.layout = layout
 
 if space.config.objspace.std.withtypeversion:
 if not is_mro_purely_of_types(w_self.mro_w):

[pypy-commit] pypy cpyext-gc-support-2: Fix merge

2016-02-22 Thread rlamy

Author: Ronan Lamy 
Branch: cpyext-gc-support-2
Changeset: r82400:8a2af16e80dc
Date: 2016-02-22 17:45 +0100
http://bitbucket.org/pypy/pypy/changeset/8a2af16e80dc/

Log:Fix merge

diff --git a/pypy/module/cpyext/tupleobject.py 
b/pypy/module/cpyext/tupleobject.py
--- a/pypy/module/cpyext/tupleobject.py
+++ b/pypy/module/cpyext/tupleobject.py
@@ -101,7 +101,7 @@
 track_reference(space, py_obj, w_obj)
 return w_obj
 
-@cpython_api([PyObject], lltype.Void, external=False)
+@cpython_api([PyObject], lltype.Void, header=None)
 def tuple_dealloc(space, py_obj):
 """Frees allocated PyTupleObject resources.
 """
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit

[pypy-commit] pypy cpyext-gc-support-2: fix references to .instancetypedef

2016-02-22 Thread rlamy

Author: Ronan Lamy 
Branch: cpyext-gc-support-2
Changeset: r82401:f7a259fbc9f7
Date: 2016-02-22 18:07 +0100
http://bitbucket.org/pypy/pypy/changeset/f7a259fbc9f7/

Log:fix references to .instancetypedef

diff --git a/pypy/module/cpyext/api.py b/pypy/module/cpyext/api.py
--- a/pypy/module/cpyext/api.py
+++ b/pypy/module/cpyext/api.py
@@ -944,7 +944,7 @@
 py_obj = static_pyobjs[i]
 w_obj = static_objs_w[i]
 w_type = space.type(w_obj)
-typedescr = get_typedescr(w_type.instancetypedef)
+typedescr = get_typedescr(w_type.layout.typedef)
 py_obj.c_ob_type = rffi.cast(PyTypeObjectPtr,
  make_ref(space, w_type))
 typedescr.attach(space, py_obj, w_obj)
@@ -1142,7 +1142,7 @@
 if not use_micronumpy:
 return use_micronumpy
 # import to register api functions by side-effect
-import pypy.module.cpyext.ndarrayobject 
+import pypy.module.cpyext.ndarrayobject
 global GLOBALS, SYMBOLS_C, separate_module_files
 GLOBALS["PyArray_Type#"]= ('PyTypeObject*', 
"space.gettypeobject(W_NDimArray.typedef)")
 SYMBOLS_C += ['PyArray_Type', '_PyArray_FILLWBYTE', '_PyArray_ZEROS']
diff --git a/pypy/module/cpyext/bufferobject.py 
b/pypy/module/cpyext/bufferobject.py
--- a/pypy/module/cpyext/bufferobject.py
+++ b/pypy/module/cpyext/bufferobject.py
@@ -25,7 +25,7 @@
 @bootstrap_function
 def init_bufferobject(space):
 "Type description of PyBufferObject"
-make_typedescr(space.w_buffer.instancetypedef,
+make_typedescr(space.w_buffer.layout.typedef,
basestruct=PyBufferObject.TO,
attach=buffer_attach,
dealloc=buffer_dealloc,
diff --git a/pypy/module/cpyext/intobject.py b/pypy/module/cpyext/intobject.py
--- a/pypy/module/cpyext/intobject.py
+++ b/pypy/module/cpyext/intobject.py
@@ -19,7 +19,7 @@
 @bootstrap_function
 def init_intobject(space):
 "Type description of PyIntObject"
-make_typedescr(space.w_int.instancetypedef,
+make_typedescr(space.w_int.layout.typedef,
basestruct=PyIntObject.TO,
attach=int_attach,
realize=int_realize)
@@ -51,7 +51,7 @@
 @cpython_api([lltype.Signed], PyObject)
 def PyInt_FromLong(space, ival):
 """Create a new integer object with a value of ival.
-
+
 """
 return space.wrap(ival)
 
diff --git a/pypy/module/cpyext/object.py b/pypy/module/cpyext/object.py
--- a/pypy/module/cpyext/object.py
+++ b/pypy/module/cpyext/object.py
@@ -31,7 +31,7 @@
 def _PyObject_NewVar(space, type, itemcount):
 w_type = from_ref(space, rffi.cast(PyObject, type))
 assert isinstance(w_type, W_TypeObject)
-typedescr = get_typedescr(w_type.instancetypedef)
+typedescr = get_typedescr(w_type.layout.typedef)
 py_obj = typedescr.allocate(space, w_type, itemcount=itemcount)
 #py_obj.c_ob_refcnt = 0 --- will be set to 1 again by PyObject_Init{Var}
 if type.c_tp_itemsize == 0:
diff --git a/pypy/module/cpyext/pyobject.py b/pypy/module/cpyext/pyobject.py
--- a/pypy/module/cpyext/pyobject.py
+++ b/pypy/module/cpyext/pyobject.py
@@ -115,7 +115,7 @@
 def init_pyobject(space):
 from pypy.module.cpyext.object import PyObject_dealloc
 # typedescr for the 'object' type
-make_typedescr(space.w_object.instancetypedef,
+make_typedescr(space.w_object.layout.typedef,
dealloc=PyObject_dealloc)
 # almost all types, which should better inherit from object.
 make_typedescr(None)
@@ -207,7 +207,7 @@
 raise InvalidPointerException(str(ref))
 w_type = from_ref(space, ref_type)
 assert isinstance(w_type, W_TypeObject)
-return get_typedescr(w_type.instancetypedef).realize(space, ref)
+return get_typedescr(w_type.layout.typedef).realize(space, ref)
 
 
 def debug_collect():
@@ -327,7 +327,7 @@
 obj.c_ob_refcnt = 1
 w_type = from_ref(space, rffi.cast(PyObject, obj.c_ob_type))
 assert isinstance(w_type, W_TypeObject)
-get_typedescr(w_type.instancetypedef).realize(space, obj)
+get_typedescr(w_type.layout.typedef).realize(space, obj)
 
 @cpython_api([PyObject], lltype.Void)
 def _Py_Dealloc(space, obj):
diff --git a/pypy/module/cpyext/stringobject.py 
b/pypy/module/cpyext/stringobject.py
--- a/pypy/module/cpyext/stringobject.py
+++ b/pypy/module/cpyext/stringobject.py
@@ -59,7 +59,7 @@
 @bootstrap_function
 def init_stringobject(space):
 "Type description of PyStringObject"
-make_typedescr(space.w_str.instancetypedef,
+make_typedescr(space.w_str.layout.typedef,
basestruct=PyStringObject.TO,
attach=string_attach,
dealloc=string_dealloc,
@@ -73,7 +73,7 @@
 interpreter object.  The buffer may be mutated, until string_realize() is
 called.  Refcount of the result is 1.
 """
-typedescr = get_typedescr(space.w_str.instancetypedef)
+typedescr = get_typedescr(space.

[pypy-commit] pypy cpyext-gc-support-2: refcounts don't fit in an 'int' (important after 8c6a66dcb994)

2016-02-22 Thread arigo

Author: Armin Rigo 
Branch: cpyext-gc-support-2
Changeset: r82402:78f8c093d168
Date: 2016-02-22 18:16 +0100
http://bitbucket.org/pypy/pypy/changeset/78f8c093d168/

Log:refcounts don't fit in an 'int' (important after 8c6a66dcb994)

diff --git a/pypy/module/cpyext/test/test_cpyext.py 
b/pypy/module/cpyext/test/test_cpyext.py
--- a/pypy/module/cpyext/test/test_cpyext.py
+++ b/pypy/module/cpyext/test/test_cpyext.py
@@ -646,8 +646,8 @@
 static PyObject* foo_pi(PyObject* self, PyObject *args)
 {
 PyObject *true_obj = Py_True;
-int refcnt = true_obj->ob_refcnt;
-int refcnt_after;
+Py_ssize_t refcnt = true_obj->ob_refcnt;
+Py_ssize_t refcnt_after;
 Py_INCREF(true_obj);
 Py_INCREF(true_obj);
 PyBool_Check(true_obj);
@@ -661,8 +661,8 @@
 {
 PyObject *true_obj = Py_True;
 PyObject *tup = NULL;
-int refcnt = true_obj->ob_refcnt;
-int refcnt_after;
+Py_ssize_t refcnt = true_obj->ob_refcnt;
+Py_ssize_t refcnt_after;
 
 tup = PyTuple_New(1);
 Py_INCREF(true_obj);
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit

[pypy-commit] pypy cpyext-gc-support-2: (ronan, arigo)

2016-02-22 Thread arigo

Author: Armin Rigo 
Branch: cpyext-gc-support-2
Changeset: r82403:f2383d0be1ba
Date: 2016-02-22 18:29 +0100
http://bitbucket.org/pypy/pypy/changeset/f2383d0be1ba/

Log:(ronan, arigo)

Fix and test for inheriting at app-level from some PyTypeObject

diff --git a/pypy/module/cpyext/test/foo.c b/pypy/module/cpyext/test/foo.c
--- a/pypy/module/cpyext/test/foo.c
+++ b/pypy/module/cpyext/test/foo.c
@@ -623,11 +623,17 @@
 };
 
 
+static PyObject *size_of_instances(PyObject *self, PyObject *t)
+{
+return PyInt_FromLong(((PyTypeObject *)t)->tp_basicsize);
+}
+
 /* List of functions exported by this module */
 
 static PyMethodDef foo_functions[] = {
 {"new",(PyCFunction)foo_new, METH_NOARGS, NULL},
 {"newCustom",  (PyCFunction)newCustom, METH_NOARGS, NULL},
+{"size_of_instances", (PyCFunction)size_of_instances, METH_O, NULL},
 {NULL,NULL}/* Sentinel */
 };
 
diff --git a/pypy/module/cpyext/test/test_typeobject.py 
b/pypy/module/cpyext/test/test_typeobject.py
--- a/pypy/module/cpyext/test/test_typeobject.py
+++ b/pypy/module/cpyext/test/test_typeobject.py
@@ -744,3 +744,25 @@
 module = self.import_module(name='foo3')
 print('calling module.Type()...')
 module.Type("X", (object,), {})
+
+def test_app_subclass_of_c_type(self):
+module = self.import_module(name='foo')
+size = module.size_of_instances(module.fooType)
+class f1(object):
+pass
+class f2(module.fooType):
+pass
+class bar(f1, f2):
+pass
+assert bar.__base__ is f2
+assert module.size_of_instances(bar) == size
+
+def test_app_cant_subclass_two_types(self):
+module = self.import_module(name='foo')
+try:
+class bar(module.fooType, module.Property):
+pass
+except TypeError as e:
+assert str(e) == 'instance layout conflicts in multiple 
inheritance'
+else:
+raise AssertionError("did not get TypeError!")
diff --git a/pypy/module/cpyext/typeobject.py b/pypy/module/cpyext/typeobject.py
--- a/pypy/module/cpyext/typeobject.py
+++ b/pypy/module/cpyext/typeobject.py
@@ -297,7 +297,7 @@
 name = rffi.charp2str(pto.c_tp_name)
 
 W_TypeObject.__init__(self, space, name,
-bases_w or [space.w_object], dict_w)
+bases_w or [space.w_object], dict_w, force_new_layout=True)
 if not space.is_true(space.issubtype(self, space.w_type)):
 self.flag_cpytype = True
 self.flag_heaptype = False
diff --git a/pypy/objspace/std/typeobject.py b/pypy/objspace/std/typeobject.py
--- a/pypy/objspace/std/typeobject.py
+++ b/pypy/objspace/std/typeobject.py
@@ -148,7 +148,7 @@
 
 @dont_look_inside
 def __init__(w_self, space, name, bases_w, dict_w,
- overridetypedef=None):
+ overridetypedef=None, force_new_layout=False):
 w_self.space = space
 w_self.name = name
 w_self.bases_w = bases_w
@@ -164,9 +164,10 @@
 w_self.flag_sequence_bug_compat = False
 
 if overridetypedef is not None:
+assert not force_new_layout
 layout = setup_builtin_type(w_self, overridetypedef)
 else:
-layout = setup_user_defined_type(w_self)
+layout = setup_user_defined_type(w_self, force_new_layout)
 w_self.layout = layout
 
 if space.config.objspace.std.withtypeversion:
@@ -1000,7 +1001,7 @@
 w_self.weakrefable = w_self.weakrefable or w_base.weakrefable
 return hasoldstylebase
 
-def create_all_slots(w_self, hasoldstylebase, w_bestbase):
+def create_all_slots(w_self, hasoldstylebase, w_bestbase, force_new_layout):
 base_layout = w_bestbase.layout
 index_next_extra_slot = base_layout.nslots
 space = w_self.space
@@ -1040,7 +1041,7 @@
 if '__del__' in dict_w:
 w_self.needsdel = True
 #
-if index_next_extra_slot == base_layout.nslots:
+if index_next_extra_slot == base_layout.nslots and not force_new_layout:
 return base_layout
 else:
 return Layout(base_layout.typedef, index_next_extra_slot,
@@ -1081,7 +1082,7 @@
 return False
 return True
 
-def setup_user_defined_type(w_self):
+def setup_user_defined_type(w_self, force_new_layout):
 if len(w_self.bases_w) == 0:
 w_self.bases_w = [w_self.space.w_object]
 w_bestbase = check_and_find_best_base(w_self.space, w_self.bases_w)
@@ -1093,7 +1094,8 @@
 w_self.flag_abstract |= w_base.flag_abstract
 
 hasoldstylebase = copy_flags_from_bases(w_self, w_bestbase)
-layout = create_all_slots(w_self, hasoldstylebase, w_bestbase)
+layout = create_all_slots(w_self, hasoldstylebase, w_bestbase,
+  force_new_layout)
 
 ensure_common_attributes(w_self)
 return layout
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.pyt

[pypy-commit] pypy regalloc: added a new test file to inspect properties of the regalloc in certain circumstances

2016-02-22 Thread plan_rich

Author: Richard Plangger 
Branch: regalloc
Changeset: r82404:b52b80aa52f6
Date: 2016-02-22 18:31 +0100
http://bitbucket.org/pypy/pypy/changeset/b52b80aa52f6/

Log:added a new test file to inspect properties of the regalloc in
certain circumstances

diff --git a/rpython/jit/backend/llsupport/test/test_regalloc_call.py 
b/rpython/jit/backend/llsupport/test/test_regalloc_call.py
new file mode 100644
--- /dev/null
+++ b/rpython/jit/backend/llsupport/test/test_regalloc_call.py
@@ -0,0 +1,87 @@
+from rpython.jit.tool.oparser import parse
+from rpython.jit.backend.x86.regalloc import RegAlloc
+from rpython.jit.backend.detect_cpu import getcpuclass
+from rpython.jit.backend.x86.arch import DEFAULT_FRAME_BYTES
+from rpython.jit.metainterp.history import TargetToken
+from rpython.jit.metainterp.resoperation import rop, ResOperation
+
+class FakeReg(object):
+def __init__(self, i):
+self.n = i
+def __repr__(self):
+return 'r%d' % self.n
+
+r1, r2, r3, r4, r5, r6, r7, r8, r9, r10 = [FakeReg(i) for i in range(1,11)]
+
+CPU = getcpuclass()
+
+def parse_loop(text):
+ops = parse(text)
+tt = None
+tt = TargetToken(ops.operations[-1].getdescr())
+ops.operations = [ResOperation(rop.LABEL, ops.inputargs, None, descr=tt)] 
+ ops.operations
+for op in ops.operations:
+if op.getopnum() == rop.JUMP:
+assert tt is not None
+op.setdescr(tt)
+return ops
+
+class FakeMachineCodeBuilder(object):
+_frame_size = DEFAULT_FRAME_BYTES
+def mark_op(self, op):
+pass
+def get_relative_pos(self): return 0
+class FakeAssembler(object):
+cpu = CPU(None, None)
+current_clt = None
+target_tokens_currently_compiling = {}
+def __init__(self):
+self.mc = FakeMachineCodeBuilder()
+def regalloc_mov(self, prev, loc): pass
+def dump(self, *args): pass
+def regalloc_perform(self, *args): pass
+def label(self): pass
+def closing_jump(self, target): pass
+
+CPURegalloc = RegAlloc
+class FakeRegAlloc(CPURegalloc):
+def __init__(self, caller_saved, callee_saved):
+self.all_regs = callee_saved[:] + callee_saved
+CPURegalloc.__init__(self, FakeAssembler(), False)
+def flush_loop(self): pass
+
+class FakeLoopToken(object):
+def __init__(self):
+self.compiled_loop_token = None
+
+class TraceAllocation(object):
+def __init__(self, trace, caller_saved, callee_saved, binding):
+self.trace = trace
+self.regalloc = FakeRegAlloc(caller_saved, callee_saved)
+self.initial_binding = binding
+looptoken = FakeLoopToken()
+gcrefs = None
+for op in trace.operations:
+for arg in op.getarglist():
+pass
+pass
+self.regalloc.prepare_loop(self.trace.inputargs, 
self.trace.operations, looptoken, gcrefs)
+self.regalloc.walk_operations(trace.inputargs, trace.operations)
+
+def initial_register(self, name):
+pass
+
+class TestRegalloc(object):
+
+def test_allocate_register_into_jump_register(self):
+ops = parse_loop("""
+[p0,i1]
+i2 = int_add(i1,i1)
+i3 = int_add(i2,i1)
+jump(p0,i2)
+""")
+trace_alloc = TraceAllocation(ops, [r1, r2], [r3, r4], {'p0': r1, 
'i1': r2})
+i2 = trace_alloc.initial_register('i2')
+i1 = trace_alloc.initial_register('i1')
+assert i2 == i1
+
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit

[pypy-commit] pypy vlen-resume: close to be merged branch

2016-02-22 Thread fijal

Author: fijal
Branch: vlen-resume
Changeset: r82405:5537192c5417
Date: 2016-02-22 18:34 +0100
http://bitbucket.org/pypy/pypy/changeset/5537192c5417/

Log:close to be merged branch

___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit

[pypy-commit] pypy default: (fijal, arigo, cfbolz) Merge vlen-resume which compresses the numberings by

2016-02-22 Thread fijal

Author: fijal
Branch: 
Changeset: r82406:9cc15a57d1c2
Date: 2016-02-22 18:35 +0100
http://bitbucket.org/pypy/pypy/changeset/9cc15a57d1c2/

Log:(fijal, arigo, cfbolz) Merge vlen-resume which compresses the
numberings by about 20% by using a variable-size encoding

diff --git a/rpython/jit/metainterp/resume.py b/rpython/jit/metainterp/resume.py
--- a/rpython/jit/metainterp/resume.py
+++ b/rpython/jit/metainterp/resume.py
@@ -27,6 +27,13 @@
 self.prev = prev
 self.boxes = boxes
 
+class TopSnapshot(Snapshot):
+__slots__ = ('vable_boxes',)
+
+def __init__(self, prev, boxes, vable_boxes):
+Snapshot.__init__(self, prev, boxes)
+self.vable_boxes = vable_boxes
+
 def combine_uint(index1, index2):
 assert 0 <= index1 < 65536
 assert 0 <= index2 < 65536
@@ -127,9 +134,11 @@
snapshot_storage):
 n = len(framestack) - 1
 if virtualizable_boxes is not None:
-boxes = virtualref_boxes + virtualizable_boxes
+virtualizable_boxes = ([virtualizable_boxes[-1]] +
+virtualizable_boxes[:-1])
 else:
-boxes = virtualref_boxes[:]
+virtualizable_boxes = []
+virtualref_boxes = virtualref_boxes[:]
 if n >= 0:
 top = framestack[n]
 _ensure_parent_resumedata(framestack, n)
@@ -138,11 +147,12 @@
 snapshot_storage.rd_frame_info_list = frame_info_list
 snapshot = Snapshot(top.parent_resumedata_snapshot,
 top.get_list_of_active_boxes(False))
-snapshot = Snapshot(snapshot, boxes)
+snapshot = TopSnapshot(snapshot, virtualref_boxes, virtualizable_boxes)
 snapshot_storage.rd_snapshot = snapshot
 else:
 snapshot_storage.rd_frame_info_list = None
-snapshot_storage.rd_snapshot = Snapshot(None, boxes)
+snapshot_storage.rd_snapshot = TopSnapshot(None, virtualref_boxes,
+   virtualizable_boxes)
 
 PENDINGFIELDSTRUCT = lltype.Struct('PendingField',
('lldescr', OBJECTPTR),
@@ -200,10 +210,12 @@
 self.v = 0
 
 def count_boxes(self, lst):
-c = 0
+snapshot = lst[0]
+assert isinstance(snapshot, TopSnapshot)
+c = len(snapshot.vable_boxes)
 for snapshot in lst:
 c += len(snapshot.boxes)
-c += 2 * (len(lst) - 1)
+c += 2 * (len(lst) - 1) + 1 + 1
 return c
 
 def append(self, item):
@@ -294,13 +306,11 @@
 state.append(tagged)
 state.n = n
 state.v = v
-state.position -= length + 2
 
-def number(self, optimizer, snapshot, frameinfo):
+def number(self, optimizer, topsnapshot, frameinfo):
 # flatten the list
-vref_snapshot = snapshot
-cur = snapshot.prev
-snapshot_list = [vref_snapshot]
+cur = topsnapshot.prev
+snapshot_list = [topsnapshot]
 framestack_list = []
 while cur:
 framestack_list.append(frameinfo)
@@ -311,19 +321,30 @@
 
 # we want to number snapshots starting from the back, but ending
 # with a forward list
-for i in range(len(snapshot_list) - 1, -1, -1):
-state.position -= len(snapshot_list[i].boxes)
-if i != 0:
-frameinfo = framestack_list[i - 1]
-jitcode_pos, pc = unpack_uint(frameinfo.packed_jitcode_pc)
-state.position -= 2
-state.append(rffi.cast(rffi.SHORT, jitcode_pos))
-state.append(rffi.cast(rffi.SHORT, pc))
+for i in range(len(snapshot_list) - 1, 0, -1):
+state.position -= len(snapshot_list[i].boxes) + 2
+frameinfo = framestack_list[i - 1]
+jitcode_pos, pc = unpack_uint(frameinfo.packed_jitcode_pc)
+state.append(rffi.cast(rffi.SHORT, jitcode_pos))
+state.append(rffi.cast(rffi.SHORT, pc))
 self._number_boxes(snapshot_list[i].boxes, optimizer, state)
+state.position -= len(snapshot_list[i].boxes) + 2
 
-numb = resumecode.create_numbering(state.current,
-   len(vref_snapshot.boxes))
+assert isinstance(topsnapshot, TopSnapshot)
+special_boxes_size = (1 + len(topsnapshot.vable_boxes) +
+  1 + len(topsnapshot.boxes))
+assert state.position == special_boxes_size
 
+state.position = 0
+state.append(rffi.cast(rffi.SHORT, len(topsnapshot.vable_boxes)))
+self._number_boxes(topsnapshot.vable_boxes, optimizer, state)
+n = len(topsnapshot.boxes)
+assert not (n & 1)
+state.append(rffi.cast(rffi.SHORT, n >> 1))
+self._number_boxes(topsnapshot.boxes, optimizer, state)
+assert state.position == special_boxes_size
+
+numb = resumecode.create_numbering(state.current)
 return numb, state.liveboxes, state.v

[pypy-commit] pypy cpyext-gc-support-2: hg merge default

2016-02-22 Thread arigo

Author: Armin Rigo 
Branch: cpyext-gc-support-2
Changeset: r82407:f174145a8b9b
Date: 2016-02-22 18:03 +
http://bitbucket.org/pypy/pypy/changeset/f174145a8b9b/

Log:hg merge default

diff --git a/.hgignore b/.hgignore
--- a/.hgignore
+++ b/.hgignore
@@ -22,6 +22,7 @@
 ^pypy/module/cpyext/test/.+\.obj$
 ^pypy/module/cpyext/test/.+\.manifest$
 ^pypy/module/test_lib_pypy/ctypes_tests/.+\.o$
+^pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test\.o$
 ^pypy/module/cppyy/src/.+\.o$
 ^pypy/module/cppyy/bench/.+\.so$
 ^pypy/module/cppyy/bench/.+\.root$
@@ -35,7 +36,6 @@
 ^pypy/module/test_lib_pypy/cffi_tests/__pycache__.+$
 ^pypy/doc/.+\.html$
 ^pypy/doc/config/.+\.rst$
-^pypy/doc/basicblock\.asc$
 ^pypy/doc/.+\.svninfo$
 ^rpython/translator/c/src/libffi_msvc/.+\.obj$
 ^rpython/translator/c/src/libffi_msvc/.+\.dll$
@@ -45,53 +45,33 @@
 ^rpython/translator/c/src/cjkcodecs/.+\.obj$
 ^rpython/translator/c/src/stacklet/.+\.o$
 ^rpython/translator/c/src/.+\.o$
-^rpython/translator/jvm/\.project$
-^rpython/translator/jvm/\.classpath$
-^rpython/translator/jvm/eclipse-bin$
-^rpython/translator/jvm/src/pypy/.+\.class$
-^rpython/translator/benchmark/docutils$
-^rpython/translator/benchmark/templess$
-^rpython/translator/benchmark/gadfly$
-^rpython/translator/benchmark/mako$
-^rpython/translator/benchmark/bench-custom\.benchmark_result$
-^rpython/translator/benchmark/shootout_benchmarks$
+^rpython/translator/llvm/.+\.so$
 ^rpython/translator/goal/target.+-c$
 ^rpython/translator/goal/.+\.exe$
 ^rpython/translator/goal/.+\.dll$
 ^pypy/goal/pypy-translation-snapshot$
 ^pypy/goal/pypy-c
-^pypy/goal/pypy-jvm
-^pypy/goal/pypy-jvm.jar
 ^pypy/goal/.+\.exe$
 ^pypy/goal/.+\.dll$
 ^pypy/goal/.+\.lib$
 ^pypy/_cache$
-^pypy/doc/statistic/.+\.html$
-^pypy/doc/statistic/.+\.eps$
-^pypy/doc/statistic/.+\.pdf$
-^rpython/translator/cli/src/pypylib\.dll$
-^rpython/translator/cli/src/query\.exe$
-^rpython/translator/cli/src/main\.exe$
+^lib-python/2.7/lib2to3/.+\.pickle$
 ^lib_pypy/__pycache__$
 ^lib_pypy/ctypes_config_cache/_.+_cache\.py$
 ^lib_pypy/ctypes_config_cache/_.+_.+_\.py$
 ^lib_pypy/_libmpdec/.+.o$
-^rpython/translator/cli/query-descriptions$
 ^pypy/doc/discussion/.+\.html$
 ^include/.+\.h$
 ^include/.+\.inl$
 ^pypy/doc/_build/.*$
 ^pypy/doc/config/.+\.html$
 ^pypy/doc/config/style\.css$
-^pypy/doc/jit/.+\.html$
-^pypy/doc/jit/style\.css$
 ^pypy/doc/image/lattice1\.png$
 ^pypy/doc/image/lattice2\.png$
 ^pypy/doc/image/lattice3\.png$
 ^pypy/doc/image/stackless_informal\.png$
 ^pypy/doc/image/parsing_example.+\.png$
 ^rpython/doc/_build/.*$
-^pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test\.o$
 ^compiled
 ^.git/
 ^release/
diff --git a/pypy/interpreter/pyparser/pytokenizer.py 
b/pypy/interpreter/pyparser/pytokenizer.py
--- a/pypy/interpreter/pyparser/pytokenizer.py
+++ b/pypy/interpreter/pyparser/pytokenizer.py
@@ -91,6 +91,7 @@
 strstart = (0, 0, "")
 for line in lines:
 lnum = lnum + 1
+line = universal_newline(line)
 pos, max = 0, len(line)
 
 if contstr:
@@ -259,3 +260,14 @@
 
 token_list.append((tokens.ENDMARKER, '', lnum, pos, line))
 return token_list
+
+
+def universal_newline(line):
+# show annotator that indexes below are non-negative
+line_len_m2 = len(line) - 2
+if line_len_m2 >= 0 and line[-2] == '\r' and line[-1] == '\n':
+return line[:line_len_m2] + '\n'
+line_len_m1 = len(line) - 1
+if line_len_m1 >= 0 and line[-1] == '\r':
+return line[:line_len_m1] + '\n'
+return line
diff --git a/pypy/interpreter/pyparser/test/test_pyparse.py 
b/pypy/interpreter/pyparser/test/test_pyparse.py
--- a/pypy/interpreter/pyparser/test/test_pyparse.py
+++ b/pypy/interpreter/pyparser/test/test_pyparse.py
@@ -158,3 +158,10 @@
 
 def test_print_function(self):
 self.parse("from __future__ import print_function\nx = print\n")
+
+def test_universal_newlines(self):
+fmt = 'stuff = """hello%sworld"""'
+expected_tree = self.parse(fmt % '\n')
+for linefeed in ["\r\n","\r"]:
+tree = self.parse(fmt % linefeed)
+assert expected_tree == tree
diff --git a/pypy/interpreter/typedef.py b/pypy/interpreter/typedef.py
--- a/pypy/interpreter/typedef.py
+++ b/pypy/interpreter/typedef.py
@@ -156,20 +156,6 @@
 get_unique_interplevel_subclass._annspecialcase_ = "specialize:memo"
 _subclass_cache = {}
 
-def enum_interplevel_subclasses(config, cls):
-"""Return a list of all the extra interp-level subclasses of 'cls' that
-can be built by get_unique_interplevel_subclass()."""
-result = []
-for flag1 in (False, True):
-for flag2 in (False, True):
-for flag3 in (False, True):
-for flag4 in (False, True):
-result.append(get_unique_interplevel_subclass(
-config, cls, flag1, flag2, flag3, flag4))
-result = dict.fromkeys(result)
-assert len(result) <= 6
-return result.keys()
-
 def _getusercls(config, cls, w

[pypy-commit] pypy cpyext-gc-support-2: "backport" of 46e8c7546570

2016-02-22 Thread arigo

Author: Armin Rigo 
Branch: cpyext-gc-support-2
Changeset: r82408:0bef74bad350
Date: 2016-02-22 18:11 +
http://bitbucket.org/pypy/pypy/changeset/0bef74bad350/

Log:"backport" of 46e8c7546570

diff --git a/pypy/module/cpyext/api.py b/pypy/module/cpyext/api.py
--- a/pypy/module/cpyext/api.py
+++ b/pypy/module/cpyext/api.py
@@ -1151,8 +1151,8 @@
 
 def setup_library(space):
 "NOT_RPYTHON"
+use_micronumpy = setup_micronumpy(space)
 export_symbols = sorted(FUNCTIONS) + sorted(SYMBOLS_C) + sorted(GLOBALS)
-use_micronumpy = setup_micronumpy(space)
 from rpython.translator.c.database import LowLevelDatabase
 db = LowLevelDatabase()
 
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit

[pypy-commit] pypy stmgc-c8: try harder to not call rgil.acquire() in STM

2016-02-22 Thread Raemi

Author: Remi Meier 
Branch: stmgc-c8
Changeset: r82409:6de86a868686
Date: 2016-02-22 19:07 +0100
http://bitbucket.org/pypy/pypy/changeset/6de86a868686/

Log:try harder to not call rgil.acquire() in STM

diff --git a/pypy/module/_cffi_backend/call_python.py 
b/pypy/module/_cffi_backend/call_python.py
--- a/pypy/module/_cffi_backend/call_python.py
+++ b/pypy/module/_cffi_backend/call_python.py
@@ -40,8 +40,10 @@
at least 8 bytes in size.
 """
 from pypy.module._cffi_backend.ccallback import reveal_callback
-from rpython.rlib import rgil
+from rpython.rlib import rgil, rgc
 
+if rgc.stm_is_enabled():
+raise NotImplementedError("XXX for STM")
 rgil.acquire()
 rffi.stackcounter.stacks_counter += 1
 llop.gc_stack_bottom(lltype.Void)   # marker for trackgcroot.py
diff --git a/pypy/module/cpyext/api.py b/pypy/module/cpyext/api.py
--- a/pypy/module/cpyext/api.py
+++ b/pypy/module/cpyext/api.py
@@ -602,7 +602,7 @@
 # Make the wrapper for the cases (1) and (2)
 def make_wrapper(space, callable, gil=None):
 "NOT_RPYTHON"
-from rpython.rlib import rgil
+from rpython.rlib import rgil, rgc
 names = callable.api_func.argnames
 argtypes_enum_ui = 
unrolling_iterable(enumerate(zip(callable.api_func.argtypes,
 [name.startswith("w_") for name in names])))
@@ -618,6 +618,8 @@
 # we hope that malloc removal removes the newtuple() that is
 # inserted exactly here by the varargs specializer
 if gil_acquire:
+if rgc.stm_is_enabled():
+raise NotImplementedError("XXX for STM")
 rgil.acquire()
 rffi.stackcounter.stacks_counter += 1
 llop.gc_stack_bottom(lltype.Void)   # marker for trackgcroot.py
diff --git a/rpython/rlib/entrypoint.py b/rpython/rlib/entrypoint.py
--- a/rpython/rlib/entrypoint.py
+++ b/rpython/rlib/entrypoint.py
@@ -56,10 +56,12 @@
 """
 def deco(func):
 source = py.code.Source("""
-from rpython.rlib import rgil
+from rpython.rlib import rgil,rgc
 
 def wrapper(%(args)s):
 # acquire the GIL
+if rgc.stm_is_enabled():
+raise NotImplementedError("XXX for STM")
 rgil.acquire()
 #
 rffi.stackcounter.stacks_counter += 1
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit

[pypy-commit] pypy stmgc-c8: remove accidentially added lines

2016-02-22 Thread Raemi

Author: Remi Meier 
Branch: stmgc-c8
Changeset: r82410:0fe6e57443a4
Date: 2016-02-22 22:36 +0100
http://bitbucket.org/pypy/pypy/changeset/0fe6e57443a4/

Log:remove accidentially added lines

diff --git a/rpython/jit/backend/x86/regalloc.py 
b/rpython/jit/backend/x86/regalloc.py
--- a/rpython/jit/backend/x86/regalloc.py
+++ b/rpython/jit/backend/x86/regalloc.py
@@ -1114,8 +1114,6 @@
 assert isinstance(size_box, ConstInt)
 size = size_box.value
 size_loc = imm(abs(size))
-segment = op.getarg(3)
-assert isinstance(segment, int)
 if size < 0:
 sign_loc = imm1
 else:
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit

[pypy-commit] pypy default: Fix test by testing with a less commonly used module.

2016-02-22 Thread mjacob

Author: Manuel Jacob 
Branch: 
Changeset: r82411:9a5c6a749b49
Date: 2016-02-22 23:27 +0100
http://bitbucket.org/pypy/pypy/changeset/9a5c6a749b49/

Log:Fix test by testing with a less commonly used module.

diff --git a/pypy/module/imp/test/test_import.py 
b/pypy/module/imp/test/test_import.py
--- a/pypy/module/imp/test/test_import.py
+++ b/pypy/module/imp/test/test_import.py
@@ -1061,12 +1061,12 @@
 py.test.skip("unresolved issues with win32 shell quoting rules")
 from pypy.interpreter.test.test_zpy import pypypath 
 extrapath = udir.ensure("pythonpath", dir=1) 
-extrapath.join("urllib.py").write("print 42\n")
+extrapath.join("sched.py").write("print 42\n")
 old = os.environ.get('PYTHONPATH', None)
 oldlang = os.environ.pop('LANG', None)
 try:
 os.environ['PYTHONPATH'] = str(extrapath)
-output = py.process.cmdexec('''"%s" "%s" -c "import urllib"''' %
+output = py.process.cmdexec('''"%s" "%s" -c "import sched"''' %
  (sys.executable, pypypath))
 assert output.strip() == '42'
 finally:
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit

[pypy-commit] pypy py3k: hg merge default

2016-02-22 Thread mjacob

Author: Manuel Jacob 
Branch: py3k
Changeset: r82412:be90c2e77207
Date: 2016-02-22 23:36 +0100
http://bitbucket.org/pypy/pypy/changeset/be90c2e77207/

Log:hg merge default

diff --git a/pypy/doc/whatsnew-head.rst b/pypy/doc/whatsnew-head.rst
--- a/pypy/doc/whatsnew-head.rst
+++ b/pypy/doc/whatsnew-head.rst
@@ -128,6 +128,7 @@
 
 Fix SSL tests by importing cpython's patch
 
+
 .. branch: remove-getfield-pure
 
 Remove pure variants of ``getfield_gc_*`` operations from the JIT. Relevant
@@ -163,3 +164,10 @@
 .. branch: windows-vmprof-support
 
 vmprof should work on Windows.
+
+
+.. branch: reorder-map-attributes
+
+When creating instances and adding attributes in several different orders
+depending on some condition, the JIT would create too much code. This is now
+fixed.
\ No newline at end of file
diff --git a/pypy/module/imp/test/test_import.py 
b/pypy/module/imp/test/test_import.py
--- a/pypy/module/imp/test/test_import.py
+++ b/pypy/module/imp/test/test_import.py
@@ -1216,12 +1216,12 @@
 py.test.skip("unresolved issues with win32 shell quoting rules")
 from pypy.interpreter.test.test_zpy import pypypath 
 extrapath = udir.ensure("pythonpath", dir=1) 
-extrapath.join("urllib.py").write("print(42)\n")
+extrapath.join("sched.py").write("print(42)\n")
 old = os.environ.get('PYTHONPATH', None)
 oldlang = os.environ.pop('LANG', None)
 try:
 os.environ['PYTHONPATH'] = str(extrapath)
-output = py.process.cmdexec('''"%s" "%s" -c "import urllib"''' %
+output = py.process.cmdexec('''"%s" "%s" -c "import sched"''' %
  (sys.executable, pypypath))
 assert output.strip() == '42'
 finally:
diff --git a/pypy/module/pypyjit/test_pypy_c/test_string.py 
b/pypy/module/pypyjit/test_pypy_c/test_string.py
--- a/pypy/module/pypyjit/test_pypy_c/test_string.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_string.py
@@ -28,7 +28,6 @@
 guard_true(i14, descr=...)
 guard_not_invalidated(descr=...)
 i16 = int_eq(i6, %d)
-guard_false(i16, descr=...)
 i15 = int_mod(i6, i10)
 i17 = int_rshift(i15, %d)
 i18 = int_and(i10, i17)
@@ -68,7 +67,6 @@
 guard_true(i11, descr=...)
 guard_not_invalidated(descr=...)
 i13 = int_eq(i6, %d) # value provided below
-guard_false(i13, descr=...)
 i15 = int_mod(i6, 10)
 i17 = int_rshift(i15, %d)# value provided below
 i18 = int_and(10, i17)
@@ -144,43 +142,6 @@
 jump(..., descr=...)
 """)
 
-def test_getattr_promote(self):
-def main(n):
-class A(object):
-def meth_a(self):
-return 1
-def meth_b(self):
-return 2
-a = A()
-
-l = ['a', 'b']
-s = 0
-for i in range(n):
-name = 'meth_' + l[i & 1]
-meth = getattr(a, name) # ID: getattr
-s += meth()
-return s
-
-log = self.run(main, [1000])
-assert log.result == main(1000)
-loops = log.loops_by_filename(self.filepath)
-assert len(loops) == 1
-for loop in loops:
-assert loop.match_by_id('getattr','''
-guard_not_invalidated?
-i32 = strlen(p31)
-i34 = int_add(5, i32)
-p35 = newstr(i34)
-strsetitem(p35, 0, 109)
-strsetitem(p35, 1, 101)
-strsetitem(p35, 2, 116)
-strsetitem(p35, 3, 104)
-strsetitem(p35, 4, 95)
-copystrcontent(p31, p35, 0, 5, i32)
-i49 = 
call_i(ConstClass(_ll_2_str_eq_nonnull__rpy_stringPtr_rpy_stringPtr), p35, 
ConstPtr(ptr48), descr=)
-guard_value(i49, 1, descr=...)
-''')
-
 def test_remove_duplicate_method_calls(self):
 def main(n):
 lst = []
diff --git a/pypy/objspace/std/mapdict.py b/pypy/objspace/std/mapdict.py
--- a/pypy/objspace/std/mapdict.py
+++ b/pypy/objspace/std/mapdict.py
@@ -1,4 +1,4 @@
-import weakref
+import weakref, sys
 
 from rpython.rlib import jit, objectmodel, debug, rerased
 from rpython.rlib.rarithmetic import intmask, r_uint
@@ -12,6 +12,11 @@
 from pypy.objspace.std.typeobject import MutableCell
 
 
+erase_item, unerase_item = rerased.new_erasing_pair("mapdict storage item")
+erase_map,  unerase_map = rerased.new_erasing_pair("map")
+erase_list, unerase_list = rerased.new_erasing_pair("mapdict storage list")
+
+
 # 
 # attribute shapes
 
@@ -20,6 +25,7 @@
 # note: we use "x * NUM_DIGITS_POW2" instead of "x << NUM_DIGITS" because
 # we want to propagate knowledge that the result cannot be negative
 
+
 class AbstractAttribute(object):
 _immutable_fields_ = ['terminator']
 cache_attrs = None
@@ -151,29 +157,124 @@
 cache[na

[pypy-commit] pypy py3.3: hg merge py3k

2016-02-22 Thread mjacob

Author: Manuel Jacob 
Branch: py3.3
Changeset: r82413:6107f05ecc93
Date: 2016-02-22 23:37 +0100
http://bitbucket.org/pypy/pypy/changeset/6107f05ecc93/

Log:hg merge py3k

diff --git a/pypy/doc/whatsnew-head.rst b/pypy/doc/whatsnew-head.rst
--- a/pypy/doc/whatsnew-head.rst
+++ b/pypy/doc/whatsnew-head.rst
@@ -128,6 +128,7 @@
 
 Fix SSL tests by importing cpython's patch
 
+
 .. branch: remove-getfield-pure
 
 Remove pure variants of ``getfield_gc_*`` operations from the JIT. Relevant
@@ -163,3 +164,10 @@
 .. branch: windows-vmprof-support
 
 vmprof should work on Windows.
+
+
+.. branch: reorder-map-attributes
+
+When creating instances and adding attributes in several different orders
+depending on some condition, the JIT would create too much code. This is now
+fixed.
\ No newline at end of file
diff --git a/pypy/module/imp/test/test_import.py 
b/pypy/module/imp/test/test_import.py
--- a/pypy/module/imp/test/test_import.py
+++ b/pypy/module/imp/test/test_import.py
@@ -992,12 +992,12 @@
 py.test.skip("unresolved issues with win32 shell quoting rules")
 from pypy.interpreter.test.test_zpy import pypypath 
 extrapath = udir.ensure("pythonpath", dir=1) 
-extrapath.join("urllib.py").write("print(42)\n")
+extrapath.join("sched.py").write("print(42)\n")
 old = os.environ.get('PYTHONPATH', None)
 oldlang = os.environ.pop('LANG', None)
 try:
 os.environ['PYTHONPATH'] = str(extrapath)
-output = py.process.cmdexec('''"%s" "%s" -c "import urllib"''' %
+output = py.process.cmdexec('''"%s" "%s" -c "import sched"''' %
  (sys.executable, pypypath))
 assert output.strip() == '42'
 finally:
diff --git a/pypy/module/pypyjit/test_pypy_c/test_string.py 
b/pypy/module/pypyjit/test_pypy_c/test_string.py
--- a/pypy/module/pypyjit/test_pypy_c/test_string.py
+++ b/pypy/module/pypyjit/test_pypy_c/test_string.py
@@ -28,7 +28,6 @@
 guard_true(i14, descr=...)
 guard_not_invalidated(descr=...)
 i16 = int_eq(i6, %d)
-guard_false(i16, descr=...)
 i15 = int_mod(i6, i10)
 i17 = int_rshift(i15, %d)
 i18 = int_and(i10, i17)
@@ -68,7 +67,6 @@
 guard_true(i11, descr=...)
 guard_not_invalidated(descr=...)
 i13 = int_eq(i6, %d) # value provided below
-guard_false(i13, descr=...)
 i15 = int_mod(i6, 10)
 i17 = int_rshift(i15, %d)# value provided below
 i18 = int_and(10, i17)
@@ -144,43 +142,6 @@
 jump(..., descr=...)
 """)
 
-def test_getattr_promote(self):
-def main(n):
-class A(object):
-def meth_a(self):
-return 1
-def meth_b(self):
-return 2
-a = A()
-
-l = ['a', 'b']
-s = 0
-for i in range(n):
-name = 'meth_' + l[i & 1]
-meth = getattr(a, name) # ID: getattr
-s += meth()
-return s
-
-log = self.run(main, [1000])
-assert log.result == main(1000)
-loops = log.loops_by_filename(self.filepath)
-assert len(loops) == 1
-for loop in loops:
-assert loop.match_by_id('getattr','''
-guard_not_invalidated?
-i32 = strlen(p31)
-i34 = int_add(5, i32)
-p35 = newstr(i34)
-strsetitem(p35, 0, 109)
-strsetitem(p35, 1, 101)
-strsetitem(p35, 2, 116)
-strsetitem(p35, 3, 104)
-strsetitem(p35, 4, 95)
-copystrcontent(p31, p35, 0, 5, i32)
-i49 = 
call_i(ConstClass(_ll_2_str_eq_nonnull__rpy_stringPtr_rpy_stringPtr), p35, 
ConstPtr(ptr48), descr=)
-guard_value(i49, 1, descr=...)
-''')
-
 def test_remove_duplicate_method_calls(self):
 def main(n):
 lst = []
diff --git a/pypy/objspace/std/mapdict.py b/pypy/objspace/std/mapdict.py
--- a/pypy/objspace/std/mapdict.py
+++ b/pypy/objspace/std/mapdict.py
@@ -1,4 +1,4 @@
-import weakref
+import weakref, sys
 
 from rpython.rlib import jit, objectmodel, debug, rerased
 from rpython.rlib.rarithmetic import intmask, r_uint
@@ -12,6 +12,11 @@
 from pypy.objspace.std.typeobject import MutableCell
 
 
+erase_item, unerase_item = rerased.new_erasing_pair("mapdict storage item")
+erase_map,  unerase_map = rerased.new_erasing_pair("map")
+erase_list, unerase_list = rerased.new_erasing_pair("mapdict storage list")
+
+
 # 
 # attribute shapes
 
@@ -20,6 +25,7 @@
 # note: we use "x * NUM_DIGITS_POW2" instead of "x << NUM_DIGITS" because
 # we want to propagate knowledge that the result cannot be negative
 
+
 class AbstractAttribute(object):
 _immutable_fields_ = ['terminator']
 cache_attrs = None
@@ -151,29 +157,124 @@
 cache[name,

[pypy-commit] pypy default: (cfbolz, fijal around) kill test belonging to 40f823984bbf

[pypy-commit] pypy default: fix fallout from 4ee6b2b67a6b

[pypy-commit] pypy.org extradoc: close numpy donation campaign

[pypy-commit] pypy reorder-map-attributes: close to-be-merged branch

[pypy-commit] pypy reorder-map-attributes: what's new entry

[pypy-commit] pypy default: merge reorder-map-attributes:

[pypy-commit] pypy vlen-resume: finish the refactoring

[pypy-commit] pypy.org pypy3-update: update pypy3 and kill pypy3 campaign, pending a new pypy3 campaign in the futue

[pypy-commit] pypy default: add requirements.txt for untranslated tests

[pypy-commit] pypy cpyext-ext: (rlamy) fix for testing - cache the function

[pypy-commit] pypy cpyext-ext: fix, test PyDictProxy object

[pypy-commit] pypy default: (cfbolz, arigato, plan_rich) progress on fixing the new introduced bug in ll2ctypes, added two tests

[pypy-commit] pypy py3.3: Remove unneeded and broken hack.

[pypy-commit] pypy stmgc-c8: Partial merge with default

[pypy-commit] pypy stmgc-c8: Merge with default

[pypy-commit] pypy stmgc-c8: fix several things from the merges (wip)

[pypy-commit] pypy stmgc-c8: readd changes lost during a merge

[pypy-commit] pypy py3.3: cpyext: Add defines for PyExc_OSError aliases.

[pypy-commit] buildbot default: run own tests in a virtualenv, document the virtualenv requirement

[pypy-commit] pypy cpyext-gc-support-2: merge heads

[pypy-commit] pypy cpyext-gc-support-2: (ronan, arigo)

[pypy-commit] pypy cpyext-gc-support-2: Fix merge

[pypy-commit] pypy cpyext-gc-support-2: fix references to .instancetypedef

[pypy-commit] pypy cpyext-gc-support-2: refcounts don't fit in an 'int' (important after 8c6a66dcb994)

[pypy-commit] pypy cpyext-gc-support-2: (ronan, arigo)

[pypy-commit] pypy regalloc: added a new test file to inspect properties of the regalloc in certain circumstances

[pypy-commit] pypy vlen-resume: close to be merged branch

[pypy-commit] pypy default: (fijal, arigo, cfbolz) Merge vlen-resume which compresses the numberings by

[pypy-commit] pypy cpyext-gc-support-2: hg merge default

[pypy-commit] pypy cpyext-gc-support-2: "backport" of 46e8c7546570

[pypy-commit] pypy stmgc-c8: try harder to not call rgil.acquire() in STM

[pypy-commit] pypy stmgc-c8: remove accidentially added lines

[pypy-commit] pypy default: Fix test by testing with a less commonly used module.

[pypy-commit] pypy py3k: hg merge default

[pypy-commit] pypy py3.3: hg merge py3k

35 matches

Site Navigation

Mail list logo

Footer information