[pypy-commit] pypy default: Added test for fix for issue 1739
Author: Ryan Gonzalez rym...@gmail.com
Branch:
Changeset: r71034:bc7d31eac294
Date: 2014-04-27 22:30 +
http://bitbucket.org/pypy/pypy/changeset/bc7d31eac294/
Log:Added test for fix for issue 1739
diff --git a/rpython/rlib/test/test_streamio.py
b/rpython/rlib/test/test_streamio.py
--- a/rpython/rlib/test/test_streamio.py
+++ b/rpython/rlib/test/test_streamio.py
@@ -1104,6 +1104,25 @@
finally:
signal(SIGALRM, SIG_DFL)
+def test_append_mode(self):
+try:
+fo = streamio.open_file_as_stream # shorthand
+x = fo('.test.file', 'w')
+x.write('abc123')
+x.close()
+
+x = fo('.test.file', 'a')
+x.write('456')
+x.close()
+x = fo('.test.file', 'r')
+assert x.read() == 'abc123456'
+x.close()
+except:
+raise
+finally:
+if os.path.exists('.test.file'):
+os.remove('.test.file')
+
# Speed test
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit
[pypy-commit] pypy default: Removed redundant 'except' clause
Author: Ryan Gonzalez rym...@gmail.com
Branch:
Changeset: r71035:f1b863443b2d
Date: 2014-04-27 23:16 +
http://bitbucket.org/pypy/pypy/changeset/f1b863443b2d/
Log:Removed redundant 'except' clause
diff --git a/rpython/rlib/test/test_streamio.py
b/rpython/rlib/test/test_streamio.py
--- a/rpython/rlib/test/test_streamio.py
+++ b/rpython/rlib/test/test_streamio.py
@@ -1117,8 +1117,6 @@
x = fo('.test.file', 'r')
assert x.read() == 'abc123456'
x.close()
-except:
-raise
finally:
if os.path.exists('.test.file'):
os.remove('.test.file')
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit
[pypy-commit] pypy default: Fix the test:
Author: Armin Rigo ar...@tunes.org
Branch:
Changeset: r71036:d63b49940eff
Date: 2014-04-28 09:30 +0200
http://bitbucket.org/pypy/pypy/changeset/d63b49940eff/
Log:Fix the test:
- make it pass (O_APPEND missing)
- make it fail before the O_APPEND change
diff --git a/rpython/rlib/streamio.py b/rpython/rlib/streamio.py
--- a/rpython/rlib/streamio.py
+++ b/rpython/rlib/streamio.py
@@ -40,7 +40,7 @@
from rpython.rlib import rposix
from rpython.rlib.rstring import StringBuilder
-from os import O_RDONLY, O_WRONLY, O_RDWR, O_CREAT, O_TRUNC
+from os import O_RDONLY, O_WRONLY, O_RDWR, O_CREAT, O_TRUNC, O_APPEND
O_BINARY = getattr(os, O_BINARY, 0)
# (basemode, plus)
diff --git a/rpython/rlib/test/test_streamio.py
b/rpython/rlib/test/test_streamio.py
--- a/rpython/rlib/test/test_streamio.py
+++ b/rpython/rlib/test/test_streamio.py
@@ -1105,21 +1105,19 @@
signal(SIGALRM, SIG_DFL)
def test_append_mode(self):
-try:
-fo = streamio.open_file_as_stream # shorthand
-x = fo('.test.file', 'w')
-x.write('abc123')
-x.close()
+tfn = str(udir.join('streamio-append-mode'))
+fo = streamio.open_file_as_stream # shorthand
+x = fo(tfn, 'w')
+x.write('abc123')
+x.close()
-x = fo('.test.file', 'a')
-x.write('456')
-x.close()
-x = fo('.test.file', 'r')
-assert x.read() == 'abc123456'
-x.close()
-finally:
-if os.path.exists('.test.file'):
-os.remove('.test.file')
+x = fo(tfn, 'a')
+x.seek(0, 0)
+x.write('456')
+x.close()
+x = fo(tfn, 'r')
+assert x.read() == 'abc123456'
+x.close()
# Speed test
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit
[pypy-commit] pypy asmosoinio/fixed-pip-installation-url-github-githu-1398674840188: fixed pip installation URL (github = githubusercontent)
Author: Asmo Soinio asmo.soi...@geckolandmarks.com Branch: asmosoinio/fixed-pip-installation-url-github-githu-1398674840188 Changeset: r71038:79ed26738adf Date: 2014-04-28 09:03 + http://bitbucket.org/pypy/pypy/changeset/79ed26738adf/ Log:fixed pip installation URL (github = githubusercontent) diff --git a/pypy/doc/getting-started.rst b/pypy/doc/getting-started.rst --- a/pypy/doc/getting-started.rst +++ b/pypy/doc/getting-started.rst @@ -76,7 +76,7 @@ .. code-block:: console $ curl -O http://python-distribute.org/distribute_setup.py -$ curl -O https://raw.github.com/pypa/pip/master/contrib/get-pip.py +$ curl -O https://raw.githubusercontent.com/pypa/pip/master/contrib/get-pip.py $ ./pypy-2.1/bin/pypy distribute_setup.py $ ./pypy-2.1/bin/pypy get-pip.py $ ./pypy-2.1/bin/pip install pygments # for example ___ pypy-commit mailing list pypy-commit@python.org https://mail.python.org/mailman/listinfo/pypy-commit
[pypy-commit] benchmarks default: tweak multithreaded raytrace a bit
Author: Remi Meier remi.me...@inf.ethz.ch Branch: Changeset: r253:bec3da064c27 Date: 2014-04-24 15:04 +0200 http://bitbucket.org/pypy/benchmarks/changeset/bec3da064c27/ Log:tweak multithreaded raytrace a bit diff --git a/multithread/raytrace/raytrace.py b/multithread/raytrace/raytrace.py --- a/multithread/raytrace/raytrace.py +++ b/multithread/raytrace/raytrace.py @@ -1,7 +1,7 @@ # From http://www.reddit.com/r/tinycode/comments/169ri9/ray_tracer_in_140_sloc_of_python_with_picture/ # Date: 14.03.2013 -from math import sqrt, pow, pi +from math import sqrt, pi from common.abstract_threading import atomic, Future, set_thread_pool, ThreadPool import time @@ -125,14 +125,14 @@ -def task(x, h, cameraPos, objs, lightSource): -time.sleep(0)# XXX -with atomic: -for y in range(h): +def task(img, x, h, cameraPos, objs, lightSource): +line = img[x] +for y in range(h): +with atomic: ray = Ray(cameraPos, (Vector(x/50.0-5,y/50.0-5,0)-cameraPos).normal()) -trace(ray, objs, lightSource, 10) -time.sleep(0)# XXX +col = trace(ray, objs, lightSource, 10) +line[y] = (col.x + col.y + col.z) / 3.0 return x @@ -157,9 +157,11 @@ lightSource = Vector(0,10,0) cameraPos = Vector(0,0,20) - +img = [] for x in range(w): -future_dispatcher(ths, x, h, cameraPos, objs, lightSource) +img.append([0.0] * h) +for x in range(w): +future_dispatcher(ths, img, x, h, cameraPos, objs, lightSource) for f in futures: print f() ___ pypy-commit mailing list pypy-commit@python.org https://mail.python.org/mailman/listinfo/pypy-commit
[pypy-commit] benchmarks default: a few tweaks
Author: Remi Meier remi.me...@inf.ethz.ch Branch: Changeset: r254:de389b63b99c Date: 2014-04-28 13:50 +0200 http://bitbucket.org/pypy/benchmarks/changeset/de389b63b99c/ Log:a few tweaks diff --git a/multithread/btree/btree.py b/multithread/btree/btree.py --- a/multithread/btree/btree.py +++ b/multithread/btree/btree.py @@ -5,8 +5,6 @@ import random -thread_local = threading.local() - import bisect @@ -195,6 +193,9 @@ def insert(self, item): ancestors = self._path_to(item) +if self._present(item, ancestors): +return False + node, index = ancestors[-1] while getattr(node, children, None): node = node.children[index] @@ -202,6 +203,7 @@ ancestors.append((node, index)) node, index = ancestors.pop() node.insert(index, item, ancestors) +return True def remove(self, item): ancestors = self._path_to(item) @@ -301,18 +303,22 @@ self._root = self.BRANCH(self, contents=seps, children=levels[-1]) +## +## +## + OPS = [BTree.__contains__] * 98 + [BTree.insert, BTree.remove] +ITEM_RANGE = 1 def task(id, tree, ops): print start task with %s ops % ops r = random.Random() r.seed(id) -thread_local.rnd = r for _ in xrange(ops): op = r.choice(OPS) -elem = r.randint(1, 1) +elem = r.randint(1, ITEM_RANGE) with atomic: op(tree, elem) @@ -331,11 +337,10 @@ operations = int(operations) set_thread_pool(ThreadPool(threads)) -thread_local.rnd = random tree = BTree(20) for _ in xrange(1000): -tree.insert(random.randint(1, 1000)) +tree.insert(random.randint(1, ITEM_RANGE)) c_len = operations // threads fs = [] diff --git a/multithread/common/abstract_threading.py b/multithread/common/abstract_threading.py --- a/multithread/common/abstract_threading.py +++ b/multithread/common/abstract_threading.py @@ -1,6 +1,6 @@ from Queue import Queue, Empty, Full -from threading import Thread, Condition, Lock -import thread, atexit, sys +from threading import Thread, Condition, Lock, local +import thread, atexit, sys, time try: from __pypy__.thread import atomic, getsegmentlimit @@ -10,6 +10,31 @@ return 1 +class TLQueue(object): +def __init__(self): +self.items = [] +self._new_items = Condition() + +def put(self, v): +self.items.append(v) +with self._new_items: +self._new_items.notify_all() + +def get(self): +items = self.items +with atomic: +if items: +return items.pop() + +while True: +with self._new_items: +with atomic: +if items: +return items.pop() + +self._new_items.wait() + + class Worker(Thread): Thread executing tasks from a given tasks queue def __init__(self, queue): @@ -29,7 +54,7 @@ class ThreadPool(object): def __init__(self, n_workers=None): -self.input_queue = Queue() +self.input_queue = TLQueue() if n_workers is None: n_workers = getsegmentlimit() self.workers = [Worker(self.input_queue) for i in range(n_workers)] diff --git a/multithread/skiplist/skiplist.py b/multithread/skiplist/skiplist.py --- a/multithread/skiplist/skiplist.py +++ b/multithread/skiplist/skiplist.py @@ -86,6 +86,7 @@ OPS = [SkipList.find] * 98 + [SkipList.insert, SkipList.remove] +ITEM_RANGE = 1 def task(id, slist, ops): print start task with %s ops % ops @@ -95,7 +96,7 @@ for _ in xrange(ops): op = r.choice(OPS) -elem = r.randint(1, 1) +elem = r.randint(1, ITEM_RANGE) with atomic: op(slist, elem) @@ -118,7 +119,7 @@ slist = SkipList() for _ in xrange(1000): -slist.insert(random.randint(1, 1000)) +slist.insert(random.randint(1, ITEM_RANGE)) c_len = operations // threads fs = [] ___ pypy-commit mailing list pypy-commit@python.org https://mail.python.org/mailman/listinfo/pypy-commit
[pypy-commit] pypy default: rlib parsing token_class extended with token_position_class
Author: Sebastian Pawlu? sebastian.paw...@gmail.com Branch: Changeset: r71040:01046366784a Date: 2014-04-28 15:48 +0200 http://bitbucket.org/pypy/pypy/changeset/01046366784a/ Log:rlib parsing token_class extended with token_position_class diff --git a/rpython/rlib/parsing/lexer.py b/rpython/rlib/parsing/lexer.py --- a/rpython/rlib/parsing/lexer.py +++ b/rpython/rlib/parsing/lexer.py @@ -107,7 +107,7 @@ self.matcher = matcher self.lineno = 0 self.columnno = 0 - + def find_next_token(self): while 1: self.state = 0 @@ -126,8 +126,8 @@ i = ~i stop = self.last_matched_index + 1 assert stop = 0 -if start == stop: -source_pos = SourcePos(i - 1, self.lineno, self.columnno) +if start == stop: +source_pos = self.token_position_class(i - 1, self.lineno, self.columnno) raise deterministic.LexerError(self.text, self.state, source_pos) source = self.text[start:stop] @@ -147,7 +147,7 @@ else: raise StopIteration return result -source_pos = SourcePos(i - 1, self.lineno, self.columnno) +source_pos = self.token_position_class(i - 1, self.lineno, self.columnno) raise deterministic.LexerError(self.text, self.state, source_pos) def adjust_position(self, token): @@ -158,7 +158,7 @@ self.columnno += len(token) else: self.columnno = token.rfind(\n) - + #def inner_loop(self, i): #while i len(self.text): #char = self.text[i] @@ -186,10 +186,15 @@ class LexingDFARunner(AbstractLexingDFARunner): def __init__(self, matcher, automaton, text, ignore, eof=False, token_class=None): -if token_class is None: + +if not token_class: self.token_class = Token +self.token_position_class = SourcePos + else: self.token_class = token_class +self.token_position_class = token_class.source_position_class + AbstractLexingDFARunner.__init__(self, matcher, automaton, text, eof) self.ignore = ignore @@ -198,7 +203,8 @@ def make_token(self, index, state, text, eof=False): assert (eof and state == -1) or 0 = state len(self.automaton.names) -source_pos = SourcePos(index, self.lineno, self.columnno) + +source_pos = self.token_position_class(index, self.lineno, self.columnno) if eof: return self.token_class(EOF, EOF, source_pos) return self.token_class(self.automaton.names[self.last_matched_state], ___ pypy-commit mailing list pypy-commit@python.org https://mail.python.org/mailman/listinfo/pypy-commit
[pypy-commit] pypy default: Merged in xando/pypy/lexer_token_position_class (pull request #236)
Author: Maciej Fijalkowski fij...@gmail.com Branch: Changeset: r71042:f725787e0d52 Date: 2014-04-28 16:02 +0200 http://bitbucket.org/pypy/pypy/changeset/f725787e0d52/ Log:Merged in xando/pypy/lexer_token_position_class (pull request #236) Extending LexingDFARunner with custom token_position_class diff --git a/rpython/rlib/parsing/lexer.py b/rpython/rlib/parsing/lexer.py --- a/rpython/rlib/parsing/lexer.py +++ b/rpython/rlib/parsing/lexer.py @@ -107,7 +107,7 @@ self.matcher = matcher self.lineno = 0 self.columnno = 0 - + def find_next_token(self): while 1: self.state = 0 @@ -126,8 +126,8 @@ i = ~i stop = self.last_matched_index + 1 assert stop = 0 -if start == stop: -source_pos = SourcePos(i - 1, self.lineno, self.columnno) +if start == stop: +source_pos = self.token_position_class(i - 1, self.lineno, self.columnno) raise deterministic.LexerError(self.text, self.state, source_pos) source = self.text[start:stop] @@ -147,7 +147,7 @@ else: raise StopIteration return result -source_pos = SourcePos(i - 1, self.lineno, self.columnno) +source_pos = self.token_position_class(i - 1, self.lineno, self.columnno) raise deterministic.LexerError(self.text, self.state, source_pos) def adjust_position(self, token): @@ -158,7 +158,7 @@ self.columnno += len(token) else: self.columnno = token.rfind(\n) - + #def inner_loop(self, i): #while i len(self.text): #char = self.text[i] @@ -186,10 +186,15 @@ class LexingDFARunner(AbstractLexingDFARunner): def __init__(self, matcher, automaton, text, ignore, eof=False, token_class=None): -if token_class is None: + +if not token_class: self.token_class = Token +self.token_position_class = SourcePos + else: self.token_class = token_class +self.token_position_class = token_class.source_position_class + AbstractLexingDFARunner.__init__(self, matcher, automaton, text, eof) self.ignore = ignore @@ -198,8 +203,10 @@ def make_token(self, index, state, text, eof=False): assert (eof and state == -1) or 0 = state len(self.automaton.names) -source_pos = SourcePos(index, self.lineno, self.columnno) + +source_pos = self.token_position_class(index, self.lineno, self.columnno) if eof: return self.token_class(EOF, EOF, source_pos) + return self.token_class(self.automaton.names[self.last_matched_state], text, source_pos) ___ pypy-commit mailing list pypy-commit@python.org https://mail.python.org/mailman/listinfo/pypy-commit
[pypy-commit] pypy lexer_token_position_class: new line
Author: Sebastian Pawlu? sebastian.paw...@gmail.com Branch: lexer_token_position_class Changeset: r71041:401c05e2904d Date: 2014-04-28 15:55 +0200 http://bitbucket.org/pypy/pypy/changeset/401c05e2904d/ Log:new line diff --git a/rpython/rlib/parsing/lexer.py b/rpython/rlib/parsing/lexer.py --- a/rpython/rlib/parsing/lexer.py +++ b/rpython/rlib/parsing/lexer.py @@ -207,5 +207,6 @@ source_pos = self.token_position_class(index, self.lineno, self.columnno) if eof: return self.token_class(EOF, EOF, source_pos) + return self.token_class(self.automaton.names[self.last_matched_state], text, source_pos) ___ pypy-commit mailing list pypy-commit@python.org https://mail.python.org/mailman/listinfo/pypy-commit
[pypy-commit] benchmarks default: more tweaks
Author: Remi Meier remi.me...@inf.ethz.ch
Branch:
Changeset: r255:2193a2976d5b
Date: 2014-04-28 16:35 +0200
http://bitbucket.org/pypy/benchmarks/changeset/2193a2976d5b/
Log:more tweaks
diff --git a/multithread/bench.py b/multithread/bench.py
--- a/multithread/bench.py
+++ b/multithread/bench.py
@@ -34,7 +34,7 @@
def get_error(times):
ts = sorted(times)[:args.k]
best = float(ts[0])
-
+
return max((t / best) - 1.0 for t in ts)
def within_error(args, times):
@@ -51,6 +51,7 @@
test = import_file(os.path.basename(args.file))
times = []
+results = []
k = 1
try:
while True:
@@ -60,14 +61,15 @@
test_time = time.time()
if args.p:
-test.run(*args.more)
+results.append(test.run(*args.more))
else:
with nostdout():
-test.run(*args.more)
+results.append(test.run(*args.more))
times.append(time.time() - test_time)
if not args.q:
print took {} s.format(times[-1])
+print returned, results[-1]
if k = args.k:
if within_error(args, times):
@@ -83,7 +85,9 @@
k += 1
finally:
if not args.q:
-print times:, times
+print == times ==\n, \n.join(map(str, times))
+print == reported results ==\n, \n.join(
+map(str, filter(None, results)))
if times:
times = sorted(times)[:args.k]
diff --git a/multithread/btree/btree.py b/multithread/btree/btree.py
--- a/multithread/btree/btree.py
+++ b/multithread/btree/btree.py
@@ -344,16 +344,18 @@
c_len = operations // threads
fs = []
+parallel_time = time.time()
for i in xrange(threads):
fs.append(Future(task, i, tree, c_len))
for f in fs:
f()
-
+parallel_time = time.time() - parallel_time
# print tree:
# print tree
# shutdown current pool
set_thread_pool(None)
+return parallel_time
diff --git a/multithread/common/abstract_threading.py
b/multithread/common/abstract_threading.py
--- a/multithread/common/abstract_threading.py
+++ b/multithread/common/abstract_threading.py
@@ -10,6 +10,49 @@
return 1
+class TLQueue_concurrent(object):
+def __init__(self):
+my_id = thread.get_ident()
+self._tl_items = {my_id : []}
+self._new_items = Condition()
+self._c = 0
+
+def put(self, v):
+# conflicts with any put() and get()s from
+# the chosen queue:
+c = (id(v) // 5) % len(self._tl_items)
+items = self._tl_items.values()[c]
+
+with self._new_items:
+items.append(v)
+self._new_items.notify_all()
+
+def _get_my_items(self):
+my_id = thread.get_ident()
+try:
+items = self._tl_items[my_id]
+except KeyError:
+items = []
+self._tl_items[my_id] = items
+return items
+
+def get(self):
+# tries first to get item from its
+# own thread-local queue
+items = self._get_my_items()
+with atomic:
+if items:
+return items.pop()
+
+while True:
+with self._new_items:
+# steal from other queues
+for its in self._tl_items.values():
+with atomic:
+if its:
+return its.pop()
+self._new_items.wait()
+
class TLQueue(object):
def __init__(self):
self.items = []
diff --git a/multithread/mandelbrot/mandelbrot.py
b/multithread/mandelbrot/mandelbrot.py
--- a/multithread/mandelbrot/mandelbrot.py
+++ b/multithread/mandelbrot/mandelbrot.py
@@ -1,5 +1,5 @@
from common.abstract_threading import atomic, Future, set_thread_pool,
ThreadPool
-import sys
+import sys, time
def calculate(a, b, im_size, max_iter=255):
@@ -11,8 +11,7 @@
real_step = (br - ar) / (width - 1)
print real/width:%s, imag/height:%s % (real_step, imag_step)
-with atomic:
-result = [[0] * width for y in xrange(height)]
+result = [[0] * width for y in xrange(height)]
for y in xrange(height):
zi = ai + y * imag_step
for x in xrange(width):
@@ -64,6 +63,7 @@
res = []
ai = -1.5
bi = ai + step
+parallel_time = time.time()
for i in xrange(threads):
res.append(Future(calculate,
a=(ar, ai + i * step),
@@ -72,9 +72,11 @@
))
res = [f() for f in res]
+parallel_time = time.time() - parallel_time
set_thread_pool(None)
-return merge_imgs(res)
+merge_imgs(res)
+return parallel_time
diff --git a/multithread/raytrace/raytrace.py b/multithread/raytrace/raytrace.py
--- a/multithread/raytrace/raytrace.py
+++ b/multithread/raytrace/raytrace.py
@@ -160,15 +160,18 @@
img
[pypy-commit] stmgc marker: in-progress
Author: Armin Rigo ar...@tunes.org
Branch: marker
Changeset: r1186:a8f77868840a
Date: 2014-04-28 15:40 +0200
http://bitbucket.org/pypy/stmgc/changeset/a8f77868840a/
Log:in-progress
diff --git a/c7/stm/contention.c b/c7/stm/contention.c
--- a/c7/stm/contention.c
+++ b/c7/stm/contention.c
@@ -196,6 +196,10 @@
/* We have to signal the other thread to abort, and wait until
it does. */
contmgr.other_pseg-pub.nursery_end = abort_category;
+if (kind == WRITE_WRITE_CONTENTION) {
+//marker_fetch_obj_write(contmgr.other_pseg-pub.segment_num,
+// obj, contmgr.other_pseg-...);
+}
int sp = contmgr.other_pseg-safe_point;
switch (sp) {
diff --git a/c7/stm/core.h b/c7/stm/core.h
--- a/c7/stm/core.h
+++ b/c7/stm/core.h
@@ -174,7 +174,7 @@
/* Temporarily stores the marker information */
char marker_self[_STM_MARKER_LEN];
char marker_other[_STM_MARKER_LEN];
-uintptr_t marker_inev[2];
+uintptr_t marker_inev[2]; /* marker where this thread became inevitable */
};
enum /* safe_point */ {
diff --git a/c7/stm/marker.c b/c7/stm/marker.c
--- a/c7/stm/marker.c
+++ b/c7/stm/marker.c
@@ -32,10 +32,9 @@
static void marker_expand(uintptr_t marker[2], char *segment_base,
char *outmarker)
{
+outmarker[0] = 0;
if (marker[0] == 0)
return; /* no marker entry found */
-if (outmarker[0] != 0)
-return; /* already collected an entry */
if (stmcb_expand_marker != NULL) {
stmcb_expand_marker(segment_base, marker[0], (object_t *)marker[1],
outmarker, _STM_MARKER_LEN);
@@ -44,9 +43,13 @@
static void marker_fetch_expand(struct stm_priv_segment_info_s *pseg)
{
+if (pseg-marker_self[0] != 0)
+return; /* already collected an entry */
+
uintptr_t marker[2];
marker_fetch(pseg-pub.running_thread, marker);
marker_expand(marker, pseg-pub.segment_base, pseg-marker_self);
+pseg-marker_other[0] = 0;
}
char *_stm_expand_marker(void)
@@ -85,33 +88,11 @@
pseg-marker_other[0] = 0;
}
-static void marker_lookup_from_thread(struct stm_priv_segment_info_s *pseg,
- object_t *obj, char *outmarker)
+static void marker_fetch_obj_write(uint8_t in_segment_num, object_t *obj,
+ uintptr_t marker[2])
{
-outmarker[0] = 0;
-
-long i;
-struct list_s *mlst = pseg-modified_old_objects;
-struct list_s *mlstm = pseg-modified_old_objects_markers;
-for (i = list_count(mlst); --i = 0; ) {
-if (list_item(mlst, i) == (uintptr_t)obj) {
-uintptr_t marker[2];
-assert(list_count(mlstm) == 2 * list_count(mlst));
-marker[0] = list_item(mlstm, i * 2 + 0);
-marker[1] = list_item(mlstm, i * 2 + 1);
-
-marker_expand(marker, pseg-pub.segment_base, outmarker);
-break;
-}
-}
-}
-
-static void marker_lookup_other_thread_write_write(uint8_t other_segment_num,
- object_t *obj)
-{
-struct stm_priv_segment_info_s *my_pseg, *other_pseg;
-char *other_segment_base = get_segment_base(other_segment_num);
-acquire_segment_lock(other_segment_base);
+char *segment_base = get_segment_base(in_segment_num);
+acquire_segment_lock(segment_base);
assert(_has_mutex());
/* here, we acquired the other thread's segment_lock, which means that:
@@ -122,12 +103,35 @@
(2) it is not mutating 'modified_old_objects' right now (we have
the global mutex_lock at this point too).
*/
+long i;
+struct stm_priv_segment_info_s *pseg = get_priv_segment(in_segment_num);
+struct list_s *mlst = pseg-modified_old_objects;
+struct list_s *mlstm = pseg-modified_old_objects_markers;
+for (i = list_count(mlst); --i = 0; ) {
+if (list_item(mlst, i) == (uintptr_t)obj) {
+assert(list_count(mlstm) == 2 * list_count(mlst));
+marker[0] = list_item(mlstm, i * 2 + 0);
+marker[1] = list_item(mlstm, i * 2 + 1);
+goto done;
+}
+}
+marker[0] = 0;
+marker[1] = 0;
+ done:
+release_segment_lock(segment_base);
+}
+
+static void marker_lookup_other_thread_write_write(uint8_t other_segment_num,
+ object_t *obj)
+{
+uintptr_t marker[2];
+marker_fetch_obj_write(other_segment_num, obj, marker);
+
+struct stm_priv_segment_info_s *my_pseg, *other_pseg;
+other_pseg = get_priv_segment(other_segment_num);
my_pseg = get_priv_segment(STM_SEGMENT-segment_num);
-other_pseg = get_priv_segment(other_segment_num);
-
-marker_lookup_from_thread(other_pseg, obj, my_pseg-marker_other);
-
-release_segment_lock(other_segment_base);
+my_pseg-marker_other[0] = 0;
+marker_expand(marker, other_pseg-pub.segment_base,
[pypy-commit] stmgc marker: hg merge default
Author: Armin Rigo ar...@tunes.org
Branch: marker
Changeset: r1187:06fa05eeb305
Date: 2014-04-28 17:15 +0200
http://bitbucket.org/pypy/stmgc/changeset/06fa05eeb305/
Log:hg merge default
diff --git a/c7/demo/demo2.c b/c7/demo/demo2.c
--- a/c7/demo/demo2.c
+++ b/c7/demo/demo2.c
@@ -44,6 +44,8 @@
visit((object_t **)n-next);
}
+void stmcb_commit_soon() {}
+
static void expand_marker(char *base, uintptr_t odd_number,
object_t *following_object,
char *outputbuf, size_t outputbufsize)
diff --git a/c7/demo/demo_largemalloc.c b/c7/demo/demo_largemalloc.c
--- a/c7/demo/demo_largemalloc.c
+++ b/c7/demo/demo_largemalloc.c
@@ -23,6 +23,8 @@
abort();
}
+void stmcb_commit_soon() {}
+
//
#define ARENA_SIZE (1024*1024*1024)
diff --git a/c7/demo/demo_random.c b/c7/demo/demo_random.c
--- a/c7/demo/demo_random.c
+++ b/c7/demo/demo_random.c
@@ -79,6 +79,8 @@
assert(n-next == *last_next);
}
+void stmcb_commit_soon() {}
+
int get_rand(int max)
{
if (max == 0)
diff --git a/c7/demo/demo_simple.c b/c7/demo/demo_simple.c
--- a/c7/demo/demo_simple.c
+++ b/c7/demo/demo_simple.c
@@ -39,6 +39,8 @@
visit((object_t **)n-next);
}
+void stmcb_commit_soon() {}
+
static sem_t done;
diff --git a/c7/stm/contention.c b/c7/stm/contention.c
--- a/c7/stm/contention.c
+++ b/c7/stm/contention.c
@@ -165,7 +165,8 @@
change_timing_state(wait_category);
-/* XXX should also tell other_pseg please commit soon */
+/* tell the other to commit ASAP */
+signal_other_to_commit_soon(contmgr.other_pseg);
dprintf((pausing...\n));
cond_signal(C_AT_SAFE_POINT);
@@ -181,6 +182,9 @@
}
else if (!contmgr.abort_other) {
+/* tell the other to commit ASAP, since it causes aborts */
+signal_other_to_commit_soon(contmgr.other_pseg);
+
dprintf((abort in contention\n));
STM_SEGMENT-nursery_end = abort_category;
if (kind == WRITE_WRITE_CONTENTION)
@@ -267,6 +271,13 @@
abort_data_structures_from_segment_num(other_segment_num);
}
dprintf((killed other thread\n));
+
+/* we should commit soon, we caused an abort */
+
//signal_other_to_commit_soon(get_priv_segment(STM_SEGMENT-segment_num));
+if (!STM_PSEGMENT-signalled_to_commit_soon) {
+STM_PSEGMENT-signalled_to_commit_soon = true;
+stmcb_commit_soon();
+}
}
}
diff --git a/c7/stm/contention.h b/c7/stm/contention.h
--- a/c7/stm/contention.h
+++ b/c7/stm/contention.h
@@ -6,7 +6,8 @@
static void inevitable_contention_management(uint8_t other_segment_num);
static inline bool is_abort(uintptr_t nursery_end) {
-return (nursery_end = _STM_NSE_SIGNAL_MAX nursery_end != NSE_SIGPAUSE);
+return (nursery_end = _STM_NSE_SIGNAL_MAX nursery_end != NSE_SIGPAUSE
+ nursery_end != NSE_SIGCOMMITSOON);
}
static inline bool is_aborting_now(uint8_t other_segment_num) {
diff --git a/c7/stm/core.c b/c7/stm/core.c
--- a/c7/stm/core.c
+++ b/c7/stm/core.c
@@ -14,13 +14,10 @@
#define EVENTUALLY(condition) \
{ \
if (!(condition)) { \
-int _i; \
-for (_i = 1; _i = NB_SEGMENTS; _i++) \
-spinlock_acquire(lock_pages_privatizing[_i]); \
+acquire_privatization_lock(); \
if (!(condition)) \
stm_fatalerror(fails: #condition); \
-for (_i = 1; _i = NB_SEGMENTS; _i++) \
-spinlock_release(lock_pages_privatizing[_i]); \
+release_privatization_lock(); \
} \
}
#endif
@@ -78,11 +75,11 @@
if (write_locks[lock_idx] == 0) {
/* A lock to prevent reading garbage from
lookup_other_thread_recorded_marker() */
-acquire_segment_lock(STM_SEGMENT-segment_base);
+acquire_marker_lock(STM_SEGMENT-segment_base);
if (UNLIKELY(!__sync_bool_compare_and_swap(write_locks[lock_idx],
0, lock_num))) {
-release_segment_lock(STM_SEGMENT-segment_base);
+release_marker_lock(STM_SEGMENT-segment_base);
goto retry;
}
@@ -99,7 +96,7 @@
list_append2(STM_PSEGMENT-modified_old_objects_markers,
marker[0], marker[1]);
-release_segment_lock(STM_SEGMENT-segment_base);
+release_marker_lock(STM_SEGMENT-segment_base);
/* We need to privatize the pages containing the object, if they
are still
[pypy-commit] stmgc marker: in-progress: clean-up
Author: Armin Rigo ar...@tunes.org
Branch: marker
Changeset: r1188:fe38fc7a7c18
Date: 2014-04-28 17:53 +0200
http://bitbucket.org/pypy/stmgc/changeset/fe38fc7a7c18/
Log:in-progress: clean-up
diff --git a/c7/stm/contention.c b/c7/stm/contention.c
--- a/c7/stm/contention.c
+++ b/c7/stm/contention.c
@@ -187,12 +187,7 @@
dprintf((abort in contention\n));
STM_SEGMENT-nursery_end = abort_category;
-if (kind == WRITE_WRITE_CONTENTION)
-marker_lookup_other_thread_write_write(other_segment_num, obj);
-else if (kind == INEVITABLE_CONTENTION)
-marker_lookup_other_thread_inev(other_segment_num);
-else if (kind == WRITE_READ_CONTENTION)
-marker_lookup_same_thread_write_read(obj);
+marker_contention_abort_self(abort_category, other_segment_num, obj);
abort_with_mutex();
}
diff --git a/c7/stm/core.c b/c7/stm/core.c
--- a/c7/stm/core.c
+++ b/c7/stm/core.c
@@ -635,8 +635,9 @@
(int)pseg-transaction_state);
}
-/* look up and preserve the marker information as a string */
-marker_fetch_expand(pseg);
+/* if we don't have marker information already, look up and preserve
+ the marker information from the shadowstack as a string */
+marker_default_for_abort(pseg);
/* throw away the content of the nursery */
long bytes_in_nursery = throw_away_nursery(pseg);
diff --git a/c7/stm/marker.c b/c7/stm/marker.c
--- a/c7/stm/marker.c
+++ b/c7/stm/marker.c
@@ -13,25 +13,40 @@
static void marker_fetch(stm_thread_local_t *tl, uintptr_t marker[2])
{
+/* fetch the current marker from the tl's shadow stack,
+ and return it in 'marker[2]'. */
struct stm_shadowentry_s *current = tl-shadowstack - 1;
struct stm_shadowentry_s *base = tl-shadowstack_base;
-/* stop walking just before shadowstack_base, which contains
- STM_STACK_MARKER_OLD which shouldn't be expanded */
-while (--current base) {
-if (((uintptr_t)current-ss) 1) {
-/* found the odd marker */
-marker[0] = (uintptr_t)current[0].ss;
-marker[1] = (uintptr_t)current[1].ss;
-return;
-}
+
+/* The shadowstack_base contains STM_STACK_MARKER_OLD, which is
+ a convenient stopper for the loop below but which shouldn't
+ be returned. */
+assert(base-ss == (object_t *)STM_STACK_MARKER_OLD);
+
+while (!(((uintptr_t)current-ss) 1)) {
+current--;
+assert(current = base);
}
-marker[0] = 0;
-marker[1] = 0;
+if (current != base) {
+/* found the odd marker */
+marker[0] = (uintptr_t)current[0].ss;
+marker[1] = (uintptr_t)current[1].ss;
+}
+else {
+/* no marker found */
+marker[0] = 0;
+marker[1] = 0;
+}
}
static void marker_expand(uintptr_t marker[2], char *segment_base,
char *outmarker)
{
+/* Expand the marker given by 'marker[2]' into a full string. This
+ works assuming that the marker was produced inside the segment
+ given by 'segment_base'. If that's from a different thread, you
+ must first acquire the corresponding 'marker_lock'. */
+assert(_has_mutex());
outmarker[0] = 0;
if (marker[0] == 0)
return; /* no marker entry found */
@@ -41,7 +56,7 @@
}
}
-static void marker_fetch_expand(struct stm_priv_segment_info_s *pseg)
+static void marker_default_for_abort(struct stm_priv_segment_info_s *pseg)
{
if (pseg-marker_self[0] != 0)
return; /* already collected an entry */
@@ -58,8 +73,10 @@
static char _result[_STM_MARKER_LEN];
uintptr_t marker[2];
_result[0] = 0;
+s_mutex_lock();
marker_fetch(STM_SEGMENT-running_thread, marker);
marker_expand(marker, STM_SEGMENT-segment_base, _result);
+s_mutex_unlock();
return _result;
}
@@ -91,8 +108,6 @@
static void marker_fetch_obj_write(uint8_t in_segment_num, object_t *obj,
uintptr_t marker[2])
{
-char *segment_base = get_segment_base(in_segment_num);
-acquire_marker_lock(segment_base);
assert(_has_mutex());
/* here, we acquired the other thread's marker_lock, which means that:
@@ -112,50 +127,58 @@
assert(list_count(mlstm) == 2 * list_count(mlst));
marker[0] = list_item(mlstm, i * 2 + 0);
marker[1] = list_item(mlstm, i * 2 + 1);
-goto done;
+return;
}
}
marker[0] = 0;
marker[1] = 0;
- done:
-release_marker_lock(segment_base);
}
-static void marker_lookup_other_thread_write_write(uint8_t other_segment_num,
- object_t *obj)
+static void marker_contention_abort_self(int category,
+ uint8_t other_segment_num,
+ object_t *obj)
{
-uintptr_t marker[2];
-
[pypy-commit] stmgc marker: Find out how to reuse the logic.
Author: Armin Rigo ar...@tunes.org
Branch: marker
Changeset: r1189:6b462cb2ba16
Date: 2014-04-28 18:04 +0200
http://bitbucket.org/pypy/stmgc/changeset/6b462cb2ba16/
Log:Find out how to reuse the logic.
diff --git a/c7/stm/contention.c b/c7/stm/contention.c
--- a/c7/stm/contention.c
+++ b/c7/stm/contention.c
@@ -187,7 +187,7 @@
dprintf((abort in contention\n));
STM_SEGMENT-nursery_end = abort_category;
-marker_contention_abort_self(abort_category, other_segment_num, obj);
+marker_contention(abort_category, false, other_segment_num, obj);
abort_with_mutex();
}
@@ -195,10 +195,7 @@
/* We have to signal the other thread to abort, and wait until
it does. */
contmgr.other_pseg-pub.nursery_end = abort_category;
-if (kind == WRITE_WRITE_CONTENTION) {
-//marker_fetch_obj_write(contmgr.other_pseg-pub.segment_num,
-// obj, contmgr.other_pseg-...);
-}
+marker_contention(abort_category, true, other_segment_num, obj);
int sp = contmgr.other_pseg-safe_point;
switch (sp) {
diff --git a/c7/stm/marker.c b/c7/stm/marker.c
--- a/c7/stm/marker.c
+++ b/c7/stm/marker.c
@@ -134,9 +134,8 @@
marker[1] = 0;
}
-static void marker_contention_abort_self(int category,
- uint8_t other_segment_num,
- object_t *obj)
+static void marker_contention(int category, bool abort_other,
+ uint8_t other_segment_num, object_t *obj)
{
uintptr_t self_marker[2];
uintptr_t other_marker[2];
@@ -148,25 +147,30 @@
char *my_segment_base = STM_SEGMENT-segment_base;
char *other_segment_base = get_segment_base(other_segment_num);
-/* I'm aborting. Collect the location for myself. It's usually
- the current location, except in a write-read abort, in which
- case it's the older location of the write. */
+acquire_marker_lock(other_segment_base);
+
+/* Collect the location for myself. It's usually the current
+ location, except in a write-read abort, in which case it's the
+ older location of the write. */
if (category == STM_TIME_RUN_ABORTED_WRITE_READ)
marker_fetch_obj_write(my_pseg-pub.segment_num, obj, self_marker);
else
marker_fetch(my_pseg-pub.running_thread, self_marker);
-marker_expand(self_marker, my_segment_base, my_pseg-marker_self);
+/* Expand this location into either my_pseg-marker_self or
+ other_pseg-marker_other, depending on who aborts. */
+marker_expand(self_marker, my_segment_base,
+ abort_other ? other_pseg-marker_other
+ : my_pseg-marker_self);
/* For some categories, we can also collect the relevant information
for the other segment. */
-acquire_marker_lock(other_segment_base);
-
switch (category) {
case STM_TIME_RUN_ABORTED_WRITE_WRITE:
marker_fetch_obj_write(other_segment_num, obj, other_marker);
break;
case STM_TIME_RUN_ABORTED_INEVITABLE:
+assert(abort_other == false);
other_marker[0] = other_pseg-marker_inev[0];
other_marker[1] = other_pseg-marker_inev[1];
break;
@@ -176,7 +180,16 @@
break;
}
-marker_expand(other_marker, other_segment_base, my_pseg-marker_other);
+marker_expand(other_marker, other_segment_base,
+ abort_other ? other_pseg-marker_self
+ : my_pseg-marker_other);
+
+if (abort_other other_pseg-marker_self[0] == 0) {
+if (category == STM_TIME_RUN_ABORTED_WRITE_READ)
+strcpy(other_pseg-marker_self, read at unknown location);
+else
+strcpy(other_pseg-marker_self, no location information);
+}
release_marker_lock(other_segment_base);
}
diff --git a/c7/stm/marker.h b/c7/stm/marker.h
--- a/c7/stm/marker.h
+++ b/c7/stm/marker.h
@@ -8,6 +8,5 @@
struct stm_priv_segment_info_s *pseg,
enum stm_time_e attribute_to, double time);
-static void marker_contention_abort_self(int category,
- uint8_t other_segment_num,
- object_t *obj);
+static void marker_contention(int category, bool abort_other,
+ uint8_t other_segment_num, object_t *obj);
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit
[pypy-commit] stmgc marker: A passing test
Author: Armin Rigo ar...@tunes.org Branch: marker Changeset: r1190:3e7dc1e81647 Date: 2014-04-28 18:08 +0200 http://bitbucket.org/pypy/stmgc/changeset/3e7dc1e81647/ Log:A passing test diff --git a/c7/test/test_marker.py b/c7/test/test_marker.py --- a/c7/test/test_marker.py +++ b/c7/test/test_marker.py @@ -299,3 +299,36 @@ assert tl.longest_marker_state == lib.STM_TIME_RUN_ABORTED_WRITE_WRITE assert ffi.string(tl.longest_marker_self) == '19' assert ffi.string(tl.longest_marker_other) == '21' + +def test_double_remote_markers_cb_write_read(self): +@ffi.callback(void(char *, uintptr_t, object_t *, char *, size_t)) +def expand_marker(base, number, ptr, outbuf, outbufsize): +s = '%d\x00' % (number,) +assert len(s) = outbufsize +outbuf[0:len(s)] = s +lib.stmcb_expand_marker = expand_marker +p = stm_allocate_old(16) +# +self.start_transaction() +assert stm_get_char(p) == '\x00'# read +tl0 = self.get_stm_thread_local() +# +self.switch(1) +self.start_transaction() +self.become_inevitable() +self.push_root(ffi.cast(object_t *, 21)) +self.push_root(ffi.cast(object_t *, ffi.NULL)) +stm_set_char(p, 'B')# write, will abort #0 +self.pop_root() +self.pop_root() +self.push_root(ffi.cast(object_t *, 23)) +self.push_root(ffi.cast(object_t *, ffi.NULL)) +self.commit_transaction() +# +py.test.raises(Conflict, self.switch, 0) +# +tl = self.get_stm_thread_local() +assert tl is tl0 +assert tl.longest_marker_state == lib.STM_TIME_RUN_ABORTED_WRITE_READ +assert ffi.string(tl.longest_marker_self)=='read at unknown location' +assert ffi.string(tl.longest_marker_other) == '21' ___ pypy-commit mailing list pypy-commit@python.org https://mail.python.org/mailman/listinfo/pypy-commit
[pypy-commit] stmgc marker: Report markers for forced pauses. Seems to work but hard to test for now...
Author: Armin Rigo ar...@tunes.org
Branch: marker
Changeset: r1191:4bde66e3b621
Date: 2014-04-28 18:36 +0200
http://bitbucket.org/pypy/stmgc/changeset/4bde66e3b621/
Log:Report markers for forced pauses. Seems to work but hard to test
for now...
diff --git a/c7/demo/demo2.c b/c7/demo/demo2.c
--- a/c7/demo/demo2.c
+++ b/c7/demo/demo2.c
@@ -51,7 +51,7 @@
char *outputbuf, size_t outputbufsize)
{
assert(following_object == NULL);
-snprintf(outputbuf, outputbufsize, %lu, odd_number);
+snprintf(outputbuf, outputbufsize, %p %lu, base, odd_number);
}
@@ -98,6 +98,18 @@
STM_START_TRANSACTION(stm_thread_local, here);
+if (stm_thread_local.longest_marker_state != 0) {
+fprintf(stderr, [%p] marker %d for %.6f seconds:\n,
+stm_thread_local,
+stm_thread_local.longest_marker_state,
+stm_thread_local.longest_marker_time);
+fprintf(stderr, \tself:\t\%s\\n\tother:\t\%s\\n,
+stm_thread_local.longest_marker_self,
+stm_thread_local.longest_marker_other);
+stm_thread_local.longest_marker_state = 0;
+stm_thread_local.longest_marker_time = 0.0;
+}
+
nodeptr_t prev = initial;
stm_read((objptr_t)prev);
diff --git a/c7/stm/contention.c b/c7/stm/contention.c
--- a/c7/stm/contention.c
+++ b/c7/stm/contention.c
@@ -162,6 +162,7 @@
itself already paused here.
*/
contmgr.other_pseg-signal_when_done = true;
+marker_contention(kind, false, other_segment_num, obj);
change_timing_state(wait_category);
@@ -178,7 +179,13 @@
if (must_abort())
abort_with_mutex();
-change_timing_state(STM_TIME_RUN_CURRENT);
+struct stm_priv_segment_info_s *pseg =
+get_priv_segment(STM_SEGMENT-segment_num);
+double elapsed =
+change_timing_state_tl(pseg-pub.running_thread,
+ STM_TIME_RUN_CURRENT);
+marker_copy(pseg-pub.running_thread, pseg,
+wait_category, elapsed);
}
else if (!contmgr.abort_other) {
@@ -187,7 +194,7 @@
dprintf((abort in contention\n));
STM_SEGMENT-nursery_end = abort_category;
-marker_contention(abort_category, false, other_segment_num, obj);
+marker_contention(kind, false, other_segment_num, obj);
abort_with_mutex();
}
@@ -195,7 +202,7 @@
/* We have to signal the other thread to abort, and wait until
it does. */
contmgr.other_pseg-pub.nursery_end = abort_category;
-marker_contention(abort_category, true, other_segment_num, obj);
+marker_contention(kind, true, other_segment_num, obj);
int sp = contmgr.other_pseg-safe_point;
switch (sp) {
diff --git a/c7/stm/marker.c b/c7/stm/marker.c
--- a/c7/stm/marker.c
+++ b/c7/stm/marker.c
@@ -134,7 +134,7 @@
marker[1] = 0;
}
-static void marker_contention(int category, bool abort_other,
+static void marker_contention(int kind, bool abort_other,
uint8_t other_segment_num, object_t *obj)
{
uintptr_t self_marker[2];
@@ -152,7 +152,7 @@
/* Collect the location for myself. It's usually the current
location, except in a write-read abort, in which case it's the
older location of the write. */
-if (category == STM_TIME_RUN_ABORTED_WRITE_READ)
+if (kind == WRITE_READ_CONTENTION)
marker_fetch_obj_write(my_pseg-pub.segment_num, obj, self_marker);
else
marker_fetch(my_pseg-pub.running_thread, self_marker);
@@ -165,11 +165,11 @@
/* For some categories, we can also collect the relevant information
for the other segment. */
-switch (category) {
-case STM_TIME_RUN_ABORTED_WRITE_WRITE:
+switch (kind) {
+case WRITE_WRITE_CONTENTION:
marker_fetch_obj_write(other_segment_num, obj, other_marker);
break;
-case STM_TIME_RUN_ABORTED_INEVITABLE:
+case INEVITABLE_CONTENTION:
assert(abort_other == false);
other_marker[0] = other_pseg-marker_inev[0];
other_marker[1] = other_pseg-marker_inev[1];
@@ -185,7 +185,7 @@
: my_pseg-marker_other);
if (abort_other other_pseg-marker_self[0] == 0) {
-if (category == STM_TIME_RUN_ABORTED_WRITE_READ)
+if (kind == WRITE_READ_CONTENTION)
strcpy(other_pseg-marker_self, read at unknown location);
else
strcpy(other_pseg-marker_self, no location information);
diff --git a/c7/stm/marker.h b/c7/stm/marker.h
--- a/c7/stm/marker.h
+++ b/c7/stm/marker.h
@@ -8,5 +8,5 @@
struct stm_priv_segment_info_s *pseg,
enum stm_time_e attribute_to, double time);
-static void marker_contention(int category, bool abort_other,
+static void marker_contention(int kind, bool abort_other,
[pypy-commit] pypy default: this workaround only necessary when using AI_NUMERICSERV
Author: Brian Kearns bdkea...@gmail.com Branch: Changeset: r71044:877e67830b30 Date: 2014-04-28 12:53 -0400 http://bitbucket.org/pypy/pypy/changeset/877e67830b30/ Log:this workaround only necessary when using AI_NUMERICSERV diff --git a/rpython/rlib/rsocket.py b/rpython/rlib/rsocket.py --- a/rpython/rlib/rsocket.py +++ b/rpython/rlib/rsocket.py @@ -1146,9 +1146,9 @@ address_to_fill=None): # port_or_service is a string, not an int (but try str(port_number)). assert port_or_service is None or isinstance(port_or_service, str) -if _c._MACOSX: -if port_or_service is None or port_or_service == '0': -port_or_service = '00' +if _c._MACOSX and flags AI_NUMERICSERV and \ +(port_or_service is None or port_or_service == '0'): +port_or_service = '00' hints = lltype.malloc(_c.addrinfo, flavor='raw', zero=True) rffi.setintfield(hints, 'c_ai_family', family) rffi.setintfield(hints, 'c_ai_socktype', socktype) ___ pypy-commit mailing list pypy-commit@python.org https://mail.python.org/mailman/listinfo/pypy-commit
[pypy-commit] pypy stmgc-c7: import stmgc/4bde66e3b621 (branch marker)
Author: Armin Rigo ar...@tunes.org
Branch: stmgc-c7
Changeset: r71045:73611e136820
Date: 2014-04-28 18:45 +0200
http://bitbucket.org/pypy/pypy/changeset/73611e136820/
Log:import stmgc/4bde66e3b621 (branch marker)
diff --git a/rpython/translator/stm/src_stm/revision
b/rpython/translator/stm/src_stm/revision
--- a/rpython/translator/stm/src_stm/revision
+++ b/rpython/translator/stm/src_stm/revision
@@ -1,1 +1,1 @@
-889897f2f5ef
+4bde66e3b621
diff --git a/rpython/translator/stm/src_stm/stm/contention.c
b/rpython/translator/stm/src_stm/stm/contention.c
--- a/rpython/translator/stm/src_stm/stm/contention.c
+++ b/rpython/translator/stm/src_stm/stm/contention.c
@@ -100,7 +100,8 @@
static void contention_management(uint8_t other_segment_num,
- enum contention_kind_e kind)
+ enum contention_kind_e kind,
+ object_t *obj)
{
assert(_has_mutex());
assert(other_segment_num != STM_SEGMENT-segment_num);
@@ -162,10 +163,12 @@
itself already paused here.
*/
contmgr.other_pseg-signal_when_done = true;
+marker_contention(kind, false, other_segment_num, obj);
change_timing_state(wait_category);
-/* XXX should also tell other_pseg please commit soon */
+/* tell the other to commit ASAP */
+signal_other_to_commit_soon(contmgr.other_pseg);
dprintf((pausing...\n));
cond_signal(C_AT_SAFE_POINT);
@@ -177,12 +180,22 @@
if (must_abort())
abort_with_mutex();
-change_timing_state(STM_TIME_RUN_CURRENT);
+struct stm_priv_segment_info_s *pseg =
+get_priv_segment(STM_SEGMENT-segment_num);
+double elapsed =
+change_timing_state_tl(pseg-pub.running_thread,
+ STM_TIME_RUN_CURRENT);
+marker_copy(pseg-pub.running_thread, pseg,
+wait_category, elapsed);
}
else if (!contmgr.abort_other) {
+/* tell the other to commit ASAP, since it causes aborts */
+signal_other_to_commit_soon(contmgr.other_pseg);
+
dprintf((abort in contention\n));
STM_SEGMENT-nursery_end = abort_category;
+marker_contention(kind, false, other_segment_num, obj);
abort_with_mutex();
}
@@ -190,6 +203,7 @@
/* We have to signal the other thread to abort, and wait until
it does. */
contmgr.other_pseg-pub.nursery_end = abort_category;
+marker_contention(kind, true, other_segment_num, obj);
int sp = contmgr.other_pseg-safe_point;
switch (sp) {
@@ -257,10 +271,18 @@
abort_data_structures_from_segment_num(other_segment_num);
}
dprintf((killed other thread\n));
+
+/* we should commit soon, we caused an abort */
+
//signal_other_to_commit_soon(get_priv_segment(STM_SEGMENT-segment_num));
+if (!STM_PSEGMENT-signalled_to_commit_soon) {
+STM_PSEGMENT-signalled_to_commit_soon = true;
+stmcb_commit_soon();
+}
}
}
-static void write_write_contention_management(uintptr_t lock_idx)
+static void write_write_contention_management(uintptr_t lock_idx,
+ object_t *obj)
{
s_mutex_lock();
@@ -271,7 +293,7 @@
assert(get_priv_segment(other_segment_num)-write_lock_num ==
prev_owner);
-contention_management(other_segment_num, WRITE_WRITE_CONTENTION);
+contention_management(other_segment_num, WRITE_WRITE_CONTENTION, obj);
/* now we return into _stm_write_slowpath() and will try again
to acquire the write lock on our object. */
@@ -280,12 +302,13 @@
s_mutex_unlock();
}
-static void write_read_contention_management(uint8_t other_segment_num)
+static void write_read_contention_management(uint8_t other_segment_num,
+ object_t *obj)
{
-contention_management(other_segment_num, WRITE_READ_CONTENTION);
+contention_management(other_segment_num, WRITE_READ_CONTENTION, obj);
}
static void inevitable_contention_management(uint8_t other_segment_num)
{
-contention_management(other_segment_num, INEVITABLE_CONTENTION);
+contention_management(other_segment_num, INEVITABLE_CONTENTION, NULL);
}
diff --git a/rpython/translator/stm/src_stm/stm/contention.h
b/rpython/translator/stm/src_stm/stm/contention.h
--- a/rpython/translator/stm/src_stm/stm/contention.h
+++ b/rpython/translator/stm/src_stm/stm/contention.h
@@ -1,11 +1,14 @@
/* Imported by rpython/translator/stm/import_stmgc.py */
-static void write_write_contention_management(uintptr_t lock_idx);
-static void write_read_contention_management(uint8_t other_segment_num);
+static void write_write_contention_management(uintptr_t lock_idx,
+ object_t *obj);
+static void
[pypy-commit] pypy stmgc-c7: Test and implementation of a way to grab the longest_abort_info from RPython
Author: Armin Rigo ar...@tunes.org
Branch: stmgc-c7
Changeset: r71047:5971a915100c
Date: 2014-04-28 19:05 +0200
http://bitbucket.org/pypy/pypy/changeset/5971a915100c/
Log:Test and implementation of a way to grab the longest_abort_info from
RPython
diff --git a/rpython/rlib/rstm.py b/rpython/rlib/rstm.py
--- a/rpython/rlib/rstm.py
+++ b/rpython/rlib/rstm.py
@@ -132,6 +132,16 @@
def pop_marker():
llop.stm_pop_marker(lltype.Void)
+def longest_abort_info():
+state = llop.stm_longest_marker_state(lltype.Signed)
+time = llop.stm_longest_marker_time(lltype.Float)
+cself = llop.stm_longest_marker_self(rffi.CCHARP)
+cother = llop.stm_longest_marker_other(rffi.CCHARP)
+return (state, time, rffi.charp2str(cself), rffi.charp2str(cother))
+
+def reset_longest_abort_info():
+llop.stm_reset_longest_marker_state(lltype.Void)
+
#
def make_perform_transaction(func, CONTAINERP):
diff --git a/rpython/rtyper/lltypesystem/lloperation.py
b/rpython/rtyper/lltypesystem/lloperation.py
--- a/rpython/rtyper/lltypesystem/lloperation.py
+++ b/rpython/rtyper/lltypesystem/lloperation.py
@@ -457,22 +457,11 @@
'stm_expand_marker': LLOp(),
'stm_setup_expand_marker_for_pypy': LLOp(),
-##'stm_allocate_nonmovable_int_adr': LLOp(sideeffects=False,
canmallocgc=True),
-##'stm_become_inevitable': LLOp(canmallocgc=True),
-##'stm_stop_all_other_threads': LLOp(canmallocgc=True),
-##'stm_partial_commit_and_resume_other_threads': LLOp(canmallocgc=True),
-##'stm_minor_collect': LLOp(canmallocgc=True),
-##'stm_major_collect': LLOp(canmallocgc=True),
-##'stm_get_tid':LLOp(canfold=True),
-##'stm_ptr_eq': LLOp(canfold=True),
-
-##'stm_weakref_allocate': LLOp(sideeffects=False, canmallocgc=True),
-
-##'stm_get_adr_of_private_rev_num':LLOp(),
-##'stm_get_adr_of_read_barrier_cache':LLOp(),
-##'stm_get_adr_of_nursery_current': LLOp(),
-##'stm_get_adr_of_nursery_nextlimit': LLOp(),
-##'stm_get_adr_of_active': LLOp(),
+'stm_longest_marker_state': LLOp(),
+'stm_longest_marker_time':LLOp(),
+'stm_longest_marker_self':LLOp(),
+'stm_longest_marker_other': LLOp(),
+'stm_reset_longest_marker_state': LLOp(),
# __ address operations __
diff --git a/rpython/translator/stm/funcgen.py
b/rpython/translator/stm/funcgen.py
--- a/rpython/translator/stm/funcgen.py
+++ b/rpython/translator/stm/funcgen.py
@@ -239,3 +239,25 @@
assert len(offsets) == 4
return 'pypy_stm_setup_expand_marker(%s, %s, %s, %s);' % (
offsets[0], offsets[1], offsets[2], offsets[3])
+
+def stm_longest_marker_state(funcgen, op):
+result = funcgen.expr(op.result)
+return '%s = (Signed)stm_thread_local.longest_marker_state;' % (result,)
+
+def stm_longest_marker_time(funcgen, op):
+result = funcgen.expr(op.result)
+return '%s = stm_thread_local.longest_marker_time;' % (result,)
+
+def stm_longest_marker_self(funcgen, op):
+result = funcgen.expr(op.result)
+return '%s = stm_thread_local.longest_marker_self;' % (result,)
+
+def stm_longest_marker_other(funcgen, op):
+result = funcgen.expr(op.result)
+return '%s = stm_thread_local.longest_marker_other;' % (result,)
+
+def stm_reset_longest_marker_state(funcgen, op):
+return ('stm_thread_local.longest_marker_state = 0;\n'
+'stm_thread_local.longest_marker_time = 0.0;\n'
+'stm_thread_local.longest_marker_self[0] = 0;\n'
+'stm_thread_local.longest_marker_other[0] = 0;')
diff --git a/rpython/translator/stm/test/test_ztranslated.py
b/rpython/translator/stm/test/test_ztranslated.py
--- a/rpython/translator/stm/test/test_ztranslated.py
+++ b/rpython/translator/stm/test/test_ztranslated.py
@@ -237,7 +237,6 @@
assert 'ok\n' in data
def test_abort_info(self):
-py.test.skip(goes away)
class Parent(object):
pass
class Foobar(Parent):
@@ -249,19 +248,12 @@
globf.xy = 100 + retry_counter
def check(_, retry_counter):
-rstm.abort_info_push(globf, ('[', 'xy', ']', 'yx'))
setxy(globf, retry_counter)
if retry_counter 3:
rstm.abort_and_retry()
-#
-last = rstm.charp_inspect_abort_info()
-if last:
-print rffi.charp2str(last)
-else:
-print 'got abort_info=NULL!'
-print int(bool(rstm.charp_inspect_abort_info()))
-#
-rstm.abort_info_pop(2)
+print rstm.longest_abort_info()
+rstm.reset_longest_abort_info()
+print rstm.longest_abort_info()
return 0
PS = lltype.Ptr(lltype.GcStruct('S', ('got_exception', OBJECTPTR)))
@@ -275,7 +267,10 @@
return 0
t, cbuilder = self.compile(main)
[pypy-commit] pypy stmgc-c7: Add a dummy stmcb_commit_soon(). Fix me!
Author: Armin Rigo ar...@tunes.org
Branch: stmgc-c7
Changeset: r71046:3e0cdc451004
Date: 2014-04-28 19:02 +0200
http://bitbucket.org/pypy/pypy/changeset/3e0cdc451004/
Log:Add a dummy stmcb_commit_soon(). Fix me!
diff --git a/rpython/translator/stm/src_stm/stmgcintf.c
b/rpython/translator/stm/src_stm/stmgcintf.c
--- a/rpython/translator/stm/src_stm/stmgcintf.c
+++ b/rpython/translator/stm/src_stm/stmgcintf.c
@@ -231,3 +231,5 @@
_pypy_stm_inev_state();
stm_become_globally_unique_transaction(stm_thread_local, for the JIT);
}
+
+void stmcb_commit_soon(void) { /*XXX FIXME*/ }
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit
[pypy-commit] pypy stmgc-c7: Give access to this info from app-level (untested so far)
Author: Armin Rigo ar...@tunes.org Branch: stmgc-c7 Changeset: r71048:9a04fbeb89a3 Date: 2014-04-28 19:08 +0200 http://bitbucket.org/pypy/pypy/changeset/9a04fbeb89a3/ Log:Give access to this info from app-level (untested so far) diff --git a/pypy/module/__pypy__/__init__.py b/pypy/module/__pypy__/__init__.py --- a/pypy/module/__pypy__/__init__.py +++ b/pypy/module/__pypy__/__init__.py @@ -39,8 +39,8 @@ '_atomic_enter': 'interp_atomic.atomic_enter', '_exclusive_atomic_enter': 'interp_atomic.exclusive_atomic_enter', '_atomic_exit':'interp_atomic.atomic_exit', -'last_abort_info': 'interp_atomic.last_abort_info', -'discard_last_abort_info': 'interp_atomic.discard_last_abort_info', +'longest_abort_info': 'interp_atomic.longest_abort_info', +'reset_longest_abort_info':'interp_atomic.reset_longest_abort_info', 'getsegmentlimit': 'interp_atomic.getsegmentlimit', } def activate(self, space): diff --git a/pypy/module/__pypy__/interp_atomic.py b/pypy/module/__pypy__/interp_atomic.py --- a/pypy/module/__pypy__/interp_atomic.py +++ b/pypy/module/__pypy__/interp_atomic.py @@ -59,8 +59,16 @@ else: return space.wrap(1) -def last_abort_info(space): -return space.w_None +def longest_abort_info(space): +if space.config.translation.stm: +from rpython.rlib.rstm import longest_abort_info +a, b, c, d = longest_abort_info() +return space.newtuple([space.wrap(a), space.wrap(b), + space.wrap(c), space.wrap(d)]) +else: +return space.w_None -def discard_last_abort_info(space): -pass +def reset_longest_abort_info(space): +if space.config.translation.stm: +from rpython.rlib.rstm import reset_longest_abort_info +reset_longest_abort_info() ___ pypy-commit mailing list pypy-commit@python.org https://mail.python.org/mailman/listinfo/pypy-commit
[pypy-commit] pypy stmgc-c7: Test and fix: the counter argument we pass to the callback
Author: Armin Rigo ar...@tunes.org
Branch: stmgc-c7
Changeset: r71049:f6dab9f69e7c
Date: 2014-04-28 19:23 +0200
http://bitbucket.org/pypy/pypy/changeset/f6dab9f69e7c/
Log:Test and fix: the counter argument we pass to the callback from
pypy_stm_perform_transaction() should start at 0.
diff --git a/rpython/translator/stm/src_stm/stmgcintf.c
b/rpython/translator/stm/src_stm/stmgcintf.c
--- a/rpython/translator/stm/src_stm/stmgcintf.c
+++ b/rpython/translator/stm/src_stm/stmgcintf.c
@@ -140,6 +140,7 @@
STM_PUSH_ROOT(stm_thread_local, arg);
while (1) {
+long counter;
if (pypy_stm_ready_atomic == 1) {
/* Not in an atomic transaction; but it might be an inevitable
@@ -156,11 +157,13 @@
declared below than this point only.
*/
while (__builtin_setjmp(jmpbuf) == 1) { /*redo setjmp*/ }
+counter = v_counter;
pypy_stm_start_transaction(jmpbuf, v_counter);
}
else {
/* In an atomic transaction */
assert(pypy_stm_nursery_low_fill_mark == (uintptr_t) -1);
+counter = v_counter;
}
/* invoke the callback in the new transaction */
@@ -168,7 +171,7 @@
assert(v_old_shadowstack == stm_thread_local.shadowstack - 1);
STM_PUSH_ROOT(stm_thread_local, arg);
-long result = v_callback(arg, v_counter);
+long result = v_callback(arg, counter);
if (result = 0)
break;
v_counter = 0;
diff --git a/rpython/translator/stm/test/test_ztranslated.py
b/rpython/translator/stm/test/test_ztranslated.py
--- a/rpython/translator/stm/test/test_ztranslated.py
+++ b/rpython/translator/stm/test/test_ztranslated.py
@@ -130,6 +130,23 @@
data, dataerr = cbuilder.cmdexec('4 5000', err=True)
assert 'check ok!' in data
+def test_retry_counter_starts_at_zero(self):
+#
+def check(foobar, retry_counter):
+print '', retry_counter, ''
+return 0
+#
+S = lltype.GcStruct('S', ('got_exception', OBJECTPTR))
+PS = lltype.Ptr(S)
+perform_transaction = rstm.make_perform_transaction(check, PS)
+def entry_point(argv):
+perform_transaction(lltype.malloc(S))
+return 0
+#
+t, cbuilder = self.compile(entry_point, backendopt=True)
+data = cbuilder.cmdexec('a b c d')
+assert ' 0 \n' in data
+
def test_bug1(self):
#
def check(foobar, retry_counter):
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit
[pypy-commit] pypy stmgc-c7: Jit Fix
Author: Armin Rigo ar...@tunes.org Branch: stmgc-c7 Changeset: r71050:92c1768af884 Date: 2014-04-28 19:59 +0200 http://bitbucket.org/pypy/pypy/changeset/92c1768af884/ Log:Jit Fix diff --git a/rpython/rlib/rstm.py b/rpython/rlib/rstm.py --- a/rpython/rlib/rstm.py +++ b/rpython/rlib/rstm.py @@ -132,6 +132,7 @@ def pop_marker(): llop.stm_pop_marker(lltype.Void) +@dont_look_inside def longest_abort_info(): state = llop.stm_longest_marker_state(lltype.Signed) time = llop.stm_longest_marker_time(lltype.Float) @@ -139,6 +140,7 @@ cother = llop.stm_longest_marker_other(rffi.CCHARP) return (state, time, rffi.charp2str(cself), rffi.charp2str(cother)) +@dont_look_inside def reset_longest_abort_info(): llop.stm_reset_longest_marker_state(lltype.Void) ___ pypy-commit mailing list pypy-commit@python.org https://mail.python.org/mailman/listinfo/pypy-commit
[pypy-commit] cffi release-0.8: clarify that cffi is distributed with pypy
Author: Matti Picus matti.pi...@gmail.com Branch: release-0.8 Changeset: r1508:0884285cc707 Date: 2014-04-28 18:32 + http://bitbucket.org/cffi/cffi/changeset/0884285cc707/ Log:clarify that cffi is distributed with pypy diff --git a/doc/source/index.rst b/doc/source/index.rst --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -45,7 +45,7 @@ Installation and Status === -Quick installation: +Quick installation (for cpython, cffi is distributed with PyPy): * ``pip install cffi`` @@ -60,10 +60,10 @@ left. It supports CPython 2.6; 2.7; 3.x (tested with 3.2 and 3.3); -and PyPy 2.0 beta2 or later. +and is distrubuted with PyPy 2.0 beta2 or later. Its speed is comparable to ctypes on CPython (a bit faster but a higher -warm-up time). It is already faster on PyPy (1.5x-2x), but not yet +warm-up time). It is already faster than ctypes on PyPy (1.5x-2x), but not yet *much* faster; stay tuned. Requirements: ___ pypy-commit mailing list pypy-commit@python.org https://mail.python.org/mailman/listinfo/pypy-commit
[pypy-commit] pypy default: rework, call the release Easier Than Ever for want of a better name
Author: Matti Picus matti.pi...@gmail.com Branch: Changeset: r71051:8bc78794a75b Date: 2014-04-28 22:49 +0300 http://bitbucket.org/pypy/pypy/changeset/8bc78794a75b/ Log:rework, call the release Easier Than Ever for want of a better name diff --git a/pypy/doc/release-2.3.0.rst b/pypy/doc/release-2.3.0.rst --- a/pypy/doc/release-2.3.0.rst +++ b/pypy/doc/release-2.3.0.rst @@ -1,11 +1,17 @@ === -PyPy 2.3 - TODO +PyPy 2.3 - Easier Than Ever === We're pleased to announce PyPy 2.3, which targets version 2.7.6 of the Python language. This release updates the stdlib from 2.7.3, jumping directly to 2.7.6. -This release also contains several bugfixes and performance improvements. +This release also contains several bugfixes and performance improvements, +many generated by real users finding corner cases our `TDD`_ methods missed. +`CFFI`_ has made it easier than ever to use existing C code with both cpython +and PyPy, easing the transition for packages like `cryptography`_, `Pillow`_ +(Python Imaging Library [Fork]), a basic port of `pygame-cffi`_, and others. + +PyPy can now be embedded in a hosting application, for instance inside `uWSGI`_ You can download the PyPy 2.3 release here: @@ -17,72 +23,112 @@ Please consider donating more so we can finish those projects! The three projects are: -* Py3k (supporting Python 3.x): the release PyPy3 2.2 is imminent. +* `Py3k`_ (supporting Python 3.x): the release PyPy3 2.2 is imminent. -* STM (software transactional memory): a preview will be released very soon, - as soon as we fix a few bugs +* `STM`_ (software transactional memory): a preview will be released very soon, + once we fix a few bugs -* NumPy: the work done is included in the PyPy 2.2 release. More details below. +* `NumPy`_ the work done is included in the PyPy 2.2 release. More details below. -.. _`Raspberry Pi Foundation`: http://www.raspberrypi.org +_`Py3k`: http://pypy.org/py3donate.html +_`STM`: http://pypy.org/tmdonate2.html +_ `Numpy`: http://pypy.org/numpydonate.html +_`TDD`: http://doc.pypy.org/en/latest/how-to-contribute.html +_`CFFI`: http://cffi.readthedocs.org +_`cryptography`: https://cryptography.io +_`Pillow`: https://pypi.python.org/pypi/Pillow/2.4.0 +_`pygame-cffi`: https://github.com/CTPUG/pygame_cffi +_`uWSGI`: http://uwsgi-docs.readthedocs.org/en/latest/PyPy.html What is PyPy? = PyPy is a very compliant Python interpreter, almost a drop-in replacement for -CPython 2.7. It's fast (`pypy 2.2 and cpython 2.7.2`_ performance comparison) +CPython 2.7. It's fast (`pypy 2.2 and cpython 2.7.2`_ performance comparison; +note that the latest cpython is not faster than cpython 2.7.2) due to its integrated tracing JIT compiler. -This release supports x86 machines running Linux 32/64, Mac OS X 64, Windows -32, or ARM (ARMv6 or ARMv7, with VFPv3). +This release supports x86 machines running Linux 32/64, Mac OS X 64, Windows, +and OpenBSD, +as well as newer ARM hardware (ARMv6 or ARMv7, with VFPv3) running Linux. -Work on the native Windows 64 is still stalling, we would welcome a volunteer -to handle that. +While we support 32 bit python on Windows, work on the native Windows 64 +bit python is still stalling, we would welcome a volunteer +to `handle that`_. -.. _`pypy 2.2 and cpython 2.7.2`: http://speed.pypy.org +_`pypy 2.2 and cpython 2.7.2`: http://speed.pypy.org +_`handle that`: http://doc.pypy.org/en/latest/windows.html#what-is-missing-for-a-full-64-bit-translation Highlights == -* Our Garbage Collector is now incremental. It should avoid almost - all pauses due to a major collection taking place. Previously, it - would pause the program (rarely) to walk all live objects, which - could take arbitrarily long if your process is using a whole lot of - RAM. Now the same work is done in steps. This should make PyPy - more responsive, e.g. in games. There are still other pauses, from - the GC and the JIT, but they should be on the order of 5 - milliseconds each. +Bugfixes + -* The JIT counters for hot code were never reset, which meant that a - process running for long enough would eventually JIT-compile more - and more rarely executed code. Not only is it useless to compile - such code, but as more compiled code means more memory used, this - gives the impression of a memory leak. This has been tentatively - fixed by decreasing the counters from time to time. +Many issues were cleaned up after being reported by users to https://bugs.pypy.org (ignore the bad SSL certificate) or on IRC at #pypy. Note that we consider +performance slowdowns as bugs. -* NumPy has been split: now PyPy only contains the core module, called - ``_numpypy``. The ``numpy`` module itself has been moved to - ``https://bitbucket.org/pypy/numpy`` and ``numpypy`` disappeared. - You need to install NumPy separately with a virtualenv: +*
[pypy-commit] pypy default: fix for windows, script still does not create proper commands
Author: Matti Picus matti.pi...@gmail.com
Branch:
Changeset: r71053:430c42d4c761
Date: 2014-04-28 23:28 +0300
http://bitbucket.org/pypy/pypy/changeset/430c42d4c761/
Log:fix for windows, script still does not create proper commands
diff --git a/pypy/tool/release/force-builds.py
b/pypy/tool/release/force-builds.py
--- a/pypy/tool/release/force-builds.py
+++ b/pypy/tool/release/force-builds.py
@@ -9,7 +9,7 @@
modified by PyPy team
-import os, sys, pwd, urllib
+import os, sys, urllib
from twisted.internet import reactor, defer
from twisted.python import log
@@ -34,6 +34,13 @@
'build-pypy-c-jit-linux-armel',
]
+def get_user():
+if sys.platform == 'win32':
+return os.environ['USERNAME']
+else:
+import pwd
+return pwd.getpwuid(os.getuid())[0]
+
def main():
#XXX: handle release tags
#XXX: handle validity checks
@@ -49,7 +56,7 @@
print 'Forcing', builder, '...'
url = http://buildbot.pypy.org/builders/; + builder + /force
args = [
-('username', pwd.getpwuid(os.getuid())[0]),
+('username', get_user()),
('revision', ''),
('submit', 'Force Build'),
('branch', branch),
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit
[pypy-commit] pypy default: whoops
Author: Matti Picus matti.pi...@gmail.com Branch: Changeset: r71054:5a6c8e00ee88 Date: 2014-04-28 23:29 +0300 http://bitbucket.org/pypy/pypy/changeset/5a6c8e00ee88/ Log:whoops diff --git a/pypy/doc/release-2.3.0.rst b/pypy/doc/release-2.3.0.rst --- a/pypy/doc/release-2.3.0.rst +++ b/pypy/doc/release-2.3.0.rst @@ -30,15 +30,15 @@ * `NumPy`_ the work done is included in the PyPy 2.2 release. More details below. -_`Py3k`: http://pypy.org/py3donate.html -_`STM`: http://pypy.org/tmdonate2.html -_ `Numpy`: http://pypy.org/numpydonate.html -_`TDD`: http://doc.pypy.org/en/latest/how-to-contribute.html -_`CFFI`: http://cffi.readthedocs.org -_`cryptography`: https://cryptography.io -_`Pillow`: https://pypi.python.org/pypi/Pillow/2.4.0 -_`pygame-cffi`: https://github.com/CTPUG/pygame_cffi -_`uWSGI`: http://uwsgi-docs.readthedocs.org/en/latest/PyPy.html +.. _`Py3k`: http://pypy.org/py3donate.html +.. _`STM`: http://pypy.org/tmdonate2.html +.. _ `Numpy`: http://pypy.org/numpydonate.html +.. _`TDD`: http://doc.pypy.org/en/latest/how-to-contribute.html +.. _`CFFI`: http://cffi.readthedocs.org +.. _`cryptography`: https://cryptography.io +.. _`Pillow`: https://pypi.python.org/pypi/Pillow/2.4.0 +.. _`pygame-cffi`: https://github.com/CTPUG/pygame_cffi +.. _`uWSGI`: http://uwsgi-docs.readthedocs.org/en/latest/PyPy.html What is PyPy? = @@ -56,8 +56,8 @@ bit python is still stalling, we would welcome a volunteer to `handle that`_. -_`pypy 2.2 and cpython 2.7.2`: http://speed.pypy.org -_`handle that`: http://doc.pypy.org/en/latest/windows.html#what-is-missing-for-a-full-64-bit-translation +.. _`pypy 2.2 and cpython 2.7.2`: http://speed.pypy.org +.. _`handle that`: http://doc.pypy.org/en/latest/windows.html#what-is-missing-for-a-full-64-bit-translation Highlights == @@ -84,7 +84,7 @@ * Fix a rpython bug with loop-unrolling that appeared in the `HippyVM`_ PHP port -`HippyVM`_: http://www.hippyvm.com +.. _`HippyVM`: http://www.hippyvm.com New Platforms and Features -- ___ pypy-commit mailing list pypy-commit@python.org https://mail.python.org/mailman/listinfo/pypy-commit
[pypy-commit] pypy default: fix test_compiler on osx buildbot
Author: Brian Kearns bdkea...@gmail.com
Branch:
Changeset: r71055:d86c4a65f830
Date: 2014-04-28 13:52 -0700
http://bitbucket.org/pypy/pypy/changeset/d86c4a65f830/
Log:fix test_compiler on osx buildbot
diff --git a/pypy/module/sys/version.py b/pypy/module/sys/version.py
--- a/pypy/module/sys/version.py
+++ b/pypy/module/sys/version.py
@@ -14,7 +14,8 @@
if platform.name == 'msvc':
COMPILER_INFO = 'MSC v.%d 32 bit' % (platform.version * 10 + 600)
-elif platform.cc is not None and platform.cc.startswith(('gcc', 'clang')):
+elif platform.cc is not None and \
+os.path.basename(platform.cc).startswith(('gcc', 'clang')):
from rpython.rtyper.tool import rffi_platform
COMPILER_INFO = 'GCC ' + rffi_platform.getdefinedstring('__VERSION__', '')
else:
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit
[pypy-commit] pypy default: kill useless ClassMethods metaclass
Author: Ronan Lamy ronan.l...@gmail.com Branch: Changeset: r71056:c8e3b8cbc843 Date: 2014-04-29 01:36 +0100 http://bitbucket.org/pypy/pypy/changeset/c8e3b8cbc843/ Log:kill useless ClassMethods metaclass diff --git a/rpython/rtyper/module/ll_os_path.py b/rpython/rtyper/module/ll_os_path.py --- a/rpython/rtyper/module/ll_os_path.py +++ b/rpython/rtyper/module/ll_os_path.py @@ -6,26 +6,24 @@ import stat import os -from rpython.tool.staticmethods import ClassMethods # Does a path exist? # This is false for dangling symbolic links. -class BaseOsPath: -__metaclass__ = ClassMethods - +class BaseOsPath(object): +@classmethod def ll_os_path_exists(cls, path): Test whether a path exists try: -st = os.stat(cls.from_rstr_nonnull(path)) +os.stat(cls.from_rstr_nonnull(path)) except OSError: return False return True +@classmethod def ll_os_path_isdir(cls, path): try: st = os.stat(cls.from_rstr_nonnull(path)) except OSError: return False return stat.S_ISDIR(st[0]) - diff --git a/rpython/tool/staticmethods.py b/rpython/tool/staticmethods.py --- a/rpython/tool/staticmethods.py +++ b/rpython/tool/staticmethods.py @@ -10,11 +10,5 @@ class StaticMethods(AbstractMethods): Metaclass that turns plain methods into staticmethods. - + decorator = staticmethod - -class ClassMethods(AbstractMethods): - -Metaclass that turns plain methods into classmethods. - -decorator = classmethod ___ pypy-commit mailing list pypy-commit@python.org https://mail.python.org/mailman/listinfo/pypy-commit
[pypy-commit] pypy default: issue1746 -- added __closure__ to functions
Author: Alex Gaynor alex.gay...@gmail.com
Branch:
Changeset: r71057:9913d8bb1a20
Date: 2014-04-28 19:58 -0700
http://bitbucket.org/pypy/pypy/changeset/9913d8bb1a20/
Log:issue1746 -- added __closure__ to functions
diff --git a/pypy/interpreter/typedef.py b/pypy/interpreter/typedef.py
--- a/pypy/interpreter/typedef.py
+++ b/pypy/interpreter/typedef.py
@@ -809,6 +809,7 @@
__dict__ = getset_func_dict,
__defaults__ = getset_func_defaults,
__globals__ = interp_attrproperty_w('w_func_globals', cls=Function),
+__closure__ = GetSetProperty(Function.fget_func_closure),
__module__ = getset___module__,
__weakref__ = make_weakref_descr(Function),
)
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit
[pypy-commit] pypy default: merged upstream
Author: Alex Gaynor alex.gay...@gmail.com Branch: Changeset: r71058:1f55d73ad8a0 Date: 2014-04-28 19:58 -0700 http://bitbucket.org/pypy/pypy/changeset/1f55d73ad8a0/ Log:merged upstream diff --git a/pypy/doc/getting-started.rst b/pypy/doc/getting-started.rst --- a/pypy/doc/getting-started.rst +++ b/pypy/doc/getting-started.rst @@ -76,7 +76,7 @@ .. code-block:: console $ curl -O http://python-distribute.org/distribute_setup.py -$ curl -O https://raw.github.com/pypa/pip/master/contrib/get-pip.py +$ curl -O https://raw.githubusercontent.com/pypa/pip/master/contrib/get-pip.py $ ./pypy-2.1/bin/pypy distribute_setup.py $ ./pypy-2.1/bin/pypy get-pip.py $ ./pypy-2.1/bin/pip install pygments # for example diff --git a/pypy/doc/release-2.3.0.rst b/pypy/doc/release-2.3.0.rst --- a/pypy/doc/release-2.3.0.rst +++ b/pypy/doc/release-2.3.0.rst @@ -1,11 +1,17 @@ === -PyPy 2.3 - TODO +PyPy 2.3 - Easier Than Ever === We're pleased to announce PyPy 2.3, which targets version 2.7.6 of the Python language. This release updates the stdlib from 2.7.3, jumping directly to 2.7.6. -This release also contains several bugfixes and performance improvements. +This release also contains several bugfixes and performance improvements, +many generated by real users finding corner cases our `TDD`_ methods missed. +`CFFI`_ has made it easier than ever to use existing C code with both cpython +and PyPy, easing the transition for packages like `cryptography`_, `Pillow`_ +(Python Imaging Library [Fork]), a basic port of `pygame-cffi`_, and others. + +PyPy can now be embedded in a hosting application, for instance inside `uWSGI`_ You can download the PyPy 2.3 release here: @@ -17,72 +23,112 @@ Please consider donating more so we can finish those projects! The three projects are: -* Py3k (supporting Python 3.x): the release PyPy3 2.2 is imminent. +* `Py3k`_ (supporting Python 3.x): the release PyPy3 2.2 is imminent. -* STM (software transactional memory): a preview will be released very soon, - as soon as we fix a few bugs +* `STM`_ (software transactional memory): a preview will be released very soon, + once we fix a few bugs -* NumPy: the work done is included in the PyPy 2.2 release. More details below. +* `NumPy`_ the work done is included in the PyPy 2.2 release. More details below. -.. _`Raspberry Pi Foundation`: http://www.raspberrypi.org +.. _`Py3k`: http://pypy.org/py3donate.html +.. _`STM`: http://pypy.org/tmdonate2.html +.. _ `Numpy`: http://pypy.org/numpydonate.html +.. _`TDD`: http://doc.pypy.org/en/latest/how-to-contribute.html +.. _`CFFI`: http://cffi.readthedocs.org +.. _`cryptography`: https://cryptography.io +.. _`Pillow`: https://pypi.python.org/pypi/Pillow/2.4.0 +.. _`pygame-cffi`: https://github.com/CTPUG/pygame_cffi +.. _`uWSGI`: http://uwsgi-docs.readthedocs.org/en/latest/PyPy.html What is PyPy? = PyPy is a very compliant Python interpreter, almost a drop-in replacement for -CPython 2.7. It's fast (`pypy 2.2 and cpython 2.7.2`_ performance comparison) +CPython 2.7. It's fast (`pypy 2.2 and cpython 2.7.2`_ performance comparison; +note that the latest cpython is not faster than cpython 2.7.2) due to its integrated tracing JIT compiler. -This release supports x86 machines running Linux 32/64, Mac OS X 64, Windows -32, or ARM (ARMv6 or ARMv7, with VFPv3). +This release supports x86 machines running Linux 32/64, Mac OS X 64, Windows, +and OpenBSD, +as well as newer ARM hardware (ARMv6 or ARMv7, with VFPv3) running Linux. -Work on the native Windows 64 is still stalling, we would welcome a volunteer -to handle that. +While we support 32 bit python on Windows, work on the native Windows 64 +bit python is still stalling, we would welcome a volunteer +to `handle that`_. .. _`pypy 2.2 and cpython 2.7.2`: http://speed.pypy.org +.. _`handle that`: http://doc.pypy.org/en/latest/windows.html#what-is-missing-for-a-full-64-bit-translation Highlights == -* Our Garbage Collector is now incremental. It should avoid almost - all pauses due to a major collection taking place. Previously, it - would pause the program (rarely) to walk all live objects, which - could take arbitrarily long if your process is using a whole lot of - RAM. Now the same work is done in steps. This should make PyPy - more responsive, e.g. in games. There are still other pauses, from - the GC and the JIT, but they should be on the order of 5 - milliseconds each. +Bugfixes + -* The JIT counters for hot code were never reset, which meant that a - process running for long enough would eventually JIT-compile more - and more rarely executed code. Not only is it useless to compile - such code, but as more compiled code means more memory used, this - gives the impression of a memory leak. This has been tentatively - fixed by decreasing the counters from time to
[pypy-commit] pypy default: add test for 9913d8bb1a20
Author: Brian Kearns bdkea...@gmail.com Branch: Changeset: r71059:0524dae88c75 Date: 2014-04-28 23:27 -0400 http://bitbucket.org/pypy/pypy/changeset/0524dae88c75/ Log:add test for 9913d8bb1a20 diff --git a/pypy/interpreter/test/test_typedef.py b/pypy/interpreter/test/test_typedef.py --- a/pypy/interpreter/test/test_typedef.py +++ b/pypy/interpreter/test/test_typedef.py @@ -387,3 +387,9 @@ # because it's a regular method, and .__objclass__ # differs from .im_class in case the method is # defined in some parent class of l's actual class + +def test_func_closure(self): +x = 2 +def f(): +return x +assert f.__closure__[0].cell_contents is x ___ pypy-commit mailing list pypy-commit@python.org https://mail.python.org/mailman/listinfo/pypy-commit
