[
https://issues.apache.org/jira/browse/ARROW-2071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16351860#comment-16351860
]
ASF GitHub Bot commented on ARROW-2071:
---------------------------------------
xhochy closed pull request #1550: ARROW-2071: [Python] Lighten serialization
tests
URL: https://github.com/apache/arrow/pull/1550
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/python/manylinux1/build_arrow.sh b/python/manylinux1/build_arrow.sh
index 4d816bec9..6bed451d2 100755
--- a/python/manylinux1/build_arrow.sh
+++ b/python/manylinux1/build_arrow.sh
@@ -81,7 +81,7 @@ for PYTHON in ${PYTHON_VERSIONS}; do
source /venv-test-${PYTHON}/bin/activate
pip install repaired_wheels/*.whl
- py.test --parquet /venv-test-${PYTHON}/lib/*/site-packages/pyarrow -v
+ py.test -v -r sxX --durations=15 --parquet
/venv-test-${PYTHON}/lib/*/site-packages/pyarrow
deactivate
mv repaired_wheels/*.whl /io/dist
diff --git a/python/pyarrow/tests/test_serialization.py
b/python/pyarrow/tests/test_serialization.py
index 23059e12d..3b1324beb 100644
--- a/python/pyarrow/tests/test_serialization.py
+++ b/python/pyarrow/tests/test_serialization.py
@@ -99,7 +99,7 @@ def assert_equal(obj1, obj2):
PRIMITIVE_OBJECTS = [
- 0, 0.0, 0.9, 1 << 62, 1 << 100, 1 << 999,
+ 0, 0.0, 0.9, 1 << 62, 1 << 999,
[1 << 100, [1 << 100]], "a", string.printable, "\u262F",
"hello world", u"hello world", u"\xff\xfe\x9c\x001\x000\x00",
None, True, False, [], (), {}, {(1, 2): 1}, {(): 2},
@@ -110,10 +110,11 @@ def assert_equal(obj1, obj2):
{"hello": set([2, 3]), "world": set([42.0]), "this": None},
np.int8(3), np.int32(4), np.int64(5),
np.uint8(3), np.uint32(4), np.uint64(5), np.float16(1.9), np.float32(1.9),
- np.float64(1.9), np.zeros([100, 100]),
- np.random.normal(size=[100, 100]), np.array(["hi", 3]),
+ np.float64(1.9), np.zeros([8, 20]),
+ np.random.normal(size=[17, 10]), np.array(["hi", 3]),
np.array(["hi", 3], dtype=object),
- np.random.normal(size=[45, 22]).T]
+ np.random.normal(size=[15, 13]).T,
+]
if sys.version_info >= (3, 0):
@@ -126,11 +127,12 @@ def assert_equal(obj1, obj2):
COMPLEX_OBJECTS = [
[[[[[[[[[[[[]]]]]]]]]]]],
- {"obj{}".format(i): np.random.normal(size=[100, 100]) for i in range(10)},
+ {"obj{}".format(i): np.random.normal(size=[4, 4]) for i in range(5)},
# {(): {(): {(): {(): {(): {(): {(): {(): {(): {(): {
# (): {(): {}}}}}}}}}}}}},
((((((((((),),),),),),),),),),
- {"a": {"b": {"c": {"d": {}}}}}]
+ {"a": {"b": {"c": {"d": {}}}}},
+]
class Foo(object):
@@ -146,7 +148,7 @@ def __eq__(self, other):
class Bar(object):
def __init__(self):
- for i, val in enumerate(PRIMITIVE_OBJECTS + COMPLEX_OBJECTS):
+ for i, val in enumerate(COMPLEX_OBJECTS):
setattr(self, "field{}".format(i), val)
@@ -161,7 +163,7 @@ def method(self, arg):
class Qux(object):
def __init__(self):
- self.objs = [Foo(), Bar(), Baz()]
+ self.objs = [Foo(1), Foo(42)]
class SubQux(Qux):
@@ -231,7 +233,7 @@ def _check_component_roundtrip(value,
context=global_serialization_context):
@pytest.yield_fixture(scope='session')
-def large_buffer(size=100*1024*1024):
+def large_buffer(size=32*1024*1024):
return pa.allocate_buffer(size)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> [Python] Reduce runtime of builds in Travis CI
> ----------------------------------------------
>
> Key: ARROW-2071
> URL: https://issues.apache.org/jira/browse/ARROW-2071
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Python
> Reporter: Wes McKinney
> Priority: Major
> Labels: pull-request-available
> Fix For: 0.9.0
>
>
> For some reason, recently each Python build has been taking about 15 minutes
> to run. I speculate this is due to VM thrashing caused by reduced resources
> on the Travis CI workers, related to the problem I fixed in ARROW-2062.
> We should experiment, but it seems like perhaps this can be fixed either by:
> * Reducing the size of the Plasma store on Travis CI
> * Disabling valgrind in Plasma tests
> The slowness could be caused by something else, though, so we should
> investigate (and have pytest report slow tests in the logs)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
