[
https://issues.apache.org/jira/browse/BEAM-9085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17036457#comment-17036457
]
Valentyn Tymofieiev commented on BEAM-9085:
-------------------------------------------
note that passing numpy==1.16.5 in requirements.txt increases the installation
time on Dataflow workers in Python 3 tests. Looking at worker startup logs, it
takes about ~8 min to install numpy form sources that are staged, which makes
Py3 tests appear slower than Py2 if judged by walltime (537s (py2) vs 977s
(py3+requirements file to downgrade numpy)).
> Performance regression in np.random.RandomState() skews performance test
> results across Python 2/3 on Dataflow
> --------------------------------------------------------------------------------------------------------------
>
> Key: BEAM-9085
> URL: https://issues.apache.org/jira/browse/BEAM-9085
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
> Reporter: Kamil Wasilewski
> Assignee: Kamil Wasilewski
> Priority: Major
>
> Tests show that the performance of core Beam operations in Python 3.x on
> Dataflow can be a few time slower than in Python 2.7. We should investigate
> what's the cause of the problem.
> Currently, we have one ParDo test that is run both in Py3 and Py2 [1]. A
> dashboard with runtime results can be found here [2].
> [1] sdks/python/apache_beam/testing/load_tests/pardo_test.py
> [2] https://apache-beam-testing.appspot.com/explore?dashboard=5678187241537536
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
