[
https://issues.apache.org/jira/browse/BEAM-8397?focusedWorklogId=336955&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-336955
]
ASF GitHub Bot logged work on BEAM-8397:
----------------------------------------
Author: ASF GitHub Bot
Created on: 31/Oct/19 17:55
Start Date: 31/Oct/19 17:55
Worklog Time Spent: 10m
Work Description: tvalentyn commented on pull request #9881: [BEAM-8397]
Fix infinite recursion errors in test_remote_runner_display_data_test on Python
3.7.
URL: https://github.com/apache/beam/pull/9881
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 336955)
Time Spent: 2.5h (was: 2h 20m)
> DataflowRunnerTest.test_remote_runner_display_data sometimes fails due to
> infinite recursion during pickling on Python 3.7.
> ---------------------------------------------------------------------------------------------------------------------------
>
> Key: BEAM-8397
> URL: https://issues.apache.org/jira/browse/BEAM-8397
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
> Reporter: Valentyn Tymofieiev
> Assignee: Valentyn Tymofieiev
> Priority: Major
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> `python ./setup.py test -s
> apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest.test_remote_runner_display_data`
> passes.
> `tox -e py37-gcp` passes if Beam depends on dill==0.3.0, but fails if Beam
> depends on dill==0.3.1.1.`python ./setup.py nosetests --tests
> 'apache_beam/runners/dataflow/dataflow_runner_test.py:DataflowRunnerTest.test_remote_runner_display_data`
> fails currently if run on master.
> The failure indicates infinite recursion during pickling:
> {noformat}
> test_remote_runner_display_data
> (apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest) ...
> Fatal Python error: Cannot recover from stack overflow.
> Current thread 0x00007f9d700ed740 (most recent call first):
> File "/usr/lib/python3.7/pickle.py", line 479 in get
> File "/usr/lib/python3.7/pickle.py", line 497 in save
> File "/usr/lib/python3.7/pickle.py", line 786 in save_tuple
> File "/usr/lib/python3.7/pickle.py", line 504 in save
> File "/usr/lib/python3.7/pickle.py", line 638 in save_reduce
> File
> "/usr/local/google/home/valentyn/tmp/py37env/lib/python3.7/site-packages/dill/_dill.py",
> line 1394 in save_function
> File "/usr/lib/python3.7/pickle.py", line 504 in save
> File "/usr/lib/python3.7/pickle.py", line 882 in _batch_setitems
> File "/usr/lib/python3.7/pickle.py", line 856 in save_dict
> File
> "/usr/local/google/home/valentyn/tmp/py37env/lib/python3.7/site-packages/dill/_dill.py",
> line 910 in save_module_dict
> File
> "/usr/local/google/home/valentyn/projects/beam/clean/beam/sdks/python/apache_beam/internal/pickler.py",
> line 198 in new_save_module_dict
> File "/usr/lib/python3.7/pickle.py", line 504 in save
> File "/usr/lib/python3.7/pickle.py", line 786 in save_tuple
> File "/usr/lib/python3.7/pickle.py", line 504 in save
> File "/usr/lib/python3.7/pickle.py", line 638 in save_reduce
> File
> "/usr/local/google/home/valentyn/projects/beam/clean/beam/sdks/python/apache_beam/internal/pickler.py",
> line 114 in wrapper
> File "/usr/lib/python3.7/pickle.py", line 504 in save
> File "/usr/lib/python3.7/pickle.py", line 771 in save_tuple
> File "/usr/lib/python3.7/pickle.py", line 504 in save
> File "/usr/lib/python3.7/pickle.py", line 638 in save_reduce
> File
> "/usr/local/google/home/valentyn/tmp/py37env/lib/python3.7/site-packages/dill/_dill.py",
> line 1137 in save_cell
> File "/usr/lib/python3.7/pickle.py", line 504 in save
> File "/usr/lib/python3.7/pickle.py", line 771 in save_tuple
> File "/usr/lib/python3.7/pickle.py", line 504 in save
> File "/usr/lib/python3.7/pickle.py", line 786 in save_tuple
> File "/usr/lib/python3.7/pickle.py", line 504 in save
> File "/usr/lib/python3.7/pickle.py", line 638 in save_reduce
> File
> "/usr/local/google/home/valentyn/tmp/py37env/lib/python3.7/site-packages/dill/_dill.py",
> line 1394 in save_function
> File "/usr/lib/python3.7/pickle.py", line 504 in save
> File "/usr/lib/python3.7/pickle.py", line 882 in _batch_setitems
> File "/usr/lib/python3.7/pickle.py", line 856 in save_dict
> File
> "/usr/local/google/home/valentyn/tmp/py37env/lib/python3.7/site-packages/dill/_dill.py",
> line 910 in save_module_dict
> File
> "/usr/local/google/home/valentyn/projects/beam/clean/beam/sdks/python/apache_beam/internal/pickler.py",
> line 198 in new_save_module_dict
> ...
> {noformat}
> cc: [~yoshiki.obata]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
