Valentyn Tymofieiev created BEAM-8397:
-----------------------------------------

             Summary: DataflowRunnerTest.test_remote_runner_display_data fails 
due to infinite recursion during pickling.
                 Key: BEAM-8397
                 URL: https://issues.apache.org/jira/browse/BEAM-8397
             Project: Beam
          Issue Type: Sub-task
          Components: sdk-py-core
            Reporter: Valentyn Tymofieiev


`python ./setup.py test -s 
apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest.test_remote_runner_display_data`
 passes.
`tox -e py37-gcp` passes if Beam depends on dill==0.3.0, but fails if Beam 
depends on dill==0.3.1.1.`python ./setup.py nosetests --tests 
'apache_beam/runners/dataflow/dataflow_runner_test.py:DataflowRunnerTest.test_remote_runner_display_data`
 fails currently if run on master.

The failure indicates infinite recursion during pickling:
{noformat}
test_remote_runner_display_data 
(apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest) ... 
Fatal Python error: Cannot recover from stack overflow.

Current thread 0x00007f9d700ed740 (most recent call first):
  File "/usr/lib/python3.7/pickle.py", line 479 in get
  File "/usr/lib/python3.7/pickle.py", line 497 in save
  File "/usr/lib/python3.7/pickle.py", line 786 in save_tuple
  File "/usr/lib/python3.7/pickle.py", line 504 in save
  File "/usr/lib/python3.7/pickle.py", line 638 in save_reduce
  File 
"/usr/local/google/home/valentyn/tmp/py37env/lib/python3.7/site-packages/dill/_dill.py",
 line 1394 in save_function
  File "/usr/lib/python3.7/pickle.py", line 504 in save
  File "/usr/lib/python3.7/pickle.py", line 882 in _batch_setitems
  File "/usr/lib/python3.7/pickle.py", line 856 in save_dict
  File 
"/usr/local/google/home/valentyn/tmp/py37env/lib/python3.7/site-packages/dill/_dill.py",
 line 910 in save_module_dict
  File 
"/usr/local/google/home/valentyn/projects/beam/clean/beam/sdks/python/apache_beam/internal/pickler.py",
 line 198 in new_save_module_dict
  File "/usr/lib/python3.7/pickle.py", line 504 in save
  File "/usr/lib/python3.7/pickle.py", line 786 in save_tuple
  File "/usr/lib/python3.7/pickle.py", line 504 in save
  File "/usr/lib/python3.7/pickle.py", line 638 in save_reduce
  File 
"/usr/local/google/home/valentyn/projects/beam/clean/beam/sdks/python/apache_beam/internal/pickler.py",
 line 114 in wrapper
  File "/usr/lib/python3.7/pickle.py", line 504 in save
  File "/usr/lib/python3.7/pickle.py", line 771 in save_tuple
  File "/usr/lib/python3.7/pickle.py", line 504 in save
  File "/usr/lib/python3.7/pickle.py", line 638 in save_reduce
  File 
"/usr/local/google/home/valentyn/tmp/py37env/lib/python3.7/site-packages/dill/_dill.py",
 line 1137 in save_cell
  File "/usr/lib/python3.7/pickle.py", line 504 in save
  File "/usr/lib/python3.7/pickle.py", line 771 in save_tuple
  File "/usr/lib/python3.7/pickle.py", line 504 in save
  File "/usr/lib/python3.7/pickle.py", line 786 in save_tuple
  File "/usr/lib/python3.7/pickle.py", line 504 in save
  File "/usr/lib/python3.7/pickle.py", line 638 in save_reduce
  File 
"/usr/local/google/home/valentyn/tmp/py37env/lib/python3.7/site-packages/dill/_dill.py",
 line 1394 in save_function
  File "/usr/lib/python3.7/pickle.py", line 504 in save
  File "/usr/lib/python3.7/pickle.py", line 882 in _batch_setitems
  File "/usr/lib/python3.7/pickle.py", line 856 in save_dict
  File 
"/usr/local/google/home/valentyn/tmp/py37env/lib/python3.7/site-packages/dill/_dill.py",
 line 910 in save_module_dict
  File 
"/usr/local/google/home/valentyn/projects/beam/clean/beam/sdks/python/apache_beam/internal/pickler.py",
 line 198 in new_save_module_dict
...
{noformat}

cc: [~lazylynx]




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Reply via email to