yoshiki obata created BEAM-9754:
-----------------------------------
Summary: Dataflow runner currently does not support Python 3.8
Key: BEAM-9754
URL: https://issues.apache.org/jira/browse/BEAM-9754
Project: Beam
Issue Type: Sub-task
Components: sdk-py-core
Reporter: yoshiki obata
Following tests fail due to the dataflow runner not supporting Python 3.8:
- test_default_ip_configuration
(apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
- test_experiment_use_multiple_sdk_containers
(apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
- test_flexrs_blank
(apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
- test_flexrs_cost
(apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
- test_flexrs_speed
(apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
- test_harness_override_absent_in_released_sdks_with_runner_v2
(apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
- test_harness_override_absent_in_unreleased_sdk
(apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
- test_harness_override_custom_in_released_sdks
(apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
- test_harness_override_custom_in_released_sdks_with_runner_v2
(apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
- test_harness_override_default_in_released_sdks
(apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
- test_harness_override_uses_base_version_in_rc_releases
(apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
- test_number_of_worker_harness_threads
(apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
- test_pinned_worker_harness_image_tag_used_in_dev_sdk
(apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
- test_pipeline_url
(apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
- test_private_ip_configuration
(apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
- test_public_ip_configuration
(apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
- test_sdk_harness_container_images_get_set
(apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
- test_set_network
(apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
- test_set_subnetwork
(apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
- test_worker_harness_image_tag_matches_base_sdk_version_of_an_rc
(apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
- test_worker_harness_image_tag_matches_released_sdk_version
(apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
- test_worker_harness_override_takes_precedence_over_sdk_defaults
(apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
- test_biqquery_read_streaming_fail
(apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest)
- test_dataflow_worker_jar_flag_adds_use_staged_worker_jar_experiment
(apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest)
- test_dataflow_worker_jar_flag_non_fnapi_noop
(apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest)
- test_min_cpu_platform_flag_is_propagated_to_experiments
(apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest)
- test_remote_runner_display_data
(apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest)
- test_remote_runner_translation
(apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest)
- test_streaming_create_translation
(apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest)
- test_streaming_engine_flag_adds_windmill_experiments
(apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest)
- test_unsupported_fnapi_features
(apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest)
- test_upload_graph_experiment
(apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest)
- test_use_fastavro_experiment_is_added_on_py3_and_onwards
(apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest)
- test_use_fastavro_experiment_is_not_added_when_use_avro_is_present
(apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest)
- test_bad_path
(apache_beam.runners.dataflow.template_runner_test.TemplatingDataflowRunnerTest)
- test_full_completion
(apache_beam.runners.dataflow.template_runner_test.TemplatingDataflowRunnerTest)
Sample error message:
{code}
======================================================================
ERROR: test_default_ip_configuration
(apache_beam.runners.dataflow.internal.apiclient_test.UtilTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/home/yob/Desktop/beam-test/beam/sdks/python/apache_beam/runners/dataflow/internal/apiclient_test.py",
line 356, in test_default_ip_configuration
env = apiclient.Environment([],
File
"/home/yob/Desktop/beam-test/beam/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
line 182, in __init__
_verify_interpreter_version_is_supported(options)
File
"/home/yob/Desktop/beam-test/beam/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
line 1140, in _verify_interpreter_version_is_supported
raise Exception(
Exception: Dataflow runner currently supports Python versions 2.7, 3.5, 3.6,
and 3.7. To ignore this requirement and start a job using a different version
of Python 3 interpreter, pass --experiment ignore_py3_minor_version pipeline
option.
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
