[
https://issues.apache.org/jira/browse/BEAM-14072?focusedWorklogId=740265&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-740265
]
ASF GitHub Bot logged work on BEAM-14072:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 11/Mar/22 19:59
Start Date: 11/Mar/22 19:59
Worklog Time Spent: 10m
Work Description: rahuliyer95 commented on pull request #17078:
URL: https://github.com/apache/beam/pull/17078#issuecomment-1065458981
I ran the integration test suite by making some changes to the build files
to allow them to run with our GCP setup.
```
./gradlew -Pdocker-pull-licenses -PpythonVersion=3.7
-PdataflowProject=twttr-udp-e2e-jobs-prod
:runners:google-cloud-dataflow-java:validatesCrossLanguageRunnerPythonUsingJava
```
The job ran with the following beam plugins,
```
--beam_plugins='apache_beam.io.filesystem.FileSystem,
apache_beam.io.hadoopfilesystem.HadoopFileSystem,
apache_beam.io.localfilesystem.LocalFileSystem,
apache_beam.io.gcp.gcsfilesystem.GCSFileSystem,
apache_beam.io.aws.s3filesystem.S3FileSystem,
apache_beam.io.azure.blobstoragefilesystem.BlobStorageFileSystem,
apache_beam.io.filesystem_test.TestingFileSystem,
apache_beam.runners.portability.sdk_container_builder.SdkContainerImageBuilder,
apache_beam.runners.portability.sdk_container_builder._SdkContainerImageLocalBuilder,
apache_beam.runners.portability.sdk_container_builder._SdkContainerImageCloudBuilder,
apache_beam.runners.interactive.display.pipeline_graph_renderer.PipelineGraphRenderer,
apache_beam.runners.interactive.display.pipeline_graph_renderer.MuteRenderer,
apache_beam.runners.interactive.display.pipeline_graph_renderer.TextRenderer,
apache_beam.runners.interactive.display.pipeline_graph_renderer.PydotRenderer
```
and failed with the following error,
```
"2022/03/11 03:53:04 Traceback (most recent call last): "
"2022/03/11 03:53:04 File
"/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py",
line 55, in _import_beam_plugins "
"2022/03/11 03:53:04 importlib.import_module(plugin) "
"2022/03/11 03:53:04 File "/usr/local/lib/python3.7/importlib/__init__.py",
line 127, in import_module "
"2022/03/11 03:53:04 return _bootstrap._gcd_import(name[level:], package,
level) "
"2022/03/11 03:53:04 File "<frozen importlib._bootstrap>", line 1006, in
_gcd_import "
"2022/03/11 03:53:04 File "<frozen importlib._bootstrap>", line 983, in
_find_and_load "
"2022/03/11 03:53:04 File "<frozen importlib._bootstrap>", line 953, in
_find_and_load_unlocked "
"2022/03/11 03:53:04 File "<frozen importlib._bootstrap>", line 219, in
_call_with_frames_removed "
"2022/03/11 03:53:04 File "<frozen importlib._bootstrap>", line 1006, in
_gcd_import "
"2022/03/11 03:53:04 File "<frozen importlib._bootstrap>", line 983, in
_find_and_load "
"2022/03/11 03:53:04 File "<frozen importlib._bootstrap>", line 967, in
_find_and_load_unlocked "
"2022/03/11 03:53:04 File "<frozen importlib._bootstrap>", line 677, in
_load_unlocked "
"2022/03/11 03:53:04 File "<frozen importlib._bootstrap_external>", line
728, in exec_module "
"2022/03/11 03:53:04 File "<frozen importlib._bootstrap>", line 219, in
_call_with_frames_removed "
"2022/03/11 03:53:04 File
"/usr/local/lib/python3.7/site-packages/apache_beam/io/filesystem_test.py",
line 34, in <module> "
"2022/03/11 03:53:04 from parameterized import param "
"2022/03/11 03:53:04 ModuleNotFoundError: No module named 'parameterized' "
```
Apache Beam SDK does not ship with test packages aka `parameterized` and
hence the import fails. With the fix of ignoring the module if the import fails
with a warning message seems to pass the integration tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 740265)
Time Spent: 50m (was: 40m)
> XVR_PythonUsingJava_Dataflow and XVR_PythonUsingJavaSQL_Dataflow failing due
> to a timeout
> -----------------------------------------------------------------------------------------
>
> Key: BEAM-14072
> URL: https://issues.apache.org/jira/browse/BEAM-14072
> Project: Beam
> Issue Type: Bug
> Components: cross-language, test-failures
> Reporter: Chamikara Madhusanka Jayalath
> Assignee: Steve Niemitz
> Priority: P2
> Time Spent: 50m
> Remaining Estimate: 0h
>
> First failures:
> https://ci-beam.apache.org/job/beam_PostCommit_XVR_PythonUsingJava_Dataflow/144/
> https://ci-beam.apache.org/job/beam_PostCommit_XVR_PythonUsingJavaSQL_Dataflow/138/
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
