See
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/3634/display/redirect?page=changes>
Changes:
[noreply] [Spark Dataset runner] Fix SparkSessionFactory to better support
running
------------------------------------------
[...truncated 861.54 KB...]
capabilities: "beam:coder:string_utf8:v1"
capabilities: "beam:coder:length_prefix:v1"
capabilities: "beam:coder:kv:v1"
capabilities: "beam:coder:iterable:v1"
capabilities: "beam:coder:state_backed_iterable:v1"
capabilities: "beam:coder:windowed_value:v1"
capabilities: "beam:coder:global_window:v1"
capabilities: "beam:coder:interval_window:v1"
capabilities: "beam:coder:row:v1"
capabilities: "beam:coder:nullable:v1"
dependencies: <
type_urn: "beam:artifact:type:file:v1"
role_urn: "beam:artifact:role:go_worker_binary:v1"
>
>
>
>
root_transform_ids: "s1"
root_transform_ids: "e2"
root_transform_ids: "e3"
root_transform_ids: "s3"
requirements: "beam:requirement:pardo:splittable_dofn:v1"
2023/01/16 17:27:21 Cross-compiling
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/sdks/go/test/integration/io/xlang/jdbc/jdbc_test.go>
as /tmp/worker-4-1673890041702579592
2023/01/16 17:27:23 Prepared job with id:
go-testjdbcio_postgresreadwrite-488-3ba37eab-3bc4-4239-ab0e-2b081207c65d and
staging token:
go-testjdbcio_postgresreadwrite-488-3ba37eab-3bc4-4239-ab0e-2b081207c65d
2023/01/16 17:27:24 Staged binary artifact with token:
2023/01/16 17:27:24 Submitted job:
go-testjdbcio_postgresreadwrite-488-3ba37eab-3bc4-4239-ab0e-2b081207c65d
2023/01/16 17:27:24 2023-01-16 17:27:24. (1): Default Python SDK image for
environment is apache/beam_python3.7_sdk:2.45.0.dev
2023/01/16 17:27:24 Job state: STOPPED
2023/01/16 17:27:24 Job state: STARTING
2023/01/16 17:27:24 Job state: RUNNING
2023/01/16 17:27:24 2023-01-16 17:27:24. (2): ==================== <function
annotate_downstream_side_inputs at 0x7f4480b690e0> ====================
2023/01/16 17:27:24 2023-01-16 17:27:24. (3): ==================== <function
fix_side_input_pcoll_coders at 0x7f4480b69200> ====================
2023/01/16 17:27:24 2023-01-16 17:27:24. (4): ==================== <function
pack_combiners at 0x7f4480b69710> ====================
2023/01/16 17:27:24 2023-01-16 17:27:24. (5): ==================== <function
lift_combiners at 0x7f4480b697a0> ====================
2023/01/16 17:27:24 2023-01-16 17:27:24. (6): ==================== <function
expand_sdf at 0x7f4480b69950> ====================
2023/01/16 17:27:24 2023-01-16 17:27:24. (7): ==================== <function
expand_gbk at 0x7f4480b699e0> ====================
2023/01/16 17:27:24 2023-01-16 17:27:24. (8): ==================== <function
sink_flattens at 0x7f4480b69b00> ====================
2023/01/16 17:27:24 2023-01-16 17:27:24. (9): ==================== <function
greedily_fuse at 0x7f4480b69b90> ====================
2023/01/16 17:27:24 2023-01-16 17:27:24. (10): ==================== <function
read_to_impulse at 0x7f4480b69c20> ====================
2023/01/16 17:27:24 2023-01-16 17:27:24. (11): ==================== <function
impulse_to_input at 0x7f4480b69cb0> ====================
2023/01/16 17:27:24 2023-01-16 17:27:24. (12): ==================== <function
sort_stages at 0x7f4480b69ef0> ====================
2023/01/16 17:27:24 2023-01-16 17:27:24. (13): ==================== <function
add_impulse_to_dangling_transforms at 0x7f4480b6a050> ====================
2023/01/16 17:27:24 2023-01-16 17:27:24. (14): ==================== <function
setup_timer_mapping at 0x7f4480b69e60> ====================
2023/01/16 17:27:24 2023-01-16 17:27:24. (15): ==================== <function
populate_data_channel_coders at 0x7f4480b69f80> ====================
2023/01/16 17:27:24 2023-01-16 17:27:24. (16): starting control server on port
40761
2023/01/16 17:27:24 2023-01-16 17:27:24. (17): starting data server on port
40837
2023/01/16 17:27:24 2023-01-16 17:27:24. (18): starting state server on port
41727
2023/01/16 17:27:24 2023-01-16 17:27:24. (19): starting logging server on port
42693
2023/01/16 17:27:24 2023-01-16 17:27:24. (20): Created Worker handler
<apache_beam.runners.portability.fn_api_runner.worker_handlers.DockerSdkWorkerHandler
object at 0x7f44801a9350> for environment go (beam:env:docker:v1,
b'\n\x16apache/beam_go_sdk:dev')
2023/01/16 17:27:24 2023-01-16 17:27:24. (21): Attempting to pull image
apache/beam_go_sdk:dev
2023/01/16 17:27:24 2023-01-16 17:27:24. (22): Unable to pull image
apache/beam_go_sdk:dev, defaulting to local image if it exists
2023/01/16 17:27:27 2023-01-16 17:27:27. (23): Waiting for docker to start up.
Current status is running
2023/01/16 17:27:27 2023-01-16 17:27:27. (24): Docker container is running.
container_id =
b'b9642619682c151083d5b7fb51e0bff98a6fd6b39a6b1be9d847b9accbf303ca', worker_id
= worker_18
2023/01/16 17:27:28 2023-01-16 17:27:28. (25): Created Worker handler
<apache_beam.runners.portability.fn_api_runner.worker_handlers.DockerSdkWorkerHandler
object at 0x7f44801d6f10> for environment HQDZEsIckAbeam:env:docker:v1
(beam:env:docker:v1, b'\n apache/beam_java8_sdk:2.45.0.dev')
2023/01/16 17:27:28 2023-01-16 17:27:28. (26): Attempting to pull image
apache/beam_java8_sdk:2.45.0.dev
2023/01/16 17:27:29 2023-01-16 17:27:29. (27): Unable to pull image
apache/beam_java8_sdk:2.45.0.dev, defaulting to local image if it exists
2023/01/16 17:27:29 2023-01-16 17:27:29. (28): Waiting for docker to start up.
Current status is running
2023/01/16 17:27:29 2023-01-16 17:27:29. (29): Docker container is running.
container_id =
b'cc0d4f0d1054922ddfcc175a20410ca0eeab2b5e4e76926d31d8237f30fd6b39', worker_id
= worker_19
2023/01/16 17:27:39 2023-01-16 17:27:39. (30): Completed job in
15.139701128005981 seconds with state DONE.
2023/01/16 17:27:39 Job state: DONE
--- PASS: TestJDBCIO_PostgresReadWrite (49.22s)
PASS
ok github.com/apache/beam/sdks/v2/go/test/integration/io/xlang/jdbc
120.140s
=== RUN TestKafkaIO_BasicReadWrite
integration.go:308: Test TestKafkaIO_BasicReadWrite is currently filtered
for runner portable
--- SKIP: TestKafkaIO_BasicReadWrite (0.00s)
PASS
ok github.com/apache/beam/sdks/v2/go/test/integration/io/xlang/kafka
6.054s
$ cd ../..
$ exit
$ exit
> Task
> :sdks:python:test-suites:direct:xlang:validatesCrossLanguageRunnerJavaUsingPython
INFO:apache_beam.runners.portability.stager:Executing command:
['/home/jenkins/.apache_beam/cache/venvs/py-3.8-beam-2.44.0-743bdccef67087512a7a190820e727aab7bdcb14/bin/python',
'-m', 'pip', 'download', '--dest', '/tmp/dataflow-requirements-cache', '-r',
'/tmp/tmpn8pjqahk/tmp_requirements.txt', '--exists-action', 'i', '--no-deps',
'--implementation', 'cp', '--abi', 'cp38', '--platform', 'manylinux2014_x86_64']
INFO:root:Default Python SDK image for environment is
apache/beam_python3.8_sdk:2.44.0
INFO:root:No image given, using default Python SDK image
INFO:root:Default Python SDK image for environment is
apache/beam_python3.8_sdk:2.44.0
INFO:root:Python SDK container image set to "apache/beam_python3.8_sdk:2.44.0"
for Docker environment
INFO:__main__:Listening for expansion requests at 37901
INFO:root:Missing pipeline option (runner). Executing pipeline using the
default runner: DirectRunner.
INFO:__main__:Shutting down expansion service.
org.apache.beam.sdk.extensions.python.PythonExternalTransformTest >
pythonTransformWithDependencies FAILED
java.lang.RuntimeException at PythonExternalTransformTest.java:72
Caused by: java.lang.RuntimeException at
PythonExternalTransformTest.java:72
Caused by: java.io.IOException at
PythonExternalTransformTest.java:72
org.apache.beam.sdk.extensions.python.transforms.DataframeTransformTest >
testDataframeSum FAILED
java.lang.RuntimeException at DataframeTransformTest.java:50
Caused by: java.lang.RuntimeException at DataframeTransformTest.java:50
java.lang.NullPointerException at Preconditions.java:980
org.apache.beam.sdk.extensions.python.transforms.PythonMapTest > testPythonMap
FAILED
java.lang.RuntimeException at PythonMapTest.java:41
Caused by: java.lang.RuntimeException at PythonMapTest.java:41
java.lang.NullPointerException at Preconditions.java:980
org.apache.beam.sdk.extensions.python.transforms.PythonMapTest >
testPythonFlatMap FAILED
java.lang.RuntimeException at PythonMapTest.java:54
Caused by: java.lang.RuntimeException at PythonMapTest.java:54
java.lang.NullPointerException at Preconditions.java:980
org.apache.beam.sdk.extensions.python.transforms.RunInferenceTransformTest >
testRunInferenceWithKVs FAILED
java.lang.RuntimeException at RunInferenceTransformTest.java:108
Caused by: java.lang.RuntimeException at
RunInferenceTransformTest.java:108
java.lang.NullPointerException at Preconditions.java:980
org.apache.beam.sdk.extensions.python.transforms.RunInferenceTransformTest >
testRunInference FAILED
java.lang.RuntimeException at RunInferenceTransformTest.java:60
Caused by: java.lang.RuntimeException at
RunInferenceTransformTest.java:60
java.lang.NullPointerException at Preconditions.java:980
15 tests completed, 6 failed
> Task
> :sdks:python:test-suites:direct:xlang:validatesCrossLanguageRunnerJavaUsingPython
> FAILED
> Task
> :sdks:python:test-suites:direct:xlang:validatesCrossLanguageRunnerPythonUsingJava
>>> RUNNING integration tests with pipeline options: --runner=PortableRunner
>>> --job_endpoint=localhost:46143 --environment_cache_millis=10000
>>> --experiments=beam_fn_api
>>> pytest options:
>>> collect markers: -m=uses_java_expansion_service
ImportError while loading conftest
'<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/sdks/python/conftest.py'.>
conftest.py:23: in <module>
from apache_beam.testing.test_pipeline import TestPipeline
/usr/lib/python3.7/os.py:223: in makedirs
mkdir(name, mode)
E OSError: [Errno 28] No space left on device:
'<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/sdks/python/apache_beam/testing/__pycache__'>
> Task
> :sdks:python:test-suites:direct:xlang:validatesCrossLanguageRunnerPythonUsingJava
> FAILED
> Task
> :sdks:python:test-suites:direct:xlang:validatesCrossLanguageRunnerPythonUsingSql
> FAILED
>>> RUNNING integration tests with pipeline options: --runner=PortableRunner
>>> --job_endpoint=localhost:46143 --environment_cache_millis=10000
>>> --experiments=beam_fn_api
>>> pytest options:
>>> collect markers: -m=xlang_sql_expansion_service
Traceback (most recent call last):
File
"<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/build/gradleenv/1922375555/bin/pytest";,>
line 8, in <module>
sys.exit(console_main())
File
"<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/build/gradleenv/1922375555/lib/python3.7/site-packages/_pytest/config/__init__.py";,>
line 190, in console_main
code = main()
File
"<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/build/gradleenv/1922375555/lib/python3.7/site-packages/_pytest/config/__init__.py";,>
line 148, in main
config = _prepareconfig(args, plugins)
File
"<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/build/gradleenv/1922375555/lib/python3.7/site-packages/_pytest/config/__init__.py";,>
line 330, in _prepareconfig
pluginmanager=pluginmanager, args=args
File
"<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/build/gradleenv/1922375555/lib/python3.7/site-packages/pluggy/_hooks.py";,>
line 265, in __call__
return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
File
"<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/build/gradleenv/1922375555/lib/python3.7/site-packages/pluggy/_manager.py";,>
line 80, in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
File
"<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/build/gradleenv/1922375555/lib/python3.7/site-packages/pluggy/_callers.py";,>
line 55, in _multicall
gen.send(outcome)
File
"<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/build/gradleenv/1922375555/lib/python3.7/site-packages/_pytest/helpconfig.py";,>
line 103, in pytest_cmdline_parse
config: Config = outcome.get_result()
File
"<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/build/gradleenv/1922375555/lib/python3.7/site-packages/pluggy/_result.py";,>
line 60, in get_result
raise ex[1].with_traceback(ex[2])
File
"<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/build/gradleenv/1922375555/lib/python3.7/site-packages/pluggy/_callers.py";,>
line 39, in _multicall
res = hook_impl.function(*args)
File
"<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/build/gradleenv/1922375555/lib/python3.7/site-packages/_pytest/config/__init__.py";,>
line 1058, in pytest_cmdline_parse
self.parse(args)
File
"<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/build/gradleenv/1922375555/lib/python3.7/site-packages/_pytest/config/__init__.py";,>
line 1346, in parse
self._preparse(args, addopts=addopts)
File
"<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/build/gradleenv/1922375555/lib/python3.7/site-packages/_pytest/config/__init__.py";,>
line 1249, in _preparse
early_config=self, args=args, parser=self._parser
File
"<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/build/gradleenv/1922375555/lib/python3.7/site-packages/pluggy/_hooks.py";,>
line 265, in __call__
return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
File
"<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/build/gradleenv/1922375555/lib/python3.7/site-packages/pluggy/_manager.py";,>
line 80, in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
File
"<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/build/gradleenv/1922375555/lib/python3.7/site-packages/pluggy/_callers.py";,>
line 60, in _multicall
return outcome.get_result()
File
"<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/build/gradleenv/1922375555/lib/python3.7/site-packages/pluggy/_result.py";,>
line 60, in get_result
raise ex[1].with_traceback(ex[2])
File
"<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/build/gradleenv/1922375555/lib/python3.7/site-packages/pluggy/_callers.py";,>
line 34, in _multicall
next(gen) # first yield
File
"<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/build/gradleenv/1922375555/lib/python3.7/site-packages/_pytest/capture.py";,>
line 141, in pytest_load_initial_conftests
capman.start_global_capturing()
File
"<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/build/gradleenv/1922375555/lib/python3.7/site-packages/_pytest/capture.py";,>
line 688, in start_global_capturing
self._global_capturing = _get_multicapture(self._method)
File
"<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/build/gradleenv/1922375555/lib/python3.7/site-packages/_pytest/capture.py";,>
line 630, in _get_multicapture
return MultiCapture(in_=FDCapture(0), out=FDCapture(1), err=FDCapture(2))
File
"<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/build/gradleenv/1922375555/lib/python3.7/site-packages/_pytest/capture.py";,>
line 388, in __init__
TemporaryFile(buffering=0),
File "/usr/lib/python3.7/tempfile.py", line 725, in TemporaryFile
prefix, suffix, dir, output_type = _sanitize_params(prefix, suffix, dir)
File "/usr/lib/python3.7/tempfile.py", line 265, in _sanitize_params
dir = gettempdir()
File "/usr/lib/python3.7/tempfile.py", line 433, in gettempdir
tempdir = _get_default_tempdir()
File "/usr/lib/python3.7/tempfile.py", line 368, in _get_default_tempdir
dirlist)
FileNotFoundError: [Errno 2] No usable temporary directory found in ['/tmp',
'/var/tmp', '/usr/tmp',
'<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/sdks/python']>
> Task :sdks:python:test-suites:direct:xlang:fnApiJobServerCleanup
Killing process at 1222043
> Task :sdks:python:test-suites:direct:xlang:validatesCrossLanguageRunnerCleanup
Stopping expansion service pid: 1239024.
Stopping expansion service pid: 1239025.
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task
':sdks:python:test-suites:direct:xlang:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at:
> file://<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/sdks/python/test-suites/direct/xlang/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task
':sdks:python:test-suites:direct:xlang:validatesCrossLanguageRunnerPythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 4
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task
':sdks:python:test-suites:direct:xlang:validatesCrossLanguageRunnerPythonUsingSql'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
See
https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during
this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 34m 30s
248 actionable tasks: 163 executed, 73 from cache, 12 up-to-date
Publishing build scan...
https://gradle.com/s/bhyw6k6qxapqq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
