See
<https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/2750/display/redirect>
Changes:
------------------------------------------
[...truncated 65.18 KB...]
warning: no files found matching 'LICENSE.python'
warning: sdist: standard file not found: should have one of README, README.rst,
README.txt, README.md
> Task :sdks:java:container:pullLicenses
Collecting pip
Using cached pip-22.2.2-py3-none-any.whl (2.0 MB)
Collecting setuptools
Using cached setuptools-65.1.1-py3-none-any.whl (1.2 MB)
Collecting wheel
Using cached wheel-0.37.1-py2.py3-none-any.whl (35 kB)
Installing collected packages: pip, setuptools, wheel
Attempting uninstall: pip
Found existing installation: pip 20.0.2
Uninstalling pip-20.0.2:
Successfully uninstalled pip-20.0.2
Attempting uninstall: setuptools
Found existing installation: setuptools 44.0.0
Uninstalling setuptools-44.0.0:
Successfully uninstalled setuptools-44.0.0
Successfully installed pip-22.2.2 setuptools-65.1.1 wheel-0.37.1
Collecting beautifulsoup4<5.0,>=4.9.0
Using cached beautifulsoup4-4.11.1-py3-none-any.whl (128 kB)
Collecting pyyaml<6.0.0,>=3.12
Using cached PyYAML-5.4.1-cp38-cp38-manylinux1_x86_64.whl (662 kB)
Collecting tenacity<6.0,>=5.0.2
Using cached tenacity-5.1.5-py2.py3-none-any.whl (34 kB)
Collecting soupsieve>1.2
Using cached soupsieve-2.3.2.post1-py3-none-any.whl (37 kB)
Collecting six>=1.9.0
Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Installing collected packages: soupsieve, six, pyyaml, tenacity, beautifulsoup4
Successfully installed beautifulsoup4-4.11.1 pyyaml-5.4.1 six-1.16.0
soupsieve-2.3.2.post1 tenacity-5.1.5
Executing python
<https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py>
--license_index=<https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/ws/src/sdks/java/container/build/reports/dependency-license/index.json>
--output_dir=<https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/ws/src/sdks/java/container/build/target/java_third_party_licenses>
--dep_url_yaml=<https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/ws/src/sdks/java/container/license_scripts/dep_urls_java.yaml>
--manual_license_path=<https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/ws/src/sdks/java/container/license_scripts/manual_licenses>
INFO:root:Pulling license for 240 dependencies using 16 threads.
INFO:root:Replaced local file URL with
file://<https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/ws/src/sdks/java/container/license_scripts/manual_licenses/xz/COPYING>
for xz-1.5
INFO:root:pull_licenses_java.py succeed. It took 8.653863 seconds with 16
threads.
Copying licenses from
<https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/ws/src/sdks/java/container/build/target/java_third_party_licenses>
to
<https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/ws/src/sdks/java/container/build/target/third_party_licenses.>
Finished license_scripts.sh
> Task :sdks:java:container:goPrepare UP-TO-DATE
> Task :release:go-licenses:java:docker
[91mgo: downloading github.com/spf13/cobra v1.4.0
go: downloading github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
go: downloading github.com/otiai10/copy v1.6.0
[0m[91mgo: downloading github.com/google/licenseclassifier
v0.0.0-20210722185704-3043a050f148
[0m[91mgo: downloading gopkg.in/src-d/go-git.v4 v4.13.1
go: downloading go.opencensus.io v0.23.0
[0m[91mgo: downloading golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2
go: downloading golang.org/x/tools v0.1.10
[0m[91mgo: downloading golang.org/x/mod
v0.6.0-dev.0.20220106191415-9b9b3d81d5e3
[0m[91mgo: downloading github.com/spf13/pflag v1.0.5
[0m[91mgo: downloading github.com/golang/groupcache
v0.0.0-20210331224755-41bb18bfe9da
[0m[91mgo: downloading gopkg.in/src-d/go-billy.v4 v4.3.2
go: downloading golang.org/x/crypto v0.0.0-20220112180741-5e0467b6c7ce
[0m[91mgo: downloading github.com/sergi/go-diff v1.2.0
[0m[91mgo: downloading github.com/jbenet/go-context
v0.0.0-20150711004518-d14ea06fba99
[0m[91mgo: downloading github.com/emirpasic/gods v1.12.0
[0m[91mgo: downloading github.com/kevinburke/ssh_config
v0.0.0-20190725054713-01f96b0aa0cd
go: downloading github.com/mitchellh/go-homedir v1.1.0
[0m[91mgo: downloading github.com/xanzy/ssh-agent v0.2.1
[0m[91mgo: downloading github.com/src-d/gcfg v1.4.0
[0m[91mgo: downloading golang.org/x/sys v0.0.0-20220111092808-5a964db01320
[0m[91mgo: downloading gopkg.in/warnings.v0 v0.1.2
[0m[91mgo: downloading golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1
[0m
> Task :runners:flink:1.15:job-server:shadowJar FROM-CACHE
> Task :sdks:java:container:java8:copyJavaThirdPartyLicenses
> Task :sdks:java:container:goBuild
/home/jenkins/go/bin/go1.18.1 build -o ./build/target/linux_amd64/boot boot.go
boot_test.go
> Task :sdks:java:container:java8:copySdkHarnessLauncher
Execution optimizations have been disabled for task
':sdks:java:container:java8:copySdkHarnessLauncher' to ensure correctness due
to the following reasons:
- Gradle detected a problem with the following location:
'<https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/ws/src/sdks/java/container/build/target'.>
Reason: Task ':sdks:java:container:java8:copySdkHarnessLauncher' uses this
output of task ':sdks:java:container:downloadCloudProfilerAgent' without
declaring an explicit or implicit dependency. This can lead to incorrect
results being produced, depending on what order the tasks are executed. Please
refer to
https://docs.gradle.org/7.4/userguide/validation_problems.html#implicit_dependency
for more details about this problem.
- Gradle detected a problem with the following location:
'<https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/ws/src/sdks/java/container/build/target'.>
Reason: Task ':sdks:java:container:java8:copySdkHarnessLauncher' uses this
output of task ':sdks:java:container:pullLicenses' without declaring an
explicit or implicit dependency. This can lead to incorrect results being
produced, depending on what order the tasks are executed. Please refer to
https://docs.gradle.org/7.4/userguide/validation_problems.html#implicit_dependency
for more details about this problem.
> Task :release:go-licenses:java:docker
Removing intermediate container 94e82b8dc25c
---> b9182ddc8c79
Step 5/8 : COPY get-licenses.sh /opt/apache/beam/
---> cdce28e84f13
Step 6/8 : ARG sdk_location
---> Running in 2266fac150a9
Removing intermediate container 2266fac150a9
---> 3b28c7eaf8ed
Step 7/8 : ENV sdk_location=$sdk_location
---> Running in e015dd3f0d3c
Removing intermediate container e015dd3f0d3c
---> fdb607618ff0
Step 8/8 : ENTRYPOINT /opt/apache/beam/get-licenses.sh
---> Running in 3a70c49fbc9f
Removing intermediate container 3a70c49fbc9f
---> d2923958fb47
Successfully built d2923958fb47
Successfully tagged golicenses-java:latest
> Task :release:go-licenses:java:dockerRun
+ rm -rf '/output/*'
+ go get github.com/apache/beam/sdks/v2/java/container
go: downloading github.com/apache/beam/sdks/v2 v2.40.0
go: downloading github.com/apache/beam v2.40.0+incompatible
go: downloading github.com/golang/protobuf v1.5.2
go: downloading google.golang.org/grpc v1.47.0
go: downloading google.golang.org/protobuf v1.28.0
go: downloading google.golang.org/genproto v0.0.0-20220602131408-e326c6e8e9c8
go: downloading golang.org/x/net v0.0.0-20220607020251-c690dde0001d
go: downloading golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a
go: downloading golang.org/x/text v0.3.7
go: added github.com/apache/beam/sdks/v2 v2.40.0
go: added github.com/golang/protobuf v1.5.2
go: added golang.org/x/net v0.0.0-20220607020251-c690dde0001d
go: added golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a
go: added golang.org/x/text v0.3.7
go: added google.golang.org/genproto v0.0.0-20220602131408-e326c6e8e9c8
go: added google.golang.org/grpc v1.47.0
go: added google.golang.org/protobuf v1.28.0
+ go-licenses save github.com/apache/beam/sdks/v2/java/container
--save_path=/output/licenses
W0821 10:03:39.207691 27 library.go:94] "golang.org/x/sys/unix" contains
non-Go code that can't be inspected for further dependencies:
/go/pkg/mod/golang.org/x/[email protected]/unix/asm_linux_amd64.s
+ go-licenses csv github.com/apache/beam/sdks/v2/java/container
+ tee /output/licenses/list.csv
W0821 10:03:43.572078 171 library.go:94] "golang.org/x/sys/unix" contains
non-Go code that can't be inspected for further dependencies:
/go/pkg/mod/golang.org/x/[email protected]/unix/asm_linux_amd64.s
github.com/apache/beam/sdks/v2/go/pkg/beam,https://github.com/apache/beam/blob/sdks/v2.40.0/sdks/go/README.md,Apache-2.0
github.com/apache/beam/sdks/v2/java/container,https://github.com/apache/beam/blob/sdks/v2.40.0/sdks/LICENSE,Apache-2.0
github.com/golang/protobuf,https://github.com/golang/protobuf/blob/v1.5.2/LICENSE,BSD-3-Clause
golang.org/x/net,https://cs.opensource.google/go/x/net/+/c690dde0:LICENSE,BSD-3-Clause
golang.org/x/sys,https://cs.opensource.google/go/x/sys/+/bc2c85ad:LICENSE,BSD-3-Clause
golang.org/x/text,https://cs.opensource.google/go/x/text/+/v0.3.7:LICENSE,BSD-3-Clause
google.golang.org/genproto/googleapis/rpc/status,https://github.com/googleapis/go-genproto/blob/e326c6e8e9c8/LICENSE,Apache-2.0
google.golang.org/grpc,https://github.com/grpc/grpc-go/blob/v1.47.0/LICENSE,Apache-2.0
google.golang.org/protobuf,https://github.com/protocolbuffers/protobuf-go/blob/v1.28.0/LICENSE,BSD-3-Clause
+ chmod -R a+w /output/licenses
> Task :release:go-licenses:java:createLicenses
> Task :sdks:java:container:java8:copyGolangLicenses
> Task :sdks:java:container:java8:dockerPrepare
> Task :sdks:java:container:java8:docker
> Task :sdks:python:test-suites:portable:py39:flinkCompatibilityMatrixLOOPBACK
[1mflink-runner-test create:
<https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python/target/.tox-flink-runner-test/flink-runner-test>
[0m[1mflink-runner-test installdeps: -rbuild-requirements.txt
[0m[1mflink-runner-test inst:
<https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python/target/.tox-flink-runner-test/.tmp/package/1/apache-beam.tar.gz>
[0m[1mflink-runner-test installed: apache-beam @
file://<https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python/target/.tox-flink-runner-test/.tmp/package/1/apache-beam.tar.gz,attrs==22.1.0,certifi==2022.6.15,cffi==1.15.1,charset-normalizer==2.1.1,cloudpickle==2.1.0,crcmod==1.7,cryptography==37.0.4,deprecation==2.1.0,dill==0.3.1.1,distlib==0.3.1,docker==6.0.0,docopt==0.6.2,execnet==1.9.0,fastavro==1.6.0,freezegun==1.2.2,greenlet==1.1.2,grpcio==1.47.0,grpcio-tools==1.37.0,hdfs==2.7.0,httplib2==0.20.4,idna==3.3,iniconfig==1.1.1,joblib==1.1.0,mock==2.0.0,mypy-protobuf==1.18,numpy==1.22.4,orjson==3.7.12,packaging==21.3,pandas==1.4.3,parameterized==0.8.1,pbr==5.10.0,pluggy==1.0.0,proto-plus==1.22.0,protobuf==3.20.1,psycopg2-binary==2.9.3,py==1.11.0,pyarrow==7.0.0,pycparser==2.21,pydot==1.4.2,PyHamcrest==1.10.1,pymongo==3.12.3,PyMySQL==1.0.2,pyparsing==3.0.9,pytest==7.1.2,pytest-forked==1.4.0,pytest-timeout==2.1.0,pytest-xdist==2.5.0,python-dateutil==2.8.2,pytz==2022.2.1,PyYAML==6.0,regex==2022.8.17,requests==2.28.1,requests-mock==1.9.3,scikit-learn==1.1.2,scipy==1.9.0,six==1.16.0,SQLAlchemy==1.4.40,tenacity==5.1.5,testcontainers==3.6.1,threadpoolctl==3.1.0,tomli==2.0.1,typing_extensions==4.3.0,urllib3==1.26.11,websocket-client==1.3.3,wrapt==1.14.1,zstandard==0.18.0>
[0m[1mflink-runner-test run-test-pre: PYTHONHASHSEED='4101988638'
[0m[1mflink-runner-test run-test-pre: commands[0] | python --version
[0mPython 3.9.10
[1mflink-runner-test run-test-pre: commands[1] | pip --version
[0mpip 22.2.1 from
<https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python/target/.tox-flink-runner-test/flink-runner-test/lib/python3.9/site-packages/pip>
(python 3.9)
[1mflink-runner-test run-test-pre: commands[2] | pip check
[0mNo broken requirements found.
[1mflink-runner-test run-test-pre: commands[3] | bash
<https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m[1mflink-runner-test run-test: commands[0] |
<https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python/scripts/pytest_validates_runner.sh>
flink-runner-test
<https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python/apache_beam/runners/portability/flink_runner_test.py>
'--flink_job_server_jar=<https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/ws/src/runners/flink/1.15/job-server/build/libs/beam-runners-flink-1.15-job-server-2.42.0-SNAPSHOT.jar>
--environment_type=LOOPBACK'
[0m============================= test session starts
==============================
platform linux -- Python 3.9.10, pytest-7.1.2, pluggy-1.0.0
cachedir: target/.tox-flink-runner-test/flink-runner-test/.pytest_cache
rootdir:
<https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python,>
configfile: pytest.ini
plugins: xdist-2.5.0, timeout-2.1.0, forked-1.4.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
collected 210 items
apache_beam/runners/portability/flink_runner_test.py ......s...s...ss... [ 9%]
..........s....s...s..s............s..ss.....ss.s..F.....s...s...ss...ss [ 43%]
s.......s....s...s..s............s..ss.....ssss........s...s...ss....... [ 77%]
......s....s...s..s............s..ss.....ss.s.. [100%]
=================================== FAILURES ===================================
__________________ FlinkRunnerTestOptimized.test_assert_that ___________________
RuntimeError: Pipeline timed out waiting for job service subprocess.
During handling of the above exception, another exception occurred:
self =
<apache_beam.runners.portability.flink_runner_test.FlinkRunnerTestOptimized
testMethod=test_assert_that>
def test_assert_that(self):
# TODO: figure out a way for fn_api_runner to parse and raise the
# underlying exception.
with self.assertRaisesRegex(Exception, 'Failed assert'):
with self.create_pipeline() as p:
> assert_that(p | beam.Create(['a', 'b']), equal_to(['a']))
E AssertionError: "Failed assert" does not match "Pipeline timed out
waiting for job service subprocess."
apache_beam/runners/portability/fn_api_runner/fn_runner_test.py:113:
AssertionError
----------------------------- Captured stderr call -----------------------------
Aug 21, 2022 10:10:12 AM
software.amazon.awssdk.regions.internal.util.EC2MetadataUtils getItems
WARNING: Unable to retrieve the requested metadata.
Aug 21, 2022 10:10:12 AM org.apache.beam.runners.jobsubmission.JobServerDriver
createArtifactStagingService
INFO: ArtifactStagingService started on localhost:43895
Aug 21, 2022 10:10:12 AM org.apache.beam.runners.jobsubmission.JobServerDriver
createExpansionService
INFO: Java ExpansionService started on localhost:50325
Aug 21, 2022 10:10:12 AM org.apache.beam.runners.jobsubmission.JobServerDriver
createJobServer
INFO: JobService started on localhost:37467
Aug 21, 2022 10:10:12 AM org.apache.beam.runners.jobsubmission.JobServerDriver
run
INFO: Job server now running, terminate with Ctrl+C
=============================== warnings summary ===============================
target/.tox-flink-runner-test/flink-runner-test/lib/python3.9/site-packages/hdfs/config.py:15
<https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python/target/.tox-flink-runner-test/flink-runner-test/lib/python3.9/site-packages/hdfs/config.py>:15:
DeprecationWarning: the imp module is deprecated in favour of importlib; see
the module's documentation for alternative uses
from imp import load_source
target/.tox-flink-runner-test/flink-runner-test/lib/python3.9/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python/target/.tox-flink-runner-test/flink-runner-test/lib/python3.9/site-packages/tenacity/_asyncio.py>:42:
DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use
"async def" instead
def call(self, fn, *args, **kwargs):
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated xml file:
<https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python/pytest_flink-runner-test.xml>
-
=========================== short test summary info ============================
FAILED
apache_beam/runners/portability/flink_runner_test.py::FlinkRunnerTestOptimized::test_assert_that
====== 1 failed, 163 passed, 46 skipped, 2 warnings in 823.96s (0:13:43) =======
[31mERROR: InvocationError for command
<https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python/scripts/pytest_validates_runner.sh>
flink-runner-test apache_beam/runners/portability/flink_runner_test.py
'--flink_job_server_jar=<https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/ws/src/runners/flink/1.15/job-server/build/libs/beam-runners-flink-1.15-job-server-2.42.0-SNAPSHOT.jar>
--environment_type=LOOPBACK' (exited with code 1)
[0m[1mflink-runner-test run-test-post: commands[0] | bash
<https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary
____________________________________
[31mERROR: flink-runner-test: commands failed
[0m
> Task :sdks:python:test-suites:portable:py39:flinkCompatibilityMatrixLOOPBACK
> FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task
':sdks:python:test-suites:portable:py39:flinkCompatibilityMatrixLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
See
https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during
this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 16m 9s
147 actionable tasks: 98 executed, 45 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/p3sc6cvlyqnmg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
