See <https://ci-beam.apache.org/job/beam_Metrics_Report/81/display/redirect?page=changes>
Changes: [benjamin.gonzalez] [BEAM-12572] Add DistinctExample test [benjamin.gonzalez] [BEAM-12572] Add DistinctExample test [benjamin.gonzalez] [BEAM-12572] Add java examples tests [benjamin.gonzalez] [BEAM-12572] Change tests collected by runner [benjamin.gonzalez] [BEAM-12572] Fix PipelineOptions, skip tests for specific runners [benjamin.gonzalez] [BEAM-12572] Skip failing tests [benjamin.gonzalez] [BEAM-12572] Fix checkstyle warnings [benjamin.gonzalez] [BEAM-12572] Skip tfidf tests on direct runner [benjamin.gonzalez] [BEAM-12572] Fix PipelineOptions [benjamin.gonzalez] [BEAM-12572] Fix checkstyle warning [Luke Cwik] [BEAM-10212] Clean-up comments, remove rawtypes usage. [benjamin.gonzalez] [BEAM-12572] Enable test with Pipeline serialization error fixed, link [benjamin.gonzalez] [BEAM-12572] Sickbay TfIdf test for DirectRunner [Kiley Sok] Add Java 17 Nexmark metrics to Grafana [yiru] . [Chamikara Madhusanka Jayalath] Updates x-lang release validation to use staged jars [yiru] . [dhuntsperger] documented maven-to-gradle conversion for Dataflow; refactored java [dhuntsperger] adding a list of example pipelines [benjamin.gonzalez] [BEAM-12572] Refactor naming for BigQueryClient [yiru] . [dhuntsperger] removing unnecessary `ls` command from maven project generation [yiru] format fix [yiru] . [yiru] make DoFn into a separate class [yiru] . [noreply] Mapped JOB_STATE_RESOURCE_CLEANING_UP to State.RUNNING. [ryanthompson591] fixed typo in typehints [mmack] [BEAM-13175] Add KinesisIO.write for AWS SDK v2. [mmack] [adhoc] Update owners for aws modules [zyichi] Remove unused prebuild_sdk_container_base_iamge option from validate [noreply] [BEAM-14051] Add a warning to gradle.properties that many Beam modules [noreply] Merge pull request #17000 from [BEAM-13881] [Playground] Increase test [noreply] Add deterministic dict coding via key sorting. (#17037) [noreply] Merge pull request #17054 from [BEAM-14075] [SnowflakeIO] Change a [dhuntsperger] fixing filename formatting in response to feedback [dhuntsperger] adding extra step emphasizing runner setupt [dhuntsperger] reorganized instructions to emphasize setup steps for runners [mmack] [BEAM-13175] Document changes to IOs in amazon-web-services2 for AWS [noreply] [BEAM-13767] Move a bunch of python tasks to use gradle configuration… [yiru] fix setting [noreply] Merge pull request #17052 from [BEAM-13818] [SnowflakeIO] Add support [noreply] Adding pydoc for StateHandler (#17091) [noreply] BEAM-3165 Bypass split if numSplit is zero (#17084) [hengfeng] feat: add more custom metrics [noreply] [BEAM-14103][Playgrounf][Bugfix] Fix google analytics id (#17092) [noreply] Minor: Make ScopedReadStateSupplier final (#16992) [noreply] [BEAM-14113] Improve SamzaJobServerDriver extensibility (#17099) [noreply] [BEAM-14116] Chunk commit requests dynamically (#17004) [noreply] Merge pull request #17079 from [BEAM-13660] Add types and queries in [noreply] [BEAM-13888] Add unit testing to ioutilx (#17058) [noreply] Merge pull request #16822 from [BEAM-13841][Playground] Add Application [noreply] Minor: Make serializableCoder warning gramatically correct english [noreply] [BEAM-14091] Fixing Interactive Beam show/collect for remote runners [noreply] [BEAM-11934] Add enable_file_dynamic_sharding to allow DataflowRunner [noreply] [BEAM-12777] Create symlink for `current` directory (#17105) [noreply] [BEAM-14020] Adding SchemaTransform, SchemaTransformProvider, [noreply] [BEAM-13015] Modify metrics to begin and reset to a non-dirty state. [noreply] [BEAM-14112] Avoid storing a generator in _CustomBigQuerySource (#17100) [noreply] Populate environment capabilities in v1beta3 protos. (#17042) [Kyle Weaver] [BEAM-12976] Test a whole pipeline using projection pushdown in BQ IO. [Kyle Weaver] [BEAM-12976] Enable projection pushdown for Java pipelines on Dataflow, [noreply] [BEAM-14038] Auto-startup for Python expansion service. (#17035) [Kyle Weaver] [BEAM-14123] Fix typo in hdfsIntegrationTest task name. [noreply] [BEAM-13893] improved coverage of jobopts package (#17003) [noreply] Merge pull request #16977 from [BEAM-12164] Added integration test for [mmack] [adhoc] Minor cleanup for aws2 tests [mmack] [BEAM-14125] Update website IO matrix to recommend aws2 IOs [noreply] [BEAM-14128] Eliminating quadratic behavior of [noreply] [BEAM-13972] Add RunInference interface (#16917) [noreply] Merge pull request #17116 from [BEAM-12164] Remove change_stream in [yiru] fix checkstyle [yiru] spotlessapply [noreply] Deprecate tags.go (#17025) [noreply] [BEAM-12753] and [BEAM-12815] Fix Flink Integration Tests (#17067) [noreply] Merge pull request #16895 from [BEAM-13882][Playground] More tests for [noreply] [BEAM-13925] Add weekly automation to update our reviewer config [noreply] Merge pull request #17076 from Beam 14082 update payground for mobile [noreply] [BEAM-13925] Assign committers in the scheduled action (#17062) [noreply] Pin setup-gcloud to v0 instead of master (#17123) [noreply] [BEAM-3304] documentation for PaneInfo in BPG (#17047) [noreply] Merge pull request #17016 from [BEAM-14049][Playground] Add new API [noreply] Merge pull request #17077 from [BEAM-14078] [Website] change link [noreply] Merge pull request #17085 from [BEAM-14077] [Website] add beam [noreply] Update Changes.md w/Go pipeline pre-process fix. [noreply] [BEAM-14098] wrapper for postgres on JDBC IO GO SDK (#17088) [noreply] Merge pull request #17023 from [BEAM-12164]: Remove child partition [noreply] [BEAM-14122] Upgrade pip-licenses dependency (#17132) ------------------------------------------ Started by timer Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on apache-beam-jenkins-8 (beam) in workspace <https://ci-beam.apache.org/job/beam_Metrics_Report/ws/> The recommended git tool is: NONE No credentials specified Wiping out workspace first. Cloning the remote Git repository Cloning repository https://github.com/apache/beam.git > git init <https://ci-beam.apache.org/job/beam_Metrics_Report/ws/src> # > timeout=10 Fetching upstream changes from https://github.com/apache/beam.git > git --version # timeout=10 > git --version # 'git version 2.25.1' > git fetch --tags --force --progress -- https://github.com/apache/beam.git > +refs/heads/*:refs/remotes/origin/* # timeout=10 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # > timeout=10 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10 Fetching upstream changes from https://github.com/apache/beam.git > git fetch --tags --force --progress -- https://github.com/apache/beam.git > +refs/heads/*:refs/remotes/origin/* > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/* # > timeout=10 > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision 096aeeff833679af3068eea2f00b280a589256e8 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f 096aeeff833679af3068eea2f00b280a589256e8 # timeout=10 Commit message: "[BEAM-14122] Upgrade pip-licenses dependency (#17132)" > git rev-list --no-walk 2cec50b0f1b87388fb408b6af3e09c22ca4a627c # timeout=10 No emails were triggered. No emails were triggered. [EnvInject] - Executing scripts and injecting environment variables after the SCM step. [EnvInject] - Injecting as environment variables the properties content SPARK_LOCAL_IP=127.0.0.1 SETUPTOOLS_USE_DISTUTILS=stdlib [EnvInject] - Variables injected successfully. [Gradle] - Launching build. [src] $ <https://ci-beam.apache.org/job/beam_Metrics_Report/ws/src/gradlew> --continue --max-****s=12 -Dorg.gradle.jvmargs=-Xms2g -Dorg.gradle.jvmargs=-Xmx4g -Dorg.gradle.vfs.watch=false -Pdocker-pull-licenses -PinfluxDb=beam_test_metrics -PinfluxHost=10.128.0.96 -PinfluxPort=8086 :beam-test-jenkins:generateMetricsReport Starting a Gradle Daemon, 2 busy Daemons could not be reused, use --status for details Configuration on demand is an incubating feature. > Task :buildSrc:compileJava NO-SOURCE > Task :buildSrc:compileGroovy FROM-CACHE > Task :buildSrc:pluginDescriptors > Task :buildSrc:processResources > Task :buildSrc:classes > Task :buildSrc:jar > Task :buildSrc:assemble > Task :buildSrc:spotlessGroovy FROM-CACHE > Task :buildSrc:spotlessGroovyCheck UP-TO-DATE > Task :buildSrc:spotlessGroovyGradle FROM-CACHE > Task :buildSrc:spotlessGroovyGradleCheck UP-TO-DATE > Task :buildSrc:spotlessCheck UP-TO-DATE > Task :buildSrc:pluginUnderTestMetadata > Task :buildSrc:compileTestJava NO-SOURCE > Task :buildSrc:compileTestGroovy NO-SOURCE > Task :buildSrc:processTestResources NO-SOURCE > Task :buildSrc:testClasses UP-TO-DATE > Task :buildSrc:test NO-SOURCE > Task :buildSrc:validatePlugins FROM-CACHE > Task :buildSrc:check UP-TO-DATE > Task :buildSrc:build > Task :beam-test-jenkins:setupVirtualenv Collecting pip Using cached pip-22.0.4-py3-none-any.whl (2.1 MB) Installing collected packages: pip Attempting uninstall: pip Found existing installation: pip 20.0.2 Uninstalling pip-20.0.2: Successfully uninstalled pip-20.0.2 Successfully installed pip-22.0.4 Collecting tox==3.20.1 Using cached tox-3.20.1-py2.py3-none-any.whl (83 kB) Requirement already satisfied: setuptools in <https://ci-beam.apache.org/job/beam_Metrics_Report/ws/src/build/gradleenv/-1871197473/lib/python3.8/site-packages> (from -r <https://ci-beam.apache.org/job/beam_Metrics_Report/ws/src/sdks/python/build-requirements.txt> (line 20)) (44.0.0) Collecting setuptools Using cached setuptools-60.10.0-py3-none-any.whl (1.1 MB) Collecting wheel>=0.36.0 Using cached wheel-0.37.1-py2.py3-none-any.whl (35 kB) Collecting grpcio-tools==1.37.0 Using cached grpcio_tools-1.37.0-cp38-cp38-manylinux2014_x86_64.whl (2.5 MB) Collecting mypy-protobuf==1.18 Using cached mypy_protobuf-1.18-py3-none-any.whl (7.3 kB) Collecting distlib==0.3.1 Using cached distlib-0.3.1-py2.py3-none-any.whl (335 kB) Collecting filelock>=3.0.0 Using cached filelock-3.6.0-py3-none-any.whl (10.0 kB) Collecting py>=1.4.17 Using cached py-1.11.0-py2.py3-none-any.whl (98 kB) Collecting virtualenv!=20.0.0,!=20.0.1,!=20.0.2,!=20.0.3,!=20.0.4,!=20.0.5,!=20.0.6,!=20.0.7,>=16.0.0 Using cached virtualenv-20.13.4-py2.py3-none-any.whl (8.7 MB) Collecting six>=1.14.0 Using cached six-1.16.0-py2.py3-none-any.whl (11 kB) Collecting toml>=0.9.4 Using cached toml-0.10.2-py2.py3-none-any.whl (16 kB) Collecting pluggy>=0.12.0 Using cached pluggy-1.0.0-py2.py3-none-any.whl (13 kB) Collecting packaging>=14 Using cached packaging-21.3-py3-none-any.whl (40 kB) Collecting grpcio>=1.37.0 Using cached grpcio-1.44.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.3 MB) Collecting protobuf<4.0dev,>=3.5.0.post1 Using cached protobuf-3.19.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.1 MB) Collecting pyparsing!=3.0.5,>=2.0.2 Using cached pyparsing-3.0.7-py3-none-any.whl (98 kB) Collecting platformdirs<3,>=2 Using cached platformdirs-2.5.1-py3-none-any.whl (14 kB) Installing collected packages: distlib, wheel, toml, six, setuptools, pyparsing, py, protobuf, pluggy, platformdirs, filelock, virtualenv, packaging, mypy-protobuf, grpcio, tox, grpcio-tools Attempting uninstall: setuptools Found existing installation: setuptools 44.0.0 Uninstalling setuptools-44.0.0: Successfully uninstalled setuptools-44.0.0 Successfully installed distlib-0.3.1 filelock-3.6.0 grpcio-1.44.0 grpcio-tools-1.37.0 mypy-protobuf-1.18 packaging-21.3 platformdirs-2.5.1 pluggy-1.0.0 protobuf-3.19.4 py-1.11.0 pyparsing-3.0.7 setuptools-60.10.0 six-1.16.0 toml-0.10.2 tox-3.20.1 virtualenv-20.13.4 wheel-0.37.1 > Task :beam-test-jenkins:generateMetricsReport py38-test create: <https://ci-beam.apache.org/job/beam_Metrics_Report/ws/src/.test-infra/jenkins/metrics_report/.tox/py38-test> py38-test installdeps: -rrequirements.txt py38-test installed: certifi==2021.10.8,charset-normalizer==2.0.12,idna==3.3,influxdb==5.3.0,Jinja2==2.11.3,MarkupSafe==2.1.1,msgpack==0.6.1,prettytable==0.7.2,python-dateutil==2.8.2,pytz==2022.1,requests==2.27.1,six==1.16.0,urllib3==1.26.9 py38-test run-test-pre: PYTHONHASHSEED='304201172' py38-test run-test-pre: commands[0] | python --version Python 3.8.10 py38-test run-test-pre: commands[1] | pip --version pip 22.0.4 from <https://ci-beam.apache.org/job/beam_Metrics_Report/ws/src/.test-infra/jenkins/metrics_report/.tox/py38-test/lib/python3.8/site-packages/pip> (python 3.8) py38-test run-test-pre: commands[2] | pip check No broken requirements found. py38-test run-test: commands[0] | python -m unittest dashboards_parser.py ... ---------------------------------------------------------------------- Ran 3 tests in 0.001s OK ___________________________________ summary ____________________________________ py38-test: commands succeeded congratulations :) py38-generate-report create: <https://ci-beam.apache.org/job/beam_Metrics_Report/ws/src/.test-infra/jenkins/metrics_report/.tox/py38-generate-report> py38-generate-report installdeps: -rrequirements.txt py38-generate-report installed: certifi==2021.10.8,charset-normalizer==2.0.12,idna==3.3,influxdb==5.3.0,Jinja2==2.11.3,MarkupSafe==2.1.1,msgpack==0.6.1,prettytable==0.7.2,python-dateutil==2.8.2,pytz==2022.1,requests==2.27.1,six==1.16.0,urllib3==1.26.9 py38-generate-report run-test-pre: PYTHONHASHSEED='477466327' py38-generate-report run-test-pre: commands[0] | python --version Python 3.8.10 py38-generate-report run-test-pre: commands[1] | pip --version pip 22.0.4 from <https://ci-beam.apache.org/job/beam_Metrics_Report/ws/src/.test-infra/jenkins/metrics_report/.tox/py38-generate-report/lib/python3.8/site-packages/pip> (python 3.8) py38-generate-report run-test-pre: commands[2] | pip check No broken requirements found. py38-generate-report run-test: commands[0] | python report_generator.py --influx-db=beam_test_metrics --influx-host=10.128.0.96 --influx-port=8086 --output-file=<https://ci-beam.apache.org/job/beam_Metrics_Report/81/artifact/src/.test-infra/jenkins/metrics_report/beam-metrics_report.html> Traceback (most recent call last): File "report_generator.py", line 23, in <module> import jinja2 File "<https://ci-beam.apache.org/job/beam_Metrics_Report/ws/src/.test-infra/jenkins/metrics_report/.tox/py38-generate-report/lib/python3.8/site-packages/jinja2/__init__.py";,> line 12, in <module> from .environment import Environment File "<https://ci-beam.apache.org/job/beam_Metrics_Report/ws/src/.test-infra/jenkins/metrics_report/.tox/py38-generate-report/lib/python3.8/site-packages/jinja2/environment.py";,> line 25, in <module> from .defaults import BLOCK_END_STRING File "<https://ci-beam.apache.org/job/beam_Metrics_Report/ws/src/.test-infra/jenkins/metrics_report/.tox/py38-generate-report/lib/python3.8/site-packages/jinja2/defaults.py";,> line 3, in <module> from .filters import FILTERS as DEFAULT_FILTERS # noqa: F401 File "<https://ci-beam.apache.org/job/beam_Metrics_Report/ws/src/.test-infra/jenkins/metrics_report/.tox/py38-generate-report/lib/python3.8/site-packages/jinja2/filters.py";,> line 13, in <module> from markupsafe import soft_unicode ImportError: cannot import name 'soft_unicode' from 'markupsafe' (<https://ci-beam.apache.org/job/beam_Metrics_Report/ws/src/.test-infra/jenkins/metrics_report/.tox/py38-generate-report/lib/python3.8/site-packages/markupsafe/__init__.py)> ERROR: InvocationError for command <https://ci-beam.apache.org/job/beam_Metrics_Report/ws/src/.test-infra/jenkins/metrics_report/.tox/py38-generate-report/bin/python> report_generator.py --influx-db=beam_test_metrics --influx-host=10.128.0.96 --influx-port=8086 --output-file=<https://ci-beam.apache.org/job/beam_Metrics_Report/81/artifact/src/.test-infra/jenkins/metrics_report/beam-metrics_report.html> (exited with code 1) ___________________________________ summary ____________________________________ ERROR: py38-generate-report: commands failed > Task :beam-test-jenkins:generateMetricsReport FAILED FAILURE: Build failed with an exception. * Where: Build file '<https://ci-beam.apache.org/job/beam_Metrics_Report/ws/src/.test-infra/jenkins/build.gradle'> line: 44 * What went wrong: Execution failed for task ':beam-test-jenkins:generateMetricsReport'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. > Run with --scan to get full insights. * Get more help at https://help.gradle.org BUILD FAILED in 56s 12 actionable tasks: 6 executed, 4 from cache, 2 up-to-date Publishing build scan... https://gradle.com/s/poscnkdbklgva Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
