See <https://ci-beam.apache.org/job/beam_Metrics_Report/24/display/redirect?page=changes>
Changes: [David Morávek] Simplify LateDataDropping runner. [sychen] Integrate BigQuery sink file loads with GroupIntoBatches [nahian97] Changing query to sql to fix doc [samuelw] [BEAM-11729] Remove calls to Class.getSimpleName() for WindowTracing [heejong] [BEAM-11520] Stage extra PyPI dependencies with generated requirements [Robert Bradshaw] [BEAM-11723] Use packed name for PairWithVoid consolidation. [Robert Bradshaw] Only do pair-with-none packing when combiner packing is desired. [Robert Bradshaw] Make eliminate_common_key_with_none private. [shehzaad] [BEAM-10961] make mongo-java-driver dependency a named variable [randomstep] [BEAM-11595] bump jackson to 2.12.1 [randomstep] [BEAM-11503] bump rockdbjni to 6.15.2 [ramazan.yapparov] Renamed build.gradle to build.gradle.kts [ramazan.yapparov] Migrated build file to kotlin [ramazan.yapparov] Reverted extra change [noreply] [BEAM-11773] Add bhulette as an owner for pandas, pyarrow, numpy [Boyuan Zhang] Using LoadingCache instead of Map to cache BundleProcessor [noreply] [BEAM-9547] Produce better errors for some groupby() and set_index() [yifanmai] Remove translations.eliminate_common_key_with_none from DataflowRunner [noreply] [BEAM-11762] Upgrades Tensorflow, numpy, and grpcio dependencies [tysonjh] [BEAM-11377] Add retries to mvn command. [noreply] [BEAM-9918] Support Tagged external PTransforms (#13872) [zyichi] Explicitly set dataflow runner v2 test shuffle mode to appliance [noreply] [BEAM-11265] make sure sample text file gets copied into archetype [noreply] [BEAM-11324] Impose consistent ordering on partitionings used in [noreply] [BEAM-11659] Extract common payload serialization behavior for Row [randomstep] [BEAM-11168] bump powermock to 2.0.9 [noreply] [BEAM-11781]: Update kafka-clients version in samza-runner (#13935) [ramazan.yapparov] Removed kotlin-dsl plugin from buildSrc [Kenneth Knowles] Maintain topological order for root transforms [kawaigin] [BEAM-11045] Updated screen diff goldens [daniel.o.programmer] Update programming guide: New python expansion service binary [Rui Wang] [BEAM-10925] Test Java UDF on columns [Kyle Weaver] [BEAM-11789] Upgrade gradle-dependency-analyze plugin. [tysonjh] [BEAM-11377] Disable mvn http keepalive. [noreply] [BEAM-11743] countif aggregate function implementation for zetasql [noreply] [BEAM-11708] Skip combinefn_lifecycle_test.test_combining_value_state [Pablo Estrada] [BEAM-11791] Continuously running microbenchmark for FnApiRunner [chamikaramj] Removes VR test [relax] update versions [relax] update autovalue version [chamikaramj] Fixes a checkstyle error in UdfTestProvider [noreply] [BEAM-11787] Do not call `translations.optimize_pipeline` in Dataflow [noreply] [BEAM-2914] Add portable merging window support to Python. (#12995) [noreply] [BEAM-11647] Fix go:goBuild gradle rules for build collision (#13958) [noreply] [BEAM-11611] Add transformation for computing approximate quantiles. [Kyle Weaver] [BEAM-10925] Don't publish udf-test-provider to Maven. [noreply] [BEAM-11780] Use vendored cloudbuild python client. (#13933) [Brian Hulette] Revert "update versions" [noreply] [BEAM-11804] Remove vendors/sdk-java-extensions-protobuf (#13968) [noreply] [BEAM-7372][BEAM-9372] cleanup python 2.x and 3.5 codepaths (#13913) [evgeny.belousov] [BEAM-11807] SDK Worker multithreading causes boto3 the KeyError [heejong] raise exception for non-file type artifacts [Brian Hulette] Revert "update autovalue version" [noreply] [BEAM-11476] Resolve flakiness (#13984) ------------------------------------------ Started by timer Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on apache-beam-jenkins-10 (beam) in workspace <https://ci-beam.apache.org/job/beam_Metrics_Report/ws/> No credentials specified Wiping out workspace first. Cloning the remote Git repository Cloning repository https://github.com/apache/beam.git > git init <https://ci-beam.apache.org/job/beam_Metrics_Report/ws/src> # > timeout=10 Fetching upstream changes from https://github.com/apache/beam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/apache/beam.git > +refs/heads/*:refs/remotes/origin/* # timeout=10 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # > timeout=10 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10 Fetching upstream changes from https://github.com/apache/beam.git > git fetch --tags --progress https://github.com/apache/beam.git > +refs/heads/*:refs/remotes/origin/* > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/* # > timeout=10 > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision e092703c9686a40abadfb61725523224be95c835 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f e092703c9686a40abadfb61725523224be95c835 # timeout=10 Commit message: "[BEAM-11805] Revert version updates from #13765 (#13972)" > git rev-list --no-walk b74fcf7b30d956fb42830d652a57b265a1546973 # timeout=10 No emails were triggered. No emails were triggered. [EnvInject] - Executing scripts and injecting environment variables after the SCM step. [EnvInject] - Injecting as environment variables the properties content SPARK_LOCAL_IP=127.0.0.1 SETUPTOOLS_USE_DISTUTILS=stdlib [EnvInject] - Variables injected successfully. [Gradle] - Launching build. [src] $ <https://ci-beam.apache.org/job/beam_Metrics_Report/ws/src/gradlew> --continue --max-****s=12 -Dorg.gradle.jvmargs=-Xms2g -Dorg.gradle.jvmargs=-Xmx4g -Pdocker-pull-licenses -PinfluxDb=beam_test_metrics -PinfluxHost=10.128.0.96 -PinfluxPort=8086 :beam-test-jenkins:generateMetricsReport Starting a Gradle Daemon, 1 busy Daemon could not be reused, use --status for details > Task :buildSrc:compileJava NO-SOURCE > Task :buildSrc:compileGroovy FROM-CACHE > Task :buildSrc:pluginDescriptors > Task :buildSrc:processResources > Task :buildSrc:classes > Task :buildSrc:jar > Task :buildSrc:assemble > Task :buildSrc:spotlessGroovy FROM-CACHE > Task :buildSrc:spotlessGroovyCheck UP-TO-DATE > Task :buildSrc:spotlessGroovyGradle FROM-CACHE > Task :buildSrc:spotlessGroovyGradleCheck UP-TO-DATE > Task :buildSrc:spotlessCheck UP-TO-DATE > Task :buildSrc:pluginUnderTestMetadata > Task :buildSrc:compileTestJava NO-SOURCE > Task :buildSrc:compileTestGroovy NO-SOURCE > Task :buildSrc:processTestResources NO-SOURCE > Task :buildSrc:testClasses UP-TO-DATE > Task :buildSrc:test NO-SOURCE > Task :buildSrc:validatePlugins FROM-CACHE > Task :buildSrc:check UP-TO-DATE > Task :buildSrc:build Configuration on demand is an incubating feature. > Task :beam-test-jenkins:setupVirtualenv Using base prefix '/usr' New python executable in <https://ci-beam.apache.org/job/beam_Metrics_Report/ws/src/build/gradleenv/-1871197473/bin/python3.8> Also creating executable in <https://ci-beam.apache.org/job/beam_Metrics_Report/ws/src/build/gradleenv/-1871197473/bin/python> Installing setuptools, pip, wheel... done. Running virtualenv with interpreter /usr/bin/python3.8 Collecting tox==3.20.1 Using cached tox-3.20.1-py2.py3-none-any.whl (83 kB) Collecting grpcio-tools==1.30.0 Using cached grpcio_tools-1.30.0-cp38-cp38-manylinux2010_x86_64.whl (2.5 MB) Collecting future==0.18.2 Using cached future-0.18.2-py3-none-any.whl Collecting mypy-protobuf==1.18 Using cached mypy_protobuf-1.18-py3-none-any.whl (7.3 kB) Collecting distlib==0.3.1 Using cached distlib-0.3.1-py2.py3-none-any.whl (335 kB) Collecting protobuf>=3.5.0.post1 Using cached protobuf-3.14.0-cp38-cp38-manylinux1_x86_64.whl (1.0 MB) Collecting grpcio>=1.30.0 Using cached grpcio-1.35.0-cp38-cp38-manylinux2014_x86_64.whl (4.1 MB) Collecting toml>=0.9.4 Using cached toml-0.10.2-py2.py3-none-any.whl (16 kB) Collecting pluggy>=0.12.0 Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB) Collecting filelock>=3.0.0 Using cached filelock-3.0.12-py3-none-any.whl (7.6 kB) Collecting py>=1.4.17 Using cached py-1.10.0-py2.py3-none-any.whl (97 kB) Collecting packaging>=14 Using cached packaging-20.9-py2.py3-none-any.whl (40 kB) Collecting virtualenv!=20.0.0,!=20.0.1,!=20.0.2,!=20.0.3,!=20.0.4,!=20.0.5,!=20.0.6,!=20.0.7,>=16.0.0 Using cached virtualenv-20.4.2-py2.py3-none-any.whl (7.2 MB) Collecting six>=1.14.0 Using cached six-1.15.0-py2.py3-none-any.whl (10 kB) Collecting pyparsing>=2.0.2 Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB) Collecting appdirs<2,>=1.4.3 Using cached appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB) Installing collected packages: six, pyparsing, filelock, distlib, appdirs, virtualenv, toml, py, protobuf, pluggy, packaging, grpcio, tox, mypy-protobuf, grpcio-tools, future Successfully installed appdirs-1.4.4 distlib-0.3.1 filelock-3.0.12 future-0.18.2 grpcio-1.35.0 grpcio-tools-1.30.0 mypy-protobuf-1.18 packaging-20.9 pluggy-0.13.1 protobuf-3.14.0 py-1.10.0 pyparsing-2.4.7 six-1.15.0 toml-0.10.2 tox-3.20.1 virtualenv-20.4.2 > Task :beam-test-jenkins:generateMetricsReport py38-test create: <https://ci-beam.apache.org/job/beam_Metrics_Report/ws/src/.test-infra/jenkins/metrics_report/.tox/py38-test> py38-test installdeps: -rrequirements.txt py38-test installed: certifi==2020.12.5,chardet==4.0.0,idna==2.10,influxdb==5.3.0,Jinja2==2.11.2,MarkupSafe==1.1.1,msgpack==0.6.1,prettytable==0.7.2,python-dateutil==2.8.1,pytz==2021.1,requests==2.25.1,six==1.15.0,urllib3==1.26.3 py38-test run-test-pre: PYTHONHASHSEED='3404288497' py38-test run-test-pre: commands[0] | python --version Python 3.8.5 py38-test run-test-pre: commands[1] | pip --version pip 21.0.1 from <https://ci-beam.apache.org/job/beam_Metrics_Report/ws/src/.test-infra/jenkins/metrics_report/.tox/py38-test/lib/python3.8/site-packages/pip> (python 3.8) py38-test run-test-pre: commands[2] | pip check No broken requirements found. py38-test run-test: commands[0] | python -m unittest dashboards_parser.py ... ---------------------------------------------------------------------- Ran 3 tests in 0.001s OK ___________________________________ summary ____________________________________ py38-test: commands succeeded congratulations :) py38-generate-report create: <https://ci-beam.apache.org/job/beam_Metrics_Report/ws/src/.test-infra/jenkins/metrics_report/.tox/py38-generate-report> py38-generate-report installdeps: -rrequirements.txt py38-generate-report installed: certifi==2020.12.5,chardet==4.0.0,idna==2.10,influxdb==5.3.0,Jinja2==2.11.2,MarkupSafe==1.1.1,msgpack==0.6.1,prettytable==0.7.2,python-dateutil==2.8.1,pytz==2021.1,requests==2.25.1,six==1.15.0,urllib3==1.26.3 py38-generate-report run-test-pre: PYTHONHASHSEED='1473004561' py38-generate-report run-test-pre: commands[0] | python --version Python 3.8.5 py38-generate-report run-test-pre: commands[1] | pip --version pip 21.0.1 from <https://ci-beam.apache.org/job/beam_Metrics_Report/ws/src/.test-infra/jenkins/metrics_report/.tox/py38-generate-report/lib/python3.8/site-packages/pip> (python 3.8) py38-generate-report run-test-pre: commands[2] | pip check No broken requirements found. py38-generate-report run-test: commands[0] | python report_generator.py --influx-db=beam_test_metrics --influx-host=10.128.0.96 --influx-port=8086 --output-file=<https://ci-beam.apache.org/job/beam_Metrics_Report/24/artifact/src/.test-infra/jenkins/metrics_report/beam-metrics_report.html> Traceback (most recent call last): File "report_generator.py", line 227, in <module> main() File "report_generator.py", line 221, in main data = get_metrics_data(client, influx_db) File "report_generator.py", line 183, in get_metrics_data data.extend(_get_query_runtime_data(client, bind_params)) File "report_generator.py", line 121, in _get_query_runtime_data for dashboard in guess_dashboard_by_measurement( File "<https://ci-beam.apache.org/job/beam_Metrics_Report/ws/src/.test-infra/jenkins/metrics_report/dashboards_parser.py";,> line 77, in guess_dashboard_by_measurement dashboards = list(Dashboard.get_dashboards_from_directory(directory)) File "<https://ci-beam.apache.org/job/beam_Metrics_Report/ws/src/.test-infra/jenkins/metrics_report/dashboards_parser.py";,> line 67, in get_dashboards_from_directory yield cls(file) File "<https://ci-beam.apache.org/job/beam_Metrics_Report/ws/src/.test-infra/jenkins/metrics_report/dashboards_parser.py";,> line 25, in __init__ self.uid, self.queries = self.get_dashboard_uid_and_queries(file) File "<https://ci-beam.apache.org/job/beam_Metrics_Report/ws/src/.test-infra/jenkins/metrics_report/dashboards_parser.py";,> line 33, in get_dashboard_uid_and_queries data = json.load(f) File "/usr/lib/python3.8/json/__init__.py", line 293, in load return loads(fp.read(), File "/usr/lib/python3.8/json/__init__.py", line 357, in loads return _default_decoder.decode(s) File "/usr/lib/python3.8/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib/python3.8/json/decoder.py", line 353, in raw_decode obj, end = self.scan_once(s, idx) json.decoder.JSONDecodeError: Expecting ',' delimiter: line 142 column 3 (char 3182) ERROR: InvocationError for command <https://ci-beam.apache.org/job/beam_Metrics_Report/ws/src/.test-infra/jenkins/metrics_report/.tox/py38-generate-report/bin/python> report_generator.py --influx-db=beam_test_metrics --influx-host=10.128.0.96 --influx-port=8086 --output-file=<https://ci-beam.apache.org/job/beam_Metrics_Report/24/artifact/src/.test-infra/jenkins/metrics_report/beam-metrics_report.html> (exited with code 1) ___________________________________ summary ____________________________________ ERROR: py38-generate-report: commands failed > Task :beam-test-jenkins:generateMetricsReport FAILED FAILURE: Build failed with an exception. * Where: Build file '<https://ci-beam.apache.org/job/beam_Metrics_Report/ws/src/.test-infra/jenkins/build.gradle'> line: 43 * What went wrong: Execution failed for task ':beam-test-jenkins:generateMetricsReport'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/6.8/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 57s 2 actionable tasks: 2 executed Publishing build scan... https://gradle.com/s/t52xsl7uwrckq Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
