See <https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/177/display/redirect?page=changes>
Changes: [echauchot] Introduce MetricsPusher in runner core to regularly push aggregated [echauchot] Instanciate MetricsPusher in runner-specific code because we need [echauchot] Improve MetricsPusher: do not aggregate metrics when not needed, improve [echauchot] Create JsonMetricsSerializer [echauchot] Stop MetricsPusher thread by observing pipeline state and improve the [echauchot] Make metrics sink configurable through PipelineOptions, pass [echauchot] Add MetricsPusher tests specific to Spark (because Spark streaming tests [echauchot] Add a MetricksPusher test to runner-core (batch and streaming are done [echauchot] Push metrics at the end of a batch pipeline in flink runner [echauchot] improve MetricsPusher lifecycle and thread safety [echauchot] Make MetricsPusher merge a list a MetricsContainerStepMaps because there [echauchot] Fix thread synchronisation and replace usages of instance variable by [echauchot] Clear dummyMetricsSink before test [echauchot] Push metrics at the end of a batch pipeline in spark runner [echauchot] Improve MetricsPusher teardown to enable multiple pipelines in a single [echauchot] Manually generate json and remove jackson [echauchot] Replace use of http client by use of java.net.HttpUrlConnection and deal [echauchot] Remove DEFAULT_PERIOD constant in favor of already existing [echauchot] Remove unneeded null check, format [echauchot] convert MetricsSink to an interface with a single writeMetrics method [echauchot] Remove MetricsSerializer base class and inline serialization in [echauchot] Dynamically create the sinks by reflection [echauchot] Split DummyMetricsSink into NoOpMetricsSink (default sink) and [echauchot] Reduce overhead when no metricsSink is provided, do not start polling [echauchot] Make MetricsPusher a regular object instead of a singleton to allow [echauchot] Explicitely start MetricsPusher from the runners [echauchot] Separate MetricsHttpSink POC to a new runners-extensions artifact and [echauchot] Fix cycle bug between teardown() and pushmetrics() [echauchot] Update MetricsPusher and TestMetricsSink to new serializable [echauchot] Use regular jackson object mapper to serialize metrics now that they are [echauchot] Give MetricsPusher a bit of time to push before assert in test [echauchot] Make MetricsPusher thread a daemon [echauchot] Fix build and clean: dependencies, rat, checkstyle, findbugs, remove [echauchot] Move build to gradle [echauchot] MetricsSink no more needs to be generic [echauchot] SparkRunnerDebugger does not need to export metrics as it does not run ------------------------------------------ GitHub pull request #4548 of commit 496f9a7b6710e04df33f0c48862b5d4b66b8871c, no merge conflicts. Setting status of 496f9a7b6710e04df33f0c48862b5d4b66b8871c to PENDING with url https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/177/ and message: 'Build started sha1 is merged.' Using context: Jenkins: Apache Flink Runner ValidatesRunner Tests [EnvInject] - Loading node environment variables. Building remotely on beam1 (beam) in workspace <https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/ws/> > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/apache/beam.git # timeout=10 Fetching upstream changes from https://github.com/apache/beam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/apache/beam.git > +refs/heads/*:refs/remotes/origin/* > +refs/pull/4548/*:refs/remotes/origin/pr/4548/* > git rev-parse refs/remotes/origin/pr/4548/merge^{commit} # timeout=10 > git rev-parse refs/remotes/origin/origin/pr/4548/merge^{commit} # timeout=10 Checking out Revision c5944d46961969154bb2df1582bb65da69cf5abb (refs/remotes/origin/pr/4548/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f c5944d46961969154bb2df1582bb65da69cf5abb Commit message: "Merge 496f9a7b6710e04df33f0c48862b5d4b66b8871c into 3eacf8b8d7301d1bd4db17874a31caf3b9949526" > git rev-list --no-walk 251e9eac1ddead8c1fcf1cc198de488e4f04e130 # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 [EnvInject] - Executing scripts and injecting environment variables after the SCM step. [EnvInject] - Injecting as environment variables the properties content SPARK_LOCAL_IP=127.0.0.1 [EnvInject] - Variables injected successfully. [Gradle] - Launching build. [src] $ <https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/ws/src/gradlew> --info --continue --max-workers=2 :beam-runners-flink_2.11:validatesRunner Initialized native services in: /home/jenkins/.gradle/native Using 2 worker leases. Starting Build Invalidating in-memory cache of /home/jenkins/.gradle/caches/4.6/fileHashes/fileHashes.bin Invalidating in-memory cache of /home/jenkins/.gradle/caches/4.6/fileHashes/resourceHashesCache.bin Settings evaluated using settings file '<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/ws/src/settings.gradle'.> Using local directory build cache for the root build (location = /home/jenkins/.gradle/caches/build-cache-1, removeUnusedEntriesAfter = 7 days). Projects loaded. Root project using build file '<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/ws/src/build.gradle'.> Included projects: [root project 'beam', project ':beam-examples-java', project ':beam-local-artifact-service-java', project ':beam-model-fn-execution', project ':beam-model-job-management', project ':beam-model-pipeline', project ':beam-runners-apex', project ':beam-runners-core-construction-java', project ':beam-runners-core-java', project ':beam-runners-direct-java', project ':beam-runners-extensions-java', project ':beam-runners-flink_2.11', project ':beam-runners-gcp-gcemd', project ':beam-runners-gcp-gcsproxy', project ':beam-runners-gearpump', project ':beam-runners-google-cloud-dataflow-java', project ':beam-runners-java-fn-execution', project ':beam-runners-local-java-core', project ':beam-runners-reference-java', project ':beam-runners-reference-job-server', project ':beam-runners-spark', project ':beam-sdks-go', project ':beam-sdks-go-container', project ':beam-sdks-go-examples', project ':beam-sdks-java-build-tools', project ':beam-sdks-java-container', project ':beam-sdks-java-core', project ':beam-sdks-java-extensions-google-cloud-platform-core', project ':beam-sdks-java-extensions-join-library', project ':beam-sdks-java-extensions-json-jackson', project ':beam-sdks-java-extensions-protobuf', project ':beam-sdks-java-extensions-sketching', project ':beam-sdks-java-extensions-sorter', project ':beam-sdks-java-extensions-sql', project ':beam-sdks-java-fn-execution', project ':beam-sdks-java-harness', project ':beam-sdks-java-io-amazon-web-services', project ':beam-sdks-java-io-amqp', project ':beam-sdks-java-io-cassandra', project ':beam-sdks-java-io-common', project ':beam-sdks-java-io-elasticsearch', project ':beam-sdks-java-io-elasticsearch-tests-2', project ':beam-sdks-java-io-elasticsearch-tests-5', project ':beam-sdks-java-io-elasticsearch-tests-common', project ':beam-sdks-java-io-file-based-io-tests', project ':beam-sdks-java-io-google-cloud-platform', project ':beam-sdks-java-io-hadoop-common', project ':beam-sdks-java-io-hadoop-file-system', project ':beam-sdks-java-io-hadoop-input-format', project ':beam-sdks-java-io-hbase', project ':beam-sdks-java-io-hcatalog', project ':beam-sdks-java-io-jdbc', project ':beam-sdks-java-io-jms', project ':beam-sdks-java-io-kafka', project ':beam-sdks-java-io-kinesis', project ':beam-sdks-java-io-mongodb', project ':beam-sdks-java-io-mqtt', project ':beam-sdks-java-io-redis', project ':beam-sdks-java-io-solr', project ':beam-sdks-java-io-tika', project ':beam-sdks-java-io-xml', project ':beam-sdks-java-javadoc', project ':beam-sdks-java-maven-archetypes-examples', project ':beam-sdks-java-maven-archetypes-starter', project ':beam-sdks-java-nexmark', project ':beam-sdks-python', project ':beam-sdks-python-container', project ':release'] Parallel execution with configuration on demand is an incubating feature. Evaluating root project 'beam' using build file '<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/ws/src/build.gradle'.> Invalidating in-memory cache of /home/jenkins/.gradle/caches/4.6/fileHashes/fileHashes.bin Invalidating in-memory cache of /home/jenkins/.gradle/caches/4.6/fileHashes/resourceHashesCache.bin Applying build_rules.gradle to beam Offline dependencies root configured at 'offline-repository' createPerformanceTestHarness with default configuration for project beam Adding 48 .gitignore exclusions to Apache Rat Evaluating project ':beam-runners-flink_2.11' using build file '<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/ws/src/runners/flink/build.gradle'.> Applying build_rules.gradle to beam-runners-flink_2.11 Offline dependencies root configured at 'offline-repository' applyJavaNature with default configuration for project beam-runners-flink_2.11 Evaluating project ':beam-sdks-java-core' using build file '<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/ws/src/sdks/java/core/build.gradle'.> Applying build_rules.gradle to beam-sdks-java-core Offline dependencies root configured at 'offline-repository' applyJavaNature with [shadowClosure:org.codehaus.groovy.runtime.ComposedClosure@a99c42c] for project beam-sdks-java-core applyAvroNature with default configuration for project beam-sdks-java-core FAILURE: Build failed with an exception. * Where: Build file '<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/ws/src/runners/flink/build.gradle'> line: 97 * What went wrong: A problem occurred evaluating project ':beam-runners-flink_2.11'. > Could not get unknown property 'sourceSets' for project > ':beam-runners-core-java' of type org.gradle.api.Project. * Try: Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org BUILD FAILED in 19s Publishing build scan... https://gradle.com/s/mcwzxe2jwhf7c Build cache (/home/jenkins/.gradle/caches/build-cache-1) has not been cleaned up in 0 days Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure Recording test results ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error? Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected]
