See
<https://builds.apache.org/job/beam_PostCommit_Python_ValidatesContainer_Dataflow/14/display/redirect?page=changes>
Changes:
[joey.baruch] Add javadoc to ConsoleIO
[zoy] Increasing BatchElements's max_batch_size to 10K
[wcn] Update generated version of Fn API code.
[Pablo] Tracking of time spent reading side inputs, and bytes read in Dataflow.
[Pablo] Fix lint issues
[Pablo] Fixing counter names
[Pablo] Using comments to improve changes
[Pablo] Placing feature behind an experiment flag.
[Pablo] Fixing lint issue
[Pablo] Rebasing
[Pablo] Added cythonization and test
[Pablo] Fixing lint issue
[Pablo] Modifying the experiment flag check to reduce performance impact.
[Pablo] Addressing comments. Profile pending.
[Pablo] Reducing impact - Tracking only Bytes if experiment is active.
[ekirpichov] Introduces the Wait transform
[Pablo] Addressing comments
[Pablo] Improved IO Target documentation
[Pablo] Improving documentation
[Pablo] Addressing comments.
[tgroh] Retrieve Environments from PTransforms
[kirpichov] [BEAM-3083] Do not call getSchema() and getTable() on every element
[altay] Use non-deprecated version of Futures.transform
[aromanenko.dev] [BEAM-3539] BigtableIO.Write javadoc of some methods is
incorrect
[iemejia] Remove unneeded profile for javadoc on Java 8
[iemejia] Remove unneeded explicit Java 8 references on maven-compiler-plugin
[iemejia] Fix doc error on hadoop-input-format ITs after move to Java 8 only
tests
[iemejia] Remove references to non-existent examples:java8 module in gradle
[iemejia] Remove references to java 7/8 only examples from the README
[iemejia] Remove some comments on Java 7/8 only stuff that don't make sense
[kenn] Add a test for an event time timer loop in ParDo
[lcwik] [BEAM-2273] Cleanup examples Maven Archetype to copy in a clean state
[XuMingmin] [BEAM-3526] KakfaIO support for finalizeCheckpoint() (#4481)
[kedin] Add Avro dependency to KafkaIO
[rangadi] Remove an unused test dependency in KafkaIO.
[lcwik] [Beam-2500] Add S3FileSystem to SDKs/Java/IO
[lcwik] implement serializing AWS credentials provider
[lcwik] fixup! Clarify error message is received from SDK harness
[iemejia] Fix modules that were activated only on Java 8 profile
[iemejia] [BEAM-3275] Fix ValidatesRunner Spark runner after the Kafka update
[iemejia] Refactor code into idiomatic Java 8 style
[iemejia] Fix missing gearpump module activated only on Java 8 profile
[iemejia] Add missing amazon-web-services module from javadoc
[jbonofre] [BEAM-2271] Exclude files not intended to be in the releases source
[jbonofre] Typo fix in MinimalWordCount example
[Mottyg1] [BEAM-675] Introduce message mapper in JmsIO
[iemejia] Update maven-shade-plugin version to 3.1.0
[iemejia] Update maven-compiler-plugin version to 3.7.0
[iemejia] Update maven-dependency-plugin version to 3.0.2
[iemejia] Update maven-surefire-plugin version to 2.20.1
[iemejia] Update maven-failsafe-plugin version to 2.20.1
[iemejia] Update maven-assembly-plugin version to 3.1.0
[iemejia] Update versions-maven-plugin version to 2.5
[iemejia] Update findbugs-maven-plugin version to 3.0.5
[iemejia] Update license-maven-plugin version to 1.14
[iemejia] Update jacoco-maven-plugin version to 0.7.9
[iemejia] Update dockerfile-maven-plugin version to 1.3.6
[iemejia] Add maven-enforcer-plugin version to pluginManagement
[iemejia] Fix warning on using directly parent.version on Nexmark
[iemejia] Remove warnings on repeated maven-jar-plugin invocation / deps
[iemejia] Remove warning on defining <prerequisites> for non-maven plugin
project
[iemejia] Update parent POM to version 19
[iemejia] Remove repeated dependency in hadoop-input-format module
[mariagh] Support argparse-style choices for ValueProvider
[jbonofre] [BEAM-3551] Add -parameters flag to the compiler
[jbonofre] [BEAM-3551] Align gradle java compile task options with the
[coheigea] BEAM-3560 - Switch to use BigInteger/BigDecimal.ZERO/ONE/TEN
[lcwik] [BEAM-2500, BEAM-3249] Add amazon-web-services gradle build rules.
[jbonofre] [maven-release-plugin] prepare branch release-2.3.0
[jbonofre] [maven-release-plugin] prepare for next development iteration
[jbonofre] Update Python SDK version post release
[iemejia] Fix dependency order error, harness must be built before container
[robertwb] [BEAM-3537] Allow more general eager in-process pipeline execution
[lcwik] [BEAM-3550] Add --awsServiceEndpoint option and use with S3 filesystem.
[lcwik] Fixes for sdks/java/core for the eclipse compiler
[lcwik] Add some m2e lifecycle things to get the various maven plugins running
[lcwik] Set the proper package for the Snippets example so eclipse won't raise
[lcwik] Add some template args and direct casts to help the eclipse compiler
[lcwik] Incorporate reviews and rebase on latest master
[lcwik] Return no environment more often
[Pablo] Logging deviation from sampling expectation. This will allow to track
[kedin] [SQL] Refactor Variance
[robertwb] [BEAM-3490] Make runtime type checking code runner agnostic.
[kedin] [Nexmark][SQL] Implement sql query 3
[github] [BEAM-3557] Sets parent pointer of AppliedPTransform objects correctly
[pawel.pk.kaczmarczyk] [BEAM-2469] Handling Kinesis shards splits and merges
[Pablo] Adding a static getter for RuntimeValueProvider.
[tgroh] Add CoderTranslatorRegistrar
[tgroh] Add slf4j_simple to the top level Gradle build
[tgroh] Implement FnService in FnApiControlClientPoolService
[tgroh] Add a Timeout to GrpcDataService#send
[tgroh] Use a Data Service in SdkHarnessClient
[github] get the query from configuration not options
[XuMingmin] [BEAM-3525] Fix KafkaIO metric (#4524)
[chamikara] Updates PTransform overriding to create a new AppliedPTransform
object
[dariusz.aniszewski] use build $WORKSPACE as pkb temp_dir and update pip and
setuptools in
[iemejia] [BEAM-3578] SQL module build breaks because of missing dependency
[Pablo] Renaming the ZERO element of DistributionResult to be IDENTITY_ELEMENT.
[kenn] google-java-format
[kenn] Fix Distinct null pointer error with speculative triggers
[kenn] Move TestCountingSource to appropriate location
[robertwb] Direct runner fixes.
[lcwik] [BEAM-2926] Add support for side inputs to the runner harness.
[kenn] Sickbay ApexRunner gradle WordCountIT
[kenn] Sickbay flakey KinesisReaderTest
[Pablo] Addressing comments.
[ehudm] Split out buffered read and write code from gcsio.
[github] Removing unnecessary code.
[lcwik] [BEAM-3249] Make sure that all java projects package tests. Also package
[lcwik] [BEAM-3249] Do not assume build directory is within build/, use the
[github] Fix undefined names: exc_info --> self.exc_info
[github] import logging for line 1163
[iemejia] [BEAM-3592] Fix spark-runner profile for Nexmark after move to Spark
2.x
[dkulp] [BEAM-3562] Update to Checkstyle 8.7
[klk] Encourage a good description in a good spot on a PR description.
[lcwik] Change info to debug statement
[cclauss] global INT64_MAX, INT64_MIN to placate linters
[tgroh] Add QueryablePipeline
[gene] Changing FileNaming to public to allow for usage in lambdas/inheritance
[robertwb] [BEAM-3207] Create a standard location to enumerate and document
URNs.
[robertwb] Reduce the flakiness of the state sampler progress metrics.
[robertwb] Revert URNs that are currently hard-coded in the Dataflow worker.
[kedin] [SQL] Inherit windowing strategy from the input in Aggregate operation
[jbonofre] [BEAM-3551] Define compiler -parameters flag in the default options
[tgroh] Add SdkHarnessClientControlService
[tgroh] Update Synchronization in FnApiControlClient
[coheigea] BEAM-3593 - Remove methods that just call super()
[lcwik] Move off of deprecated method in Guava.
[tgroh] Add a LocalArtifactStagingLocation
[tgroh] Add LocalArtifactStagingLocation#forExisting
[tgroh] Add an ArtifactRetrievalService interface
[tgroh] Implement a Local ArtifactRetrievalService
[chamikara] Adds a ReadAll transform to tfrecordio.
[rangadi] KafkaIO : move source and sink implemenations into own files.
[rangadi] minor
[kedin] [SQL] Add SqlTypeCoder, replace java.sql.Types
[Pablo] Moving User metrics to be in the PTransform proto for Fn API.
[mairbek] Update cloud spanner library to 0.29.0
[mairbek] Fix test
[mairbek] More google-cloud-platform whitelisting
[mairbek] pom updates to make maven happy
[mairbek] Update netty deps
[ccy] [BEAM-3566] Replace apply_* hooks in DirectRunner with
[ccy] Address reviewer comments
[klk] google-java-format
[klk] Fix empty window assignments in Nexmark
[klk] Fix empty window assignment in FlattenEvaluatorFactoryTest
[klk] Switch DataflowRunner to its own private ValueInEmptyWindows
[klk] Remove deprecated valueInEmptyWindows
[jiangkai] Covariance Functions
[aljoscha.krettek] Remove erroneous cast in FlinkStreamingTransformTranslators
[aljoscha.krettek] [BEAM-3186] Correctly use deserialized timerService in Flink
Runner
[lcwik] Adjust gradle build dirs and hints to help IntelliJ (#4583)
[coheigea] BEAM-3618 - Remove extraneous "return" statement
[robertwb] [BEAM-3183] Allow a callable as input to runner.run().
[sidhom] Fix gradle java sdk image build
[kenn] Add MoreFutures utility
[kenn] Switch runners/java-fn-execution from Guava futures to Java 8 futures
[kenn] Switch DataflowRunner from Guava futures to Java 8 futures
[kenn] Switch gcp-core from Guava futures to Java 8 futures
[kenn] Switch runners/core-construction-java from Guava futures to Java 8
[kenn] Switch AWS IO from Guava futures to Java 8 futures
[kenn] Switch BigTableIO from Guava futures to Java 8 futures
[kenn] Deprecate DoFnTester
[mairbek] Fixed broken test
[ekirpichov] Adds PositionT and claim callback to RestrictionTracker
[ekirpichov] Changes OutputAndTimeBounded invoker to start checkpoint timer
after
[ekirpichov] Compresses encoded GrowthState with Snappy - about 2x-3x more
compact
[ekirpichov] InMemoryStateInternals.copy clones the values using the coder
[ekirpichov] Final fixups
[ekirpichov] Bump worker to 20180205
[klk] Sickbay flaky KinesisIO tests
[klk] Remove DoFnTester from core SDK tests
[aljoscha.krettek] [BEAM-2806] Fix pipeline translation mode recognition in
Flink Runner
[github] Consistantly show Python and pip versions in tests
[jbonofre] Revert "Reinstate proto round trip in Java DirectRunner"
[tgroh] Update Assign Window URN Constant Name
[coheigea] BEAM-3624 - Remove collapsible if statements
[kenn] Sickbay ApexRunner ParDoTranslatorTest.testAssertionFailure
[kenn] Switch FullWindowedValueCoder to bypass validation
[kedin] Refactor BeamRecordType and BeamRecord
[sidhom] Allow docker tag root to be specified as in Maven image build
[herohde] [BEAM-3457] Upgrade gogradle and fix thrift resolution issue
[herohde] [BEAM-3457] Add Go Gradle precommit
[ccy] [BEAM-3635] Infer type hints on PTransformOverrides
[robertwb] [BEAM-3625] Enable DoFn params in Map, Filter, etc.
[kedin] Rename BeamRecord -> Row, BeamRecordType -> RowType
[iemejia] Add missing gradle build config for sdks/java/extensions/sketching
[iemejia] Fix type on shadowTest when it should be testShadow
[ccy] Update snippets to fix pickling and clarify encoding issues
[wcn] Modify BufferedElementCountingOutputStream to use a buffer pool for its
[klk] Fix stable name errors in HBaseIOTest
[aromanenko.dev] [BEAM-3291] Add Kinesis write transform
[coheigea] Remove unused private variables.
[ehudm] Add and migrate to HdfsCLI library for Python SDK.
[klk] Fix typo in gradle idea hints
[dkulp] [BEAM-3639] Update to gradle 4.5.1
[github] Revert "Update cloud spanner library to 0.29.0"
[kedin] [Schema Generation] Generate BeamRecordTypes based on pojos.
[alan] [BEAM-3524] Automate testing using python sdk container built at head
[apilloud] [BEAM-410] Sort PriorityQueue<QuantileBuffer> with explicit
comparator
[robertwb] Disable verbose typecoder warnings.
[kenn] Increase gradle logging to --info
[ccy] Add switchable DirectRunner which uses the fast FnApiRunner when
------------------------------------------
[...truncated 95.06 KB...]
copying apache_beam/runners/dataflow/internal/names.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/dataflow/internal
copying apache_beam/runners/dataflow/internal/clients/__init__.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/dataflow/internal/clients
copying apache_beam/runners/dataflow/internal/clients/dataflow/__init__.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/dataflow/internal/clients/dataflow
copying
apache_beam/runners/dataflow/internal/clients/dataflow/dataflow_v1b3_client.py
-> apache-beam-2.4.0.dev0/apache_beam/runners/dataflow/internal/clients/dataflow
copying
apache_beam/runners/dataflow/internal/clients/dataflow/dataflow_v1b3_messages.py
->
apache-beam-2.4.0.dev0/apache_beam/runners/dataflow/internal/clients/dataflow
copying
apache_beam/runners/dataflow/internal/clients/dataflow/message_matchers.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/dataflow/internal/clients/dataflow
copying
apache_beam/runners/dataflow/internal/clients/dataflow/message_matchers_test.py
-> apache-beam-2.4.0.dev0/apache_beam/runners/dataflow/internal/clients/dataflow
copying apache_beam/runners/dataflow/native_io/__init__.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/dataflow/native_io
copying apache_beam/runners/dataflow/native_io/iobase.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/dataflow/native_io
copying apache_beam/runners/dataflow/native_io/iobase_test.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/dataflow/native_io
copying apache_beam/runners/dataflow/native_io/streaming_create.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/dataflow/native_io
copying apache_beam/runners/direct/__init__.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/bundle_factory.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/clock.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/consumer_tracking_pipeline_visitor.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/consumer_tracking_pipeline_visitor_test.py
-> apache-beam-2.4.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/direct_metrics.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/direct_metrics_test.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/direct_runner.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/direct_runner_test.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/evaluation_context.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/executor.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/helper_transforms.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/sdf_direct_runner.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/sdf_direct_runner_test.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/transform_evaluator.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/util.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/watermark_manager.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/experimental/__init__.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/experimental
copying apache_beam/runners/experimental/python_rpc_direct/__init__.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/experimental/python_rpc_direct
copying
apache_beam/runners/experimental/python_rpc_direct/python_rpc_direct_runner.py
-> apache-beam-2.4.0.dev0/apache_beam/runners/experimental/python_rpc_direct
copying apache_beam/runners/experimental/python_rpc_direct/server.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/experimental/python_rpc_direct
copying apache_beam/runners/job/__init__.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/job
copying apache_beam/runners/job/manager.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/job
copying apache_beam/runners/job/utils.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/job
copying apache_beam/runners/portability/__init__.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/fn_api_runner.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/fn_api_runner_test.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/maptask_executor_runner.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/maptask_executor_runner_test.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/universal_local_runner.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/universal_local_runner_main.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/universal_local_runner_test.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/test/__init__.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/test
copying apache_beam/runners/worker/__init__.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/bundle_processor.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/data_plane.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/data_plane_test.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/log_handler.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/log_handler_test.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/logger.pxd ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/logger.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/logger_test.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/opcounters.pxd ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/opcounters.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/opcounters_test.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/operation_specs.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/operations.pxd ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/operations.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/sdk_worker.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/sdk_worker_main.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/sdk_worker_main_test.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/sdk_worker_test.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/sideinputs.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/sideinputs_test.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/statesampler.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/statesampler_fast.pyx ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/statesampler_slow.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/statesampler_test.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/testing/__init__.py ->
apache-beam-2.4.0.dev0/apache_beam/testing
copying apache_beam/testing/pipeline_verifiers.py ->
apache-beam-2.4.0.dev0/apache_beam/testing
copying apache_beam/testing/pipeline_verifiers_test.py ->
apache-beam-2.4.0.dev0/apache_beam/testing
copying apache_beam/testing/test_pipeline.py ->
apache-beam-2.4.0.dev0/apache_beam/testing
copying apache_beam/testing/test_pipeline_test.py ->
apache-beam-2.4.0.dev0/apache_beam/testing
copying apache_beam/testing/test_stream.py ->
apache-beam-2.4.0.dev0/apache_beam/testing
copying apache_beam/testing/test_stream_test.py ->
apache-beam-2.4.0.dev0/apache_beam/testing
copying apache_beam/testing/test_utils.py ->
apache-beam-2.4.0.dev0/apache_beam/testing
copying apache_beam/testing/test_utils_test.py ->
apache-beam-2.4.0.dev0/apache_beam/testing
copying apache_beam/testing/util.py ->
apache-beam-2.4.0.dev0/apache_beam/testing
copying apache_beam/testing/util_test.py ->
apache-beam-2.4.0.dev0/apache_beam/testing
copying apache_beam/testing/data/standard_coders.yaml ->
apache-beam-2.4.0.dev0/apache_beam/testing/data
copying apache_beam/testing/data/trigger_transcripts.yaml ->
apache-beam-2.4.0.dev0/apache_beam/testing/data
copying apache_beam/transforms/__init__.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py ->
apache-beam-2.4.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py ->
apache-beam-2.4.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py ->
apache-beam-2.4.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py ->
apache-beam-2.4.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py ->
apache-beam-2.4.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py ->
apache-beam-2.4.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py ->
apache-beam-2.4.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py ->
apache-beam-2.4.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py ->
apache-beam-2.4.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py ->
apache-beam-2.4.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py ->
apache-beam-2.4.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py ->
apache-beam-2.4.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py ->
apache-beam-2.4.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py ->
apache-beam-2.4.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd ->
apache-beam-2.4.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py ->
apache-beam-2.4.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py ->
apache-beam-2.4.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.4.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py ->
apache-beam-2.4.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py ->
apache-beam-2.4.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py ->
apache-beam-2.4.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py ->
apache-beam-2.4.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.4.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py ->
apache-beam-2.4.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py ->
apache-beam-2.4.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py ->
apache-beam-2.4.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.4.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd ->
apache-beam-2.4.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py ->
apache-beam-2.4.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py ->
apache-beam-2.4.0.dev0/apache_beam/utils
Writing apache-beam-2.4.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.4.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nocapture \
--processes=1 \
--process-timeout=900 \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1"
<https://builds.apache.org/job/beam_PostCommit_Python_ValidatesContainer_Dataflow/ws/src/sdks/python/container/local/lib/python2.7/site-packages/setuptools/dist.py>:355:
UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Python_ValidatesContainer_Dataflow/ws/src/sdks/python/apache_beam/io/gcp/gcsio.py>:166:
DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT)
... ok
----------------------------------------------------------------------
Ran 1 test in 409.506s
OK
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG
Untagged: us.gcr.io/apache-beam-testing/jenkins/python:20180209-225422
Untagged:
us.gcr.io/apache-beam-testing/jenkins/python@sha256:476fc0b8f7ce4f921b4f58b231be7dcd68dc95c1ffaeedb6dc76da3fc24da445
Deleted: sha256:1da514b23b50ded2309578bcffc5a7a5d638d5688fdd74e6c4121f11f59524cb
Deleted: sha256:151ccc635d38c07bb9255eb72b3ebaa29fccd4060463e85d19724c9c03cc4581
Deleted: sha256:4516f57595e9725c46eb6020e86bcc36751354d276196246ebbd198bf39517a5
Deleted: sha256:215e9fff65f1f9975d25db737bc2625ce893c73f5fa6acd9cc9cfe483751a420
Deleted: sha256:b85ca9bc2c0e5bf610e96f5e8cdffb833309e68ec6d8ffd4f4ce31f9d3bc06f5
Deleted: sha256:dcfb0f644fc5f2fa93c2446f1869f223dcbdc2ce942f7a19e316666d7c702704
Deleted: sha256:bc36941632d7d34d8e591d122a91139e75987a6495f7e10446f29acd6a5fef2d
Deleted: sha256:45f736826751a5e99ce6dc1623e6fc219ca0bb1b5d1e2d54349f35c2cd6d72ab
gcloud container images delete $CONTAINER:$TAG --quiet
Usage: gcloud container [optional flags] <group | command>
group may be clusters | node-pools | operations
command may be get-server-config
Deploy and manage clusters of machines for running containers.
flags:
Run `gcloud container --help`
for the full list of available flags for this command.
global flags:
Run `gcloud -h` for a description of flags available to all commands.
command groups:
clusters Deploy and teardown Google Container Engine clusters.
node-pools Create and delete operations for Google Container
Engine node pools.
operations Get and list operations for Google Container Engine
clusters.
commands:
get-server-config Get Container Engine server config.
For more detailed information on this command and its flags, run:
gcloud container --help
ERROR: (gcloud.container) Invalid choice: 'images'.
Valid choices are [clusters, get-server-config, node-pools, operations].
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
