See
<https://builds.apache.org/job/beam_PostCommit_Python_ValidatesContainer_Dataflow/20/display/redirect?page=changes>
Changes:
[jb] [BEAM-3668] Quick workaround fix for netty conflict waiting better fix
------------------------------------------
[...truncated 94.18 KB...]
copying
apache_beam/runners/dataflow/internal/clients/dataflow/dataflow_v1b3_client.py
-> apache-beam-2.4.0.dev0/apache_beam/runners/dataflow/internal/clients/dataflow
copying
apache_beam/runners/dataflow/internal/clients/dataflow/dataflow_v1b3_messages.py
->
apache-beam-2.4.0.dev0/apache_beam/runners/dataflow/internal/clients/dataflow
copying
apache_beam/runners/dataflow/internal/clients/dataflow/message_matchers.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/dataflow/internal/clients/dataflow
copying
apache_beam/runners/dataflow/internal/clients/dataflow/message_matchers_test.py
-> apache-beam-2.4.0.dev0/apache_beam/runners/dataflow/internal/clients/dataflow
copying apache_beam/runners/dataflow/native_io/__init__.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/dataflow/native_io
copying apache_beam/runners/dataflow/native_io/iobase.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/dataflow/native_io
copying apache_beam/runners/dataflow/native_io/iobase_test.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/dataflow/native_io
copying apache_beam/runners/dataflow/native_io/streaming_create.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/dataflow/native_io
copying apache_beam/runners/direct/__init__.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/bundle_factory.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/clock.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/consumer_tracking_pipeline_visitor.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/consumer_tracking_pipeline_visitor_test.py
-> apache-beam-2.4.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/direct_metrics.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/direct_metrics_test.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/direct_runner.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/direct_runner_test.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/evaluation_context.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/executor.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/helper_transforms.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/sdf_direct_runner.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/sdf_direct_runner_test.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/transform_evaluator.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/util.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/watermark_manager.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/experimental/__init__.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/experimental
copying apache_beam/runners/experimental/python_rpc_direct/__init__.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/experimental/python_rpc_direct
copying
apache_beam/runners/experimental/python_rpc_direct/python_rpc_direct_runner.py
-> apache-beam-2.4.0.dev0/apache_beam/runners/experimental/python_rpc_direct
copying apache_beam/runners/experimental/python_rpc_direct/server.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/experimental/python_rpc_direct
copying apache_beam/runners/job/__init__.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/job
copying apache_beam/runners/job/manager.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/job
copying apache_beam/runners/job/utils.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/job
copying apache_beam/runners/portability/__init__.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/fn_api_runner.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/fn_api_runner_test.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/maptask_executor_runner.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/maptask_executor_runner_test.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/universal_local_runner.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/universal_local_runner_main.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/universal_local_runner_test.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/test/__init__.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/test
copying apache_beam/runners/worker/__init__.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/bundle_processor.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/data_plane.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/data_plane_test.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/log_handler.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/log_handler_test.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/logger.pxd ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/logger.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/logger_test.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/opcounters.pxd ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/opcounters.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/opcounters_test.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/operation_specs.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/operations.pxd ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/operations.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/sdk_worker.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/sdk_worker_main.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/sdk_worker_main_test.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/sdk_worker_test.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/sideinputs.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/sideinputs_test.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/statesampler.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/statesampler_fast.pyx ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/statesampler_slow.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/statesampler_test.py ->
apache-beam-2.4.0.dev0/apache_beam/runners/worker
copying apache_beam/testing/__init__.py ->
apache-beam-2.4.0.dev0/apache_beam/testing
copying apache_beam/testing/pipeline_verifiers.py ->
apache-beam-2.4.0.dev0/apache_beam/testing
copying apache_beam/testing/pipeline_verifiers_test.py ->
apache-beam-2.4.0.dev0/apache_beam/testing
copying apache_beam/testing/test_pipeline.py ->
apache-beam-2.4.0.dev0/apache_beam/testing
copying apache_beam/testing/test_pipeline_test.py ->
apache-beam-2.4.0.dev0/apache_beam/testing
copying apache_beam/testing/test_stream.py ->
apache-beam-2.4.0.dev0/apache_beam/testing
copying apache_beam/testing/test_stream_test.py ->
apache-beam-2.4.0.dev0/apache_beam/testing
copying apache_beam/testing/test_utils.py ->
apache-beam-2.4.0.dev0/apache_beam/testing
copying apache_beam/testing/test_utils_test.py ->
apache-beam-2.4.0.dev0/apache_beam/testing
copying apache_beam/testing/util.py ->
apache-beam-2.4.0.dev0/apache_beam/testing
copying apache_beam/testing/util_test.py ->
apache-beam-2.4.0.dev0/apache_beam/testing
copying apache_beam/testing/data/standard_coders.yaml ->
apache-beam-2.4.0.dev0/apache_beam/testing/data
copying apache_beam/testing/data/trigger_transcripts.yaml ->
apache-beam-2.4.0.dev0/apache_beam/testing/data
copying apache_beam/transforms/__init__.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py ->
apache-beam-2.4.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py ->
apache-beam-2.4.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py ->
apache-beam-2.4.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py ->
apache-beam-2.4.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py ->
apache-beam-2.4.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py ->
apache-beam-2.4.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py ->
apache-beam-2.4.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py ->
apache-beam-2.4.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py ->
apache-beam-2.4.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py ->
apache-beam-2.4.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py ->
apache-beam-2.4.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py ->
apache-beam-2.4.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py ->
apache-beam-2.4.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py ->
apache-beam-2.4.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py ->
apache-beam-2.4.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd ->
apache-beam-2.4.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py ->
apache-beam-2.4.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py ->
apache-beam-2.4.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.4.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py ->
apache-beam-2.4.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py ->
apache-beam-2.4.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py ->
apache-beam-2.4.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py ->
apache-beam-2.4.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.4.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py ->
apache-beam-2.4.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py ->
apache-beam-2.4.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py ->
apache-beam-2.4.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.4.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd ->
apache-beam-2.4.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py ->
apache-beam-2.4.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py ->
apache-beam-2.4.0.dev0/apache_beam/utils
Writing apache-beam-2.4.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.4.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nocapture \
--processes=1 \
--process-timeout=900 \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1"
<https://builds.apache.org/job/beam_PostCommit_Python_ValidatesContainer_Dataflow/ws/src/sdks/python/container/local/lib/python2.7/site-packages/setuptools/dist.py>:355:
UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Python_ValidatesContainer_Dataflow/ws/src/sdks/python/apache_beam/io/gcp/gcsio.py>:166:
DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT)
... ok
----------------------------------------------------------------------
Ran 1 test in 445.711s
OK
# Delete the container locally and remotely
docker rmi $CONTAINER:$TAG
Untagged: us.gcr.io/apache-beam-testing/jenkins/python:20180210-174822
Untagged:
us.gcr.io/apache-beam-testing/jenkins/python@sha256:295bf498480317abe53869b062e279f82066fd9231c082e91514c9be108b0534
Deleted: sha256:33bbc6a04dfde8543534371dc2447412de6f43155f5e33ac4ec777e0b8200a4f
Deleted: sha256:85f2d833c13ea1e58d93dd06df63556f4805077c24a6b01cdcd440b7c9102858
Deleted: sha256:8503c00c85a7d125ccb15f8c2d2aa7020fb6dc134f37f5373f758e4db18de043
Deleted: sha256:de5da3f54473d4e21af7bc7ddcb6bca8d5192903c93617c8c4df5aa0fab18260
Deleted: sha256:c271eb02eb9adb6e737a046e850a5719c1ee06e4c68b586db35b26deacc7d750
Deleted: sha256:5e5ce1803f0fd9a3b262ab65ec48aa853914dbc2805c09f205b641b83f93022c
Deleted: sha256:842fcad8189ab558bbc7ebad7e788f6db3ac0d4126be6df433ead49c30cc4bfc
Deleted: sha256:c8dfa7adbcca05ec0bf382f3a0240f7235e422b9ed90807c6496a59a49bdcd31
gcloud container images delete $CONTAINER:$TAG --quiet
Usage: gcloud container [optional flags] <group | command>
group may be clusters | node-pools | operations
command may be get-server-config
Deploy and manage clusters of machines for running containers.
flags:
Run `gcloud container --help`
for the full list of available flags for this command.
global flags:
Run `gcloud -h` for a description of flags available to all commands.
command groups:
clusters Deploy and teardown Google Container Engine clusters.
node-pools Create and delete operations for Google Container
Engine node pools.
operations Get and list operations for Google Container Engine
clusters.
commands:
get-server-config Get Container Engine server config.
For more detailed information on this command and its flags, run:
gcloud container --help
ERROR: (gcloud.container) Invalid choice: 'images'.
Valid choices are [clusters, get-server-config, node-pools, operations].
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
