See
<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/4921/display/redirect?page=changes>
Changes:
[milantracy] [BEAM-8406] Add support for JSON format text tables
------------------------------------------
[...truncated 593.96 KB...]
copying apache_beam/utils/profiler.py ->
apache-beam-2.18.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py ->
apache-beam-2.18.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.18.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py ->
apache-beam-2.18.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py ->
apache-beam-2.18.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py ->
apache-beam-2.18.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py ->
apache-beam-2.18.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py ->
apache-beam-2.18.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py ->
apache-beam-2.18.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.18.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd ->
apache-beam-2.18.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py ->
apache-beam-2.18.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py ->
apache-beam-2.18.0.dev0/apache_beam/utils
Writing apache-beam-2.18.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.18.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1"
setup.py:198: UserWarning: You are using Apache Beam with Python 2. New
releases of Apache Beam will soon support Python 3 only.
'You are using Apache Beam with Python 2. '
<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/setuptools/dist.py>:476:
UserWarning: Normalizing '2.18.0.dev' to '2.18.0.dev0'
normalized_version,
running nosetests
running egg_info
<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:58:
UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/__init__.py>:84:
UserWarning: You are using Apache Beam with Python 2. New releases of Apache
Beam will soon support Python 3 only.
'You are using Apache Beam with Python 2. '
No handlers could be found for logger
"apache_beam.io.gcp.datastore.v1.datastoreio"
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
WARNING:apache_beam.runners.interactive.interactive_environment:Interactive
Beam requires Python 3.5.3+.
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies
required for Interactive Beam PCollection visualization are not available,
please use: `pip install apache-beam[interactive]` to install necessary
dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use
Interactive Beam features when you are not in an interactive environment such
as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python
2.7 interpreter.
<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/trigger_test.py>:517:
YAMLLoadWarning: calling yaml.load_all() without Loader=... is deprecated, as
the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full
details.
for spec in yaml.load_all(open(transcript_filename)):
WARNING:apache_beam.options.pipeline_options:--region not set; will default to
us-central1. Future releases of Beam will require the user to set --region
explicitly, or else have a default set via the gcloud tool.
https://cloud.google.com/compute/docs/regions-zones
WARNING:apache_beam.options.pipeline_options:--region not set; will default to
us-central1. Future releases of Beam will require the user to set --region
explicitly, or else have a default set via the gcloud tool.
https://cloud.google.com/compute/docs/regions-zones
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT)
... ok
WARNING:apache_beam.options.pipeline_options:--region not set; will default to
us-central1. Future releases of Beam will require the user to set --region
explicitly, or else have a default set via the gcloud tool.
https://cloud.google.com/compute/docs/regions-zones
WARNING:apache_beam.options.pipeline_options:--region not set; will default to
us-central1. Future releases of Beam will require the user to set --region
explicitly, or else have a default set via the gcloud tool.
https://cloud.google.com/compute/docs/regions-zones
test_metrics_fnapi_it
(apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest)
... FAIL
======================================================================
FAIL: test_metrics_fnapi_it
(apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py";,>
line 60, in test_metrics_fnapi_it
self.assertFalse(errors, str(errors))
AssertionError: Unable to match metrics for matcher namespace:
'apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline.UserMetricsDoFn'
name: 'distribution_values' step: 'metrics' attempted: sum_value: <100>
count_value: <4> min_value: <0> max_value: <100> committed: sum_value: <100>
count_value: <4> min_value: <0> max_value: <100>
Actual MetricResults:
MetricResult(key=MetricKey(step=metrics,
metric=MetricName(namespace=apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline.UserMetricsDoFn,
name=total_values), labels={}), committed=100, attempted=100)
MetricResult(key=MetricKey(step=generatedPtransform-172,
metric=MetricName(namespace=apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline.UserMetricsDoFn,
name=total_values), labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=metrics,
metric=MetricName(namespace=apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline.UserMetricsDoFn,
name=distribution_values), labels={}), committed=DistributionResult(sum=100,
count=3, min=0, max=100), attempted=DistributionResult(sum=100, count=3, min=0,
max=100))
MetricResult(key=MetricKey(step=generatedPtransform-172,
metric=MetricName(namespace=apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline.UserMetricsDoFn,
name=distribution_values), labels={}), committed=DistributionResult(sum=0,
count=1, min=0, max=0), attempted=DistributionResult(sum=0, count=1, min=0,
max=0))
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=ElementCount), labels={'output_user_name':
u'Create/MaybeReshuffle/Reshuffle/AddRandomKeys-out0', 'original_name':
u'Create/MaybeReshuffle/Reshuffle/AddRandomKeys-out0-ElementCount'}),
committed=4, attempted=4)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=TotalPdUsage), labels={'original_name': u'Service-pd_gb_seconds'}),
committed=35405, attempted=35405)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=ElementCount), labels={'output_user_name': u'GroupByKey/Reify-out0',
'original_name': u'GroupByKey/Reify-out0-ElementCount'}), committed=4,
attempted=4)
MetricResult(key=MetricKey(step=Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys,
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_FinishBundle),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=ElementCount), labels={'output_user_name': u'Create/Map(decode)-out0',
'original_name': u'Create/Map(decode)-out0-ElementCount'}), committed=3,
attempted=3)
MetricResult(key=MetricKey(step=s07, metric=MetricName(namespace=dataflow/v1b3,
name=ExecutionTime_FinishBundle), labels={}), committed=201, attempted=201)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=MeanByteCount), labels={'output_user_name':
u'Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow-out0',
'original_name':
u'Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow-out0-MeanByteCount'}),
committed=34, attempted=34)
MetricResult(key=MetricKey(step=Create/Impulse,
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_StartBundle),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=MeanByteCount), labels={'output_user_name': u'GroupByKey/Read-out0',
'original_name': u'GroupByKey/Read-out0-MeanByteCount'}), committed=22,
attempted=22)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=ExecutionStepProgress), labels={'original_name':
u'dax_workflow_stage_progress_/workflows/wf-2019-11-28_17_56_43-746395101226374261/phases/graph_runner/step-invocations/F20-invoke-5966526490717910510/map-tasks-completion/map-tasks',
'execution_step': u'F20'}), committed=1, attempted=1)
MetricResult(key=MetricKey(step=map_to_common_key,
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_FinishBundle),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=MeanByteCount), labels={'output_user_name':
u'Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys-out0', 'original_name':
u'Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys-out0-MeanByteCount'}),
committed=15, attempted=15)
MetricResult(key=MetricKey(step=Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps),
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_ProcessTimers),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys,
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_ProcessElement),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=MeanByteCount), labels={'output_user_name':
u'Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)-out0',
'original_name':
u'Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)-out0-MeanByteCount'}),
committed=29, attempted=29)
MetricResult(key=MetricKey(step=s07, metric=MetricName(namespace=dataflow/v1b3,
name=ExecutionTime_StartBundle), labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps),
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_FinishBundle),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=ElementCount), labels={'output_user_name':
u'Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Read-out0',
'original_name':
u'Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Read-out0-ElementCount'}),
committed=4, attempted=4)
MetricResult(key=MetricKey(step=Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey,
metric=MetricName(namespace=dataflow/v1b3, name=ShuffleBytesWritten),
labels={'original_name': u's5-write-shuffle14-ByteCount'}), committed=118,
attempted=118)
MetricResult(key=MetricKey(step=s03, metric=MetricName(namespace=dataflow/v1b3,
name=ExecutionTime_ProcessTimers), labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=ElementCount), labels={'output_user_name': u'Create/FlatMap(<lambda at
core.py:2532>)-out0', 'original_name': u'Create/FlatMap(<lambda at
core.py:2532>)-out0-ElementCount'}), committed=4, attempted=4)
MetricResult(key=MetricKey(step=GroupByKey,
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_FinishBundle),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=CurrentVcpuCount), labels={'original_name': u'Service-cpu_num'}),
committed=1, attempted=1)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=MeanByteCount), labels={'output_user_name': u'metrics-out0',
'original_name': u'metrics-out0-MeanByteCount'}), committed=14, attempted=14)
MetricResult(key=MetricKey(step=metrics,
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_ProcessElement),
labels={}), committed=2996, attempted=2996)
MetricResult(key=MetricKey(step=Create/MaybeReshuffle/Reshuffle/AddRandomKeys,
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_ProcessElement),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=s05, metric=MetricName(namespace=dataflow/v1b3,
name=ExecutionTime_FinishBundle), labels={}), committed=8842, attempted=8842)
MetricResult(key=MetricKey(step=Create/Map(decode),
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_ProcessElement),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps),
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_StartBundle),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys,
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_StartBundle),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=ElementCount), labels={'output_user_name':
u'GroupByKey/GroupByWindow-out0', 'original_name':
u'GroupByKey/GroupByWindow-out0-ElementCount'}), committed=1, attempted=1)
MetricResult(key=MetricKey(step=Create/Map(decode),
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_FinishBundle),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=ElementCount), labels={'output_user_name': u'map_to_common_key-out0',
'original_name': u'map_to_common_key-out0-ElementCount'}), committed=4,
attempted=4)
MetricResult(key=MetricKey(step=Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps),
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_ProcessTimers),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=map_to_common_key,
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_ProcessTimers),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=s05, metric=MetricName(namespace=dataflow/v1b3,
name=ExecutionTime_StartBundle), labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=Create/FlatMap(<lambda at core.py:2532>),
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_StartBundle),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey,
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_FinishBundle),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=GroupByKey,
metric=MetricName(namespace=dataflow/v1b3, name=ShuffleBytesRead),
labels={'original_name': u's11-read-shuffle7-ByteCount'}), committed=68,
attempted=68)
MetricResult(key=MetricKey(step=metrics,
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_ProcessTimers),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=GroupByKey,
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_WriteShuffle),
labels={}), committed=6804, attempted=6804)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=MeanByteCount), labels={'output_user_name': u'Create/FlatMap(<lambda at
core.py:2532>)-out0', 'original_name': u'Create/FlatMap(<lambda at
core.py:2532>)-out0-MeanByteCount'}), committed=15, attempted=15)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=ExecutionStepProgress), labels={'original_name':
u'dax_workflow_stage_progress_/workflows/wf-2019-11-28_17_56_43-746395101226374261/phases/graph_runner/step-invocations/F19-invoke-5966526490717911210/map-tasks-completion/map-tasks',
'execution_step': u'F19'}), committed=1, attempted=1)
MetricResult(key=MetricKey(step=m_out,
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_StartBundle),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=Create/Impulse,
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_ProcessElement),
labels={}), committed=200, attempted=200)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=TotalSsdUsage), labels={'original_name': u'Service-pd_ssd_gb_seconds'}),
committed=0, attempted=0)
MetricResult(key=MetricKey(step=Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey,
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_ProcessTimers),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=TotalMemoryUsage), labels={'original_name': u'Service-mem_mb_seconds'}),
committed=543824, attempted=543824)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=MeanByteCount), labels={'output_user_name':
u'Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)-out0',
'original_name':
u'Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)-out0-MeanByteCount'}),
committed=26, attempted=26)
MetricResult(key=MetricKey(step=Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey,
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_WriteShuffle),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=GroupByKey,
metric=MetricName(namespace=dataflow/v1b3, name=ShuffleBytesWritten),
labels={'original_name': u's11-write-shuffle5-ByteCount'}), committed=68,
attempted=68)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=ElementCount), labels={'output_user_name': u'metrics-out0',
'original_name': u'metrics-out0-ElementCount'}), committed=3, attempted=3)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=ElementCount), labels={'output_user_name':
u'Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)-out0',
'original_name':
u'Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)-out0-ElementCount'}),
committed=3, attempted=3)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=ElementCount), labels={'output_user_name':
u'Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow-out0',
'original_name':
u'Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow-out0-ElementCount'}),
committed=4, attempted=4)
MetricResult(key=MetricKey(step=Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps),
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_ProcessElement),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=GroupByKey,
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_ProcessTimers),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=MeanByteCount), labels={'output_user_name':
u'Create/MaybeReshuffle/Reshuffle/AddRandomKeys-out0', 'original_name':
u'Create/MaybeReshuffle/Reshuffle/AddRandomKeys-out0-MeanByteCount'}),
committed=26, attempted=26)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=CurrentSsdUsage), labels={'original_name': u'Service-pd_ssd_gb'}),
committed=0, attempted=0)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=CurrentPdUsage), labels={'original_name': u'Service-pd_gb'}),
committed=250, attempted=250)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=ElementCount), labels={'output_user_name':
u'Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys-out0', 'original_name':
u'Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys-out0-ElementCount'}),
committed=3, attempted=3)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=CurrentMemoryUsage), labels={'original_name': u'Service-mem_mb'}),
committed=3840, attempted=3840)
MetricResult(key=MetricKey(step=GroupByKey,
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_StartBundle),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=ElementCount), labels={'output_user_name': u'GroupByKey/Read-out0',
'original_name': u'GroupByKey/Read-out0-ElementCount'}), committed=1,
attempted=1)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=BillableShuffleDataProcessed), labels={'original_name':
u'Service-shuffle_service_chargeable_gb'}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=Create/MaybeReshuffle/Reshuffle/AddRandomKeys,
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_StartBundle),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=Create/Map(decode),
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_StartBundle),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps),
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_ProcessElement),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=TotalVcpuTime), labels={'original_name': u'Service-cpu_num_seconds'}),
committed=141, attempted=141)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=TotalShuffleDataProcessed), labels={'original_name':
u'Service-shuffle_service_actual_gb'}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=s03, metric=MetricName(namespace=dataflow/v1b3,
name=ExecutionTime_StartBundle), labels={}), committed=200, attempted=200)
MetricResult(key=MetricKey(step=s07, metric=MetricName(namespace=dataflow/v1b3,
name=ExecutionTime_ProcessTimers), labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=m_out,
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_ProcessTimers),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=Create/FlatMap(<lambda at core.py:2532>),
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_ProcessElement),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey,
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_ProcessElement),
labels={}), committed=406, attempted=406)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=MeanByteCount), labels={'output_user_name':
u'GroupByKey/GroupByWindow-out0', 'original_name':
u'GroupByKey/GroupByWindow-out0-MeanByteCount'}), committed=25, attempted=25)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=ElementCount), labels={'output_user_name': u'm_out-out0', 'original_name':
u'm_out-out0-ElementCount'}), committed=5, attempted=5)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=MeanByteCount), labels={'output_user_name': u'Create/Impulse-out0',
'original_name': u'Create/Impulse-out0-MeanByteCount'}), committed=14,
attempted=14)
MetricResult(key=MetricKey(step=Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys,
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_ProcessTimers),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps),
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_FinishBundle),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=m_out,
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_FinishBundle),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=MeanByteCount), labels={'output_user_name':
u'Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Reify-out0',
'original_name':
u'Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Reify-out0-MeanByteCount'}),
committed=45, attempted=45)
MetricResult(key=MetricKey(step=Create/MaybeReshuffle/Reshuffle/AddRandomKeys,
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_ProcessTimers),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=ElementCount), labels={'output_user_name':
u'Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)-out0',
'original_name':
u'Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)-out0-ElementCount'}),
committed=4, attempted=4)
MetricResult(key=MetricKey(step=map_to_common_key,
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_StartBundle),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=GroupByKey,
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_ProcessElement),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=metrics,
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_FinishBundle),
labels={}), committed=1995, attempted=1995)
MetricResult(key=MetricKey(step=Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey,
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_ReadShuffle),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=GroupByKey,
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_ReadShuffle),
labels={}), committed=203, attempted=203)
MetricResult(key=MetricKey(step=Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps),
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_StartBundle),
labels={}), committed=57, attempted=57)
MetricResult(key=MetricKey(step=Create/Impulse,
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_FinishBundle),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=s05, metric=MetricName(namespace=dataflow/v1b3,
name=ExecutionTime_ProcessTimers), labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey,
metric=MetricName(namespace=dataflow/v1b3, name=ShuffleBytesRead),
labels={'original_name': u's5-read-shuffle16-ByteCount'}), committed=118,
attempted=118)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=MeanByteCount), labels={'output_user_name': u'Create/Map(decode)-out0',
'original_name': u'Create/Map(decode)-out0-MeanByteCount'}), committed=14,
attempted=14)
MetricResult(key=MetricKey(step=Create/MaybeReshuffle/Reshuffle/AddRandomKeys,
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_FinishBundle),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=ExecutionStepProgress), labels={'original_name':
u'dax_workflow_stage_progress_/workflows/wf-2019-11-28_17_56_43-746395101226374261/phases/graph_runner/step-invocations/F18-invoke-5966526490717911910/map-tasks-completion/map-tasks',
'execution_step': u'F18'}), committed=1, attempted=1)
MetricResult(key=MetricKey(step=Create/Impulse,
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_ProcessTimers),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=s05, metric=MetricName(namespace=dataflow/v1b3,
name=ExecutionTime_ProcessElement), labels={}), committed=2004, attempted=2004)
MetricResult(key=MetricKey(step=s07, metric=MetricName(namespace=dataflow/v1b3,
name=ExecutionTime_ProcessElement), labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=MeanByteCount), labels={'output_user_name': u'GroupByKey/Reify-out0',
'original_name': u'GroupByKey/Reify-out0-MeanByteCount'}), committed=31,
attempted=31)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=MeanByteCount), labels={'output_user_name': u'map_to_common_key-out0',
'original_name': u'map_to_common_key-out0-MeanByteCount'}), committed=18,
attempted=18)
MetricResult(key=MetricKey(step=Create/FlatMap(<lambda at core.py:2532>),
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_ProcessTimers),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=s03, metric=MetricName(namespace=dataflow/v1b3,
name=ExecutionTime_ProcessElement), labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=Create/FlatMap(<lambda at core.py:2532>),
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_FinishBundle),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=m_out,
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_ProcessElement),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=ElementCount), labels={'output_user_name':
u'Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Reify-out0',
'original_name':
u'Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Reify-out0-ElementCount'}),
committed=4, attempted=4)
MetricResult(key=MetricKey(step=Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey,
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_StartBundle),
labels={}), committed=200, attempted=200)
MetricResult(key=MetricKey(step=metrics,
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_StartBundle),
labels={}), committed=1978, attempted=1978)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=MeanByteCount), labels={'output_user_name': u'm_out-out0',
'original_name': u'm_out-out0-MeanByteCount'}), committed=15, attempted=15)
MetricResult(key=MetricKey(step=Create/Map(decode),
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_ProcessTimers),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=TotalStreamingDataProcessed), labels={'original_name':
u'Service-streaming_service_gb'}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=ElementCount), labels={'output_user_name': u'Create/Impulse-out0',
'original_name': u'Create/Impulse-out0-ElementCount'}), committed=1,
attempted=1)
MetricResult(key=MetricKey(step=s03, metric=MetricName(namespace=dataflow/v1b3,
name=ExecutionTime_FinishBundle), labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=map_to_common_key,
metric=MetricName(namespace=dataflow/v1b3, name=ExecutionTime_ProcessElement),
labels={}), committed=0, attempted=0)
MetricResult(key=MetricKey(step=, metric=MetricName(namespace=dataflow/v1b3,
name=MeanByteCount), labels={'output_user_name':
u'Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Read-out0',
'original_name':
u'Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Read-out0-MeanByteCount'}),
committed=30, attempted=30)
-------------------- >> begin captured stdout << ---------------------
Worker logs:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-11-28_17_56_43-746395101226374261?project=apache-beam-testing
--------------------- >> end captured stdout << ----------------------
----------------------------------------------------------------------
XML: nosetests-python2.7_sdk.xml
----------------------------------------------------------------------
XML:
<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 2 tests in 1096.980s
FAILED (failures=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/python2.7_sdk:20191129-013600
Untagged:
us.gcr.io/apache-beam-testing/jenkins/python2.7_sdk@sha256:b8502c43d957534512daafb257c968ae563239d8e1958ea0cfe08b688c5df5f0
Deleted: sha256:e65c97ee009b03848fa66c38028ade209d6ce2d637aa746b2e9487aa5326fb11
Deleted: sha256:3b190c2cb82bcfbaf96e51eaf5b334b3ef5f4efae5bf5b2d9fd0aa392c31f354
Deleted: sha256:f585c28c5879f66ac25d36c400a30bb82e8f0753110b9a819cb24e934999e525
Deleted: sha256:496281d10c156b1d7ddfb01481396ef0529117e2b3f0ebeb80a2fb372e492dde
Deleted: sha256:0b6ad9d2e13a983e6d287ebd521b276a8f89edae1246149438f6b778edf1ed3b
Deleted: sha256:38c771ea79a7a3be7633b2932e7fe66753710efe0441c6d0c8f4fa40b2cba84b
Deleted: sha256:4003aa9ec6f66daece0c7ca000dcfe343d9463180113dda6a963d6b325cbaac1
Deleted: sha256:47b08f645823868d1b23ed74221cdb1c063ca9be763cab54fee2f13d74d31446
Deleted: sha256:76706883322b25497d3971a5faa8c40ddb412b0c23ed9666c0e6c012371c3e33
Deleted: sha256:f69d97b7be139faf7905eaa0614c2964cfd4ee0236219ab3d211fb45e9ff180f
Deleted: sha256:6e22dcd1277159ca27eadb6b00fa61e321218efc198ee47468766d0d712fda2b
Deleted: sha256:dd7344bf4aa496587ee9b591cf255c4a3ccf8ca946a90edb25d665841350d382
Deleted: sha256:8b81a52a7dc3d8f48add014662f7ae7becf47cc222c73db41bc049b739e9174c
Deleted: sha256:56c6ebe862f1d4e47156230aac8abdf63c0eb09296626d10c074f1163c1b76db
Deleted: sha256:8be6167a05e609ae939fc20ae7ef370a1ad280d322bf789682d335c00c5f16f5
Deleted: sha256:c78f8bb1d7ec987eafe56cadffe89eece526e5ee8d5fcfe15f324fa379e98ad2
Deleted: sha256:4cd56a06e979240df551c1cbc7b82929d1b6d799a15b6a4d9b9b0e8cf9ad1d01
Deleted: sha256:91cc26ea266a0917499b86e13224adb93be300a9966717dc614905b4ce4b42de
Deleted: sha256:85013cf7a79442b8020c802d5c61b6a28943045ece2848362e23be6fd0765aa3
Deleted: sha256:cf1a0c88d92adbd7f5e9198f7b7773c715d7fe27acd4227240aa407f33d94992
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/python2.7_sdk@sha256:b8502c43d957534512daafb257c968ae563239d8e1958ea0cfe08b688c5df5f0
Associated tags:
- 20191129-013600
Tags:
- us.gcr.io/apache-beam-testing/jenkins/python2.7_sdk:20191129-013600
Deleted [us.gcr.io/apache-beam-testing/jenkins/python2.7_sdk:20191129-013600].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/python2.7_sdk@sha256:b8502c43d957534512daafb257c968ae563239d8e1958ea0cfe08b688c5df5f0].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
